feat(runtime): HOS_DEV_DLL_SWAP, a renderer built alone taken at the next start, checked against the game first

dyst commented

2026-11-13 11:43:44 +00:00

Member

Why

Trying a change to the renderer means quitting the game, building the whole game folder again and starting it, or copying the DLL in by hand with nothing to say whether it pairs with the executable. On Linux the renderer reaches the host through weak hos_* symbols, so one from another tree fails at dlopen, or calls through NULL. This builds the renderer alone while the game keeps running, checks it against the game, and has the game take it at its next start.

What changed

runtime: dev_dll_swap in host/rx_backend.c, called by load_vulkan before LoadLibraryA. With HOS_DEV_DLL_SWAP=1, a <renderer>.new beside the game (longshot_vk.dll.new, longshot_vk.so.new on Linux, the name LoadLibraryA's shim opens) is renamed over the renderer first, logged either way. The old process has ended by then (a restart waits for it: HOS_RESTART_OF), so nothing holds the old file. Off, nothing is looked for: a release never swaps.
script: scripts/stage_renderer.py GAME_DIR [RENDERER] (new, standard library) copies the renderer (default: the work folder's sa/build/ one) whole to <name>.new beside the executable (bin/ when the game has one), only when it pairs. Linux: every name it leaves for the loader (nm -D --undefined-only), weak ones too, unless versioned (the C and C++ runtime's) or the start files' own (__gmon_start__, _ITM_*...), is one the executable exports. Windows: every function it imports from the executable is one the executable exports (llvm-objdump -p); today the renderer imports none (the host hands it a table). One that doesn't pair isn't staged, and the names are said.
tests: tests/test_stage_renderer.py (new): the readers on nm and llvm-objdump output, the staging, a real Linux executable and shared object paired, and dev_dll_swap cut out of host/rx_backend.c and run with the switch on and off, a renderer staged and not: on Windows with the real Win32 calls, on Linux over host/posix/win32_posix.c, where it swaps the .so.

Flags and settings

Flag	What it does	Default now	Recommendation	Helps on	Doesn't help on	Untested on
`HOS_DEV_DLL_SWAP`	At the start, a staged `<renderer>.new` beside the game takes the renderer's place before it is loaded	off	off by default (a developer switch)	trying a renderer change without quitting the game for a whole game-folder build	players (nothing staged)	Linux, in the game (should help the same: the swap's code ran there over the POSIX shim)

Measurements

main: f41db065 (2026-11-13)
this PR: feat/renderer-swap @ 2a333461
Hardware: RTX 3080 (NVIDIA 621.14), Ryzen 9 3900X, 64 GB, Windows 11
Scene: Kokiri Forest from a save (the title's Continue), headless, muted; both builds from one work folder
Method: from a comment added to gpu/vk/vk_menu.cpp while the game is in play to being in play again on the new renderer, by the script's clock, rounds interleaved (main, PR, main, PR...). main: quit, make_game.py (the game folder built again; the lift skipped), launch, Continue. This PR: RX_VK_ONLY=1 nx2/build.py --standalone (the renderer alone) while the game plays on, scripts/stage_renderer.py, longshot_ctl.py relaunch (the game started with HOS_DEV_DLL_SWAP=1), Continue. The PR side was this commit on feat/relaunch @ 0fa72f49 (#97; the same patch), since a staged renderer is taken at the next start.
How to reproduce: as the method, with a save copied into each game folder.

	main @ `f41db065`	this PR @ 2a333461
renderer edit to in play again (s), median of 3	73.2	51.6
of it, building (s), median of 3	16.9	16.2

main 1: edit to play 266.2 s (the game folder built in 223.9 s, the lift run again)
pr 1:   edit to play 177.4 s (the renderer built in 137.3 s, every renderer file compiled; staged; relaunched: answering again after 1.8 s)
main 2: edit to play 73.9 s (the game folder built in 16.9 s)
pr 2:   edit to play 47.3 s (the renderer built in 13.1 s; staged; relaunched: answering again after 1.4 s)
main 3: edit to play 73.2 s (the game folder built in 16.4 s)
pr 3:   edit to play 51.6 s (the renderer built in 16.8 s; staged; relaunched: answering again after 1.6 s)
main 4: edit to play 69.0 s (the game folder built in 17.2 s)
pr 4:   edit to play 56.8 s (the renderer built in 16.2 s; staged; relaunched: answering again after 1.8 s)
the log: "HOS_DEV_DLL_SWAP: ...longshot_vk.dll.new took the old renderer's place"; errors in the PR runs' logs: 0

The medians are of rounds 2 to 4. Round 1 is left out on both sides: main's build ran the lift again, because the PR's first build from the other checkout had changed the shared work folder's lift key, and the PR's was its first RX_VK_ONLY build there, which compiles every renderer file once. In a warm work folder building the renderer alone is no faster than the game folder's build (everything else is cached); the ~22 s a loop saved is the game left running through the build and relaunched over the bridge, against quit, launch and wait. Rebased since onto main c67cd4a0: of its commits, one touches host/rx_backend.c (#63, the write marks' collection), not the loader this changes, and none touch the build's renderer path, so the numbers hold.

Testing

Tested: on the Windows PC above: the loop in the game (four rounds, the log saying the staged renderer took the old one's place, play in Kokiri Forest, no errors in the logs); tests/run_all.py, the whole suite, on this branch and on main c67cd4a0 (496 passed and 5 failed on main, 497 passed and 5 failed here, no failure main doesn't have: test_builder_dlss_update, test_check_third_party, test_fpx_sticky, test_release and test_spawn_emit fail on main too); tests/test_stage_renderer.py on Windows and on Linux (Docker, python:3.12 with gcc and binutils: the readers, the staging, a real executable and shared object paired, and dev_dll_swap run over the POSIX shim, swapping the .so; a copy with the .dll-to-.so rename removed fails it), three times on this branch's head.

Not tested: the swap in the game on Linux (its code ran there over the shim, not in a game build); a game folder with a launcher (bin/), where the script stages beside bin/longshot.

Needs other testers:

Linux: a game started with HOS_DEV_DLL_SWAP=1, a renderer staged with scripts/stage_renderer.py, then longshot_ctl.py relaunch, expecting "HOS_DEV_DLL_SWAP: ... took the old renderer's place" in the log and the game playing (T-PZ2F)

Screenshots

Nothing on screen.

Risk and rollback

With the switch set and a stale or wrong .new beside the game, that renderer loads in place of the old one: the script refuses one that doesn't pair, but a file copied by hand isn't checked. Without the switch nothing changes: no file is looked for. Unset HOS_DEV_DLL_SWAP (or delete the .new) to turn it off; reverting the two commits removes it.

Reviewer notes

The switch reference (docs/development.md, the README) isn't updated here: this PR carries code and tests, and the flag is noted under Flags and settings.

The dev-loop proposal's B1, its second part (https://github.com/NotRexed/Hookshot/pull/2636). It is independent of the relaunch PR (#97) in code, and measured with it, since a staged renderer is taken at the next start. build.py's check_vk_symbols checks the renderer alone at build time; this pairs it with the executable it will run under.

Issues

See https://github.com/NotRexed/Hookshot/pull/2636

Checklist

For everyone

One topic.
The title is a Conventional Commit (type(scope): description).
Rebased on current main.
python -I -B tests/run_all.py run and passing (any failure that also fails on main is named above).
Tested in the game, or what wasn't is named under Testing.
Testing filled in: what was tested, what wasn't, and what needs other testers (with test requests).
Measured: main and this PR at named commits, on the same PC, scene and settings, at least 2 interleaved runs each, the median reported.
No game files, dumps, build output or generated files.
third_party/ untouched (a new folder with a PROVENANCE.md instead).
The license is OK: GPL-3.0-or-later, nothing incompatible and nothing decompiled.

If applicable

AI assistance recorded with Assisted-by: / Co-Authored-By: trailers.
A game-free test added for the fix, failing on main and passing here.
Host code builds on both Windows (llvm-mingw) and Linux, with UTF-8 paths.

🤖 Generated with Claude Code

## Why Trying a change to the renderer means quitting the game, building the whole game folder again and starting it, or copying the DLL in by hand with nothing to say whether it pairs with the executable. On Linux the renderer reaches the host through weak `hos_*` symbols, so one from another tree fails at `dlopen`, or calls through NULL. This builds the renderer alone while the game keeps running, checks it against the game, and has the game take it at its next start. ## What changed - runtime: `dev_dll_swap` in `host/rx_backend.c`, called by `load_vulkan` before `LoadLibraryA`. With `HOS_DEV_DLL_SWAP=1`, a `<renderer>.new` beside the game (`longshot_vk.dll.new`, `longshot_vk.so.new` on Linux, the name `LoadLibraryA`'s shim opens) is renamed over the renderer first, logged either way. The old process has ended by then (a restart waits for it: `HOS_RESTART_OF`), so nothing holds the old file. Off, nothing is looked for: a release never swaps. - script: `scripts/stage_renderer.py GAME_DIR [RENDERER]` (new, standard library) copies the renderer (default: the work folder's `sa/build/` one) whole to `<name>.new` beside the executable (`bin/` when the game has one), only when it pairs. Linux: every name it leaves for the loader (`nm -D --undefined-only`), weak ones too, unless versioned (the C and C++ runtime's) or the start files' own (`__gmon_start__`, `_ITM_*`...), is one the executable exports. Windows: every function it imports from the executable is one the executable exports (`llvm-objdump -p`); today the renderer imports none (the host hands it a table). One that doesn't pair isn't staged, and the names are said. - tests: `tests/test_stage_renderer.py` (new): the readers on `nm` and `llvm-objdump` output, the staging, a real Linux executable and shared object paired, and `dev_dll_swap` cut out of `host/rx_backend.c` and run with the switch on and off, a renderer staged and not: on Windows with the real Win32 calls, on Linux over `host/posix/win32_posix.c`, where it swaps the `.so`. ## Flags and settings | Flag | What it does | Default now | Recommendation | Helps on | Doesn't help on | Untested on | |---|---|---|---|---|---|---| | `HOS_DEV_DLL_SWAP` | At the start, a staged `<renderer>.new` beside the game takes the renderer's place before it is loaded | off | off by default (a developer switch) | trying a renderer change without quitting the game for a whole game-folder build | players (nothing staged) | Linux, in the game (should help the same: the swap's code ran there over the POSIX shim) | ## Measurements main: f41db065 (2026-11-13) this PR: feat/renderer-swap @ 2a333461 Hardware: RTX 3080 (NVIDIA 621.14), Ryzen 9 3900X, 64 GB, Windows 11 Scene: Kokiri Forest from a save (the title's Continue), headless, muted; both builds from one work folder Method: from a comment added to `gpu/vk/vk_menu.cpp` while the game is in play to being in play again on the new renderer, by the script's clock, rounds interleaved (main, PR, main, PR...). main: quit, `make_game.py` (the game folder built again; the lift skipped), launch, Continue. This PR: `RX_VK_ONLY=1 nx2/build.py --standalone` (the renderer alone) while the game plays on, `scripts/stage_renderer.py`, `longshot_ctl.py relaunch` (the game started with `HOS_DEV_DLL_SWAP=1`), Continue. The PR side was this commit on feat/relaunch @ 0fa72f49 (#97; the same patch), since a staged renderer is taken at the next start. How to reproduce: as the method, with a save copied into each game folder. | | main @ f41db065 | this PR @ 2a333461 | |---|---|---| | renderer edit to in play again (s), median of 3 | 73.2 | 51.6 | | of it, building (s), median of 3 | 16.9 | 16.2 | ``` main 1: edit to play 266.2 s (the game folder built in 223.9 s, the lift run again) pr 1: edit to play 177.4 s (the renderer built in 137.3 s, every renderer file compiled; staged; relaunched: answering again after 1.8 s) main 2: edit to play 73.9 s (the game folder built in 16.9 s) pr 2: edit to play 47.3 s (the renderer built in 13.1 s; staged; relaunched: answering again after 1.4 s) main 3: edit to play 73.2 s (the game folder built in 16.4 s) pr 3: edit to play 51.6 s (the renderer built in 16.8 s; staged; relaunched: answering again after 1.6 s) main 4: edit to play 69.0 s (the game folder built in 17.2 s) pr 4: edit to play 56.8 s (the renderer built in 16.2 s; staged; relaunched: answering again after 1.8 s) the log: "HOS_DEV_DLL_SWAP: ...longshot_vk.dll.new took the old renderer's place"; errors in the PR runs' logs: 0 ``` The medians are of rounds 2 to 4. Round 1 is left out on both sides: main's build ran the lift again, because the PR's first build from the other checkout had changed the shared work folder's lift key, and the PR's was its first `RX_VK_ONLY` build there, which compiles every renderer file once. In a warm work folder building the renderer alone is no faster than the game folder's build (everything else is cached); the ~22 s a loop saved is the game left running through the build and relaunched over the bridge, against quit, launch and wait. Rebased since onto main c67cd4a0: of its commits, one touches `host/rx_backend.c` (#63, the write marks' collection), not the loader this changes, and none touch the build's renderer path, so the numbers hold. ## Testing **Tested:** on the Windows PC above: the loop in the game (four rounds, the log saying the staged renderer took the old one's place, play in Kokiri Forest, no errors in the logs); `tests/run_all.py`, the whole suite, on this branch and on main c67cd4a0 (496 passed and 5 failed on main, 497 passed and 5 failed here, no failure main doesn't have: test_builder_dlss_update, test_check_third_party, test_fpx_sticky, test_release and test_spawn_emit fail on main too); `tests/test_stage_renderer.py` on Windows and on Linux (Docker, python:3.12 with gcc and binutils: the readers, the staging, a real executable and shared object paired, and `dev_dll_swap` run over the POSIX shim, swapping the `.so`; a copy with the `.dll`-to-`.so` rename removed fails it), three times on this branch's head. **Not tested:** the swap in the game on Linux (its code ran there over the shim, not in a game build); a game folder with a launcher (`bin/`), where the script stages beside `bin/longshot`. **Needs other testers:** - Linux: a game started with `HOS_DEV_DLL_SWAP=1`, a renderer staged with `scripts/stage_renderer.py`, then `longshot_ctl.py relaunch`, expecting "HOS_DEV_DLL_SWAP: ... took the old renderer's place" in the log and the game playing (T-PZ2F) ## Screenshots Nothing on screen. ## Risk and rollback With the switch set and a stale or wrong `.new` beside the game, that renderer loads in place of the old one: the script refuses one that doesn't pair, but a file copied by hand isn't checked. Without the switch nothing changes: no file is looked for. Unset `HOS_DEV_DLL_SWAP` (or delete the `.new`) to turn it off; reverting the two commits removes it. ## Reviewer notes The switch reference (docs/development.md, the README) isn't updated here: this PR carries code and tests, and the flag is noted under Flags and settings. The dev-loop proposal's B1, its second part (https://github.com/NotRexed/Hookshot/pull/2636). It is independent of the relaunch PR (#97) in code, and measured with it, since a staged renderer is taken at the next start. `build.py`'s `check_vk_symbols` checks the renderer alone at build time; this pairs it with the executable it will run under. ## Issues See https://github.com/NotRexed/Hookshot/pull/2636 ## Checklist ### For everyone - [x] One topic. - [x] The title is a Conventional Commit (`type(scope): description`). - [x] Rebased on current `main`. - [x] `python -I -B tests/run_all.py` run and passing (any failure that also fails on `main` is named above). - [x] Tested in the game, or what wasn't is named under Testing. - [x] Testing filled in: what was tested, what wasn't, and what needs other testers (with test requests). - [x] Measured: main and this PR at named commits, on the same PC, scene and settings, at least 2 interleaved runs each, the median reported. - [x] No game files, dumps, build output or generated files. - [x] `third_party/` untouched (a new folder with a `PROVENANCE.md` instead). - [x] The license is OK: GPL-3.0-or-later, nothing incompatible and nothing decompiled. ### If applicable - [x] AI assistance recorded with `Assisted-by:` / `Co-Authored-By:` trailers. - [x] A game-free test added for the fix, failing on `main` and passing here. - [x] Host code builds on both Windows (llvm-mingw) and Linux, with UTF-8 paths. 🤖 Generated with [Claude Code](https://claude.com/claude-code)

dyst added 2 commits

2026-11-13 11:43:44 +00:00

feat(runtime): HOS_DEV_DLL_SWAP, a renderer built since taken at the next start, checked against the game first 48ff283ddf

Trying a renderer change meant building the whole game again (the lift checked, everything linked) or copying the DLL in
by hand while the game was closed, with nothing to say whether it paired with the executable: on Linux the renderer
reaches the host through weak hos_* symbols, so one from another tree fails at dlopen or calls through NULL.
scripts/stage_renderer.py checks the pairing (Linux: every name the renderer leaves for the loader, weak ones too, is one
the executable exports; Windows: its imports from the executable) and copies it whole beside the game as <name>.new;
with HOS_DEV_DLL_SWAP=1 the game takes it in the old one's place as it starts (host/rx_backend.c dev_dll_swap), so
longshot_ctl.py relaunch runs it. Off, nothing is looked for. The dev-loop proposal's B1, its second part
(NotRexed/Hookshot#2636).

Assisted-by: Claude Opus 5.5
Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>

test(stage_renderer): dev_dll_swap run on Linux too, over the POSIX shim, swapping the .so

CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Has been skipped

Details

CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Has been skipped

Details

CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Has been skipped

Details

Third-party parts / The same as upstream (pull_request) Has been skipped

Details

CI / Conventional Commit title (pull_request) Successful in 8s

Details

CI / Workflow lint (actionlint) (pull_request) Successful in 11s

Details

Third-party parts / As their notes say (pull_request) Successful in 12s

Details

PR description / Description follows the template (pull_request_target) Successful in 6s

Details

Website / Website on Cloudflare (pull_request_target) Has been skipped

Details

91c5273355

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>

dyst commented

2026-11-13 11:44:04 +00:00

Author

Member

Test request T-PZ2F opened for the Linux run (it needs the 'testing wanted' label to be claimed).

dyst changed title from ~~WIP: feat(runtime): HOS_DEV_DLL_SWAP, a renderer built alone taken at the next start, checked against the game first~~ to feat(runtime): HOS_DEV_DLL_SWAP, a renderer built alone taken at the next start, checked against the game first

2026-11-13 11:44:06 +00:00