fix(build): a .def or .inl edit compiles the files that include it again, so updates and packs no longer keep an old switch table #1602

Merged
HumanBean merged 5 commits from klimovich008/Longshot:fix/header-ext-deps into main 2026-11-19 07:07:07 +00:00
Member

Type: fix · Risk: low · Tested on: Windows 11, RX 9070 XT (AMD 26.11.1), Ryzen 7 5700X3D · Needs: Linux, any GPU (a clean and an incremental build after a switches.def change)

TL;DR

The build decides what to compile again from the header files each source reads, but it only counted .h, .hpp and .inc. The switch table, gpu/vk/switches.def, is none of those, so a new switch row compiled nothing again: an incremental build, a developer's pack and a player's update kept the old table, and only a clean build had the new row. This PR counts .def and .inl too. In a real build after a row change, main compiled 0 objects and its renderer lacked the row; with this PR 8 objects compiled and the row reached the renderer. A new game-free test fails on main and passes here.

Breakdown

  • Who it helps: every player who updates (their update keeps compiled objects) and every developer who builds incrementally or makes packs, whenever a switch row (or any future .def/.inl include) changes.
  • What was wrong: the build's list of header extensions left out .def and .inl, so an edit to gpu/vk/switches.def changed no object's key and the old objects came back from the cache.
  • What changes: a .def or .inl edit under the watched folders now compiles again the files that read it, like a .h edit.
  • What does not change: objects that read no .def/.inl keep exactly their keys; the lift key (no re-lift), the shader-cache stamp (no re-prepare), eden, sirit and the prebuilt third-party objects; no switch, setting or default.
  • How it was checked: real builds with and without the fix after a switch row change (the row searched in the built DLL), a game-free test with broken variants, the full test suite against clean main, the game started and played on the fix.
  • What is still open: a Linux build (Test Farm asked); the offline shader baker's own stale-exe reuse (nx2/bake_ds/bake_all.py), a separate follow-up.
  • Risk: low: two extension lists gain two entries; the one cost is a one-time recompile of the host and renderer objects on the update that ships it (below).

Why

gpu/vk/switches.h includes gpu/vk/switches.def twice, and vk_main.cpp and vk_shader.cpp read the switch table through it. nx2/build.py keys each host and renderer object by the headers it read (from the compiler's depfile, or, without one, by a hash of every header under the watched folders), and both of those only took .h, .hpp and .inc. So adding a row to switches.def left every key as it was: the build reused the old vk_main.o and vk_shader.o, and the game said "<NAME>: not a switch of this build" for a switch that was in the source. We hit it with a dev pack that added a switch row. A clean build was right; incremental builds, packs (scripts/dev_packs.py) and players' updates were not.

No issue filed for it (searched the tracker and open PRs for switches.def, HEADER_EXTS, depfile, tree_headers, stale switch). Related: #62 (open) adds a host/switches.def and counts .def for it, in version.py's SOURCE_EXT (the same line as here) and in the host's header list as it was before the depfile keys (#673); with this PR merged, its host/switches.def would be counted by both key routes already, and its version.py line drops out. #1527 (open) only edits rows of gpu/vk/switches.def; with this PR its row changes reach incremental builds and updates.

How to reproduce

Seen on: Windows 11, llvm-mingw 20261104 (any OS: the lists are plain Python)
Settings and switches: defaults
Steps: build the game; change a row of gpu/vk/switches.def (for example a switch's summary text); build again in the same folder (or make a dev pack over a base).
Expected: vk_main.cpp and vk_shader.cpp compile again and the new row is in the renderer.
Actual: nothing compiles; the renderer keeps the old table.

What changed

The two extension lists that say what an #include can name gain .inl and .def, and the build uses one list for both of its key routes so they can't drift again.

  • build: nx2/build.py: HEADER_EXTS = (".h", ".hpp", ".inc", ".inl", ".def") (the depfile route, renderer_deps), and tree_headers (the folders' hash, the key without a depfile) uses HEADER_EXTS instead of its own tuple.
  • build: nx2/version.py: SOURCE_EXT gains .inl and .def, so the source fingerprint in the log's first line and longshot_version.txt changes with a switch row too. version.py and build.py are not lift inputs (make_game.lift_sources()), so no re-lift.
  • tests: tests/test_header_ext_deps.py (new, standard library only, no compiler): a fake compiler that follows includes and writes a clang-style depfile, driven through tree_headers → renderer_deps → object_key → compile_one → rekey_depfiles as the build runs them.

Measurements

Verdict: a build-correctness fix, no speed claim. After a switch row change, main compiles nothing and ships the old table; this PR compiles the 2 includers (plus the usual stamped few) and the row reaches the renderer. The one cost is a one-time recompile on the update that ships it.

Hardware: Windows 11, Ryzen 7 5700X3D (16 threads), RX 9070 XT, llvm-mingw 20261104, no DLSS
Scene: none (builds)
Method: dev packs (scripts/dev_packs.py) over a base built at main ef6038ab, in a throwaway worktree; the row change puts a marker into one switch's summary text in switches.def (never committed); the marker is counted in the built longshot_vk.dll. Objects compiled from the build's log. The checker's runs; the implementer's earlier runs gave the same counts.
How to reproduce: as How to reproduce above, then search the DLL for the new row's text.
Baseline: the builds ran on main ef6038ab, the base our packs were built over that day. Since then main (876dc360) changed nx2/build.py only in how lifted chunks find their function definitions (lifted_c.definitions), not in the object keys, the depfile route or the folders' hash, and the rebased commits are the same diff (git range-diff), so the counts hold. The existing checks that read build.py/version.py (test_renderer_depfiles.py, test_host_depfiles.py, test_version*.py) pass unchanged.

Before: main @ ef6038ab (build.py as on main) After: this PR @ 3253d200 (same fix as this head; rebased since, build.py's change unchanged)
Objects compiled after a row change, depfiles present (a developer's incremental build) 0 8: vk_main.cpp, vk_shader.cpp (the includers), hos_main.c, vk_menu.cpp (the version stamp), windows_stop.c, vk_hangdump.cpp, msvc_oldnames.c, mingw_msvc_shims.c (keyed by the folders' hash: their depfiles name nothing under the watched folders); 23 compiler-seconds
New row in longshot_vk.dll no (old table) yes (second marker 1, first 0)
Objects compiled after a row change, no usable depfiles (as a player's update: keep_translation deletes sa/build with the depfiles) — 105: 65 host .c, 40 renderer .cpp; eden, sirit, lifted 0; lift 0 s; 239 compiler-seconds, 37 s wall
New row in longshot_vk.dll, that build — yes
The fix alone, switches.def as on main (what ships), after the above — 0 (all from the object cache)

What a player's update pays once: the update that ships this compiles the 105 host and renderer objects again, about 240 compiler-seconds (37 s of wall time on 16 threads here; other core counts not measured), because the folders' hash now includes switches.def. No re-lift, no shader re-prepare, no eden/sirit or prebuilt third-party rebuild. Those same 105 objects already compile again on any release that changes any header under host/ or gpu/ (rc.6 → rc.7 changed 11), so on a normal release it costs nothing extra.

chk5-ctl   main's build.py + row change A   renderer alone  compiled 0     marker A in DLL: 0
chk5-fix   fix + row change A (blob-keyed)  game            compiled 105   238.9 compiler-s  marker A: 1
chk5-fix2  same + row change B (depfiles)   game            compiled 8     23.1 compiler-s   marker B: 1, A: 0
chk5-fix0  fix alone                        game            compiled 0

Flags and settings

None.

Testing

Tested:

  • In the game, Windows 11, RX 9070 XT, AMD 26.11.1: the fix alone (switches.def as on main) built as a pack over main ef6038ab, started, loaded a save and played Hyrule Field for the standard 20 s window: 72.9 fps, game clock 1.000, no new error-like line kind against three same-day runs of main, no "not a switch" for any switch of the build. (A start, not a measurement: the fix changes no game code.)
  • The real builds in Measurements: the new row reaches longshot_vk.dll only with the fix.
  • tests/test_header_ext_deps.py (8 checks, about 1 s): a new switch row recompiles its includer and leaves a file that doesn't read it alone; an .inl edit recompiles its includer (through the depfile route, not only the hash); the folders' hash changes with a .def/.inl edit; objects reading only .h files, and a folder with no .def/.inl, keep exactly their old keys; the fingerprint counts .def/.inl; every #include "..." under host/ and gpu/ ends with a counted extension. On main 5 of 8 fail; here 8 of 8 pass. Broken variants (the old list) bring the stale object back and must fail; six mutations of the fix (each list reverted, .inl or .def dropped, .DEF in the wrong case, the old SOURCE_EXT) each fail the test.
  • python -I -B tests/run_all.py --jobs 4 (Windows, llvm-mingw clang 20261104) on this branch (rebased, at e3749b3b: the head but for one condition in the new test's fake compiler, which passes on the head) and on main @ 876dc360: branch 684 passed, 46 failed, 51 skipped; main 683 passed, 46 failed, 51 skipped. The 46 failures are the same list on both (they fail on this Windows checkout on main too, among them test_auto_handheld.py and test_bug_report_defaults.py); the extra pass is the new test. Before the rebase, the same against main 430eef4f. test_version.py, test_version_long_paths.py, test_renderer_depfiles.py and test_host_depfiles.py pass.
  • mise run lint: clean apart from the SC2016 note at .forgejo/workflows/build-windows.yml:82 that main has too.

Not tested:

  • Linux and a cross build (RX_TARGET=windows on Linux): the lists are plain strings with no OS branch, and the test is standard-library only, but no Linux build was run.
  • An actual update from the previous release in the Windows builder; the build with no usable depfiles above is its equivalent on this PC.
  • The one-time recompile's time on other core counts.

Must be tested:

  • linux, any GPU: a clean build of this branch, then change a row of gpu/vk/switches.def (for example add a word to one switch's summary) and build again in the same folder, expecting the second build to compile vk_main.cpp and vk_shader.cpp again and the game to start with no error lines; on main the second build compiles neither (Test Farm: @Beanstalk, the comment below)

Risks

What could break: a key that changes when it shouldn't (an object compiled again for nothing) or one that doesn't when it should. Objects that read no .def/.inl keep their exact keys (tested); the folders' hash for the host and renderer gains one file. So the few objects keyed by that hash even with a depfile (their depfile names no header under the watched folders: msvc_oldnames.c, mingw_msvc_shims.c, windows_stop.c, vk_hangdump.cpp) also compile again on a switches.def edit, as they already do on any header edit there. A .def used only by the linker (a module definition file) under a watched folder would also count and cost a recompile when edited; none exists today.

Turning it off: revert the commit.

Compatibility: on the update that ships this, the host and renderer objects compile once more (about 240 compiler-seconds here, see Measurements), as on any release that changes a header under host/ or gpu/. No re-lift, no shader or pipeline cache change, no settings change, no MKWVK_API_VERSION change.

Left out, follow-ups:

  • nx2/bake_ds/bake_all.py:78: the offline shader baker reuses an existing bake_ds exe whatever changed in its sources (it includes switches.h), so after a switch change its table can be stale. A developer tool, not reached by make_game.py, build.py or either builder; not a key, so a separate fix.
  • scripts/dev_packs.py: its HEADERS list has no .def (a switches.def change counts as a link input, so a pack keeps both modules: conservative, correct), and its scan for host files that include gpu/ headers reads only .c/.h/.cpp (a host .inl including switches.def would route a pack to the renderer alone; no such file exists).

Checklist

Everyone

  • One topic.
  • The title is a Conventional Commit (type(scope): description) of the status line's type.
  • Why gives the reason plainly and links the issues that fit (the tracker searched).
  • Tested on my own PC where it runs (in the game, for a change the game runs), as Testing's Tested says; what
    can't be run by hand is in Testing's Not testable by hand, with why.
  • Measurements has a verdict, or one line saying why there is nothing to measure.
  • Nothing gets worse that I know of: any cost (speed, latency, memory, a setup that worked before) is measured and
    in Risks.
  • Testing names what was tested, what wasn't, and what must still be tested, where and how.
  • Rebased on current main.
  • python -I -B tests/run_all.py passes (any failure that also fails on main is named).
  • No game files, dumps, build output or generated files.
  • third_party/ untouched (a new folder with a PROVENANCE.md instead).
  • The license is OK: GPL-3.0-or-later, nothing incompatible and nothing decompiled.

When it applies

  • AI assistance recorded with Assisted-by: / Co-Authored-By: trailers.
  • A game-free test added for the fix, failing on main and passing here.

🤖 Generated with Claude Code

**Type:** fix · **Risk:** low · **Tested on:** Windows 11, RX 9070 XT (AMD 26.11.1), Ryzen 7 5700X3D · **Needs:** Linux, any GPU (a clean and an incremental build after a `switches.def` change) ## TL;DR The build decides what to compile again from the header files each source reads, but it only counted `.h`, `.hpp` and `.inc`. The switch table, `gpu/vk/switches.def`, is none of those, so a new switch row compiled nothing again: an incremental build, a developer's pack and a player's update kept the old table, and only a clean build had the new row. This PR counts `.def` and `.inl` too. In a real build after a row change, main compiled 0 objects and its renderer lacked the row; with this PR 8 objects compiled and the row reached the renderer. A new game-free test fails on main and passes here. ## Breakdown - **Who it helps:** every player who updates (their update keeps compiled objects) and every developer who builds incrementally or makes packs, whenever a switch row (or any future `.def`/`.inl` include) changes. - **What was wrong:** the build's list of header extensions left out `.def` and `.inl`, so an edit to `gpu/vk/switches.def` changed no object's key and the old objects came back from the cache. - **What changes:** a `.def` or `.inl` edit under the watched folders now compiles again the files that read it, like a `.h` edit. - **What does not change:** objects that read no `.def`/`.inl` keep exactly their keys; the lift key (no re-lift), the shader-cache stamp (no re-prepare), eden, sirit and the prebuilt third-party objects; no switch, setting or default. - **How it was checked:** real builds with and without the fix after a switch row change (the row searched in the built DLL), a game-free test with broken variants, the full test suite against clean main, the game started and played on the fix. - **What is still open:** a Linux build (Test Farm asked); the offline shader baker's own stale-exe reuse (`nx2/bake_ds/bake_all.py`), a separate follow-up. - **Risk:** low: two extension lists gain two entries; the one cost is a one-time recompile of the host and renderer objects on the update that ships it (below). ## Why `gpu/vk/switches.h` includes `gpu/vk/switches.def` twice, and `vk_main.cpp` and `vk_shader.cpp` read the switch table through it. `nx2/build.py` keys each host and renderer object by the headers it read (from the compiler's depfile, or, without one, by a hash of every header under the watched folders), and both of those only took `.h`, `.hpp` and `.inc`. So adding a row to `switches.def` left every key as it was: the build reused the old `vk_main.o` and `vk_shader.o`, and the game said "`<NAME>`: not a switch of this build" for a switch that was in the source. We hit it with a dev pack that added a switch row. A clean build was right; incremental builds, packs (`scripts/dev_packs.py`) and players' updates were not. No issue filed for it (searched the tracker and open PRs for switches.def, HEADER_EXTS, depfile, tree_headers, stale switch). Related: #62 (open) adds a `host/switches.def` and counts `.def` for it, in `version.py`'s `SOURCE_EXT` (the same line as here) and in the host's header list as it was before the depfile keys (#673); with this PR merged, its `host/switches.def` would be counted by both key routes already, and its `version.py` line drops out. #1527 (open) only edits rows of `gpu/vk/switches.def`; with this PR its row changes reach incremental builds and updates. ## How to reproduce **Seen on:** Windows 11, llvm-mingw 20261104 (any OS: the lists are plain Python) **Settings and switches:** defaults **Steps:** build the game; change a row of `gpu/vk/switches.def` (for example a switch's summary text); build again in the same folder (or make a dev pack over a base). **Expected:** `vk_main.cpp` and `vk_shader.cpp` compile again and the new row is in the renderer. **Actual:** nothing compiles; the renderer keeps the old table. ## What changed The two extension lists that say what an `#include` can name gain `.inl` and `.def`, and the build uses one list for both of its key routes so they can't drift again. - build: `nx2/build.py`: `HEADER_EXTS = (".h", ".hpp", ".inc", ".inl", ".def")` (the depfile route, `renderer_deps`), and `tree_headers` (the folders' hash, the key without a depfile) uses `HEADER_EXTS` instead of its own tuple. - build: `nx2/version.py`: `SOURCE_EXT` gains `.inl` and `.def`, so the source fingerprint in the log's first line and `longshot_version.txt` changes with a switch row too. `version.py` and `build.py` are not lift inputs (`make_game.lift_sources()`), so no re-lift. - tests: `tests/test_header_ext_deps.py` (new, standard library only, no compiler): a fake compiler that follows includes and writes a clang-style depfile, driven through `tree_headers` → `renderer_deps` → `object_key` → `compile_one` → `rekey_depfiles` as the build runs them. ## Measurements **Verdict:** a build-correctness fix, no speed claim. After a switch row change, main compiles nothing and ships the old table; this PR compiles the 2 includers (plus the usual stamped few) and the row reaches the renderer. The one cost is a one-time recompile on the update that ships it. Hardware: Windows 11, Ryzen 7 5700X3D (16 threads), RX 9070 XT, llvm-mingw 20261104, no DLSS Scene: none (builds) Method: dev packs (`scripts/dev_packs.py`) over a base built at main ef6038ab, in a throwaway worktree; the row change puts a marker into one switch's summary text in `switches.def` (never committed); the marker is counted in the built `longshot_vk.dll`. Objects compiled from the build's log. The checker's runs; the implementer's earlier runs gave the same counts. How to reproduce: as How to reproduce above, then search the DLL for the new row's text. Baseline: the builds ran on main ef6038ab, the base our packs were built over that day. Since then main (876dc360) changed `nx2/build.py` only in how lifted chunks find their function definitions (`lifted_c.definitions`), not in the object keys, the depfile route or the folders' hash, and the rebased commits are the same diff (`git range-diff`), so the counts hold. The existing checks that read `build.py`/`version.py` (`test_renderer_depfiles.py`, `test_host_depfiles.py`, `test_version*.py`) pass unchanged. | | Before: main @ ef6038ab (build.py as on main) | After: this PR @ 3253d200 (same fix as this head; rebased since, build.py's change unchanged) | |---|---|---| | Objects compiled after a row change, depfiles present (a developer's incremental build) | 0 | 8: `vk_main.cpp`, `vk_shader.cpp` (the includers), `hos_main.c`, `vk_menu.cpp` (the version stamp), `windows_stop.c`, `vk_hangdump.cpp`, `msvc_oldnames.c`, `mingw_msvc_shims.c` (keyed by the folders' hash: their depfiles name nothing under the watched folders); 23 compiler-seconds | | New row in `longshot_vk.dll` | no (old table) | yes (second marker 1, first 0) | | Objects compiled after a row change, no usable depfiles (as a player's update: `keep_translation` deletes `sa/build` with the depfiles) | — | 105: 65 host `.c`, 40 renderer `.cpp`; eden, sirit, lifted 0; lift 0 s; 239 compiler-seconds, 37 s wall | | New row in `longshot_vk.dll`, that build | — | yes | | The fix alone, `switches.def` as on main (what ships), after the above | — | 0 (all from the object cache) | **What a player's update pays once:** the update that ships this compiles the 105 host and renderer objects again, about 240 compiler-seconds (37 s of wall time on 16 threads here; other core counts not measured), because the folders' hash now includes `switches.def`. No re-lift, no shader re-prepare, no eden/sirit or prebuilt third-party rebuild. Those same 105 objects already compile again on any release that changes any header under `host/` or `gpu/` (rc.6 → rc.7 changed 11), so on a normal release it costs nothing extra. ``` chk5-ctl main's build.py + row change A renderer alone compiled 0 marker A in DLL: 0 chk5-fix fix + row change A (blob-keyed) game compiled 105 238.9 compiler-s marker A: 1 chk5-fix2 same + row change B (depfiles) game compiled 8 23.1 compiler-s marker B: 1, A: 0 chk5-fix0 fix alone game compiled 0 ``` ### Flags and settings None. ## Testing **Tested:** - In the game, Windows 11, RX 9070 XT, AMD 26.11.1: the fix alone (switches.def as on main) built as a pack over main ef6038ab, started, loaded a save and played Hyrule Field for the standard 20 s window: 72.9 fps, game clock 1.000, no new error-like line kind against three same-day runs of main, no "not a switch" for any switch of the build. (A start, not a measurement: the fix changes no game code.) - The real builds in Measurements: the new row reaches `longshot_vk.dll` only with the fix. - `tests/test_header_ext_deps.py` (8 checks, about 1 s): a new switch row recompiles its includer and leaves a file that doesn't read it alone; an `.inl` edit recompiles its includer (through the depfile route, not only the hash); the folders' hash changes with a `.def`/`.inl` edit; objects reading only `.h` files, and a folder with no `.def`/`.inl`, keep exactly their old keys; the fingerprint counts `.def`/`.inl`; every `#include "..."` under `host/` and `gpu/` ends with a counted extension. On main 5 of 8 fail; here 8 of 8 pass. Broken variants (the old list) bring the stale object back and must fail; six mutations of the fix (each list reverted, `.inl` or `.def` dropped, `.DEF` in the wrong case, the old `SOURCE_EXT`) each fail the test. - `python -I -B tests/run_all.py --jobs 4` (Windows, llvm-mingw clang 20261104) on this branch (rebased, at e3749b3b: the head but for one condition in the new test's fake compiler, which passes on the head) and on main @ 876dc360: branch 684 passed, 46 failed, 51 skipped; main 683 passed, 46 failed, 51 skipped. The 46 failures are the same list on both (they fail on this Windows checkout on main too, among them `test_auto_handheld.py` and `test_bug_report_defaults.py`); the extra pass is the new test. Before the rebase, the same against main 430eef4f. `test_version.py`, `test_version_long_paths.py`, `test_renderer_depfiles.py` and `test_host_depfiles.py` pass. - `mise run lint`: clean apart from the SC2016 note at `.forgejo/workflows/build-windows.yml:82` that main has too. **Not tested:** - Linux and a cross build (`RX_TARGET=windows` on Linux): the lists are plain strings with no OS branch, and the test is standard-library only, but no Linux build was run. - An actual update from the previous release in the Windows builder; the build with no usable depfiles above is its equivalent on this PC. - The one-time recompile's time on other core counts. **Must be tested:** - linux, any GPU: a clean build of this branch, then change a row of `gpu/vk/switches.def` (for example add a word to one switch's summary) and build again in the same folder, expecting the second build to compile `vk_main.cpp` and `vk_shader.cpp` again and the game to start with no error lines; on main the second build compiles neither (Test Farm: @Beanstalk, the comment below) ## Risks **What could break:** a key that changes when it shouldn't (an object compiled again for nothing) or one that doesn't when it should. Objects that read no `.def`/`.inl` keep their exact keys (tested); the folders' hash for the host and renderer gains one file. So the few objects keyed by that hash even with a depfile (their depfile names no header under the watched folders: `msvc_oldnames.c`, `mingw_msvc_shims.c`, `windows_stop.c`, `vk_hangdump.cpp`) also compile again on a `switches.def` edit, as they already do on any header edit there. A `.def` used only by the linker (a module definition file) under a watched folder would also count and cost a recompile when edited; none exists today. **Turning it off:** revert the commit. **Compatibility:** on the update that ships this, the host and renderer objects compile once more (about 240 compiler-seconds here, see Measurements), as on any release that changes a header under `host/` or `gpu/`. No re-lift, no shader or pipeline cache change, no settings change, no `MKWVK_API_VERSION` change. Left out, follow-ups: - `nx2/bake_ds/bake_all.py:78`: the offline shader baker reuses an existing `bake_ds` exe whatever changed in its sources (it includes `switches.h`), so after a switch change its table can be stale. A developer tool, not reached by `make_game.py`, `build.py` or either builder; not a key, so a separate fix. - `scripts/dev_packs.py`: its `HEADERS` list has no `.def` (a `switches.def` change counts as a link input, so a pack keeps both modules: conservative, correct), and its scan for host files that include `gpu/` headers reads only `.c/.h/.cpp` (a host `.inl` including `switches.def` would route a pack to the renderer alone; no such file exists). ## Checklist ### Everyone - [x] One topic. - [x] The title is a Conventional Commit (`type(scope): description`) of the status line's type. - [x] Why gives the reason plainly and links the issues that fit (the tracker searched). - [x] Tested on my own PC where it runs (in the game, for a change the game runs), as Testing's Tested says; what can't be run by hand is in Testing's Not testable by hand, with why. - [x] Measurements has a verdict, or one line saying why there is nothing to measure. - [x] Nothing gets worse that I know of: any cost (speed, latency, memory, a setup that worked before) is measured and in Risks. - [x] Testing names what was tested, what wasn't, and what must still be tested, where and how. - [x] Rebased on current `main`. - [x] `python -I -B tests/run_all.py` passes (any failure that also fails on `main` is named). - [x] No game files, dumps, build output or generated files. - [x] `third_party/` untouched (a new folder with a `PROVENANCE.md` instead). - [x] The license is OK: GPL-3.0-or-later, nothing incompatible and nothing decompiled. ### When it applies - [x] AI assistance recorded with `Assisted-by:` / `Co-Authored-By:` trailers. - [x] A game-free test added for the fix, failing on `main` and passing here. 🤖 Generated with [Claude Code](https://claude.com/claude-code)
nx2/build.py keyed the objects by the .h, .hpp and .inc files they read (HEADER_EXTS, and the folders'
blob in tree_headers), so a new row in gpu/vk/switches.def compiled nothing again: an update with the
old object cache, or a pack over a base, kept the old switch table in vk_main.o and vk_shader.o. Both
now count .def and .inl too; an object that reads neither keeps its key.

tests/test_header_ext_deps.py: a new switch row and an .inl edit through the depfile keys, the blob,
key stability for the rest, a broken variant, and every extension host/ and gpu/ include.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
A build with another gpu/vk/switches.def row named the same sources' fingerprint in its first log
line (nx2/version.py). The test's fake compiler writes its depfile in UTF-8 with '$' escaped, as
clang does, so a temp path with other letters or a '$' does not drop it to the folders' blob.
(gpt-6.2-sol review of f98d4dbd, points 3 and 4)

Co-Authored-By: Claude Opus 5.5 <[email protected]>
The .inl case reads a header too, so with .inl left out its key is that header's and not the
folders' blob, which an .inl edit changes anyway; its broken variant must keep the old object.
The include check matches names as the build does (case counts, no extension is not counted).
The fake compiler keeps a POSIX name's backslash and opens past-260 paths on Windows as
extended-length ones. (gpt-6.2-sol review of 7e7e4e58)

Co-Authored-By: Claude Opus 5.5 <[email protected]>
(gpt-6.2-sol review of 8f204aca)

Co-Authored-By: Claude Opus 5.5 <[email protected]>
test(build): the fake compiler opens a long path on a network share as \\?\UNC\
Some checks are pending
CI / Conventional Commit title (pull_request) Blocked by required conditions
CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Blocked by required conditions
CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Blocked by required conditions
CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Blocked by required conditions
CI / Workflow lint (actionlint) (pull_request) Blocked by required conditions
Guide screenshots / Guide images and local UI source review (pull_request) Blocked by required conditions
Third-party parts / As their notes say (pull_request) Blocked by required conditions
Third-party parts / The same as upstream (pull_request) Blocked by required conditions
Website / Website on Cloudflare (pull_request_target) Has been skipped
PR description / Description follows the template (pull_request_target) Successful in 4s
beanstalk/approval not approved yet
26fe663d29
A temporary folder on a share (\\server\share\...) past 248 characters
needs the \\?\UNC\ prefix, not \\?\ before the share's two backslashes;
a path that is already extended-length (\\?\...) is opened as it is.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Author
Member

Tested in game: Windows 11, RX 9070 XT (AMD 26.11.1), Ryzen 7 5700X3D: the fix alone built as a dev pack over main ef6038ab (switches.def as on main, so what ships), started, loaded a save and played Hyrule Field for 20 s: 72.9 fps, game clock 1.000, no new error-like line against three same-day runs of main, no 'not a switch' line. Separately, packs with a switch row changed: the new row is in longshot_vk.dll only with the fix (main: 0 objects compiled, old table).

Tested in game: Windows 11, RX 9070 XT (AMD 26.11.1), Ryzen 7 5700X3D: the fix alone built as a dev pack over main ef6038ab (switches.def as on main, so what ships), started, loaded a save and played Hyrule Field for 20 s: 72.9 fps, game clock 1.000, no new error-like line against three same-day runs of main, no 'not a switch' line. Separately, packs with a switch row changed: the new row is in longshot_vk.dll only with the fix (main: 0 objects compiled, old table).
klimovich008 changed title from WIP: fix(build): a .def or .inl edit compiles the files that include it again, so updates and packs no longer keep an old switch table to fix(build): a .def or .inl edit compiles the files that include it again, so updates and packs no longer keep an old switch table 2026-11-18 15:23:14 +00:00
Author
Member

@Beanstalk linux, any GPU: build this branch clean, start the game to the title screen, then add a word to one switch's summary text in gpu/vk/switches.def and build again in the same folder. Expect the second build to compile vk_main.cpp and vk_shader.cpp again (on main it compiles neither) and the game to start with no error lines. Fail on a build error, a second build that compiles neither file, a crash or error lines in the log.

@Beanstalk linux, any GPU: build this branch clean, start the game to the title screen, then add a word to one switch's summary text in gpu/vk/switches.def and build again in the same folder. Expect the second build to compile vk_main.cpp and vk_shader.cpp again (on main it compiles neither) and the game to start with no error lines. Fail on a build error, a second build that compiles neither file, a crash or error lines in the log.
Member

@klimovich008 only members of BigBeans can start the farm. One of them can approve this request by reacting 👍 to it or to this comment.

@klimovich008 only members of `BigBeans` can start the farm. One of them can approve this request by reacting 👍 to it or to this comment.
HumanBean left a comment

From the code and the evidence this looks right. I'm leaving it as a comment, not an approval, only because the Linux build named under Must be tested hasn't run yet. The Test Farm request is waiting for a BigBean's 👍, and CI is held until then too.

What I checked

  • The keys follow the includes and nothing else. HEADER_EXTS is now the one list for both key routes: renderer_deps (from the depfile) and tree_headers (the folders' blob). An object's key changes only when its depfile names a .def/.inl under its roots, or, without a depfile, when the blob changes. Today the only such file in the tree is gpu/vk/switches.def. So steady-state builds do no extra work. A switches.def edit compiles its two includers, plus the four objects that were already blob-keyed and already recompile on any header edit there. The second skip in host_blob (eden, sirit, third_party) still works, because the walk is unchanged.
  • The other tree_headers callers. vk_lib_deps (eden, sirit, compat), vdeps, xess_deps and pdeps cover folders that hold no .def/.inl today, so eden, sirit and the WebGPU side keep their keys, as the PR says. ngx_deps walks the local DLSS SDK's include/, which isn't in the tree. If that SDK ships an .inl, the worst case is that vk_dlss.cpp compiles again once.
  • The one-time cost on players' updates. The host blob now contains switches.def, so blob-keyed host and renderer objects compile again once. That is 105 objects, about 37 s of wall time on 16 threads. Any release that touches a header under host/ or gpu/ costs the same already, so there is no lasting cost to game build time.
  • Windows/Linux and UTF-8. Only two tuples changed. No path handling changed, and the normcase/startswith filter is untouched. The test uses a temp path with a space in it, writes the depfile as UTF-8 and escapes names as clang does.
  • The test. It drives the real key functions (tree_headers → renderer_deps → object_key → compile_one → rekey_depfiles) with a fake compiler that follows includes, in a temporary folder with OBJ_CACHE patched. Each case has a negative control that runs the old list and must keep the stale object. The repo-wide scan for included extensions that aren't counted is a useful guard. tests/test_dlss_sdk_cache.py and tests/test_shader_stamp_cache.py take the HEADER_EXTS assignment out with ast, and it is still a plain assignment, so they are unaffected.
  • nx2/version.py. SOURCE_EXT gains .def/.inl. The fingerprint only feeds the log line and longshot_version.txt, both computed by the same function on the same build, so the crash dialog's check that they are equal still holds. version.py is not a lift input, so nothing is lifted again.

Measurements: this is a build-correctness fix and claims no speed-up. The numbers come from the PR's own fix, measured as dev packs over a main base. After a row change, main compiles 0 objects and the DLL keeps the old table. With the fix, 8 objects compile (23 compiler-seconds) and the new row is in longshot_vk.dll. The one-time update cost is 105 objects, 239 compiler-seconds, 37 s wall on 16 threads. The fix alone, with switches.def as on main, compiles 0 objects. The direction is right and nothing gets slower. The cost on other core counts isn't measured, which is fine for a one-time cost.

Still needed before merge: the Linux run the PR asks for: a clean build, a row change in gpu/vk/switches.def, then a second build that compiles vk_main.cpp and vk_shader.cpp. Either the Test Farm (a BigBean approves the request above) or a Linux developer by hand.

Overlaps:

  • #62 edits the same SOURCE_EXT line in nx2/version.py. Whichever lands second needs a one-line rebase, and after this PR, 62's .def addition there drops out.
  • #1527 changes rows of switches.def and only reaches incremental builds and updates once this is in, so this one should go first.
From the code and the evidence this looks right. I'm leaving it as a comment, not an approval, only because the Linux build named under *Must be tested* hasn't run yet. The Test Farm request is waiting for a BigBean's 👍, and CI is held until then too. **What I checked** - **The keys follow the includes and nothing else.** `HEADER_EXTS` is now the one list for both key routes: `renderer_deps` (from the depfile) and `tree_headers` (the folders' blob). An object's key changes only when its depfile names a `.def`/`.inl` under its roots, or, without a depfile, when the blob changes. Today the only such file in the tree is `gpu/vk/switches.def`. So steady-state builds do no extra work. A `switches.def` edit compiles its two includers, plus the four objects that were already blob-keyed and already recompile on any header edit there. The second `skip` in `host_blob` (`eden`, `sirit`, `third_party`) still works, because the walk is unchanged. - **The other `tree_headers` callers.** `vk_lib_deps` (eden, sirit, compat), `vdeps`, `xess_deps` and `pdeps` cover folders that hold no `.def`/`.inl` today, so eden, sirit and the WebGPU side keep their keys, as the PR says. `ngx_deps` walks the local DLSS SDK's `include/`, which isn't in the tree. If that SDK ships an `.inl`, the worst case is that `vk_dlss.cpp` compiles again once. - **The one-time cost on players' updates.** The host blob now contains `switches.def`, so blob-keyed host and renderer objects compile again once. That is 105 objects, about 37 s of wall time on 16 threads. Any release that touches a header under `host/` or `gpu/` costs the same already, so there is no lasting cost to game build time. - **Windows/Linux and UTF-8.** Only two tuples changed. No path handling changed, and the `normcase`/`startswith` filter is untouched. The test uses a temp path with a space in it, writes the depfile as UTF-8 and escapes names as clang does. - **The test.** It drives the real key functions (`tree_headers` → `renderer_deps` → `object_key` → `compile_one` → `rekey_depfiles`) with a fake compiler that follows includes, in a temporary folder with `OBJ_CACHE` patched. Each case has a negative control that runs the old list and must keep the stale object. The repo-wide scan for included extensions that aren't counted is a useful guard. `tests/test_dlss_sdk_cache.py` and `tests/test_shader_stamp_cache.py` take the `HEADER_EXTS` assignment out with `ast`, and it is still a plain assignment, so they are unaffected. - **`nx2/version.py`.** `SOURCE_EXT` gains `.def`/`.inl`. The fingerprint only feeds the log line and `longshot_version.txt`, both computed by the same function on the same build, so the crash dialog's check that they are equal still holds. `version.py` is not a lift input, so nothing is lifted again. **Measurements:** this is a build-correctness fix and claims no speed-up. The numbers come from the PR's own fix, measured as dev packs over a main base. After a row change, main compiles 0 objects and the DLL keeps the old table. With the fix, 8 objects compile (23 compiler-seconds) and the new row is in `longshot_vk.dll`. The one-time update cost is 105 objects, 239 compiler-seconds, 37 s wall on 16 threads. The fix alone, with `switches.def` as on main, compiles 0 objects. The direction is right and nothing gets slower. The cost on other core counts isn't measured, which is fine for a one-time cost. **Still needed before merge:** the Linux run the PR asks for: a clean build, a row change in `gpu/vk/switches.def`, then a second build that compiles `vk_main.cpp` and `vk_shader.cpp`. Either the Test Farm (a BigBean approves the request above) or a Linux developer by hand. **Overlaps:** - https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/62 edits the same `SOURCE_EXT` line in `nx2/version.py`. Whichever lands second needs a one-line rebase, and after this PR, 62's `.def` addition there drops out. - https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/1527 changes rows of `switches.def` and only reaches incremental builds and updates once this is in, so this one should go first.
HumanBean deleted branch fix/header-ext-deps 2026-11-19 07:07:07 +00:00
Some checks are pending
CI / Conventional Commit title (pull_request) Blocked by required conditions
CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Blocked by required conditions
CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Blocked by required conditions
CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Blocked by required conditions
CI / Workflow lint (actionlint) (pull_request) Blocked by required conditions
Guide screenshots / Guide images and local UI source review (pull_request) Blocked by required conditions
Third-party parts / As their notes say (pull_request) Blocked by required conditions
Third-party parts / The same as upstream (pull_request) Blocked by required conditions
Website / Website on Cloudflare (pull_request_target) Has been skipped
PR description / Description follows the template (pull_request_target) Successful in 4s
beanstalk/approval not approved yet
Some workflows are waiting to be reviewed.
Sign in to join this conversation.
No reviewers
No milestone
No project
No assignees
3 participants
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
MagicBeanDev/Longshot!1602
No description provided.