WIP: fix(vk): the program cache's stamp holds only the switches the translation reads, in name order #2144

Draft
dyst wants to merge 1 commit from dyst/Longshot:fix/translator-stamp-env into main
Member

Summary

Setting RX_VK_MESH_CONV, the renderer's HOS_AB switch, at any value, made the game drop its whole shader program cache. It then translated every program again and prepared the shaders from cold: on this PC, 214 s to the first frame instead of 6 s. Unsetting it did the same again. The program cache's stamp took every variable beginning with RX_VK_MESH, and with it seven switches that change no translated program:

  • the draw's RX_VK_MESH_ARGS, _CONV, _COUNT_WORD, _FIRST_WORD and _IND_COUNT_WORD;
  • the mesh pipelines' _LANES and _OPT.

The stamp also hashed the switches in the environment's order. That order comes from longshot_env.txt's lines and from _putenv, which appends, so the same switches in another order dropped every cache too.

The stamp now holds the nine RX_VK_MESH_ switches the translation reads, by name, and hashes the entries in name order. The list moves out of vk_shader.cpp into gpu/vk/translator_env.h. No translator stamp covers that file (nx2/translator_files.py), so a later change to the list moves the stamp only for runs that set a name it takes or leaves, not every player's cache.

tests/test_program_stamp_knobs.py checks the list both ways:

  • every switch the translator reads is held, including EDEN_ reads through EnvSwitch and the names key_listed reads, which it did not see before;
  • the seven and RX_VK_GPL are not held.

It also compiles the header alone.

On main: start the game with RX_VK_MESH_CONV=1. The log says vk: program cache …: 0 program keys kept and the shaders are prepared before the first frame. Start it again without the switch, and the same happens.

Read first: gpu/vk/translator_env.h, translator_stamp in gpu/vk/vk_shader.cpp, tests/test_program_stamp_knobs.py.

Rebased onto main a2dc53859. Main's #2103 had since left RX_VK_GP_TABLE out of the loop this PR moves: its value reaches the stamp as graphics_table_loads, in the flags, and by its name too, setting it to its default prepared every shader again. It is translator_env.h's held_elsewhere now, and the check holds each such name to what in translator_stamp holds it (HELD_BY). The runs and checks below were made before the rebase, at 5b146577f; they have not run again.

Overlaps:

  • #1561 adds a value to RX_VK_MESH_FB_LAYOUT, which stays held by name: its description's "the RX_VK_MESH sweep" is this list now.
  • #2096's new RX_VK_NATIVE_NARROW is held by the RX_VK_NATIVE family.
  • #1542 edits translator_stamp's ABI words, not these lines.
  • #1510 is about the stamp's sizeof(Program), another of its inputs; not this.
  • #737 (one shader cache store for every game folder) keys its store by this stamp in vk_progcache.cpp, not by these lines. A stamp that moves less often keeps its store's programs longer.

Testing

Tested: Windows 11, RTX 3080 (NVIDIA 621.14), from the kokiri-start save with Skip title screen, through the PC's job queue. Two runtimes over main 971f77f52's base, each prepared at its making: main's alone, and this branch's pack with the switch table PR's change beside it (fix/switch-table-misses, a separate code path). Each run went to the 900th present, in the order: without the switch, RX_VK_MESH_CONV=1, without it again (Measurements).

  • main: RX_VK_MESH_CONV=1 and then going back to the default each gave 0 program keys kept, 0 KB and a preparation before the first frame.
  • this PR: 47867 program keys kept, 145508 KB in all three, with no preparation.
  • The runtime's own first start prepared from cold once, as the stamp moved: 0 program keys kept, then 47,867 programs in about 3 minutes.
  • Checks:
    • tests/test_program_stamp_knobs.py passes: 19 switches held, 2 log only, none of the 8. With RX_VK_MESH put back as a family, it fails and names the seven.
    • test_prepared_stamps, test_mesh_fallback_layout (a Linux check) and test_switch_table read the list from the header now.
    • vk_shader.cpp compiles (syntax, llvm-mingw).
    • Full suite (tests/run_all.py, llvm-mingw, both PRs' commits together): 770 passed, 108 failed, 59 skipped. Main 971f77f52 on this PC: 767 passed, 112 failed. No check failed here that passed on that main. Main's #2171 has since repaired many of its checks, so this comparison is against 971f77f52 only; the suite has not run on current main.
  • CI: the Website build and Guide screenshots jobs fail on main 1101b6127e's own push too (the guide's review hashes for menu files main changed); not this PR's.

Still to test:

  • Linux, any GPU: build this PR and start it with RX_VK_MESH_CONV=1 unset and set, expecting the build to pass and both starts to reach the title (@Beanstalk: the farm warms each start apart, so the cache kept across starts is Windows' result above)

Measurements

Verdict: a run that sets one of the seven keeps the program cache: 6 s to the first frame instead of 214 s, and back to the default 13 s instead of 203 s.

Setup: Windows 11, RTX 3080 (NVIDIA 621.14), 24 preparation workers. Both runtimes were over the base of main 971f77f52, each prepared at its making. Each run went from the kokiri-start save to the 900th present. The column is the time from launch to the first frame, and the program cache's line in the log.

Before: main @ 971f77f52 After: this PR @ 5b146577f
Without the switch (the runtime's first run) 77 s, 47,867 programs kept 68 s, 47,867 programs kept
Then with RX_VK_MESH_CONV=1 214 s, 0 kept: every program translated and prepared again 6 s, 47,867 kept
Then without it again 203 s, 0 kept: translated and prepared again 13 s, 47,867 kept

No switch is added and no default changes: translator_env.h names nine existing RX_VK_MESH_ switches (RX_VK_MESH_FB_LAYOUT, _FIRST, _GLOBAL_ATTRS, _NOCULL, _PAGES, _SHADERS, _SHARED_WARP, _SHARED_WARP_CHECK, _STRICT_FMUL) one by one, where the stamp took them by their family before.

Risks

  • One stamp move for every player: this edits vk_shader.cpp, a translator source, so its LONGSHOT_SHADER_STAMP moves once. At the first start after updating, every player translates and prepares again, as with any release that touches the translator; the driver's pipelines and the notes carry over (#1927). Later changes to the list move no player's stamp.
  • Two pipeline switches leave the stamp: RX_VK_MESH_LANES and RX_VK_MESH_OPT shape mesh pipelines but no program. Flipping one no longer prepares again from cold; the pipelines it changes are compiled at their first draw. No record holds them yet, as for the other pipeline-shaping switches (RX_VK_NOOPT, RX_VK_COMPUTE_PIN).
  • A new RX_VK_MESH_ translator switch must be named in translator_env.h; the check fails otherwise.
  • Turning it off: reverting is enough.

Checklist

Ready for review

  • One topic, on current main; the title is a Conventional Commit.
  • Tested on my own PC, in the game for a change the game runs, as Testing says.
  • mise run test passes (any failure that also fails on main is named).
  • Measured against main, or Measurements says why there is nothing to measure.
  • No game files, build output or hand-edited generated files; third_party/ untouched; nothing GPL-incompatible.

When it applies

  • A fix: a game-free test that fails on main and passes here.
  • A check under tests/ that reads changed source is updated, not weakened.
  • Host code builds on Windows (llvm-mingw) and Linux, with UTF-8 paths. Windows: built and run here. Linux: waits on the Test Farm run in Still to test.
  • Docs updated.
  • AI assistance recorded with Assisted-by: / Co-Authored-By: trailers.

Ready to merge

  • Everything in Still to test has run, its results in Tested.
  • Tier 1 hardware it can affect is tested (docs/dev/standards.md, "Hardware tiers").
  • Approved in review, every comment answered.
  • CI passes on the head; on current main, measured again or a comment says why the numbers still hold.

🤖 Generated with Claude Code

## Summary Setting `RX_VK_MESH_CONV`, the renderer's `HOS_AB` switch, at any value, made the game drop its whole shader program cache. It then translated every program again and prepared the shaders from cold: on this PC, 214 s to the first frame instead of 6 s. Unsetting it did the same again. The program cache's stamp took every variable beginning with `RX_VK_MESH`, and with it seven switches that change no translated program: - the draw's `RX_VK_MESH_ARGS`, `_CONV`, `_COUNT_WORD`, `_FIRST_WORD` and `_IND_COUNT_WORD`; - the mesh pipelines' `_LANES` and `_OPT`. The stamp also hashed the switches in the environment's order. That order comes from `longshot_env.txt`'s lines and from `_putenv`, which appends, so the same switches in another order dropped every cache too. The stamp now holds the nine `RX_VK_MESH_` switches the translation reads, by name, and hashes the entries in name order. The list moves out of `vk_shader.cpp` into `gpu/vk/translator_env.h`. No translator stamp covers that file (`nx2/translator_files.py`), so a later change to the list moves the stamp only for runs that set a name it takes or leaves, not every player's cache. `tests/test_program_stamp_knobs.py` checks the list both ways: - every switch the translator reads is held, including `EDEN_` reads through `EnvSwitch` and the names `key_listed` reads, which it did not see before; - the seven and `RX_VK_GPL` are not held. It also compiles the header alone. On main: start the game with `RX_VK_MESH_CONV=1`. The log says `vk: program cache …: 0 program keys kept` and the shaders are prepared before the first frame. Start it again without the switch, and the same happens. Read first: `gpu/vk/translator_env.h`, `translator_stamp` in `gpu/vk/vk_shader.cpp`, `tests/test_program_stamp_knobs.py`. Rebased onto main `a2dc53859`. Main's #2103 had since left `RX_VK_GP_TABLE` out of the loop this PR moves: its value reaches the stamp as `graphics_table_loads`, in the flags, and by its name too, setting it to its default prepared every shader again. It is `translator_env.h`'s `held_elsewhere` now, and the check holds each such name to what in `translator_stamp` holds it (`HELD_BY`). The runs and checks below were made before the rebase, at `5b146577f`; they have not run again. Overlaps: - #1561 adds a value to `RX_VK_MESH_FB_LAYOUT`, which stays held by name: its description's "the `RX_VK_MESH` sweep" is this list now. - #2096's new `RX_VK_NATIVE_NARROW` is held by the `RX_VK_NATIVE` family. - #1542 edits `translator_stamp`'s ABI words, not these lines. - #1510 is about the stamp's `sizeof(Program)`, another of its inputs; not this. - #737 (one shader cache store for every game folder) keys its store by this stamp in `vk_progcache.cpp`, not by these lines. A stamp that moves less often keeps its store's programs longer. ## Testing **Tested:** Windows 11, RTX 3080 (NVIDIA 621.14), from the `kokiri-start` save with Skip title screen, through the PC's job queue. Two runtimes over main `971f77f52`'s base, each prepared at its making: main's alone, and this branch's pack with the switch table PR's change beside it (`fix/switch-table-misses`, a separate code path). Each run went to the 900th present, in the order: without the switch, `RX_VK_MESH_CONV=1`, without it again (Measurements). - **main:** `RX_VK_MESH_CONV=1` and then going back to the default each gave `0 program keys kept, 0 KB` and a preparation before the first frame. - **this PR:** `47867 program keys kept, 145508 KB` in all three, with no preparation. - The runtime's own first start prepared from cold once, as the stamp moved: `0 program keys kept`, then 47,867 programs in about 3 minutes. - **Checks:** - `tests/test_program_stamp_knobs.py` passes: 19 switches held, 2 log only, none of the 8. With `RX_VK_MESH` put back as a family, it fails and names the seven. - `test_prepared_stamps`, `test_mesh_fallback_layout` (a Linux check) and `test_switch_table` read the list from the header now. - `vk_shader.cpp` compiles (syntax, llvm-mingw). - **Full suite** (`tests/run_all.py`, llvm-mingw, both PRs' commits together): 770 passed, 108 failed, 59 skipped. Main `971f77f52` on this PC: 767 passed, 112 failed. No check failed here that passed on that main. Main's #2171 has since repaired many of its checks, so this comparison is against `971f77f52` only; the suite has not run on current main. - **CI:** the Website build and Guide screenshots jobs fail on main `1101b6127e`'s own push too (the guide's review hashes for menu files main changed); not this PR's. **Still to test:** - Linux, any GPU: build this PR and start it with `RX_VK_MESH_CONV=1` unset and set, expecting the build to pass and both starts to reach the title (@Beanstalk: the farm warms each start apart, so the cache kept across starts is Windows' result above) ## Measurements **Verdict:** a run that sets one of the seven keeps the program cache: 6 s to the first frame instead of 214 s, and back to the default 13 s instead of 203 s. Setup: Windows 11, RTX 3080 (NVIDIA 621.14), 24 preparation workers. Both runtimes were over the base of main `971f77f52`, each prepared at its making. Each run went from the `kokiri-start` save to the 900th present. The column is the time from launch to the first frame, and the program cache's line in the log. | | Before: main @ 971f77f52 | After: this PR @ 5b146577f | |---|---|---| | Without the switch (the runtime's first run) | 77 s, 47,867 programs kept | 68 s, 47,867 programs kept | | Then with `RX_VK_MESH_CONV=1` | 214 s, 0 kept: every program translated and prepared again | 6 s, 47,867 kept | | Then without it again | 203 s, 0 kept: translated and prepared again | 13 s, 47,867 kept | No switch is added and no default changes: `translator_env.h` names nine existing `RX_VK_MESH_` switches (`RX_VK_MESH_FB_LAYOUT`, `_FIRST`, `_GLOBAL_ATTRS`, `_NOCULL`, `_PAGES`, `_SHADERS`, `_SHARED_WARP`, `_SHARED_WARP_CHECK`, `_STRICT_FMUL`) one by one, where the stamp took them by their family before. ## Risks - **One stamp move for every player:** this edits `vk_shader.cpp`, a translator source, so its `LONGSHOT_SHADER_STAMP` moves once. At the first start after updating, every player translates and prepares again, as with any release that touches the translator; the driver's pipelines and the notes carry over (#1927). Later changes to the list move no player's stamp. - **Two pipeline switches leave the stamp:** `RX_VK_MESH_LANES` and `RX_VK_MESH_OPT` shape mesh pipelines but no program. Flipping one no longer prepares again from cold; the pipelines it changes are compiled at their first draw. No record holds them yet, as for the other pipeline-shaping switches (`RX_VK_NOOPT`, `RX_VK_COMPUTE_PIN`). - **A new `RX_VK_MESH_` translator switch** must be named in `translator_env.h`; the check fails otherwise. - **Turning it off:** reverting is enough. ## Checklist ### Ready for review - [x] One topic, on current `main`; the title is a Conventional Commit. - [x] Tested on my own PC, in the game for a change the game runs, as Testing says. - [x] `mise run test` passes (any failure that also fails on `main` is named). - [x] Measured against `main`, or Measurements says why there is nothing to measure. - [x] No game files, build output or hand-edited generated files; `third_party/` untouched; nothing GPL-incompatible. ### When it applies - [x] A fix: a game-free test that fails on `main` and passes here. - [x] A check under `tests/` that reads changed source is updated, not weakened. - [ ] Host code builds on Windows (llvm-mingw) and Linux, with UTF-8 paths. Windows: built and run here. Linux: waits on the Test Farm run in Still to test. - [x] Docs updated. - [x] AI assistance recorded with `Assisted-by:` / `Co-Authored-By:` trailers. ### Ready to merge - [ ] Everything in Still to test has run, its results in Tested. - [ ] Tier 1 hardware it can affect is tested (docs/dev/standards.md, "Hardware tiers"). - [ ] Approved in review, every comment answered. - [ ] CI passes on the head; on current `main`, measured again or a comment says why the numbers still hold. 🤖 Generated with [Claude Code](https://claude.com/claude-code)
fix(vk): the program cache's stamp holds only the switches the translation reads, in name order
Some checks failed
CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Has been skipped
CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Has been skipped
CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Has been skipped
Third-party parts / The same as upstream (pull_request) Has been skipped
Website / Website on Cloudflare (pull_request) Has been skipped
CI / Conventional Commit title (pull_request) Successful in 8s
Guide screenshots / Guide images and local UI source review (pull_request) Failing after 13s
CI / Workflow lint (actionlint) (pull_request) Successful in 14s
Third-party parts / As their notes say (pull_request) Successful in 15s
Website / Website on Cloudflare (pull_request_target) Has been skipped
CI / Website build and checks (pull_request) Failing after 24s
beanstalk/approval can't be tested as asked: see the farm's comment
PR description / Description follows the template (pull_request_target) Successful in 19s
5b146577fb
The stamp took every set variable beginning with RX_VK_MESH, and so the draw's RX_VK_MESH_ARGS, _CONV, _COUNT_WORD,
_FIRST_WORD and _IND_COUNT_WORD and the mesh pipelines' _LANES and _OPT, none of which changes a translated program: setting
one, at any value, translated every program again and prepared from cold. RX_VK_MESH_CONV is the renderer's HOS_AB switch, so
an A/B that set it in the environment ran that side with a cold cache. The stamp now holds the nine RX_VK_MESH_ switches the
translation reads by name. It also hashed the entries in the environment's order, which longshot_env.txt's lines and
_putenv's appending set: the same switches in another order dropped every cache. They are hashed in name order now.

The list moves out of vk_shader.cpp into gpu/vk/translator_env.h, a file of no translator stamp (nx2/translator_files.py):
a change to it moves the stamp only for runs that set a name it takes or leaves, not every player's cache. This change
edits vk_shader.cpp, so its own LONGSHOT_SHADER_STAMP moves once, as every release that touches the translator does.

tests/test_program_stamp_knobs.py reads the list from the header, checks it both ways (every switch the translator reads,
EDEN_ reads through EnvSwitch and key_listed's lists included, is held; the seven RX_VK_MESH_ switches and RX_VK_GPL are not),
and compiles the header alone: held(), the name order, a removed switch left out.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Author
Member

Tested in game: Windows 11, RTX 3080, NVIDIA 621.14, from the kokiri-start save to the 900th present, over main 971f77f52's base. Runs in the order without the switch, RX_VK_MESH_CONV=1, without it again: main kept 47,867 programs, then 0 and 0, preparing from cold twice (214 s and 203 s to the first frame); this PR kept 47,867 in all three (68 s, 6 s, 13 s).

Tested in game: Windows 11, RTX 3080, NVIDIA 621.14, from the kokiri-start save to the 900th present, over main 971f77f52's base. Runs in the order without the switch, RX_VK_MESH_CONV=1, without it again: main kept 47,867 programs, then 0 and 0, preparing from cold twice (214 s and 203 s to the first frame); this PR kept 47,867 in all three (68 s, 6 s, 13 s).
Member

Possibly related, for context:

  • issue #1908 BUG: Vulkan Shader Preparation and Pipeline Cache Generation Extremely Slow on Linux — Reports slow shader preparation and cache reuse on Linux
  • issue #1510 The shader cache key includes the size of the program struct, so adding any member to it makes every player prepare all shaders again — Open issue on shader cache stamp invalidation; related cache-stamp correctness work
  • PR #737 feat(vk): one shader cache store for every game folder, so a build whose translation is unchanged prepares nothing — Open shared shader-cache store is keyed by this stamp
Possibly related, for context: - issue #1908 BUG: Vulkan Shader Preparation and Pipeline Cache Generation Extremely Slow on Linux — Reports slow shader preparation and cache reuse on Linux - issue #1510 The shader cache key includes the size of the program struct, so adding any member to it makes every player prepare all shaders again — Open issue on shader cache stamp invalidation; related cache-stamp correctness work - PR #737 feat(vk): one shader cache store for every game folder, so a build whose translation is unchanged prepares nothing — Open shared shader-cache store is keyed by this stamp
Author
Member

@Beanstalk Linux, any GPU: build this PR and start the game three times: without switches, with RX_VK_MESH_CONV=1, and without it again, each to the title or the first save. Expecting the build to pass, vk: program cache …: N program keys kept with the same N at all three starts once the first start has prepared, and no shader preparation at the second and third start.

@Beanstalk Linux, any GPU: build this PR and start the game three times: without switches, with RX_VK_MESH_CONV=1, and without it again, each to the title or the first save. Expecting the build to pass, `vk: program cache …: N program keys kept` with the same N at all three starts once the first start has prepared, and no shader preparation at the second and third start.
Member

Beanstalk test farm

Commit b5bf87d · approved by nobody yet · dashboard

A member of BigBeans can review this commit and comment @Beanstalk to test it.

<!-- beanstalk:sticky --> ### Beanstalk test farm Commit `b5bf87d` · approved by nobody yet · [dashboard](https://beanstalk.magicbean.dev/prs/2144) A member of `BigBeans` can review this commit and comment `@Beanstalk` to test it.
dyst force-pushed fix/translator-stamp-env from 5b146577fb
Some checks failed
CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Has been skipped
CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Has been skipped
CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Has been skipped
Third-party parts / The same as upstream (pull_request) Has been skipped
Website / Website on Cloudflare (pull_request) Has been skipped
CI / Conventional Commit title (pull_request) Successful in 8s
Guide screenshots / Guide images and local UI source review (pull_request) Failing after 13s
CI / Workflow lint (actionlint) (pull_request) Successful in 14s
Third-party parts / As their notes say (pull_request) Successful in 15s
Website / Website on Cloudflare (pull_request_target) Has been skipped
CI / Website build and checks (pull_request) Failing after 24s
beanstalk/approval can't be tested as asked: see the farm's comment
PR description / Description follows the template (pull_request_target) Successful in 19s
to b5bf87d0b1
Some checks failed
CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Has been skipped
CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Has been skipped
CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Has been skipped
Third-party parts / The same as upstream (pull_request) Has been skipped
Website / Website on Cloudflare (pull_request) Has been skipped
CI / Conventional Commit title (pull_request) Successful in 7s
Guide screenshots / Guide images and local UI source review (pull_request) Failing after 14s
CI / Workflow lint (actionlint) (pull_request) Successful in 15s
Third-party parts / As their notes say (pull_request) Successful in 17s
beanstalk/approval not approved yet
CI / Website build and checks (pull_request) Failing after 21s
PR description / Description follows the template (pull_request_target) Successful in 6s
2026-11-22 03:20:00 +00:00
Compare
Contributor

@dyst — this one looks ready for a second pair of eyes: it is clean on current main a2dc53859, the switch list is held by name and in name order, and the Windows numbers (47,867 programs kept, 6 s vs 214 s to the first frame) are the right kind of evidence for a high-priority cache bug.

Two notes so we don't duplicate anything:

  • We are not opening a competing PR or adopting this. If you want, one of us can review it or run the Linux/any-GPU start through the job queue when a game run is free.
  • We have no test idea to add: tests/test_program_stamp_knobs.py already checks the list both ways (held and not held) and compiles translator_env.h alone.

For the record, #1906 is the object-cache report, not this stamp; and #1542's commit 4 edits translator_stamp's ABI words, not this list — so neither covers #2144. Ping us if you want the extra review or the run.

@dyst — this one looks ready for a second pair of eyes: it is clean on current main a2dc53859, the switch list is held by name and in name order, and the Windows numbers (47,867 programs kept, 6 s vs 214 s to the first frame) are the right kind of evidence for a high-priority cache bug. Two notes so we don't duplicate anything: - We are not opening a competing PR or adopting this. If you want, one of us can review it or run the Linux/any-GPU start through the job queue when a game run is free. - We have no test idea to add: tests/test_program_stamp_knobs.py already checks the list both ways (held and not held) and compiles translator_env.h alone. For the record, #1906 is the object-cache report, not this stamp; and #1542's commit 4 edits translator_stamp's ABI words, not this list — so neither covers #2144. Ping us if you want the extra review or the run.
dazemc left a comment

Read of b5bf87d0 (the rebase plus held_elsewhere): correctness and what a Linux run still needs. The core fix is right: the nine held names are exactly the sixteen RX_VK_MESH_* rows minus the seven draw/pipeline ones, entries() sorts by name and still drops switches::find_gone names (gpu/vk/translator_env.h:64-76), and held()/find_gone are plain strncmp/strcmp. _environ is used exactly as main used it (gpu/vk/vk_shader.cpp:1473), so nothing in the diff is platform-specific.

One correctness gap in held_elsewhere. HELD_BY asserts that graphics_table_loads holds RX_VK_GP_TABLE (tests/test_program_stamp_knobs.py:48), and the header leaves the name out on that strength (gpu/vk/translator_env.h:41-43). But the translator reads the name itself too: gpu/vk/eden/shader_recompiler/frontend/maxwell/translate_program.cpp:202 is static const bool table_check{std::getenv("RX_VK_GP_TABLE") != nullptr && std::atoi(...) == 2};, and with it true the program gets a storage descriptor at c0+0xff0 (:204-213). The flag only holds table_loads_wanted(...) (gpu/vk/vk_coopmat.cpp:672-675), which returns 0 whenever the device has no host-memory import or no buffer device addresses — the case the code logs for at :698-700. There, RX_VK_GP_TABLE=2 and unset both put 0 in the flags at vk_shader.cpp:1429, yet table_check is true for one and false for the other, so they store different Program::Info.storage_buffers_descriptors under one stamp. main skipped the name with a !strncmp too, so the gap is not introduced here; what is new is the test asserting the premise. Fix: fold table_check into the flags beside graphics_table_loads, or leave the name in the stamp (it is a diagnostic, so a cold translate when it is set costs only that run).

The reference page now under-describes the list. docs/dev/reference/switches.md:79-83 says the stamp holds "families ... and names one by one" and that the check keeps "a draw or pipeline switch" out. It does not mention held_elsewhere, so a reader adding an RX_VK_GP_* switch believes the family holds it when RX_VK_GP_TABLE is the exception; and the "draw or pipeline switch must not" half is only checked for the eight NOT_HELD names. A sentence naming held_elsewhere (and the eight, as today's check) would make the page true.

Linux start: not verifiable from code, and not run at this head. Nothing in the diff is platform-specific (above), and the header ships: scripts/linux/appimage.sh:78 copies gpu/vk/ whole, so no packaging list needs it. But the description's suite and runs are from 5b146577f, before the rebase, and held_elsewhere with its fixture case has run nowhere. A Linux run would need:

  1. python -I -B tests/test_program_stamp_knobs.py --cxx g++ at b5bf87d0: the fixture compiles translator_env.h alone, and with glibc's g++ that is the only Linux-only compile surface in the diff.
  2. One build, then three starts from one game folder — unset, RX_VK_MESH_CONV=1, unset — reading the vk: program cache … N program keys kept line each time. The first start after the update is the cold one (the stamp moves once); the second and third are the ones that prove the fix.

The Test Farm warms each start apart, so it can prove the build and that both settings reach the title, not that the cache was kept. That half is a single machine's run (the job queue), which is what the Windows table already shows and Linux still does not.

**Read of `b5bf87d0` (the rebase plus `held_elsewhere`): correctness and what a Linux run still needs.** The core fix is right: the nine held names are exactly the sixteen `RX_VK_MESH_*` rows minus the seven draw/pipeline ones, `entries()` sorts by name and still drops `switches::find_gone` names (`gpu/vk/translator_env.h:64-76`), and `held()`/`find_gone` are plain `strncmp`/`strcmp`. `_environ` is used exactly as main used it (`gpu/vk/vk_shader.cpp:1473`), so nothing in the diff is platform-specific. **One correctness gap in `held_elsewhere`.** `HELD_BY` asserts that `graphics_table_loads` holds `RX_VK_GP_TABLE` (`tests/test_program_stamp_knobs.py:48`), and the header leaves the name out on that strength (`gpu/vk/translator_env.h:41-43`). But the translator reads the name itself too: `gpu/vk/eden/shader_recompiler/frontend/maxwell/translate_program.cpp:202` is `static const bool table_check{std::getenv("RX_VK_GP_TABLE") != nullptr && std::atoi(...) == 2};`, and with it true the program gets a storage descriptor at c0+0xff0 (`:204-213`). The flag only holds `table_loads_wanted(...)` (`gpu/vk/vk_coopmat.cpp:672-675`), which returns 0 whenever the device has no host-memory import or no buffer device addresses — the case the code logs for at `:698-700`. There, `RX_VK_GP_TABLE=2` and unset both put `0` in the flags at `vk_shader.cpp:1429`, yet `table_check` is true for one and false for the other, so they store different `Program::Info.storage_buffers_descriptors` under one stamp. main skipped the name with a `!strncmp` too, so the gap is not introduced here; what is new is the test asserting the premise. Fix: fold `table_check` into the flags beside `graphics_table_loads`, or leave the name in the stamp (it is a diagnostic, so a cold translate when it is set costs only that run). **The reference page now under-describes the list.** `docs/dev/reference/switches.md:79-83` says the stamp holds "families ... and names one by one" and that the check keeps "a draw or pipeline switch" out. It does not mention `held_elsewhere`, so a reader adding an `RX_VK_GP_*` switch believes the family holds it when `RX_VK_GP_TABLE` is the exception; and the "draw or pipeline switch must not" half is only checked for the eight `NOT_HELD` names. A sentence naming `held_elsewhere` (and the eight, as today's check) would make the page true. **Linux start: not verifiable from code, and not run at this head.** Nothing in the diff is platform-specific (above), and the header ships: `scripts/linux/appimage.sh:78` copies `gpu/vk/` whole, so no packaging list needs it. But the description's suite and runs are from `5b146577f`, before the rebase, and `held_elsewhere` with its fixture case has run nowhere. A Linux run would need: 1. `python -I -B tests/test_program_stamp_knobs.py --cxx g++` at `b5bf87d0`: the fixture compiles `translator_env.h` alone, and with glibc's g++ that is the only Linux-only compile surface in the diff. 2. One build, then three starts from one game folder — unset, `RX_VK_MESH_CONV=1`, unset — reading the `vk: program cache … N program keys kept` line each time. The first start after the update is the cold one (the stamp moves once); the second and third are the ones that prove the fix. The Test Farm warms each start apart, so it can prove the build and that both settings reach the title, not that the cache was kept. That half is a single machine's run (the job queue), which is what the Windows table already shows and Linux still does not.
dazemc left a comment

Read-only review — from the diff at the head, main a2dc5385 and the sources. No checks or game run from me (the PC is held); the PR is WIP, so this is a comment, not an approval.

Scope, today. I found no missed translation input, and the seven draw/pipeline switches are correctly out. RX_VK_GP_TABLE is out by name and its value is in flags (graphics_table_loads); RX_VK_MISS_LOG and RX_VK_BLOCK_PROBE are correctly held — they change the emitted program (the c0+0xff0 descriptor, 16 KB slices). The other inputs are held by value where the name is not (WARP64, NO_F16, FRAG_SHUFFLE_FALLBACK, HANDLE_TABLES, SIM_TURF_POOLS, FORCE_MESH_FALLBACK and the MMA/NaN/invariant values).

The gap I would close before ready. The Risk says "A new RX_VK_MESH_ translator switch must be named in translator_env.h; the check fails otherwise." For 7 of the 9 names it would not fail:

  • tests/test_program_stamp_knobs.py scans gpu/vk/eden/shader_recompiler/** and only the two names in SHADER_SWITCHES from vk_shader.cpp (plus key_listed).
  • _PAGES, _GLOBAL_ATTRS, _SHARED_WARP and _SHARED_WARP_CHECK are read in gpu/vk/vk_coopmat.cpp:708-716; _SHADERS in gpu/vk/vk_main.cpp:416; _STRICT_FMUL through gpu/vk/device_policy.h:741; _FB_LAYOUT in gpu/vk/vk_shader.cpp:1280. Only _FIRST (spirv_emit_context.cpp:580) and _NOCULL (vk_shader.cpp) are seen; _FB_LAYOUT has its own presence check in test_mesh_fallback_layout.py.
  • So a new RX_VK_MESH_* that changes the translation but is read in vk_coopmat.cpp, vk_main.cpp or vk_draw.cpp passes this check and leaves a stale cache — the case the old family caught. Scanning gpu/vk for RX_VK_MESH_ (or all RX_VK_/EDEN_) reads and requiring each to be in the list or NOT_HELD would close it, and would also prove the nine are reads.

Smaller, no stale cache. Five of the nine (_PAGES, _GLOBAL_ATTRS, _SHARED_WARP, _SHARED_WARP_CHECK, _STRICT_FMUL) already reach the stamp in flags; with the name held too, setting one to the value the device already uses still drops the whole cache — the needless drop this PR removes for _CONV. held_elsewhere/HELD_BY is the mechanism (it is why GP_TABLE is there).

Checked and sound. Name order: entries() sorts on the name before =, and std::string::compare is byte-wise, so it is locale-independent; the fixture's EDEN… < RX_VK… order matches, and the comparator is a strict weak order (only duplicate environ names are equivalent). Old stamp: a mismatch drops the cache whole, and the stamp moves once for every player because vk_shader.cpp is a translator source, so the name order adds no second move. translator_env.h is not in nx2/translator_files.py's DIRS/FILES, so later list edits move no cache. #1542 commit 4 edits other lines of the same function — no conflict, but if both land close together it is two re-preparations, not one (one line in Risks); #2177 is the test runner; #2199 sees one more gpu/vk/ header in vk_shader.cpp's depfile, one recompile. Linux: no locale, no Vulkan, no pointer-size assumption; the header compiles alone under -std=c++20 -Wall -Werror, as the fixture does. A Linux start proves only the end-to-end "cache kept across starts" (the N program keys kept line), not the stamp itself.

Evidence on the head. The description says the runs and checks were made at 5b146577f and "have not run again" after the rebase to a2dc5385, and the rebase changed the very check (HELD_BY/graphics_table_loads). python -I -B tests/test_program_stamp_knobs.py is game-free and takes seconds; test_mesh_fallback_layout, test_prepared_stamps and test_switch_table are the other changed ones. Measurements are on 971f77f52; the status line already warns about that.

**Read-only review** — from the diff at the head, main `a2dc5385` and the sources. No checks or game run from me (the PC is held); the PR is WIP, so this is a comment, not an approval. **Scope, today.** I found no missed translation input, and the seven draw/pipeline switches are correctly out. `RX_VK_GP_TABLE` is out by name and its value is in `flags` (`graphics_table_loads`); `RX_VK_MISS_LOG` and `RX_VK_BLOCK_PROBE` are correctly held — they change the emitted program (the c0+0xff0 descriptor, 16 KB slices). The other inputs are held by value where the name is not (`WARP64`, `NO_F16`, `FRAG_SHUFFLE_FALLBACK`, `HANDLE_TABLES`, `SIM_TURF_POOLS`, `FORCE_MESH_FALLBACK` and the MMA/NaN/invariant values). **The gap I would close before ready.** The Risk says "A new `RX_VK_MESH_` translator switch must be named in `translator_env.h`; the check fails otherwise." For 7 of the 9 names it would not fail: - `tests/test_program_stamp_knobs.py` scans `gpu/vk/eden/shader_recompiler/**` and only the two names in `SHADER_SWITCHES` from `vk_shader.cpp` (plus `key_listed`). - `_PAGES`, `_GLOBAL_ATTRS`, `_SHARED_WARP` and `_SHARED_WARP_CHECK` are read in `gpu/vk/vk_coopmat.cpp:708-716`; `_SHADERS` in `gpu/vk/vk_main.cpp:416`; `_STRICT_FMUL` through `gpu/vk/device_policy.h:741`; `_FB_LAYOUT` in `gpu/vk/vk_shader.cpp:1280`. Only `_FIRST` (`spirv_emit_context.cpp:580`) and `_NOCULL` (`vk_shader.cpp`) are seen; `_FB_LAYOUT` has its own presence check in `test_mesh_fallback_layout.py`. - So a new `RX_VK_MESH_*` that changes the translation but is read in `vk_coopmat.cpp`, `vk_main.cpp` or `vk_draw.cpp` passes this check and leaves a stale cache — the case the old family caught. Scanning `gpu/vk` for `RX_VK_MESH_` (or all `RX_VK_`/`EDEN_`) reads and requiring each to be in the list or `NOT_HELD` would close it, and would also prove the nine are reads. **Smaller, no stale cache.** Five of the nine (`_PAGES`, `_GLOBAL_ATTRS`, `_SHARED_WARP`, `_SHARED_WARP_CHECK`, `_STRICT_FMUL`) already reach the stamp in `flags`; with the name held too, setting one to the value the device already uses still drops the whole cache — the needless drop this PR removes for `_CONV`. `held_elsewhere`/`HELD_BY` is the mechanism (it is why `GP_TABLE` is there). **Checked and sound.** Name order: `entries()` sorts on the name before `=`, and `std::string::compare` is byte-wise, so it is locale-independent; the fixture's `EDEN… < RX_VK…` order matches, and the comparator is a strict weak order (only duplicate `environ` names are equivalent). Old stamp: a mismatch drops the cache whole, and the stamp moves once for every player because `vk_shader.cpp` is a translator source, so the name order adds no second move. `translator_env.h` is not in `nx2/translator_files.py`'s `DIRS`/`FILES`, so later list edits move no cache. #1542 commit 4 edits other lines of the same function — no conflict, but if both land close together it is two re-preparations, not one (one line in Risks); #2177 is the test runner; #2199 sees one more `gpu/vk/` header in `vk_shader.cpp`'s depfile, one recompile. Linux: no locale, no Vulkan, no pointer-size assumption; the header compiles alone under `-std=c++20 -Wall -Werror`, as the fixture does. A Linux start proves only the end-to-end "cache kept across starts" (the `N program keys kept` line), not the stamp itself. **Evidence on the head.** The description says the runs and checks were made at `5b146577f` and "have not run again" after the rebase to `a2dc5385`, and the rebase changed the very check (`HELD_BY`/`graphics_table_loads`). `python -I -B tests/test_program_stamp_knobs.py` is game-free and takes seconds; `test_mesh_fallback_layout`, `test_prepared_stamps` and `test_switch_table` are the other changed ones. Measurements are on `971f77f52`; the status line already warns about that.
Contributor

Correction to my review just above (07:14). It says RX_VK_GP_TABLE is held by graphics_table_loads, and that I found no missed translation input. The review at 07:10 is right and mine is wrong on that point: the translator also reads the name itself (gpu/vk/eden/shader_recompiler/frontend/maxwell/translate_program.cpp:202), and on a device without host-memory import or buffer device addresses graphics_table_loads is 0 for both RX_VK_GP_TABLE=2 and unset while table_check differs, so two different programs share one stamp.

My MESH-scope finding stands on its own (the knob test sees only 2 of the 9 RX_VK_MESH_* reads); the held_elsewhere gap at 07:10 is the stronger one. Please read the two together.

**Correction to my review just above (07:14).** It says `RX_VK_GP_TABLE` is held by `graphics_table_loads`, and that I found no missed translation input. The review at 07:10 is right and mine is wrong on that point: the translator also reads the name itself (`gpu/vk/eden/shader_recompiler/frontend/maxwell/translate_program.cpp:202`), and on a device without host-memory import or buffer device addresses `graphics_table_loads` is 0 for both `RX_VK_GP_TABLE=2` and unset while `table_check` differs, so two different programs share one stamp. My MESH-scope finding stands on its own (the knob test sees only 2 of the 9 `RX_VK_MESH_*` reads); the `held_elsewhere` gap at 07:10 is the stronger one. Please read the two together.
Some checks failed
CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Has been skipped
CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Has been skipped
CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Has been skipped
Third-party parts / The same as upstream (pull_request) Has been skipped
Website / Website on Cloudflare (pull_request) Has been skipped
CI / Conventional Commit title (pull_request) Successful in 7s
Guide screenshots / Guide images and local UI source review (pull_request) Failing after 14s
CI / Workflow lint (actionlint) (pull_request) Successful in 15s
Third-party parts / As their notes say (pull_request) Successful in 17s
beanstalk/approval not approved yet
CI / Website build and checks (pull_request) Failing after 21s
PR description / Description follows the template (pull_request_target) Successful in 6s
This pull request is marked as a work in progress.
View command line instructions

Checkout

From your project repository, check out a new branch and test the changes.
git fetch -u 'fix/translator-stamp-env:dyst-fix/translator-stamp-env'
git switch 'dyst-fix/translator-stamp-env'
Sign in to join this conversation.
No reviewers
No milestone
No project
No assignees
4 participants
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
MagicBeanDev/Longshot!2144
No description provided.