WIP: perf(vk): a draw keeps its program across texture registrations and uniform updates that do not touch its answers #795

Draft
dyst wants to merge 2 commits from dyst/Longshot:perf/vk-program-pick-fast into main
Member

Type: perf · Risk: low · Tested on: Windows 11 (the game-free checks; the game A/B is next) · Needs: a game run on NVIDIA, AMD and Intel

Why

A draw finds its shader program variant through a fast path that only held while no texture had been registered and no uniform updated since, and the game does both between most draws as it streams. So most draws of the same program with the same textures went through the full pick (a hash, a map lookup, rebuilding the answers' handles) for the same answer. Found in an audit of the renderer's per-draw path; no issue on the tracker.

What changed

For programs whose texture answers are all read from c2, the fast path now holds while the picks entry it came from is unchanged and none of the slots its answers name was registered anew. Programs with handle programs or c3+ handles, and compute, keep the exact checks.

  • shaders: Pick::gen, ProgFast::pick/pick_gen, pick_slots_kept and the slots path in program_for (gpu/vk/vk_shader.cpp)
  • tests: tests/test_pick_slots.py replays random binds, registrations, updates and draws through the real program_for and checks the same program for every draw

Measurements

Verdict: not measured yet: A/B runs in the game (this PR against main on one base, interleaved) are next and will replace this section.

Flags and settings

One switch for an A/B.

Flag What it is Before this PR After this PR Effect Helps / doesn't help / untested Recommendation
RX_VK_PICK_REG_GEN=1 The fast path only while nothing was registered or updated - unset On: the old checks measurement pending leave unset; =1 for an A/B

Testing

Tested: Windows 11, llvm-mingw (the builder's v2.0.14 tools): the new and changed checks; every check that reads the
changed code; -fsyntax-only of the changed renderer files with the build's flags and headers; python -I -B tests/run_all.py on this branch and on main @ aa95d35f, same PC and environment: the same results apart from the new test_pick_slots; the failures main has here are named in the first comment.

Not tested: the game yet. This PR is WIP until its A/B is in (next: this PR against main on one dev_packs base of
main @ aa95d35f, on an RTX 3080 with a Ryzen 9 3900X, Windows 11; AMD and Intel need other testers).

Must be tested:

  • Any OS and GPU: Kokiri Forest and Castle Town a few minutes, expecting the same picture and the log's vk: program picks: line counting most kept picks by their slots (Test Farm: @Beanstalk any, those scenes, the log line)

Risks

What could break: A program kept whose variant should have changed with a texture's type: a material drawn with the wrong variant (wrong colours) until its next full pick. The slots its answers depend on are checked by version.

Turning it off: RX_VK_PICK_REG_GEN=1

Compatibility: gpu/vk/vk_shader.cpp is one of the shader translator's stamped files: a player's shader caches are translated again after the update (as with every change to it). No settings or vk ABI change.

Checklist

Everyone

  • One topic.
  • The title is a Conventional Commit (type(scope): description) of the status line's type.
  • Why gives the reason plainly and links the issues that fit (the tracker searched).
  • Measurements has a verdict, or one line saying why there is nothing to measure.
  • Testing names what was tested, what wasn't, and what must still be tested, where and how.
  • Rebased on current main.
  • python -I -B tests/run_all.py passes (any failure that also fails on main is named).
  • No game files, dumps, build output or generated files.
  • third_party/ untouched (a new folder with a PROVENANCE.md instead).
  • The license is OK: GPL-3.0-or-later, nothing incompatible and nothing decompiled.

When it applies

  • AI assistance recorded with Assisted-by: / Co-Authored-By: trailers.
  • Switches documented (docs/development.md; the README for a menu row) and shipped at their recommended value.
  • A game-free test added for the fix, failing on main and passing here.

🤖 Generated with Claude Code

**Type:** perf · **Risk:** low · **Tested on:** Windows 11 (the game-free checks; the game A/B is next) · **Needs:** a game run on NVIDIA, AMD and Intel ## Why A draw finds its shader program variant through a fast path that only held while no texture had been registered and no uniform updated since, and the game does both between most draws as it streams. So most draws of the same program with the same textures went through the full pick (a hash, a map lookup, rebuilding the answers' handles) for the same answer. Found in an audit of the renderer's per-draw path; no issue on the tracker. ## What changed For programs whose texture answers are all read from c2, the fast path now holds while the picks entry it came from is unchanged and none of the slots its answers name was registered anew. Programs with handle programs or c3+ handles, and compute, keep the exact checks. - shaders: `Pick::gen`, `ProgFast::pick`/`pick_gen`, `pick_slots_kept` and the slots path in `program_for` (`gpu/vk/vk_shader.cpp`) - tests: `tests/test_pick_slots.py` replays random binds, registrations, updates and draws through the real `program_for` and checks the same program for every draw ## Measurements **Verdict:** not measured yet: A/B runs in the game (this PR against main on one base, interleaved) are next and will replace this section. ### Flags and settings One switch for an A/B. | Flag | What it is | Before this PR | After this PR | Effect | Helps / doesn't help / untested | Recommendation | |---|---|---|---|---|---|---| | `RX_VK_PICK_REG_GEN=1` | The fast path only while nothing was registered or updated | - | unset | On: the old checks | measurement pending | leave unset; `=1` for an A/B | ## Testing **Tested:** Windows 11, llvm-mingw (the builder's v2.0.14 tools): the new and changed checks; every check that reads the changed code; `-fsyntax-only` of the changed renderer files with the build's flags and headers; python -I -B tests/run_all.py on this branch and on main @ aa95d35f, same PC and environment: the same results apart from the new `test_pick_slots`; the failures main has here are named in the first comment. **Not tested:** the game yet. This PR is WIP until its A/B is in (next: this PR against main on one dev_packs base of main @ aa95d35f, on an RTX 3080 with a Ryzen 9 3900X, Windows 11; AMD and Intel need other testers). **Must be tested:** - Any OS and GPU: Kokiri Forest and Castle Town a few minutes, expecting the same picture and the log's `vk: program picks:` line counting most kept picks `by their slots` (Test Farm: @Beanstalk any, those scenes, the log line) ## Risks **What could break:** A program kept whose variant should have changed with a texture's type: a material drawn with the wrong variant (wrong colours) until its next full pick. The slots its answers depend on are checked by version. **Turning it off:** `RX_VK_PICK_REG_GEN=1` **Compatibility:** `gpu/vk/vk_shader.cpp` is one of the shader translator's stamped files: a player's shader caches are translated again after the update (as with every change to it). No settings or vk ABI change. ## Checklist ### Everyone - [x] One topic. - [x] The title is a Conventional Commit (`type(scope): description`) of the status line's type. - [x] Why gives the reason plainly and links the issues that fit (the tracker searched). - [x] Measurements has a verdict, or one line saying why there is nothing to measure. - [x] Testing names what was tested, what wasn't, and what must still be tested, where and how. - [x] Rebased on current `main`. - [x] `python -I -B tests/run_all.py` passes (any failure that also fails on `main` is named). - [x] No game files, dumps, build output or generated files. - [x] `third_party/` untouched (a new folder with a `PROVENANCE.md` instead). - [x] The license is OK: GPL-3.0-or-later, nothing incompatible and nothing decompiled. ### When it applies - [x] AI assistance recorded with `Assisted-by:` / `Co-Authored-By:` trailers. - [x] Switches documented (docs/development.md; the README for a menu row) and shipped at their recommended value. - [x] A game-free test added for the fix, failing on `main` and passing here. 🤖 Generated with [Claude Code](https://claude.com/claude-code)
perf(vk): a draw keeps its program across texture registrations and uniform updates that do not touch its answers
Some checks failed
CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Has been skipped
CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Has been skipped
CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Has been skipped
Third-party parts / The same as upstream (pull_request) Has been skipped
PR description / Description follows the template (pull_request_target) Successful in 5s
CI / Conventional Commit title (pull_request) Successful in 3s
CI / Workflow lint (actionlint) (pull_request) Successful in 6s
Guide screenshots / Guide images and local UI source review (pull_request) Failing after 6s
Third-party parts / As their notes say (pull_request) Successful in 10s
Website / Website on Cloudflare (pull_request_target) Has been skipped
62e59cc76d
The program pick's fast path (the last two picks, prog_fast) stood only while no texture at all had been registered and no
uniform updated since, and the game does both between most draws as it streams: a draw of the same program with the same
textures went through the full pick (the inputs' hash, the map, the answers' handles and a compare) for the same answer.

A kept pick now remembers the picks entry it came from (Pick::gen moves with the entry's program and handles). For a graphics
program whose answers are all read from c2 (answer_stage_mask != ~0) and whose stages' textures are the same, the fast path
stands while that entry is unchanged and none of the slots its handles name was registered anew (pick_slots_kept), and keeps
the program as the full pick would. Programs with handle programs or c3+ handles, and compute, keep the exact checks.
RX_VK_PICK_REG_GEN=1: as before.

tests/test_pick_slots.py replays random streams of binds, registrations, uniform updates and draws through the real
program_for and checks the same program is picked for every draw with and without the switch.

Assisted-by: Claude Opus 5.5
Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
Author
Member

The checks on this PC (Windows 11, RTX 3080, Ryzen 9 3900X, llvm-mingw from the builder's v2.0.14 tools): python -I -B tests/run_all.py on main @ aa95d35f fails these 19 here, and so does this branch: test_appimage_player_build.py, test_bake_portability.py, test_cpm_discovery.py, test_cpp_preflight.py, test_crash_dialog.py, test_env_on.py, test_fpx_sticky.py, test_lift_cleanup_error.py, test_linux_preflight.py, test_make_game_cache.py, test_make_game_prepare.py, test_missing_nup.py, test_oot_state_tap.py, test_options_budget_once.py, test_readback_late.py, test_spawn_emit.py, test_tested_configurations.py, test_vk_pgo.py, test_walk_key.py. Any other failure in a full run on this branch was rerun on its own and passed (the full runs shared the PC with dev_packs builds: timing checks).

Next: an interleaved A/B in the game against main on one dev_packs base of aa95d35f; the numbers will replace the Measurements section.

The checks on this PC (Windows 11, RTX 3080, Ryzen 9 3900X, llvm-mingw from the builder's v2.0.14 tools): `python -I -B tests/run_all.py` on main @ aa95d35f fails these 19 here, and so does this branch: `test_appimage_player_build.py`, `test_bake_portability.py`, `test_cpm_discovery.py`, `test_cpp_preflight.py`, `test_crash_dialog.py`, `test_env_on.py`, `test_fpx_sticky.py`, `test_lift_cleanup_error.py`, `test_linux_preflight.py`, `test_make_game_cache.py`, `test_make_game_prepare.py`, `test_missing_nup.py`, `test_oot_state_tap.py`, `test_options_budget_once.py`, `test_readback_late.py`, `test_spawn_emit.py`, `test_tested_configurations.py`, `test_vk_pgo.py`, `test_walk_key.py`. Any other failure in a full run on this branch was rerun on its own and passed (the full runs shared the PC with dev_packs builds: timing checks). Next: an interleaved A/B in the game against main on one dev_packs base of aa95d35f; the numbers will replace the Measurements section.
HumanBean force-pushed perf/vk-program-pick-fast from 62e59cc76d
Some checks failed
CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Has been skipped
CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Has been skipped
CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Has been skipped
Third-party parts / The same as upstream (pull_request) Has been skipped
PR description / Description follows the template (pull_request_target) Successful in 5s
CI / Conventional Commit title (pull_request) Successful in 3s
CI / Workflow lint (actionlint) (pull_request) Successful in 6s
Guide screenshots / Guide images and local UI source review (pull_request) Failing after 6s
Third-party parts / As their notes say (pull_request) Successful in 10s
Website / Website on Cloudflare (pull_request_target) Has been skipped
to 71eca88277
Some checks failed
CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Has been skipped
CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Has been skipped
CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Has been skipped
Third-party parts / The same as upstream (pull_request) Has been skipped
Website / Website on Cloudflare (pull_request) Has been skipped
PR description / Description follows the template (pull_request_target) Has been cancelled
CI / Conventional Commit title (pull_request) Has been cancelled
CI / Workflow lint (actionlint) (pull_request) Has been cancelled
Guide screenshots / Guide images and local UI source review (pull_request) Has been cancelled
Third-party parts / As their notes say (pull_request) Has been cancelled
2026-11-15 18:46:45 +00:00
Compare
Owner

Rebased onto main @ 7bacf846. One conflict in gpu/vk/vk_shader.cpp (program_for): main added a comment block above Pick& pk = picks[key] while this PR moved that line below the keep lambda; I kept the PR's order and moved the comment with the line. Pushed as 71eca882. No tests were run here; the test boxes stay yours.

Rebased onto `main` @ 7bacf846. One conflict in gpu/vk/vk_shader.cpp (program_for): main added a comment block above `Pick& pk = picks[key]` while this PR moved that line below the `keep` lambda; I kept the PR's order and moved the comment with the line. Pushed as 71eca882. No tests were run here; the test boxes stay yours.
fix(vk): a kept pick names its entry of picks by key, as picks is a FlatMap since #231
Some checks failed
CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Has been skipped
CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Has been skipped
CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Has been skipped
Third-party parts / The same as upstream (pull_request) Has been skipped
PR description / Description follows the template (pull_request_target) Successful in 4s
CI / Conventional Commit title (pull_request) Successful in 3s
CI / Workflow lint (actionlint) (pull_request) Successful in 5s
Guide screenshots / Guide images and local UI source review (pull_request) Failing after 5s
Third-party parts / As their notes say (pull_request) Successful in 8s
Website / Website on Cloudflare (pull_request_target) Has been skipped
2b66bff0f6
The rebase onto main (#231's flat caches) left ProgFast holding a pointer into picks across draws. FlatMap's inserts
move every entry (flat_map.h), so after the table grew, pick_slots_kept read a freed array. A kept pick now keeps its
entry's key and looks it up again; the lookup's pointer is used at once, before any insert. The kept picks still
forget their entries at the 65,536-entry clear. tests/test_pick_slots.py follows (and includes flat_map.h, which its
fixture needs since the rebase).

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Owner

A follow-up to this morning's rebase, pushed as 2b66bff0 on top of it.

The rebase onto main brought in #231 (#231), which made picks a FlatMap. A FlatMap insert can move every entry (flat_map.h), where an unordered_map never moved one. This PR's ProgFast::pick kept a pointer into picks from one draw to the next, so once a draw with a new key made the table grow, pick_slots_kept read f.pick->gen and f.pick->handles from the freed array. That isn't a text conflict, so neither the rebase nor the forge showed it.

The fix: a kept pick keeps its entry's key (has_pick, pick_key) and pick_slots_kept looks the entry up again (picks.find), returning it for immediate use before any insert. keep() takes the key alongside the entry. At the 65,536-entry clear the kept picks still forget their entries, as before. tests/test_pick_slots.py follows: its needles name the key, it checks no Pick pointer is kept, and its fixture includes flat_map.h, which it needed since the rebase anyway.

No tests were run here; the test boxes stay yours. Please check it against your intent, in particular that pick_slots_kept now does one FlatMap lookup per kept pick, which should be cheap next to the full pick it saves.

A follow-up to this morning's rebase, pushed as 2b66bff0 on top of it. The rebase onto `main` brought in #231 (https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/231), which made `picks` a `FlatMap`. A `FlatMap` insert can move every entry (`flat_map.h`), where an `unordered_map` never moved one. This PR's `ProgFast::pick` kept a pointer into `picks` from one draw to the next, so once a draw with a new key made the table grow, `pick_slots_kept` read `f.pick->gen` and `f.pick->handles` from the freed array. That isn't a text conflict, so neither the rebase nor the forge showed it. The fix: a kept pick keeps its entry's key (`has_pick`, `pick_key`) and `pick_slots_kept` looks the entry up again (`picks.find`), returning it for immediate use before any insert. `keep()` takes the key alongside the entry. At the 65,536-entry clear the kept picks still forget their entries, as before. `tests/test_pick_slots.py` follows: its needles name the key, it checks no `Pick` pointer is kept, and its fixture includes `flat_map.h`, which it needed since the rebase anyway. No tests were run here; the test boxes stay yours. Please check it against your intent, in particular that `pick_slots_kept` now does one `FlatMap` lookup per kept pick, which should be cheap next to the full pick it saves.
Author
Member

Thanks @HumanBean: 2b66bff0 is the right fix, and I missed this. I reviewed the first version assuming picks was an unordered_map, whose entries never move. Since #231 it is a FlatMap, and ProgFast::pick held a pointer into it from one draw to the next, so a grown table left that pointer on freed memory.

Read against the intent, the fix keeps it:

  • A kept pick keeps its entry's key and gen, and pick_slots_kept finds the entry again with picks.find. The pointer it returns is used at once, by the keep before any insert.
  • The gen check still catches an entry overwritten under the same key.
  • Both kept picks forget their entries at the 65,536-entry clear, so a remade entry starting its gen again cannot be taken for the old one.
  • The cost is one FlatMap lookup per kept pick, small next to the full pick it saves.

This stays WIP. Its tests haven't been run on the rebased branch (test_pick_slots with flat_map.h, and the vk_shader.cpp checks), and it has no measurement yet: the one RX_VK_BENCH run so far is too few to call. The stamped-file cost holds: shipping it re-translates players' shader caches.

Thanks @HumanBean: 2b66bff0 is the right fix, and I missed this. I reviewed the first version assuming picks was an unordered_map, whose entries never move. Since #231 it is a FlatMap, and `ProgFast::pick` held a pointer into it from one draw to the next, so a grown table left that pointer on freed memory. Read against the intent, the fix keeps it: - A kept pick keeps its entry's key and gen, and `pick_slots_kept` finds the entry again with `picks.find`. The pointer it returns is used at once, by the keep before any insert. - The gen check still catches an entry overwritten under the same key. - Both kept picks forget their entries at the 65,536-entry clear, so a remade entry starting its gen again cannot be taken for the old one. - The cost is one FlatMap lookup per kept pick, small next to the full pick it saves. This stays WIP. Its tests haven't been run on the rebased branch (test_pick_slots with flat_map.h, and the vk_shader.cpp checks), and it has no measurement yet: the one RX_VK_BENCH run so far is too few to call. The stamped-file cost holds: shipping it re-translates players' shader caches.
Some checks failed
CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Has been skipped
CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Has been skipped
CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Has been skipped
Third-party parts / The same as upstream (pull_request) Has been skipped
PR description / Description follows the template (pull_request_target) Successful in 4s
CI / Conventional Commit title (pull_request) Successful in 3s
CI / Workflow lint (actionlint) (pull_request) Successful in 5s
Guide screenshots / Guide images and local UI source review (pull_request) Failing after 5s
Third-party parts / As their notes say (pull_request) Successful in 8s
Website / Website on Cloudflare (pull_request_target) Has been skipped
This pull request is marked as a work in progress.
This branch is out-of-date with the base branch
View command line instructions

Checkout

From your project repository, check out a new branch and test the changes.
git fetch -u 'perf/vk-program-pick-fast:dyst-perf/vk-program-pick-fast'
git switch 'dyst-perf/vk-program-pick-fast'
Sign in to join this conversation.
No reviewers
No milestone
No project
No assignees
2 participants
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
MagicBeanDev/Longshot!795
No description provided.