The shader cache key includes the size of the program struct, so adding any member to it makes every player prepare all shaders again #1510

Open
opened 2026-11-18 01:46:00 +00:00 by mooki · 1 comment
Member

Type: correctness · Area: renderer, shader caches · Part of: #1493

TL;DR

The key of the shader caches (translator_stamp()) hashes sizeof(Program), although a Program is never written to the cache as it lies in memory. Adding any member to Program therefore changes the key, and the first start after the update prepares all 38,809 programs again: 251.6 s on a Steam Deck in a tester's run of #223, caught in review before it merged. The cache already has a format number (kProgramCacheFormat) and compile-time layout checks for what it really writes. I propose to take sizeof(Program) out of the key and add a test that fails if a struct size enters it again.

Breakdown

  • Who it affects: every player at the first start after an update that grew Program, and anyone who adds a member to it without knowing the key moves. Slow PCs and the Steam Deck pay the most.
  • What is wrong: the key mixes the sizes of three types and the C++ library into one hash (vk_shader.cpp:1291-1300). Program is written member by member, never as one block. The few small types written as they lie have size and offset checks, and Shader::Info is covered by the recompiler's files and its own size. So the size of the whole struct only measures members nobody stores.
  • What we propose: remove sizeof(Program) from the key. Keep the format number and the existing checks as the only way a cache layout change invalidates it. Add a test that reads the key's inputs and fails on a struct size that is not written as it lies. Add a library version for libstdc++, which the key has none for (:1297).
  • What does not change: which translations invalidate the cache (the translator files, the GPU facts, the switches the stamp lists), and the cache files themselves.
  • How we would prove it: a source check that fails today, a pairing of two builds that differ by one dummy Program member (the log line that drops caches appears today, not after), and the one re-prepare this change itself costs, shipped with a release that already re-prepares.

Evidence

Code, on main (310e7c9df):

  • gpu/vk/vk_shader.cpp:1289-1300: abi[] = sizeof(Shader::Info), sizeof(Program), sizeof(std::vector<uint32_t>), then the C++ library (libc++ and Microsoft's by version; 3u, 0u for libstdc++, no version). The comment says the records keep Shader::Info as it lies and another library crashed at start.
  • gpu/vk/vk_progcache.cpp:284-295, 368-388: a record is put_head (members one by one), then per stage its two stage words, the SPIR-V, c1, and put_info (Shader::Info by sizeof, plus its lists). Nothing writes a Program as one block.
  • vk_progcache.cpp:46-74: kProgramCacheFormat = 1, static_asserts for each type written as it lies (Program::Answer, answers, questions, Stage::c1, the head's words), and program_cache_stamp() = translator_stamp() mixed with the format. tests/test_program_cache_format.py:97-113 fails if a member put_head writes has no assert.
  • Everything keyed by the stamp: the program file and index (vk_progcache.cpp:157), the pipeline cache files (vk_draw.cpp:1034), the pipeline notes and state lists (:1753, 1798, 10688-10803), the prepare marker (:9391-9593) and the module ids (vk_programs.cpp:156). One changed input drops all of it.
  • Guards today are a comment (vk_progcache.cpp:874-879), side tables in vk_progcache.cpp (namespace module_remake) and a test that forbids five member names (tests/test_linux_pressure.py:120-126). A new member with another name passes.

What happened (the 251.6 s):

  • #223, comment of 13 Nov, a Steam Deck OLED: after updating to that PR's commit the log reads 4 files made for another translator dropped (306 MB), 0 program keys kept, then prepared 38809 shader programs (46597 new, 70 failed, 9687 compute pipelines; ...) in 251.6 s. Going back to the build without it did the same. LONGSHOT_SHADER_STAMP and kProgramCacheFormat had not moved; sizeof(Program) had, because the PR added four members. The same comment gives the cause. The members were moved out of Program in review (comment of 14 Nov), and #223 merged with Program unchanged (f1b17d59d).
  • It can happen without a translator change. Since sizeof(Program) entered the key (40c3f851b, 7 Nov), two merged commits grew Program and touched no file the translator stamp hashes: #41 (13 Nov, desc_counts) and #404 (14 Nov, compute_optimized, optimize_owed). Whether each moved sizeof was not measured, and a release may have changed the translator anyway. [INFERENCE]
  • Scale of a full re-prepare: about 5 minutes on a Ryzen 9 7900X with an RTX 3090 (#1390's description); 251.6 s on the Deck above.

What is already done, so this issue is only the rest:

  • Commit 2ed98c468 (11 Nov) gave the cache a format number apart from the translator stamp. #370 (merged 17 Nov) took vk_api.h out of the stamp. #1418 (merged) keeps the pipeline marker across an update that changes no shader. #1390 (merged 17 Nov) deleted 15 switches the translation read and says its merge re-prepares every player once.

Proposed change

  1. Delete (uint32_t)sizeof(Program) from abi[]. Keep sizeof(Shader::Info), sizeof(std::vector<uint32_t>) and the library words: they describe bytes written as they lie.
  2. Put the library version for libstdc++ in the key (a version macro), so a cache from another libstdc++ is not read. #737 already needs this for its store folder name.
  3. Replace the five-name guard in tests/test_linux_pressure.py with a test of the rule itself (below). The side tables can stay; they are not required any more.
  4. Ship it with a release that re-prepares anyway: this change moves the key once. If #1390 has not gone out in a release yet, it rides with that.

How we would test it

  • Game-free, fails today: a source check in the project's style that translator_stamp() contains no sizeof of a struct that program_cache_save does not write as one block. On main it finds sizeof(Program).
  • Game-free: keep tests/test_program_cache_format.py. Add a fingerprint of the static_assert block next to the format number in the test, so changing a written layout without raising kProgramCacheFormat fails the test and names both.
  • In the game, any GPU, any OS: three builds started on one prepared folder: main; main with one unused member added to Program; this change with the same member added. The second drops the caches at its first start (made for another translator dropped, 0 program keys kept); the third keeps them. A yes or no in the log, not a timing. No speed claim is made.
  • Timing of the re-prepare itself is not part of this issue.
  • #223 (merged): the event above, in comments 15271 and 15711. Commit 2ed98c468: the format number. #370, #1418, #1390: the stamp's other inputs, listed above.
  • #737 (open): one shader cache store keyed by the stamp; it adds the toolchain to its folder name for the libstdc++ gap.
  • #1448 (open): changes what the translation reads from the device, so also moves the key on purpose.
  • #1488 (open, work in progress): says in its description that the stamp is unchanged.
  • #1493: the umbrella. Its line "hashing sizeof(Program) once cost Steam Deck players 251.6 s" is a tester's run in review, not a shipped release.

New on this forge; not opened on GitHub.

**Type:** correctness · **Area:** renderer, shader caches · **Part of:** #1493 ## TL;DR The key of the shader caches (`translator_stamp()`) hashes `sizeof(Program)`, although a `Program` is never written to the cache as it lies in memory. Adding any member to `Program` therefore changes the key, and the first start after the update prepares all 38,809 programs again: 251.6 s on a Steam Deck in a tester's run of #223, caught in review before it merged. The cache already has a format number (`kProgramCacheFormat`) and compile-time layout checks for what it really writes. I propose to take `sizeof(Program)` out of the key and add a test that fails if a struct size enters it again. ## Breakdown - **Who it affects:** every player at the first start after an update that grew `Program`, and anyone who adds a member to it without knowing the key moves. Slow PCs and the Steam Deck pay the most. - **What is wrong:** the key mixes the sizes of three types and the C++ library into one hash (`vk_shader.cpp:1291-1300`). `Program` is written member by member, never as one block. The few small types written as they lie have size and offset checks, and `Shader::Info` is covered by the recompiler's files and its own size. So the size of the whole struct only measures members nobody stores. - **What we propose:** remove `sizeof(Program)` from the key. Keep the format number and the existing checks as the only way a cache layout change invalidates it. Add a test that reads the key's inputs and fails on a struct size that is not written as it lies. Add a library version for libstdc++, which the key has none for (`:1297`). - **What does not change:** which translations invalidate the cache (the translator files, the GPU facts, the switches the stamp lists), and the cache files themselves. - **How we would prove it:** a source check that fails today, a pairing of two builds that differ by one dummy `Program` member (the log line that drops caches appears today, not after), and the one re-prepare this change itself costs, shipped with a release that already re-prepares. ## Evidence Code, on main (310e7c9df): - `gpu/vk/vk_shader.cpp:1289-1300`: `abi[]` = `sizeof(Shader::Info)`, `sizeof(Program)`, `sizeof(std::vector<uint32_t>)`, then the C++ library (libc++ and Microsoft's by version; `3u, 0u` for libstdc++, no version). The comment says the records keep `Shader::Info` as it lies and another library crashed at start. - `gpu/vk/vk_progcache.cpp:284-295, 368-388`: a record is `put_head` (members one by one), then per stage its two stage words, the SPIR-V, `c1`, and `put_info` (`Shader::Info` by `sizeof`, plus its lists). Nothing writes a `Program` as one block. - `vk_progcache.cpp:46-74`: `kProgramCacheFormat = 1`, static_asserts for each type written as it lies (`Program::Answer`, `answers`, `questions`, `Stage::c1`, the head's words), and `program_cache_stamp()` = `translator_stamp()` mixed with the format. `tests/test_program_cache_format.py:97-113` fails if a member `put_head` writes has no assert. - Everything keyed by the stamp: the program file and index (`vk_progcache.cpp:157`), the pipeline cache files (`vk_draw.cpp:1034`), the pipeline notes and state lists (`:1753, 1798, 10688-10803`), the prepare marker (`:9391-9593`) and the module ids (`vk_programs.cpp:156`). One changed input drops all of it. - Guards today are a comment (`vk_progcache.cpp:874-879`), side tables in `vk_progcache.cpp` (`namespace module_remake`) and a test that forbids five member names (`tests/test_linux_pressure.py:120-126`). A new member with another name passes. What happened (the 251.6 s): - #223, comment of 13 Nov, a Steam Deck OLED: after updating to that PR's commit the log reads `4 files made for another translator dropped (306 MB)`, `0 program keys kept`, then `prepared 38809 shader programs (46597 new, 70 failed, 9687 compute pipelines; ...) in 251.6 s`. Going back to the build without it did the same. `LONGSHOT_SHADER_STAMP` and `kProgramCacheFormat` had not moved; `sizeof(Program)` had, because the PR added four members. The same comment gives the cause. The members were moved out of `Program` in review (comment of 14 Nov), and #223 merged with `Program` unchanged (f1b17d59d). - It can happen without a translator change. Since `sizeof(Program)` entered the key (40c3f851b, 7 Nov), two merged commits grew `Program` and touched no file the translator stamp hashes: #41 (13 Nov, `desc_counts`) and #404 (14 Nov, `compute_optimized`, `optimize_owed`). Whether each moved `sizeof` was not measured, and a release may have changed the translator anyway. [INFERENCE] - Scale of a full re-prepare: about 5 minutes on a Ryzen 9 7900X with an RTX 3090 (#1390's description); 251.6 s on the Deck above. What is already done, so this issue is only the rest: - Commit 2ed98c468 (11 Nov) gave the cache a format number apart from the translator stamp. #370 (merged 17 Nov) took `vk_api.h` out of the stamp. #1418 (merged) keeps the pipeline marker across an update that changes no shader. #1390 (merged 17 Nov) deleted 15 switches the translation read and says its merge re-prepares every player once. ## Proposed change 1. Delete `(uint32_t)sizeof(Program)` from `abi[]`. Keep `sizeof(Shader::Info)`, `sizeof(std::vector<uint32_t>)` and the library words: they describe bytes written as they lie. 2. Put the library version for libstdc++ in the key (a version macro), so a cache from another libstdc++ is not read. #737 already needs this for its store folder name. 3. Replace the five-name guard in `tests/test_linux_pressure.py` with a test of the rule itself (below). The side tables can stay; they are not required any more. 4. Ship it with a release that re-prepares anyway: this change moves the key once. If #1390 has not gone out in a release yet, it rides with that. ## How we would test it - Game-free, fails today: a source check in the project's style that `translator_stamp()` contains no `sizeof` of a struct that `program_cache_save` does not write as one block. On main it finds `sizeof(Program)`. - Game-free: keep `tests/test_program_cache_format.py`. Add a fingerprint of the static_assert block next to the format number in the test, so changing a written layout without raising `kProgramCacheFormat` fails the test and names both. - In the game, any GPU, any OS: three builds started on one prepared folder: main; main with one unused member added to `Program`; this change with the same member added. The second drops the caches at its first start (`made for another translator dropped`, `0 program keys kept`); the third keeps them. A yes or no in the log, not a timing. No speed claim is made. - Timing of the re-prepare itself is not part of this issue. ## Related - #223 (merged): the event above, in comments 15271 and 15711. Commit 2ed98c468: the format number. #370, #1418, #1390: the stamp's other inputs, listed above. - #737 (open): one shader cache store keyed by the stamp; it adds the toolchain to its folder name for the libstdc++ gap. - #1448 (open): changes what the translation reads from the device, so also moves the key on purpose. - #1488 (open, work in progress): says in its description that the stamp is unchanged. - #1493: the umbrella. Its line "hashing `sizeof(Program)` once cost Steam Deck players 251.6 s" is a tester's run in review, not a shipped release. New on this forge; not opened on GitHub.
Collaborator

Possibly related, for context:

  • issue #1493 Renderer rewrite — Umbrella renderer rework issue; the new issue is a specific cache-stamp defect
  • PR #223 fix(vk): give memory back under Linux memory pressure and after the precompile — The review run exposed the cache invalidation and Program layout concern
  • PR #370 perf(vk): the shader caches' stamp leaves out vk_api.h, so a menu, input or audio change keeps every player's translated programs — Related shader translator-stamp/cache invalidation work
  • PR #737 feat(vk): one shader cache store for every game folder, so a build whose translation is unchanged prepares nothing — Open shared shader-cache store work also uses the stamp
Possibly related, for context: - issue #1493 Renderer rewrite — Umbrella renderer rework issue; the new issue is a specific cache-stamp defect - PR #223 fix(vk): give memory back under Linux memory pressure and after the precompile — The review run exposed the cache invalidation and Program layout concern - PR #370 perf(vk): the shader caches' stamp leaves out vk_api.h, so a menu, input or audio change keeps every player's translated programs — Related shader translator-stamp/cache invalidation work - PR #737 feat(vk): one shader cache store for every game folder, so a build whose translation is unchanged prepares nothing — Open shared shader-cache store work also uses the stamp
Sign in to join this conversation.
No milestone
No project
No assignees
2 participants
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
MagicBeanDev/Longshot#1510
No description provided.