MagicBeanDev/Hookshot

Archived

Watch

Fork 0

Prepare: compute programs laid out as play lays them out; every compute program and its texture variants; predicted graphics pipelines #772

Merged

BlueHeisenberg merged 5 commits from prepare-plus into main

2026-11-08 21:37:42 +00:00

BlueHeisenberg commented

2026-11-08 20:42:42 +00:00

(Migrated from github.com)

The bug: prepared compute programs had the wrong storage buffers

--prepare-shaders (and a release build's first start, the same prepare_run) translates each compute program of the
shader census with nothing bound. A compute program's c0 layout tells where its storage buffers, textures and images
are, and cs_layout reads that layout from the bound program's control section. With none bound it fell back to
Maxwell's table. The program was kept in hookshot_programs.bin under its right key (the key comes from its code) but
with the wrong storage buffers, and play took it from there unchecked, because the key and texture answers matched. Main
has shipped this since 2a414f9, which put compute programs in the census.

Evidence (origin/main 465e096, llvm-mingw make_game, RTX 5090): a cold play translates 123 compute programs on the test
route. After --prepare-shaders, 32 of those 123 have a prepared record with the same key and texture answers but
different SPIR-V. With this PR all 123 are byte-identical to play's.

Fix: while the prepare translates a compute program, cs_layout reads the census's own control section
(vk_bank.cpp, seed_cs_control).

The census reads agl's archives, networks and the movie upscaler

Main's census finds 31,985 programs (3,808 compute). On the test route it misses 25 of the 123 compute programs and 103
of the 893 graphics stages that play meets. This PR's census also reads:

agl's BAHS archives through their stage list and program table, which gives their graphics programs too, in
ApplicationPackage and in Lib/ (agl, tera and gsys resources);
agl's NDSR networks;
the movie upscaler's NDVE.

It finds 38,809 programs (4,128 compute), a strict superset of main's, and covers 123/123 and 893/893.

Each compute program's texture variants

A compute program's texture questions (Program::answers) are answered in play by the textures bound, which nothing
knows before the game. So the prepare kept one variant of each, and play translated and compiled the others itself.
The questions are few: each is tried with every answer it can take, and an answer that changes the SPIR-V makes a
variant. Variants of two answers and more come fewest first, up to 64 per program (seed_compute). Each variant is kept
and its pipeline warmed as before. Result: 10,340 compute pipelines for 4,128 programs (main: 3,808).

Predicted graphics pipelines (needs local notes)

Most graphics programs are named by no note in hookshot_pipelines.bin, so their pipelines were built when first
drawn. After a prepare that prepared, each kept program variant that no note names takes the vertex layout and targets
the notes predict (vk_predict.cpp): those of notes with the same vertex or fragment module, else those of programs
with the same inputs and outputs. Its pipelines are built once, for the driver's own cache (predict_run).

They are not written to mkw_vk_pipelines.bin, which every start reads whole, and they don't stay in memory: each
program is borrowed from hookshot_programs.bin and let go.
--prepare-shaders shows them as a part of its progress (0.7 to 0.9). RX_VK_NO_PREDICT=1 leaves them out.

How to test

hookshot.exe --prepare-shaders (or bin\hookshot.exe --prepare-shaders in a folder with the launcher). The output is
unchanged: @@stage shaders, @@progress, @@done shaders, exit 0 (or @@error and 3). Then start the game. The log
shows:

vk: prepared N shader programs (... compute variants, ... programs with texture variants, ... at the limit);
vk: predicted N pipelines from M notes ... when there are notes.

Measurements

Method: RTX 5090, driver 0x9b404000. Clean llvm-mingw make_game builds of origin/main 465e096 and of this branch on it.
Each run had fresh caches and a unique exe name, so the driver's cache was cold for it, and played the same 150 s input
route from the same save. A had RX_VK_PRECOMPILE=0; B and C ran --prepare-shaders twice, then started the game
normally.

run	slow compute compiles (>= 200 ms)	longest	compute programs translated in play	dispatches left out while compiling	compute pipelines prepared
A: main, cold, no prepare	28	2.7 s	123	40	0
B: main, `--prepare-shaders`	10	1.4 s	39	40	3,808
C: this PR, `--prepare-shaders`	0	-	0	0	10,340

Costs, B vs C (no notes):

	B: main	C: this PR
`--prepare-shaders`, first run	72 s	249 s
`--prepare-shaders`, second run	5 s	7 s
`hookshot_programs.bin`	5.5 GB	6.5 GB
`mkw_vk_pipelines.bin`	85 MB	487 MB
every start's precompile	0.6 s	1.6 s

With 953 notes from one play, the prepare also predicted and built 39,877 pipelines in 239 s (420 s in all), and
mkw_vk_pipelines.bin stayed at 516 MB.

Notes

Graphics prediction needs local notes. A fresh install has none (v2.0.2 ships no hookshot_pipelines_seed.bin;
that release was checked), so it built 0 pipelines in A/B/C. It pays off once notes exist, from play or from a shipped
seed. A partial sign: with 953 notes from an earlier play, the play after the prepare had 1 ms of pipeline time and 8
new pipelines in its slow frames, against C's 58 ms and 94. The 953 noted pipelines were also precompiled in that run,
so the drop is not the prediction's alone.
All times are from an RTX 5090. The first prepare now compiles 10,340 compute pipelines instead of 3,808, so its time
follows the driver's compiler; it was not measured on AMD or Intel, where single culling compiles have taken
seconds.
The measurements were made on 465e096. Rebased onto 45620e8, the prediction's record reader needed one fix: #640's
revert removed the mesh-as-compute fields (449382e). The branch compiles there but was not measured again.
About 97 of the larger census's programs don't translate on their own, and the prepare counts them as failed. Play
translates them when it meets them, as before.
Graphics programs whose variant depends on the bound textures are still translated in play: 3 on this route in C.

## The bug: prepared compute programs had the wrong storage buffers `--prepare-shaders` (and a release build's first start, the same `prepare_run`) translates each compute program of the shader census with nothing bound. A compute program's c0 layout tells where its storage buffers, textures and images are, and `cs_layout` reads that layout from the bound program's control section. With none bound it fell back to Maxwell's table. The program was kept in `hookshot_programs.bin` under its right key (the key comes from its code) but with the wrong storage buffers, and play took it from there unchecked, because the key and texture answers matched. Main has shipped this since 2a414f9, which put compute programs in the census. Evidence (origin/main 465e096, llvm-mingw make_game, RTX 5090): a cold play translates 123 compute programs on the test route. After `--prepare-shaders`, 32 of those 123 have a prepared record with the same key and texture answers but different SPIR-V. With this PR all 123 are byte-identical to play's. Fix: while the prepare translates a compute program, `cs_layout` reads the census's own control section (`vk_bank.cpp`, `seed_cs_control`). ## The census reads agl's archives, networks and the movie upscaler Main's census finds 31,985 programs (3,808 compute). On the test route it misses 25 of the 123 compute programs and 103 of the 893 graphics stages that play meets. This PR's census also reads: - agl's BAHS archives through their stage list and program table, which gives their graphics programs too, in `ApplicationPackage` and in `Lib/` (agl, tera and gsys resources); - agl's NDSR networks; - the movie upscaler's NDVE. It finds 38,809 programs (4,128 compute), a strict superset of main's, and covers 123/123 and 893/893. ## Each compute program's texture variants A compute program's texture questions (`Program::answers`) are answered in play by the textures bound, which nothing knows before the game. So the prepare kept one variant of each, and play translated and compiled the others itself. The questions are few: each is tried with every answer it can take, and an answer that changes the SPIR-V makes a variant. Variants of two answers and more come fewest first, up to 64 per program (`seed_compute`). Each variant is kept and its pipeline warmed as before. Result: 10,340 compute pipelines for 4,128 programs (main: 3,808). ## Predicted graphics pipelines (needs local notes) Most graphics programs are named by no note in `hookshot_pipelines.bin`, so their pipelines were built when first drawn. After a prepare that prepared, each kept program variant that no note names takes the vertex layout and targets the notes predict (`vk_predict.cpp`): those of notes with the same vertex or fragment module, else those of programs with the same inputs and outputs. Its pipelines are built once, for the driver's own cache (`predict_run`). - They are not written to `mkw_vk_pipelines.bin`, which every start reads whole, and they don't stay in memory: each program is borrowed from `hookshot_programs.bin` and let go. - `--prepare-shaders` shows them as a part of its progress (0.7 to 0.9). `RX_VK_NO_PREDICT=1` leaves them out. ## How to test `hookshot.exe --prepare-shaders` (or `bin\hookshot.exe --prepare-shaders` in a folder with the launcher). The output is unchanged: `@@stage shaders`, `@@progress`, `@@done shaders`, exit 0 (or `@@error` and 3). Then start the game. The log shows: - `vk: prepared N shader programs (... compute variants, ... programs with texture variants, ... at the limit)`; - `vk: predicted N pipelines from M notes ...` when there are notes. ## Measurements Method: RTX 5090, driver 0x9b404000. Clean llvm-mingw make_game builds of origin/main 465e096 and of this branch on it. Each run had fresh caches and a unique exe name, so the driver's cache was cold for it, and played the same 150 s input route from the same save. A had `RX_VK_PRECOMPILE=0`; B and C ran `--prepare-shaders` twice, then started the game normally. | run | slow compute compiles (>= 200 ms) | longest | compute programs translated in play | dispatches left out while compiling | compute pipelines prepared | |---|---|---|---|---|---| | A: main, cold, no prepare | 28 | 2.7 s | 123 | 40 | 0 | | B: main, `--prepare-shaders` | 10 | 1.4 s | 39 | 40 | 3,808 | | C: this PR, `--prepare-shaders` | **0** | - | **0** | **0** | 10,340 | Costs, B vs C (no notes): | | B: main | C: this PR | |---|---|---| | `--prepare-shaders`, first run | 72 s | 249 s | | `--prepare-shaders`, second run | 5 s | 7 s | | `hookshot_programs.bin` | 5.5 GB | 6.5 GB | | `mkw_vk_pipelines.bin` | 85 MB | 487 MB | | every start's precompile | 0.6 s | 1.6 s | With 953 notes from one play, the prepare also predicted and built 39,877 pipelines in 239 s (420 s in all), and `mkw_vk_pipelines.bin` stayed at 516 MB. ## Notes - **Graphics prediction needs local notes.** A fresh install has none (v2.0.2 ships no `hookshot_pipelines_seed.bin`; that release was checked), so it built 0 pipelines in A/B/C. It pays off once notes exist, from play or from a shipped seed. A partial sign: with 953 notes from an earlier play, the play after the prepare had 1 ms of pipeline time and 8 new pipelines in its slow frames, against C's 58 ms and 94. The 953 noted pipelines were also precompiled in that run, so the drop is not the prediction's alone. - All times are from an RTX 5090. The first prepare now compiles 10,340 compute pipelines instead of 3,808, so its time follows the driver's compiler; it was not measured on AMD or Intel, where single culling compiles have taken seconds. - The measurements were made on 465e096. Rebased onto 45620e8, the prediction's record reader needed one fix: #640's revert removed the mesh-as-compute fields (449382e). The branch compiles there but was not measured again. - About 97 of the larger census's programs don't translate on their own, and the prepare counts them as failed. Play translates them when it meets them, as before. - Graphics programs whose variant depends on the bound textures are still translated in play: 3 on this route in C.

❤️ 2

klimovich008 commented

2026-11-08 21:17:12 +00:00

(Migrated from github.com)

Independent check of the core compute-layout fix on Windows, using the actual bank parser and table lookup with identical synthetic control bytes:

Runtime: SSBO base 0x60, count 5, layout end 0xc8.
v2.0.3 blank preparation: 0x10, 0, 0x10.
This branch: preparation matches runtime. Fails on the old code, passes now.

Small regression test and runner entry, based on this PR's inspected head, are available to cherry-pick. The test also checks clearing the override and thread isolation; removing production thread_local makes it fail. Run python -I -B tests/run_all.py --strict prepare_cs_layout.

Tested with Windows Clang/MSVC ABI. Linux and the full census, variant enumeration, prediction and GPU outputs were not tested here. The test contains no game data.

Independent check of the core compute-layout fix on Windows, using the actual bank parser and table lookup with identical synthetic control bytes: - Runtime: SSBO base `0x60`, count `5`, layout end `0xc8`. - v2.0.3 blank preparation: `0x10`, `0`, `0x10`. - This branch: preparation matches runtime. Fails on the old code, passes now. [Small regression test and runner entry](https://github.com/klimovich008/Hookshot/commit/102ce25239631e474031bd0a63ad65ed9f255dd9), based on this PR's inspected head, are available to cherry-pick. The test also checks clearing the override and thread isolation; removing production `thread_local` makes it fail. Run `python -I -B tests/run_all.py --strict prepare_cs_layout`. Tested with Windows Clang/MSVC ABI. Linux and the full census, variant enumeration, prediction and GPU outputs were not tested here. The test contains no game data.

👍 2

klimovich008 commented

2026-11-08 22:23:08 +00:00

(Migrated from github.com)

Tested on AMD (RX 9070 XT, Ryzen 7 5700X3D, Windows 11), with this PR's head 4ca308d. I built only its renderer and paired it with a v2.0.3 game exe, since host/ is unchanged between them. The census came from this PR's shader_census.py.

It fixes a visible bug here. With stock v2.0.3 after the first-start preparation, in Hyrule Field from the same save and camera at the same frames:

the trees in front of the castle wall have no leaves;
the foreground grass is short and flat.

With a fresh, play-only cache (no preparation), the same v2.0.3 build draws leafy trees and tall grass, as v2.0.2 does. With this PR's --prepare-shaders cache, the trees and grass are correct again. So the wrong compute layouts this PR fixes are what removes foliage on AMD.

But the preparation is much slower on this card. --prepare-shaders exited 0 after 2,758 s (46 min). Stock v2.0.3's full first-start preparation took 197 s here, and you measured 249 s on an RTX 5090. Afterwards the caches were hookshot_programs.bin 7.4 GB and mkw_vk_pipelines.bin 611 MB, and each start's precompile built 10,107 compute pipelines in 5.6 s. It ran on the default 8 preparation threads (hardware_concurrency/2). I didn't keep the per-phase output, so I can't say yet whether the time is in the compute texture variants or in the driver step. I can re-run with per-phase timing if that helps.

Tested on AMD (RX 9070 XT, Ryzen 7 5700X3D, Windows 11), with this PR's head 4ca308d. I built only its renderer and paired it with a v2.0.3 game exe, since `host/` is unchanged between them. The census came from this PR's `shader_census.py`. **It fixes a visible bug here.** With stock v2.0.3 after the first-start preparation, in Hyrule Field from the same save and camera at the same frames: - the trees in front of the castle wall have no leaves; - the foreground grass is short and flat. With a fresh, play-only cache (no preparation), the same v2.0.3 build draws leafy trees and tall grass, as v2.0.2 does. With this PR's `--prepare-shaders` cache, the trees and grass are correct again. So the wrong compute layouts this PR fixes are what removes foliage on AMD. **But the preparation is much slower on this card.** `--prepare-shaders` exited 0 after **2,758 s (46 min)**. Stock v2.0.3's full first-start preparation took 197 s here, and you measured 249 s on an RTX 5090. Afterwards the caches were `hookshot_programs.bin` 7.4 GB and `mkw_vk_pipelines.bin` 611 MB, and each start's precompile built 10,107 compute pipelines in 5.6 s. It ran on the default 8 preparation threads (hardware_concurrency/2). I didn't keep the per-phase output, so I can't say yet whether the time is in the compute texture variants or in the driver step. I can re-run with per-phase timing if that helps.

klimovich008 commented

2026-11-08 22:49:24 +00:00

(Migrated from github.com)

Confirmed on RX 9070 XT with the inspected renderer: preparation exits successfully, but cached compute precompilation reports two different invalid-module forms:

08737: integer sampled type with a float OpImageSampleDrefExplicitLod result.
04965: OpTypeImage uses F32 with R32ui.

Exact cached-record pipeline replays reproduce these in 0.313 s and 0.565 s, using about 100–123 MiB of working set. Both driver calls still return success, so successful preparation alone does not certify valid SPIR-V.

The underlying texture-pass constraints predate this PR; expanded variants expose them. #834 fixes the depth sampled type with a real pass/emitter regression (old fails, fixed passes; ordinary float shaders stay byte-identical). The unsigned storage-format path is separate work. These results do not establish a cause for the 46-minute preparation time or a corrected gameplay result.

Fresh-translation follow-up: with #834 and #848 together on the #772 renderer, the two real compute programs now build four variants each on RX 9070 XT with Khronos validation enabled and no VUIDs. All eight generated SPIR-V modules pass spirv-val. These checks cover module/pipeline construction; draw output, full-game integration and total preparation time remain untested with this build.

Confirmed on RX 9070 XT with the inspected renderer: preparation exits successfully, but cached compute precompilation reports two different invalid-module forms: - `08737`: integer sampled type with a float `OpImageSampleDrefExplicitLod` result. - `04965`: `OpTypeImage` uses F32 with R32ui. Exact cached-record pipeline replays reproduce these in 0.313 s and 0.565 s, using about 100–123 MiB of working set. Both driver calls still return success, so successful preparation alone does not certify valid SPIR-V. The underlying texture-pass constraints predate this PR; expanded variants expose them. #834 fixes the depth sampled type with a real pass/emitter regression (old fails, fixed passes; ordinary float shaders stay byte-identical). The unsigned storage-format path is separate work. These results do not establish a cause for the 46-minute preparation time or a corrected gameplay result. Fresh-translation follow-up: with #834 and #848 together on the #772 renderer, the two real compute programs now build four variants each on RX 9070 XT with Khronos validation enabled and no VUIDs. All eight generated SPIR-V modules pass `spirv-val`. These checks cover module/pipeline construction; draw output, full-game integration and total preparation time remain untested with this build.

lucasmazzoni45-del commented

2026-11-08 23:33:32 +00:00

(Migrated from github.com)

Tested v2.0.5 (includes #772) on RX 7800 XT / Windows 11: the prepare ends with 165 failed, all does not lower ... unsupported native ...:

40x stage 0 "native constant LEA carry mode"
17x stage 5 "native formatted surface load mode"
12x stage 1 "unsupported native opcode"
3x stage 1 "native texel fetch mode or descriptor binding", 3x stage 1 "native integer min/max mode"
2x stage 5 "unsupported native opcode", 2x stage 5 "native program uses lowering scratch registers", 2x stage 5 "native global load uniform addressing mode"
1x stage 6 "native stage buffer access mode", 1x stage 5 "native program has no unconditional EXIT"

In play the failed count stays at 179 (165 + the usual ~10 upscale networks + a few) and no draw is skipped, so they look unused here (they come from the wider census). On v2.0.4 (31,985 programs) it was 0 failed. Also, the prediction built 0 graphics pipelines on the first start after the update (see #865).

Tested v2.0.5 (includes #772) on RX 7800 XT / Windows 11: the prepare ends with `165 failed`, all `does not lower ... unsupported native ...`: - 40x stage 0 "native constant LEA carry mode" - 17x stage 5 "native formatted surface load mode" - 12x stage 1 "unsupported native opcode" - 3x stage 1 "native texel fetch mode or descriptor binding", 3x stage 1 "native integer min/max mode" - 2x stage 5 "unsupported native opcode", 2x stage 5 "native program uses lowering scratch registers", 2x stage 5 "native global load uniform addressing mode" - 1x stage 6 "native stage buffer access mode", 1x stage 5 "native program has no unconditional EXIT" In play the failed count stays at 179 (165 + the usual ~10 upscale networks + a few) and no draw is skipped, so they look unused here (they come from the wider census). On v2.0.4 (31,985 programs) it was `0 failed`. Also, the prediction built 0 graphics pipelines on the first start after the update (see #865).