fix(vk): the first start of the game after a preparation warms the pipeline cache itself #2869

Closed

github-sync wants to merge 1 commit from github/pr/2869 into main

github-sync commented

2026-11-12 19:00:49 +00:00

Owner

@hckrman101 opened this pull request on GitHub · 2026-11-12 18:00 UTC · original
Merged on GitHub in 8cf9a524d9

Draft: not yet run in a game. The fuller version of #2865 (which is the one line that was played once). If this is verified, #2865 is closed in its favour.

After #2760, programs are loaded when the game creates them (where the GPU has mesh shaders), and a start trusts hookshot_pipelines.bin.built: every listed pipeline is in the pipeline cache, so nothing is built and each load asks the caches. The --prepare-shaders process left that marker after its own build. When the play that follows is not answered from that work, the session compiles its pipelines in play with draws left out.

What it changes

The marker is left only by a process that plays (warm_bounded::leaves_marker). The preparation still builds the listed pipelines, which fills the driver's own cache, and stops there; the first start of the game builds them again before the title (bounded, as in #2760) and writes the marker.
Compute pipelines are in that build: every kept compute program's pipeline, as a first start in start mode makes them. A load at a create call only asks the caches, and on a new install found none (490-666 compute pipelines in no cache in the runs below).
A start that dropped the pipeline cache's files (#2684) drops the marker with them, so an upgraded folder does not load at create from an empty cache.
The safety net also sees left-out draws and pipeline waits: over 2,000 draws or dispatches left out for a pipeline that was not ready, or over 1,000 ms of waits for shaders or pipelines, in one session, and the next start loads before the title again. Until now it counted waits for loads only and did not fire in a session with 46,026 draws left out. Clean tours had 0 and 0 ms; the worst clean figure seen on an RX 580 was 273 left out.

Evidence so far

RTX 3090, NVIDIA 620.70, Linux, a fresh folder (build, --prepare-shaders, new game), played by hand:

	main	the marker rule alone (#2865)
graphics pipelines the loads at create found in no cache	4,718	0
draws and dispatches left out	46,026	40
waits for shaders or pipelines	29 (3.5 s)	0
compute pipelines in no cache	666	490

Limits of that evidence:

In both runs the preparation used a different NVIDIA driver shader cache folder (a test queue's private one) than the play that followed (nearly empty). That is the state after a driver update or a cleared driver cache, not a normal Builder preparation. Whether a normal preparation on NVIDIA is answered in play on main is not known yet.
With the marker rule the first start compiled 25,337 pipelines in the play process in 166 s on that cold driver cache. With a driver cache that holds the preparation's work the same build took 11-15 s in earlier runs. The time the compute pipelines add is not measured.
This commit itself has not run in a game. The compute build, the #2684 rule and the wider safety net are untested outside unit tests.

Tests

test_warm_bounded (fails on main for the marker rule) and test_load_default (the safety net's new rule) pass; 196 touched tests pass locally on Linux. Failing there: test_program_cache_one_variant, which fails on main until #2861, and two Windows-only tests named explicitly. The full suite was not run.

Still wanted before it is ready

The player sequence with one driver cache for the preparation and the play, and again with the driver cache emptied.
A check at each start that asks the caches for a sample of the listed pipelines and builds before the title only when they do not answer, instead of always building at the first start.

> [@hckrman101](https://github.com/hckrman101) opened this pull request on GitHub · 2026-11-12 18:00 UTC · [original](https://github.com/NotRexed/Hookshot/pull/2869) > **Merged on GitHub** in https://git.magicbean.dev/MagicBeanDev/Hookshot/commit/8cf9a524d9ed9d27f430022be3d5c80813f9e45a **Draft: not yet run in a game.** The fuller version of #2865 (which is the one line that was played once). If this is verified, #2865 is closed in its favour. After #2760, programs are loaded when the game creates them (where the GPU has mesh shaders), and a start trusts `hookshot_pipelines.bin.built`: every listed pipeline is in the pipeline cache, so nothing is built and each load asks the caches. The `--prepare-shaders` process left that marker after its own build. When the play that follows is not answered from that work, the session compiles its pipelines in play with draws left out. ## What it changes - **The marker is left only by a process that plays** (`warm_bounded::leaves_marker`). The preparation still builds the listed pipelines, which fills the driver's own cache, and stops there; the first start of the game builds them again before the title (bounded, as in #2760) and writes the marker. - **Compute pipelines are in that build:** every kept compute program's pipeline, as a first start in start mode makes them. A load at a create call only asks the caches, and on a new install found none (490-666 compute pipelines in no cache in the runs below). - **A start that dropped the pipeline cache's files (#2684) drops the marker with them,** so an upgraded folder does not load at create from an empty cache. - **The safety net also sees left-out draws and pipeline waits:** over 2,000 draws or dispatches left out for a pipeline that was not ready, or over 1,000 ms of waits for shaders or pipelines, in one session, and the next start loads before the title again. Until now it counted waits for loads only and did not fire in a session with 46,026 draws left out. Clean tours had 0 and 0 ms; the worst clean figure seen on an RX 580 was 273 left out. ## Evidence so far RTX 3090, NVIDIA 620.70, Linux, a fresh folder (build, `--prepare-shaders`, new game), played by hand: | | main | the marker rule alone (#2865) | |---|---|---| | graphics pipelines the loads at create found in no cache | 4,718 | 0 | | draws and dispatches left out | 46,026 | 40 | | waits for shaders or pipelines | 29 (3.5 s) | 0 | | compute pipelines in no cache | 666 | 490 | **Limits of that evidence:** - In both runs the preparation used a different NVIDIA driver shader cache folder (a test queue's private one) than the play that followed (nearly empty). That is the state after a driver update or a cleared driver cache, not a normal Builder preparation. **Whether a normal preparation on NVIDIA is answered in play on main is not known yet.** - With the marker rule the first start compiled 25,337 pipelines in the play process in 166 s on that cold driver cache. With a driver cache that holds the preparation's work the same build took 11-15 s in earlier runs. The time the compute pipelines add is not measured. - **This commit itself has not run in a game.** The compute build, the #2684 rule and the wider safety net are untested outside unit tests. ## Tests `test_warm_bounded` (fails on main for the marker rule) and `test_load_default` (the safety net's new rule) pass; 196 touched tests pass locally on Linux. Failing there: `test_program_cache_one_variant`, which fails on main until #2861, and two Windows-only tests named explicitly. The full suite was not run. ## Still wanted before it is ready - The player sequence with one driver cache for the preparation and the play, and again with the driver cache emptied. - A check at each start that asks the caches for a sample of the listed pipelines and builds before the title only when they do not answer, instead of always building at the first start.

github-sync added 1 commit

2026-11-12 19:00:49 +00:00

fix(vk): the first start of the game after a preparation warms the pipeline cache itself 16a4cea979

With programs loaded at their create calls, a start trusts hookshot_pipelines.bin.built:
every noted pipeline is in the pipeline cache, so nothing is built and each load asks
the caches. The preparation's own process (--prepare-shaders) left that marker after its
warm, and in the first session after it the asks were not answered (RTX 3090, a fresh
folder: "the driver's shader cache kept 0 of 32", 3,720 of 3,880 graphics and 469 compute
pipelines in no cache, up to 218 draws left out a frame). What a process that plays
builds is answered from then on.

- The marker is left only by a process that plays (warm_bounded::leaves_marker). The
  preparation's process still warms, which fills the driver's own cache, and stops there.
- The warm also builds every kept compute program's pipeline, as a first start in start
  mode does: a load at a create call only asks, and on a new install found none.
- A start that dropped the pipeline cache's files (#2684) drops the marker with them.
- The safety net also trips on what such a session looks like: over 2,000 draws or
  dispatches left out for a pipeline that was not ready, or over a second of waits for
  shaders or pipelines, in one session (0 and 0 ms in every clean tour; 46,026 and 3.5 s
  in the session above), and the next start loads before the title again.

Co-Authored-By: Claude Fable 5.2 <[email protected]>

github-sync referenced this pull request from a commit

2026-11-12 19:00:50 +00:00

fix(vk): the first start of the game after a preparation warms the pipeline cache itself (#2869)

github-sync referenced this pull request from a commit

2026-11-12 19:00:50 +00:00

fix(vk): the load default's safety net counts only what the loads found in no cache (#2884)

github-sync referenced this pull request

2026-11-12 19:00:59 +00:00

fix(vk): the load default's safety net counts only what the loads found in no cache #2884

github-sync referenced this pull request from a commit

2026-11-12 19:01:09 +00:00

fix(vk): the load default's safety net counts only what the loads found in no cache