Stop new areas freezing while AMD compiles their pipelines #700

Closed

Palayop7239 wants to merge 4 commits from fix/amd-waiting-draw-pipelines into main

Palayop7239 commented

2026-11-08 17:47:33 +00:00

(Migrated from github.com)

What

A draw that has to wait for its graphics pipeline (its targets were made this frame, so it cannot be left out) now builds the pipeline itself instead of waiting for the background job:

From the pipeline cache only (VK_PIPELINE_CREATE_FAIL_ON_PIPELINE_COMPILE_REQUIRED_BIT). A pipeline met in an earlier session is there, whole, in milliseconds, and the worker's build is then skipped.
On AMD, a quick build without the driver's optimizer and outside the cache. The worker's full build replaces it when it is done, and a quick one never replaces a full one. RX_VK_QUICK_PIPELINES=1 turns this on for any GPU, =0 turns it off.
Otherwise, if no worker has begun the full build, the draw takes it from the queue and builds it on its own thread. The worker then leaves it alone.

Also:

A whole pipeline the optimizer fails on is built again without it, rather than left out. On an RX 6650 XT, 6eaa646d07ea4e43 and 5fa56485a6b07619 failed with -13 at every start; both now build in under 0.5 s.
Graphics pipelines that take 200 ms or more are logged (SLOW compile), as compute pipelines already are.

Why

Before this change, the waiting draw waited for a job queued behind the others and built with the optimizer, so a new area's first frame stood still for as long as all its pipelines' compiles added up. In one session the stall time matched the total compile time (38.5 s against 37.6 s): the pipelines were built one after another while the other workers sat idle. AMD's optimizer takes seconds, sometimes minutes, on some of OoT's shaders (Lichii #200, #210). After v2.0.2 changed the translator, every cached pipeline missed and this hit all of them at once.

Likely the same problem as #681 (image freezes when moving between areas, sound and input keep working) and #427 (shader stutter on AMD). It also explains part of #643.

Testing

RX 6650 XT, AMD driver 32.0.21045.16010, Ryzen 7 5700X, Windows 11, Hookshot v2.0.2 with only hookshot_vk.dll rebuilt (same build flags, the shader caches kept):

	Before	After
Stalls over 1 s in one session	16	3
Total time stalled	166 s	4.5 s
Longest stall	61.9 s	1.7 s

Program 7060695868b1daa0, which caused the 62 s stall, was built the quick way in 0.95 s.

Caveats:

The second session also had a warmer driver cache than the first. The quick builds explain the stalls that remained, the cache explains much of the rest.
The cache lookup (step 1) was added after that session. It was played in a later one, below.

Cache lookup, played. v2.0.2 with only this PR (plus a log line on each waiting draw, test build only), same card. Started with RX_VK_PRECOMPILE=0, so every pipeline met earlier had to come through here. That is the state after the notes are dropped. About 4 minutes: title screen, play outdoors, the pause menu.

How the waiting draw got its pipeline	Draws	Time
From the pipeline cache	187	0.1–0.4 ms each
Not in the cache (lookup cost), then a quick build	17	lookup 0–23 ms, build 13 ms–1.35 s
Full build, or a worker's	0

No SLOW cache lookup line.

What is left:

The first pause menu of the version stood still for 11.2 s. That frame met 11 pipelines no cache had. Each was built quick, one after another, at about 1.0–1.35 s each. For these shaders the optimizer is not the cost: their full builds on the workers took about as long (0.7–1.9 s). Before this PR, that frame would also have waited behind the 165 builds already queued. They are noted now, so the precompile builds them at the next start.
Draws that may be left out still wait in the background queue, behind slow builds, even when their pipeline is in the cache. Here that was up to 136 draws a frame, with the world drawn wrong for a while. This PR leaves that rule alone. A normal start precompiles them first.

This branch has main merged in (66892e6, after the CI overhaul in #695). hookshot_vk.dll compiles from main with this PR and the two related ones merged in (build.py with RX_VK_ONLY=1, llvm-mingw from the v2.0.2 builder).

tests/test_pipeline_wait.py compiles pipeline_build_waited with stand-ins for the driver. It covers:

a cache hit, and a device without cache control;
quick build on and off;
a worker that already began the build;
a full build landing while the quick one is made;
a failed quick build.

It also checks the source for the queued job's claim, where the draw site calls this, the quick build's flags and cache, the retry without the optimizer, and the log. It fails if a quick pipeline may overwrite a full one, or if a cache hit doesn't spare the worker. tests/run_all.py picks it up and gives it --cxx g++, like the other compiled checks.

🤖 Generated with Claude Code

## What A draw that has to wait for its graphics pipeline (its targets were made this frame, so it cannot be left out) now builds the pipeline itself instead of waiting for the background job: 1. **From the pipeline cache only** (`VK_PIPELINE_CREATE_FAIL_ON_PIPELINE_COMPILE_REQUIRED_BIT`). A pipeline met in an earlier session is there, whole, in milliseconds, and the worker's build is then skipped. 2. **On AMD, a quick build without the driver's optimizer** and outside the cache. The worker's full build replaces it when it is done, and a quick one never replaces a full one. `RX_VK_QUICK_PIPELINES=1` turns this on for any GPU, `=0` turns it off. 3. **Otherwise**, if no worker has begun the full build, the draw takes it from the queue and builds it on its own thread. The worker then leaves it alone. Also: - A whole pipeline the optimizer **fails** on is built again without it, rather than left out. On an RX 6650 XT, `6eaa646d07ea4e43` and `5fa56485a6b07619` failed with `-13` at every start; both now build in under 0.5 s. - Graphics pipelines that take 200 ms or more are logged (`SLOW compile`), as compute pipelines already are. ## Why Before this change, the waiting draw waited for a job queued behind the others and built with the optimizer, so a new area's first frame stood still for as long as all its pipelines' compiles added up. In one session the stall time matched the total compile time (38.5 s against 37.6 s): the pipelines were built one after another while the other workers sat idle. AMD's optimizer takes seconds, sometimes minutes, on some of OoT's shaders (Lichii #200, #210). After v2.0.2 changed the translator, every cached pipeline missed and this hit all of them at once. Likely the same problem as #681 (image freezes when moving between areas, sound and input keep working) and #427 (shader stutter on AMD). It also explains part of #643. ## Testing RX 6650 XT, AMD driver 32.0.21045.16010, Ryzen 7 5700X, Windows 11, Hookshot v2.0.2 with only `hookshot_vk.dll` rebuilt (same build flags, the shader caches kept): | | Before | After | |---|---|---| | Stalls over 1 s in one session | 16 | 3 | | Total time stalled | 166 s | 4.5 s | | Longest stall | 61.9 s | 1.7 s | Program `7060695868b1daa0`, which caused the 62 s stall, was built the quick way in 0.95 s. Caveats: - The second session also had a warmer driver cache than the first. The quick builds explain the stalls that remained, the cache explains much of the rest. - The cache lookup (step 1) was added after that session. It was played in a later one, below. **Cache lookup, played.** v2.0.2 with only this PR (plus a log line on each waiting draw, test build only), same card. Started with `RX_VK_PRECOMPILE=0`, so every pipeline met earlier had to come through here. That is the state after the notes are dropped. About 4 minutes: title screen, play outdoors, the pause menu. | How the waiting draw got its pipeline | Draws | Time | |---|---|---| | From the pipeline cache | 187 | 0.1–0.4 ms each | | Not in the cache (lookup cost), then a quick build | 17 | lookup 0–23 ms, build 13 ms–1.35 s | | Full build, or a worker's | 0 | | No `SLOW cache lookup` line. What is left: - **The first pause menu of the version stood still for 11.2 s.** That frame met 11 pipelines no cache had. Each was built quick, one after another, at about 1.0–1.35 s each. For these shaders the optimizer is not the cost: their full builds on the workers took about as long (0.7–1.9 s). Before this PR, that frame would also have waited behind the 165 builds already queued. They are noted now, so the precompile builds them at the next start. - **Draws that may be left out still wait in the background queue**, behind slow builds, even when their pipeline is in the cache. Here that was up to 136 draws a frame, with the world drawn wrong for a while. This PR leaves that rule alone. A normal start precompiles them first. This branch has `main` merged in (66892e6, after the CI overhaul in #695). `hookshot_vk.dll` compiles from `main` with this PR and the two related ones merged in (`build.py` with `RX_VK_ONLY=1`, llvm-mingw from the v2.0.2 builder). `tests/test_pipeline_wait.py` compiles `pipeline_build_waited` with stand-ins for the driver. It covers: - a cache hit, and a device without cache control; - quick build on and off; - a worker that already began the build; - a full build landing while the quick one is made; - a failed quick build. It also checks the source for the queued job's claim, where the draw site calls this, the quick build's flags and cache, the retry without the optimizer, and the log. It fails if a quick pipeline may overwrite a full one, or if a cache hit doesn't spare the worker. `tests/run_all.py` picks it up and gives it `--cxx g++`, like the other compiled checks. 🤖 Generated with [Claude Code](https://claude.com/claude-code)

shayderrr commented

2026-11-08 18:26:38 +00:00

(Migrated from github.com)