fix(vk): multi-second freezes and white foliage without mesh shaders: fallback pipelines on a worker and in the precompile #1512

Merged

rakefire-impala merged 1 commit from fix/mesh-fallback-freezes-pipelines-on-worker into main

2026-11-10 14:46:57 +00:00

rakefire-impala commented

2026-11-10 05:43:55 +00:00

(Migrated from github.com)

Independent of the floor-grass and allocator PRs; they merge cleanly in any order. Part of splitting up #1023.

Problem

fallback_compute_pipelines made a program's task and mesh compute pipelines in its first draw, holding one global lock. Every recording thread stopped behind it. On an RX 5500M the game froze for seconds in the first room with foliage, and the door at spawn drew white until its pipelines were done.

Change

On a worker. fallback_pipelines_ready builds them on a worker, exactly as dispatch() builds a compute program's pipeline: compute_state 0/1/2 (otherwise only compute programs use it), and RX_VK_COMPUTE_SERIAL as there. The draw that starts a build waits up to 50 ms for it, which covers NVIDIA's compiles. A draw while it's still compiling is left out, as a dispatch is.
#1317's guarantee kept. fallback_compute_pipelines is now the build step. Its lock covers #1317's failure set and the program's pipelines, but not the compiles, so several programs compile at once. A failure is still remembered by key and variant before the return, and never retried. tests/test_mesh_fallback_failed_once.py passes unchanged.
Before play. precompile_run builds every fallback program's pipelines alongside the compute programs'. program_cache_load_some gets a fallbacks flag, set only by the precompile, that loads every program translated for the fallback (kind 4), whatever the compute list from #1417 says. Right after an update is exactly when those are cold. Only a GPU without mesh shaders has kind-4 programs.

The first start after a translator change compiles them all: on this laptop, about 15 minutes for 1,300 programs. The precompile's existing stall rule lets the game start while the last ones finish in the background. Later starts take them from the driver's cache in seconds.

Who it affects

Only GPUs without mesh shaders. Only they have fallback (kind-4) programs; on a GPU with mesh shaders nothing here runs.
The cost is moved, not added: the compiles that froze the game mid-play now happen before it, once per translator change. Measured on the RX 5500M on 2026-11-10: 885 s for the first start after a translator change (1,300 programs, cold), then 7 s on the next starts (1,381 programs from the driver's cache).

Tests

tests/test_mesh_fallback.py checks the worker, the draw, the lock's scope, the precompile and the program cache's load. The check fails on main without this change.
test_precompile_compute_guard.py extracts the precompile's compute branch, which is byte-for-byte unchanged.

Tested on


Tested on	RX 5500M (RDNA1, 4 GB VRAM, 16 GB RAM), Windows 11, AMD driver; Hookshot main 0c412ad (v2.0.11-383) with these PRs, 2026-11-10
Result	No freezes when foliage first appears, and the door isn't white at spawn. Log: 0 fallback draws left out while compiling; `precompile: ... 1300 mesh fallback programs in 2934 ms` (driver cache warm)

On a GPU without mesh shaders, the fallback compiled its compute pipelines inside the first draw, under one lock. AMD's compiler takes up to 12 s on one, so the game froze, and foliage and the door drew white until each was done. This builds them on a worker, and before play in the precompile. Independent of the floor-grass and allocator PRs; they merge cleanly in any order. Part of splitting up #1023. ## Problem `fallback_compute_pipelines` made a program's task and mesh compute pipelines in its first draw, holding one global lock. Every recording thread stopped behind it. On an RX 5500M the game froze for seconds in the first room with foliage, and the door at spawn drew white until its pipelines were done. ## Change - **On a worker.** `fallback_pipelines_ready` builds them on a worker, exactly as `dispatch()` builds a compute program's pipeline: `compute_state` 0/1/2 (otherwise only compute programs use it), and `RX_VK_COMPUTE_SERIAL` as there. The draw that starts a build waits up to 50 ms for it, which covers NVIDIA's compiles. A draw while it's still compiling is left out, as a dispatch is. - **#1317's guarantee kept.** `fallback_compute_pipelines` is now the build step. Its lock covers #1317's failure set and the program's pipelines, but **not the compiles**, so several programs compile at once. A failure is still remembered by key and variant before the return, and never retried. `tests/test_mesh_fallback_failed_once.py` passes unchanged. - **Before play.** `precompile_run` builds every fallback program's pipelines alongside the compute programs'. `program_cache_load_some` gets a `fallbacks` flag, set only by the precompile, that loads every program translated for the fallback (kind 4), **whatever the compute list from #1417 says**. Right after an update is exactly when those are cold. Only a GPU without mesh shaders has kind-4 programs. The first start after a translator change compiles them all: on this laptop, about 15 minutes for 1,300 programs. The precompile's existing stall rule lets the game start while the last ones finish in the background. Later starts take them from the driver's cache in seconds. ## Who it affects - **Only GPUs without mesh shaders.** Only they have fallback (kind-4) programs; on a GPU with mesh shaders nothing here runs. - **The cost is moved, not added:** the compiles that froze the game mid-play now happen before it, once per translator change. Measured on the RX 5500M on 2026-11-10: 885 s for the first start after a translator change (1,300 programs, cold), then **7 s** on the next starts (1,381 programs from the driver's cache). ## Tests - `tests/test_mesh_fallback.py` checks the worker, the draw, the lock's scope, the precompile and the program cache's load. The check fails on main without this change. - `test_precompile_compute_guard.py` extracts the precompile's compute branch, which is byte-for-byte unchanged. ## Tested on | | | |---|---| | Tested on | RX 5500M (RDNA1, 4 GB VRAM, 16 GB RAM), Windows 11, AMD driver; Hookshot main 0c412ad (v2.0.11-383) with these PRs, 2026-11-10 | | Result | No freezes when foliage first appears, and the door isn't white at spawn. Log: 0 fallback draws left out while compiling; `precompile: ... 1300 mesh fallback programs in 2934 ms` (driver cache warm) |

Commenting is not possible because the repository is archived.

No reviewers