WIP: fix(macos): let MoltenVK's precompile go on behind the title and name slow compiles

mooki commented

2026-11-15 02:17:42 +00:00

Member

Type: fix · Risk: medium · Tested on: macOS 26.7.1, Apple M4 Max (MoltenVK 1.4.2), arm64 and x86_64 under Rosetta · Needs: Windows, Linux (build and start)

Why

On a Mac the title never came: the start's precompile (every noted pipeline and compute program, about 42,000 items) sat for
minutes with nothing finishing, because Metal's compiler service takes minutes over the game's largest compute programs (one
took 6.5 minutes) and every precompile worker held one of them. The rule that lets the game start while the last stragglers
build behind it waits until every item has been handed out, and with 37,600 of 42,234 still to hand out it could never apply
(run #12: stalled at 4,600 for 160 s and at 7,698 for 80 s, no title). Nothing in the log said which compile was holding the
start, and a compile that failed when the compiler service crashed was kept failed in its pipeline cache. This PR lets the
title start on MoltenVK while the rest goes on behind it, names slow compiles, and tries a failed one once more. It is part of
the macOS port. See #420.

How to reproduce

Seen on: macOS 26.7.1, Apple M4 Max, MoltenVK 1.4.2 (main does not run on macOS yet: seen on the macOS port's integration
build a7711f3c, which carries the open macOS PRs)
Settings and switches: defaults (start mode), the program cache carried from earlier starts
Steps: start the game and wait for the title after the shader preparation.
Expected: the title comes once the precompile has built what it can, the slowest compiles going on behind it.
Actual: the precompile stalls for minutes with all 16 workers inside the compiler service, and the title never comes
(run #12, reports/2026-11-14-mac-startup-12.md; analysis in reports/2026-11-14-precompile-stall.md §1).

What changed

On MoltenVK (a new vendor fix, compiler_service), the precompile hands work out in an order made for Metal's compiler
service, lets the game start once nothing has finished for RX_VK_PRECOMPILE_STALL_S (30 s) whatever is left, or once only
compiles older than RX_VK_SLOW_COMPILE_S (10 s) remain, and builds the rest behind the game on a quarter of the workers,
every item still built. Every compile past the slow time is named while it runs and when it ends, and a compile that fails
with VK_ERROR_INITIALIZATION_FAILED is tried once more with a new pipeline cache. Every other driver keeps the precompile's
worker, its wait rule and its log line as they are.

gpu/vk/precompile_handoff.h (new, no Vulkan): due (the classic rule and the compiler service's), wait (the precompile's
wait loop, moved, with the rule as a parameter), Slots (what each worker has in hand, since when), Behind (only cap
workers take new items behind the game; the last out says so), behind_workers (a quarter, 2 to 4), order (noted
pipelines first, then mesh fallback and compute programs by SPIR-V size, smallest first, what the slow list names last),
Watch (names a running compile at the slow time and each doubling), slow_line/slow_parse.
gpu/vk/vk_draw.cpp precompile_run: the classic worker unchanged; behind_worker under compiler_service (its order,
its slot, a compute program taken through compute_state as a dispatch takes it, a noted pipeline made behind the game put
where a draw's lookup takes it, pipeline_keep_ready, never a null one); the wait through precompile_handoff::wait; the
worker caches (#650) merged by the last worker behind the game, when none compiles with them.
create_watched (around create_cached, which is unchanged): cached_graphics and cached_compute take it inside #252's
Intel gate where compiles are watched (compiles_watched: compiler_service, or RX_VK_SLOW_COMPILE_S set) and keep main's
create_cached line everywhere else. It adds the slow-compile watch (compile_watch_begin/_end, a watcher thread, the slow
list cache/longshot_pipelines.bin.slow) and compile_again (a new pipeline cache, one more compile, the line … compiled once more with a new pipeline cache: made|not made). An ask (FAIL_ON_PIPELINE_COMPILE_REQUIRED) is neither watched nor
tried again.
what each compile is: a thread-local Tagged scope at the eight sites that compile (the precompile's and the warm's noted
pipelines, the mesh fallback's compute stages, prepare_run's compute warm, the warm's and the dispatch's compute builds);
Stage::words (gpu/vk/vk_shader.h), the SPIR-V size, noted where the program cache sees SPIR-V (gpu/vk/vk_progcache.cpp).
gpu/vk/vk_vendor.h compiler_service (MoltenVK only, RX_VK_COMPILER_SERVICE); docs/development.md rows for
RX_VK_PRECOMPILE_STALL_S (undocumented until now), RX_VK_COMPILER_SERVICE and RX_VK_SLOW_COMPILE_S.
tests: tests/test_precompile_handoff.py + tests/precompile_handoff_test.cpp (new): the header with workers whose create
blocks on the giants (run #12's shape: the classic rule does not hand off, the compiler service's does after 1-2.5 s, every
item built once, at most the share behind the game, the stragglers named), the watch, the slow list and the order;
vk_draw.cpp's own create_watched and compile_again against a made-up driver that keeps a failed library in its cache as
MoltenVK does; vk_vendor.h; and the source's wiring.

Measurements

Verdict: fixes the stall: with this change the title and the menu came (run #17: handed off at 1,308 s with 42,043 of
42,135 done, 16 giants in hand, the rest behind the game on 4 workers; later integration runs #18, #19 and #21 handed off after
626-694 s, the first present 4.5-5 s later), where run #12 without it stalled with no title. The slow-compile lines name the
giants (run #14: 57 compiles named, 44 of them finished after 12.9-23.4 s, 13 still running at the quit). Measured on
integration builds before the rebase; the hand-off itself is unchanged by it (the adaptation adds #252's quick builds, which
apply only on Intel's one-at-a-time builds).

Baseline: main does not run on macOS yet, so both columns are the macOS port's integration builds, arm64, MoltenVK 1.4.2, start
mode, the program cache carried. Before: a7711f3c (run #12), without the hand-off. After: 4e472987 (run #17, integration 23 plus
this topic). One run each; run #17's timing is load-affected (full suites held the suite slot throughout).

Hardware: macOS 26.7.1, Apple M4 Max (40-core GPU), MoltenVK 1.4.2, 16-core CPU, 64 GB
Scene: a start to the title, windowless, defaults
Method: the game log's vk: precompile: and vk: start-up: lines and the bridge's menu-ready, one run each
How to reproduce: a start on a macOS build with RX_VK_COMPILER_SERVICE=0 (as before) against unset

	Before: integration @ a7711f3c (run #12)	After: integration @ 4e472987 (run #17)
precompile	stalled at 4,600 for about 160 s and at 7,698 for about 80 s; 10,043 of 42,234 at the quit (412 s)	42,043 of 42,135 done at 1,308 s, then the hand-off
title and menu	never	came; menu-ready
giants named	none	16 `in hand when the game started` lines, 15 `behind the game … finished` (7 made)

#17: vk: precompile: in this order for Metal's compiler service (RX_VK_COMPILER_SERVICE): 30329 noted pipelines, then 2099 mesh fallback programs and 9707 compute programs by their SPIR-V size, and last 18 an earlier run found slow (16 in …/game/longshot_pipelines.bin.slow)
#17: vk: precompile: 42043 of 42135 done (42059 handed out), nothing finished for longer than the stall time: the game starts, the rest go on behind it on 4 workers, every one of them still built (RX_VK_PRECOMPILE_STALL_S, RX_VK_SLOW_COMPILE_S)
#18: vk: precompile: 42134 of 42234 done (42234 handed out), every item still running has run for longer than the slow time: the game starts, the rest go on behind it on 4 workers, every one of them still built (RX_VK_PRECOMPILE_STALL_S, RX_VK_SLOW_COMPILE_S)

Run reports: reports/2026-11-14-mac-startup-12.md, -14.md, -17.md, -18.md, -19.md, -21.md.

Flags and settings

One new switch that turns the change off, and two switches with new defaults on MoltenVK; nothing changes elsewhere.

Flag	What it is	Before this PR	After this PR	Effect	Helps / doesn't help / untested	Recommendation
`RX_VK_COMPILER_SERVICE`	`=0`: the precompile, its wait and its compiles as on every other driver; `=1`: the hand-off, the order, the watch and the retry on any driver	(no switch)	on for MoltenVK only	the title comes while the giants build behind it	helps: MoltenVK; untested: other drivers with `=1`	on for MoltenVK as it ships; `=0` only to compare
`RX_VK_SLOW_COMPILE_S`	seconds after which a compile is named and written to the slow list; `0`: none	(no switch)	10 with `compiler_service`, none elsewhere	log lines, the slow list	helps: finding the giants	as it ships
`RX_VK_PRECOMPILE_STALL_S`	seconds with nothing finished before the game starts (0: wait for all)	30, once every item is handed out	30; with `compiler_service` whatever is left	the title is not held by a few giants	helps: MoltenVK	as it ships

Testing

Tested: on macOS 26.7.1, Apple M4 Max, at this head (main 6c9edffe plus #650, #651 and this PR):

tests/test_precompile_handoff.py --arch arm64 and --arch x86_64 (the fixtures run under Rosetta): pass on both (the
hand-off fixture and compile_again: all checks). On main 6c9edffe with the test files copied in: fails on both (no
precompile_handoff.h, no create_watched/compile_again, 35 source checks name what is missing). On this branch's base
(#650 with #651 merged, f9f166f7) with the test files: fails on both, the same way. reports/PubWave-files/PubPrep/proof-03/
(p37-head-ph-*, p37-base-ph-*, p37-pre-ph-*).
At this head, also pass: main's own test_pcache_fetch_lock (#190, both arches) and test_intel_compiler_gate (#252),
test_fallback_pipeline_state (it compiles build_graphics cut from the source), test_thread_stack (both arches) and
test_worker_caches (both arches). reports/PubWave-files/PubPrep/proof-03/.
Every check that reads a file changed on this branch since main (this PR, #650 and #651; 298 checks), through
tools/ls-suite.py: 279 pass, 19 fail; 16 of them fail on main 6c9edffe on macOS too (reports/Baseline-files/main-6c9edffe/),
and the other 3 (test_precompile_compute_guard, test_shader_module_guard: "Windows structured exceptions required";
test_present_aspect_vulkan: no Vulkan library to link on this Mac) are skipped on macOS in a full run and fail only because
they were named. reports/PubWave-files/PubPrep/p37-related-03/, p37-related-03-compare.txt.
vk_draw.cpp, vk_main.cpp and vk_progcache.cpp compiled whole for arm64 and x86_64 with the renderer's flags, with the
macOS compile pieces of #476, #490 and #492 applied: 6 of 6. reports/PubWave-files/PubPrep/p37-tu-stack-03/.
git merge-tree of this head with each open macOS PR's head (#431, #437, #438, #476, #479, #480, #490, #491, #492): no
conflict.
In the game (integration builds carrying these commits, before the rebase): runs #14, #17, #18, #19 and #21, above.

Not tested: the game at this head (main does not run on macOS without the other macOS PRs), the hand-off on x86_64 under
Rosetta in the game, a compile failure that the second try saves (run #14's two retries were both not made (-3): bracket
nesting in Metal, fixed elsewhere), and Windows and Linux (no change in behaviour there: compiler_service is off, the classic
worker and its rule are unchanged; not built or run on them: its test request covers the build and a start).

Must be tested:

macOS, Apple silicon, on the rebased macOS stack: a start to the title from a seeded cache with RX_VK_COMPILER_SERVICE=0
against unset, expecting the hand-off line, in hand when the game started lines and the title with it, and the old stall
without it (by hand: the parent's GPU slot).
Windows and Linux, any GPU: build this PR (head 7ef7cff9, it carries #650 and #651) and main 6c9edffe the same way and start
each to the title, expecting the same start-up lines as main, no in this order for Metal's compiler service line, and the
precompile finishing before the title as on main (by hand: test request T-XXXX)

Risks

What could break: on MoltenVK the title can come while giants still build: a draw or dispatch that needs one is left out
until it is ready, as with play's own builds; a behind-the-game compile holds a worker until the process ends (a quit ends the
process without joining them). With RX_VK_PCACHE_LOAD=1 on a Mac a play-cache save waits for compiles counted in its
generation (see reports/2026-11-14-precompile-stall.md, Addendum). Other drivers: a Tagged scope (a thread-local set) at
each compile site and Stage::words written where the program cache reads SPIR-V; the precompile's wait loop is moved into
precompile_handoff::wait with the same expression in the same order.

Turning it off: RX_VK_COMPILER_SERVICE=0, or revert the PR.

Compatibility: Nothing changes for players on Windows and Linux: no shader stamp (Stage::words is not in the translator
stamp, which takes sizeof(Program)), cache format or MKWVK_API_VERSION change. On macOS a new small file,
cache/longshot_pipelines.bin.slow, beside the notes.

Reviewer notes

Review 89606310..7ef7cff9 (5 commits; in git's notation f9f166f7..7ef7cff9): the hand-off and the retry, the test's
messages on a tree without them, the docs rows, the graphics names at the callers, and the adaptation to main (7ef7cff9).
Depends on #650 (per-worker pipeline caches: worker_caches_merge, worker_cache_calls, pcache_files_unused) and #651
(renderer threads with 16 MiB stacks: thread_stack::thread, which the watcher thread uses). The branch is #650's head
(078c9a7a) with #651's head (7707c634) merged (f9f166f7): 4 blocks in gpu/vk/vk_draw.cpp (the include block and the
thread lines of prepare_run, precompile_at_create and precompile_run), each side's lines kept.
The topic's four commits were cherry-picked with -x from macos/precompile-stall (77d4971c f9a179c0 194a230d 528784dc),
made on the integration branch. Conflicts in the pick:
- gpu/vk/vk_draw.cpp cached_graphics/cached_compute: #252 (57f4eaec) put Intel's gate and #190 (f41a5c59)
  guarded_*_held there; kept, with create_watched where compiles are watched (the adaptation commit keeps main's
  create_cached line for every other driver).
- gpu/vk/vk_draw.cpp prepare_run's compute warm: #252's gate and #650's rework (create_cached under the gate); the
  caches' path goes through create_watched there where compiles are watched, named.
- gpu/vk/vk_draw.cpp the dispatch's inline build: the integration branch's name plain (#663's rename) is p on main.
- vk_progcache.cpp (3), vk_shader.h, vk_vendor.h (3): the integration branch's context held #643's set_for_tables and
  #644's array_length/set_for_lengths; only this PR's words and compiler_service lines were kept.
The semantic adaptation is its own commit, 7ef7cff9: cached_* and the warm keep main's (and #650's) create_cached lines
and go through create_watched only where compiles are watched, so every other driver's pipelines are made as on main and
main's own source checks (test_intel_compiler_gate, test_pcache_fetch_lock) read what they read there; behind_worker
builds compute pipelines as main's classic worker does since #252 (Intel's quick builds, optimize_owed); the test's source
checks follow main's spellings; the slow list is in cache/ since #321 (76699aad). tools/ls-run.py's carry list should
carry cache/longshot_pipelines.bin.slow.
Sibling macOS PRs: #643, #644 (vk_vendor.h, vk_progcache.cpp, vk_shader.h: each side's fields and lines next to these),
#663 (renames the dispatch's p to plain: this PR's Tagged line follows it), #648 (vk_progcache.cpp), #423 (log lines
beside the precompile's: its RX_VK_FAIL_REASONS summary, folded into #423 by the owner).
Conflicts with main: none. With the open macOS PRs (#431, #437, #438, #476, #479, #480, #490, #491, #492): none
(git merge-tree).

Issues

See #420

Checklist

Everyone

One topic.
The title is a Conventional Commit (type(scope): description) of the status line's type.
Why gives the reason plainly and links the issues that fit (the tracker searched).
Measurements has a verdict, or one line saying why there is nothing to measure.
Testing names what was tested, what wasn't, and what must still be tested, where and how. (Test requests not open yet.)
Rebased on current main (main e090cc48).
python -I -B tests/run_all.py passes (any failure that also fails on main is named). (On the wave's merge of every macOS PR built with this one, 453eb678: 553 passed, 32 failed, 18 skipped against main 6c9edffe's 531/32/17 on this Mac; the failures are main's own except test_split_ub_flush, which #658's tree at the time broke and its head fixes; test_collect_wide now passes; the new tests pass, test_precompile_handoff among them, and test_darwin_audio_service, Darwin-only, is skipped by the runner; reports/PubWave-files/waveA/.)
No game files, dumps, build output or generated files.
third_party/ untouched (a new folder with a PROVENANCE.md instead).
The license is OK: GPL-3.0-or-later, nothing incompatible and nothing decompiled.

When it applies

AI assistance recorded with Assisted-by: / Co-Authored-By: trailers.
Switches documented (docs/development.md; the README for a menu row) and shipped at their recommended value.
A game-free test added for the fix, failing on main and passing here.
Docs updated.

**Type:** fix · **Risk:** medium · **Tested on:** macOS 26.7.1, Apple M4 Max (MoltenVK 1.4.2), arm64 and x86_64 under Rosetta · **Needs:** Windows, Linux (build and start) ## Why On a Mac the title never came: the start's precompile (every noted pipeline and compute program, about 42,000 items) sat for minutes with nothing finishing, because Metal's compiler service takes minutes over the game's largest compute programs (one took 6.5 minutes) and every precompile worker held one of them. The rule that lets the game start while the last stragglers build behind it waits until every item has been handed out, and with 37,600 of 42,234 still to hand out it could never apply (run #12: stalled at 4,600 for 160 s and at 7,698 for 80 s, no title). Nothing in the log said which compile was holding the start, and a compile that failed when the compiler service crashed was kept failed in its pipeline cache. This PR lets the title start on MoltenVK while the rest goes on behind it, names slow compiles, and tries a failed one once more. It is part of the macOS port. See #420. ## How to reproduce **Seen on:** macOS 26.7.1, Apple M4 Max, MoltenVK 1.4.2 (main does not run on macOS yet: seen on the macOS port's integration build a7711f3c, which carries the open macOS PRs) **Settings and switches:** defaults (start mode), the program cache carried from earlier starts **Steps:** start the game and wait for the title after the shader preparation. **Expected:** the title comes once the precompile has built what it can, the slowest compiles going on behind it. **Actual:** the precompile stalls for minutes with all 16 workers inside the compiler service, and the title never comes (run #12, `reports/2026-11-14-mac-startup-12.md`; analysis in `reports/2026-11-14-precompile-stall.md` §1). ## What changed On MoltenVK (a new vendor fix, `compiler_service`), the precompile hands work out in an order made for Metal's compiler service, lets the game start once nothing has finished for `RX_VK_PRECOMPILE_STALL_S` (30 s) whatever is left, or once only compiles older than `RX_VK_SLOW_COMPILE_S` (10 s) remain, and builds the rest behind the game on a quarter of the workers, every item still built. Every compile past the slow time is named while it runs and when it ends, and a compile that fails with `VK_ERROR_INITIALIZATION_FAILED` is tried once more with a new pipeline cache. Every other driver keeps the precompile's worker, its wait rule and its log line as they are. - `gpu/vk/precompile_handoff.h` (new, no Vulkan): `due` (the classic rule and the compiler service's), `wait` (the precompile's wait loop, moved, with the rule as a parameter), `Slots` (what each worker has in hand, since when), `Behind` (only `cap` workers take new items behind the game; the last out says so), `behind_workers` (a quarter, 2 to 4), `order` (noted pipelines first, then mesh fallback and compute programs by SPIR-V size, smallest first, what the slow list names last), `Watch` (names a running compile at the slow time and each doubling), `slow_line`/`slow_parse`. - `gpu/vk/vk_draw.cpp` `precompile_run`: the classic `worker` unchanged; `behind_worker` under `compiler_service` (its order, its slot, a compute program taken through `compute_state` as a dispatch takes it, a noted pipeline made behind the game put where a draw's lookup takes it, `pipeline_keep_ready`, never a null one); the wait through `precompile_handoff::wait`; the worker caches (#650) merged by the last worker behind the game, when none compiles with them. - `create_watched` (around `create_cached`, which is unchanged): `cached_graphics` and `cached_compute` take it inside #252's Intel gate where compiles are watched (`compiles_watched`: `compiler_service`, or `RX_VK_SLOW_COMPILE_S` set) and keep main's `create_cached` line everywhere else. It adds the slow-compile watch (`compile_watch_begin`/`_end`, a watcher thread, the slow list `cache/longshot_pipelines.bin.slow`) and `compile_again` (a new pipeline cache, one more compile, the line `… compiled once more with a new pipeline cache: made|not made`). An ask (`FAIL_ON_PIPELINE_COMPILE_REQUIRED`) is neither watched nor tried again. - what each compile is: a thread-local `Tagged` scope at the eight sites that compile (the precompile's and the warm's noted pipelines, the mesh fallback's compute stages, `prepare_run`'s compute warm, the warm's and the dispatch's compute builds); `Stage::words` (`gpu/vk/vk_shader.h`), the SPIR-V size, noted where the program cache sees SPIR-V (`gpu/vk/vk_progcache.cpp`). - `gpu/vk/vk_vendor.h` `compiler_service` (MoltenVK only, `RX_VK_COMPILER_SERVICE`); `docs/development.md` rows for `RX_VK_PRECOMPILE_STALL_S` (undocumented until now), `RX_VK_COMPILER_SERVICE` and `RX_VK_SLOW_COMPILE_S`. - tests: `tests/test_precompile_handoff.py` + `tests/precompile_handoff_test.cpp` (new): the header with workers whose create blocks on the giants (run #12's shape: the classic rule does not hand off, the compiler service's does after 1-2.5 s, every item built once, at most the share behind the game, the stragglers named), the watch, the slow list and the order; `vk_draw.cpp`'s own `create_watched` and `compile_again` against a made-up driver that keeps a failed library in its cache as MoltenVK does; `vk_vendor.h`; and the source's wiring. ## Measurements **Verdict:** fixes the stall: with this change the title and the menu came (run #17: handed off at 1,308 s with 42,043 of 42,135 done, 16 giants in hand, the rest behind the game on 4 workers; later integration runs #18, #19 and #21 handed off after 626-694 s, the first present 4.5-5 s later), where run #12 without it stalled with no title. The slow-compile lines name the giants (run #14: 57 compiles named, 44 of them finished after 12.9-23.4 s, 13 still running at the quit). Measured on integration builds before the rebase; the hand-off itself is unchanged by it (the adaptation adds #252's quick builds, which apply only on Intel's one-at-a-time builds). Baseline: main does not run on macOS yet, so both columns are the macOS port's integration builds, arm64, MoltenVK 1.4.2, start mode, the program cache carried. Before: a7711f3c (run #12), without the hand-off. After: 4e472987 (run #17, integration 23 plus this topic). One run each; run #17's timing is load-affected (full suites held the suite slot throughout). Hardware: macOS 26.7.1, Apple M4 Max (40-core GPU), MoltenVK 1.4.2, 16-core CPU, 64 GB Scene: a start to the title, windowless, defaults Method: the game log's `vk: precompile:` and `vk: start-up:` lines and the bridge's menu-ready, one run each How to reproduce: a start on a macOS build with `RX_VK_COMPILER_SERVICE=0` (as before) against unset | | Before: integration @ a7711f3c (run #12) | After: integration @ 4e472987 (run #17) | |---|---|---| | precompile | stalled at 4,600 for about 160 s and at 7,698 for about 80 s; 10,043 of 42,234 at the quit (412 s) | 42,043 of 42,135 done at 1,308 s, then the hand-off | | title and menu | never | came; menu-ready | | giants named | none | 16 `in hand when the game started` lines, 15 `behind the game … finished` (7 made) | ``` #17: vk: precompile: in this order for Metal's compiler service (RX_VK_COMPILER_SERVICE): 30329 noted pipelines, then 2099 mesh fallback programs and 9707 compute programs by their SPIR-V size, and last 18 an earlier run found slow (16 in …/game/longshot_pipelines.bin.slow) #17: vk: precompile: 42043 of 42135 done (42059 handed out), nothing finished for longer than the stall time: the game starts, the rest go on behind it on 4 workers, every one of them still built (RX_VK_PRECOMPILE_STALL_S, RX_VK_SLOW_COMPILE_S) #18: vk: precompile: 42134 of 42234 done (42234 handed out), every item still running has run for longer than the slow time: the game starts, the rest go on behind it on 4 workers, every one of them still built (RX_VK_PRECOMPILE_STALL_S, RX_VK_SLOW_COMPILE_S) ``` Run reports: `reports/2026-11-14-mac-startup-12.md`, `-14.md`, `-17.md`, `-18.md`, `-19.md`, `-21.md`. ### Flags and settings One new switch that turns the change off, and two switches with new defaults on MoltenVK; nothing changes elsewhere. | Flag | What it is | Before this PR | After this PR | Effect | Helps / doesn't help / untested | Recommendation | |---|---|---|---|---|---|---| | `RX_VK_COMPILER_SERVICE` | `=0`: the precompile, its wait and its compiles as on every other driver; `=1`: the hand-off, the order, the watch and the retry on any driver | (no switch) | on for MoltenVK only | the title comes while the giants build behind it | helps: MoltenVK; untested: other drivers with `=1` | on for MoltenVK as it ships; `=0` only to compare | | `RX_VK_SLOW_COMPILE_S` | seconds after which a compile is named and written to the slow list; `0`: none | (no switch) | 10 with `compiler_service`, none elsewhere | log lines, the slow list | helps: finding the giants | as it ships | | `RX_VK_PRECOMPILE_STALL_S` | seconds with nothing finished before the game starts (0: wait for all) | 30, once every item is handed out | 30; with `compiler_service` whatever is left | the title is not held by a few giants | helps: MoltenVK | as it ships | ## Testing **Tested:** on macOS 26.7.1, Apple M4 Max, at this head (main 6c9edffe plus #650, #651 and this PR): - `tests/test_precompile_handoff.py --arch arm64` and `--arch x86_64` (the fixtures run under Rosetta): pass on both (the hand-off fixture and `compile_again`: all checks). On main 6c9edffe with the test files copied in: fails on both (no `precompile_handoff.h`, no `create_watched`/`compile_again`, 35 source checks name what is missing). On this branch's base (#650 with #651 merged, `f9f166f7`) with the test files: fails on both, the same way. `reports/PubWave-files/PubPrep/proof-03/` (`p37-head-ph-*`, `p37-base-ph-*`, `p37-pre-ph-*`). - At this head, also pass: main's own `test_pcache_fetch_lock` (#190, both arches) and `test_intel_compiler_gate` (#252), `test_fallback_pipeline_state` (it compiles `build_graphics` cut from the source), `test_thread_stack` (both arches) and `test_worker_caches` (both arches). `reports/PubWave-files/PubPrep/proof-03/`. - Every check that reads a file changed on this branch since main (this PR, #650 and #651; 298 checks), through `tools/ls-suite.py`: 279 pass, 19 fail; 16 of them fail on main 6c9edffe on macOS too (`reports/Baseline-files/main-6c9edffe/`), and the other 3 (`test_precompile_compute_guard`, `test_shader_module_guard`: "Windows structured exceptions required"; `test_present_aspect_vulkan`: no Vulkan library to link on this Mac) are skipped on macOS in a full run and fail only because they were named. `reports/PubWave-files/PubPrep/p37-related-03/`, `p37-related-03-compare.txt`. - `vk_draw.cpp`, `vk_main.cpp` and `vk_progcache.cpp` compiled whole for arm64 and x86_64 with the renderer's flags, with the macOS compile pieces of #476, #490 and #492 applied: 6 of 6. `reports/PubWave-files/PubPrep/p37-tu-stack-03/`. - `git merge-tree` of this head with each open macOS PR's head (#431, #437, #438, #476, #479, #480, #490, #491, #492): no conflict. - In the game (integration builds carrying these commits, before the rebase): runs #14, #17, #18, #19 and #21, above. **Not tested:** the game at this head (main does not run on macOS without the other macOS PRs), the hand-off on x86_64 under Rosetta in the game, a compile failure that the second try saves (run #14's two retries were both `not made (-3)`: bracket nesting in Metal, fixed elsewhere), and Windows and Linux (no change in behaviour there: `compiler_service` is off, the classic worker and its rule are unchanged; not built or run on them: its test request covers the build and a start). **Must be tested:** - macOS, Apple silicon, on the rebased macOS stack: a start to the title from a seeded cache with `RX_VK_COMPILER_SERVICE=0` against unset, expecting the hand-off line, `in hand when the game started` lines and the title with it, and the old stall without it (by hand: the parent's GPU slot). - Windows and Linux, any GPU: build this PR (head 7ef7cff9, it carries #650 and #651) and main 6c9edffe the same way and start each to the title, expecting the same start-up lines as main, no `in this order for Metal's compiler service` line, and the precompile finishing before the title as on main (by hand: test request T-XXXX) ## Risks **What could break:** on MoltenVK the title can come while giants still build: a draw or dispatch that needs one is left out until it is ready, as with play's own builds; a behind-the-game compile holds a worker until the process ends (a quit ends the process without joining them). With `RX_VK_PCACHE_LOAD=1` on a Mac a play-cache save waits for compiles counted in its generation (see `reports/2026-11-14-precompile-stall.md`, Addendum). Other drivers: a `Tagged` scope (a thread-local set) at each compile site and `Stage::words` written where the program cache reads SPIR-V; the precompile's wait loop is moved into `precompile_handoff::wait` with the same expression in the same order. **Turning it off:** `RX_VK_COMPILER_SERVICE=0`, or revert the PR. **Compatibility:** Nothing changes for players on Windows and Linux: no shader stamp (`Stage::words` is not in the translator stamp, which takes `sizeof(Program)`), cache format or `MKWVK_API_VERSION` change. On macOS a new small file, `cache/longshot_pipelines.bin.slow`, beside the notes. ## Reviewer notes - Review `89606310..7ef7cff9` (5 commits; in git's notation `f9f166f7..7ef7cff9`): the hand-off and the retry, the test's messages on a tree without them, the docs rows, the graphics names at the callers, and the adaptation to main (`7ef7cff9`). - Depends on #650 (per-worker pipeline caches: `worker_caches_merge`, `worker_cache_calls`, `pcache_files_unused`) and #651 (renderer threads with 16 MiB stacks: `thread_stack::thread`, which the watcher thread uses). The branch is #650's head (`078c9a7a`) with #651's head (`7707c634`) merged (`f9f166f7`): 4 blocks in `gpu/vk/vk_draw.cpp` (the include block and the thread lines of `prepare_run`, `precompile_at_create` and `precompile_run`), each side's lines kept. - The topic's four commits were cherry-picked with `-x` from `macos/precompile-stall` (`77d4971c f9a179c0 194a230d 528784dc`), made on the integration branch. Conflicts in the pick: - `gpu/vk/vk_draw.cpp` `cached_graphics`/`cached_compute`: #252 (`57f4eaec`) put Intel's gate and #190 (`f41a5c59`) `guarded_*_held` there; kept, with `create_watched` where compiles are watched (the adaptation commit keeps main's `create_cached` line for every other driver). - `gpu/vk/vk_draw.cpp` `prepare_run`'s compute warm: #252's gate and #650's rework (`create_cached` under the gate); the caches' path goes through `create_watched` there where compiles are watched, named. - `gpu/vk/vk_draw.cpp` the dispatch's inline build: the integration branch's name `plain` (#663's rename) is `p` on main. - `vk_progcache.cpp` (3), `vk_shader.h`, `vk_vendor.h` (3): the integration branch's context held #643's `set_for_tables` and #644's `array_length`/`set_for_lengths`; only this PR's `words` and `compiler_service` lines were kept. - The semantic adaptation is its own commit, `7ef7cff9`: `cached_*` and the warm keep main's (and #650's) `create_cached` lines and go through `create_watched` only where compiles are watched, so every other driver's pipelines are made as on main and main's own source checks (`test_intel_compiler_gate`, `test_pcache_fetch_lock`) read what they read there; `behind_worker` builds compute pipelines as main's classic worker does since #252 (Intel's quick builds, `optimize_owed`); the test's source checks follow main's spellings; the slow list is in `cache/` since #321 (`76699aad`). `tools/ls-run.py`'s carry list should carry `cache/longshot_pipelines.bin.slow`. - Sibling macOS PRs: #643, #644 (`vk_vendor.h`, `vk_progcache.cpp`, `vk_shader.h`: each side's fields and lines next to these), #663 (renames the dispatch's `p` to `plain`: this PR's `Tagged` line follows it), #648 (`vk_progcache.cpp`), #423 (log lines beside the precompile's: its `RX_VK_FAIL_REASONS` summary, folded into #423 by the owner). - Conflicts with main: none. With the open macOS PRs (#431, #437, #438, #476, #479, #480, #490, #491, #492): none (`git merge-tree`). ## Issues See #420 ## Checklist ### Everyone - [x] One topic. - [x] The title is a Conventional Commit (`type(scope): description`) of the status line's type. - [x] Why gives the reason plainly and links the issues that fit (the tracker searched). - [x] Measurements has a verdict, or one line saying why there is nothing to measure. - [ ] Testing names what was tested, what wasn't, and what must still be tested, where and how. (Test requests not open yet.) - [x] Rebased on current `main` (main e090cc48). - [x] `python -I -B tests/run_all.py` passes (any failure that also fails on `main` is named). (On the wave's merge of every macOS PR built with this one, `453eb678`: 553 passed, 32 failed, 18 skipped against main 6c9edffe's 531/32/17 on this Mac; the failures are main's own except `test_split_ub_flush`, which #658's tree at the time broke and its head fixes; `test_collect_wide` now passes; the new tests pass, `test_precompile_handoff` among them, and `test_darwin_audio_service`, Darwin-only, is skipped by the runner; `reports/PubWave-files/waveA/`.) - [x] No game files, dumps, build output or generated files. - [x] `third_party/` untouched (a new folder with a `PROVENANCE.md` instead). - [x] The license is OK: GPL-3.0-or-later, nothing incompatible and nothing decompiled. ### When it applies - [x] AI assistance recorded with `Assisted-by:` / `Co-Authored-By:` trailers. - [x] Switches documented (docs/development.md; the README for a menu row) and shipped at their recommended value. - [x] A game-free test added for the fix, failing on `main` and passing here. - [x] Docs updated.

mooki added 14 commits

2026-11-15 02:17:43 +00:00

fix(macos): the shader preparation runs on every usable CPU on macOS 1843f6f809

prepare_run's first part (every shader translated, and each compute
variant compiled for the pipeline cache) ran on half the CPUs
everywhere: prepare_cpus gives half_cpus unless the Windows process heap
is the segment heap. Half exists for the NT heap's lock that an
in-process driver compiler queued on, and for whole-PC freezes when
those compilers ran on every CPU at High priority. Neither holds on
macOS: MoltenVK converts SPIR-V to MSL in a few ms and hands every
compile to Metal's compiler service (MTLCompilerService, a launchd
service at nice 0), so the workers mostly wait on it. The Mac's first
start logged "8 workers for the shader programs (16 CPUs usable; half
the CPUs)".

prepare_workers::count takes an `every` argument (default false): each
usable CPU gets a worker, at least half_cpus and 1, at most kMaxWorkers.
kEveryCpu is true where __APPLE__ is defined, and prepare_cpus passes
it; the preparation's log line says "every one: macOS". In a synthetic
MoltenVK 1.4.2 benchmark on an M4 Max (16 CPUs, macOS 26.7.1) 16 threads
compiled 56-61 % more compute and 35-39 % more graphics pipelines a
second than 8, but only with a pipeline cache per thread; with the one
shared cache LS uses today the rate is flat from 4 to 16 threads. The
next commit gives each worker its own cache.

tests/test_prepare_every_cpu.py compiles the real prepare_workers.h and
prepare_cpus() cut from vk_progcache.cpp: on this compiler's target
prepare_cpus() gives 16 of 16 usable CPUs on macOS (8 elsewhere); count
with and without `every`; prepare_cpus with kEveryCpu forced either way.
On the base it fails: prepare_cpus() gives 8 of 16 on macOS.

Windows and Linux: unchanged (kEveryCpu false; the existing
test_prepare_workers and test_prepare_cpus cases are untouched).

Assisted-by: Claude Opus 5.5
Co-Authored-By: Claude Opus 5.5 <[email protected]>
(cherry picked from commit 3573e736e15078554543e4740c5e2d53c719fa89)

fix(macos): a pipeline cache per preparation worker on MoltenVK 55749103ac

MoltenVK 1.4.2 holds one mutex per VkPipelineCache across the SPIR-V to
MSL conversion and Metal's library compile (MVKPipeline.mm
getShaderLibrary, _shaderCacheLock). Every worker of the start's
preparation compiled with the one start cache, pcache, so they compiled
one pipeline at a time: the Mac's first start (mac-arm64-a5f2d6f9-01,
M4 Max) logged strict pairs of Metal compiles every ~195 ms, 5.05
compute variants a second, an estimated 33 minutes for the compute
programs alone. A synthetic benchmark on that Mac (PrepSpeed, MoltenVK
1.4.2, macOS 26.7.1): one shared cache stays at 10.5 compute and 9.6
graphics pipelines a second from 4 to 16 threads; a cache per thread,
merged after, gives 31.5 / 29.7 at 8 threads and 49-51 / 40-41 at 16.

gpu/vk/worker_caches.h: a Set of empty caches, one per worker of a
phase, that a worker enters with Scope; create() asks the shared cache
first under VK_PIPELINE_CREATE_FAIL_ON_PIPELINE_COMPILE_REQUIRED_BIT
and compiles what it lacks with the worker's own; merge_into() merges
them into the start cache with one vkMergePipelineCaches in slot order
(MoltenVK retains each Metal library, no compile). The caches are not
externally synchronized, so a shared slot stays correct.

vk_draw.cpp:
- create_cached: before play, a worker in a phase with a set goes
  through worker_caches::create (pcache asked first, so a saved file's
  or an earlier phase's pipelines are found as before); every use of
  pcache holds pcache_mu shared, and worker_caches_merge holds it alone
  (vkMergePipelineCaches's dstCache is externally synchronized:
  vk.xml externsync without INTERNALLY_SYNCHRONIZED_MERGE). A save's
  reads of pcache's data hold it shared too.
- prepare_run (the compute warm), precompile_run, precompile_warm (both
  passes) and precompile_at_create make a set under worker_caches_on,
  enter it in their workers and merge it right after the join; the
  precompile's stragglers left running are merged as far as they got.
- precompile_run, with the caches on, sorts the notes by program
  variant and hands a variant's notes to one worker in runs of up to
  32 (worker_caches::runs/claim), so its shader libraries are compiled
  once, not by every worker that met one of its notes at the same time.
- In play, with the caches on, a compile goes into a new cache of its
  own merged into the generation's at once under the generation's lock
  (made_alone), which the asks of the generation hold shared: the asks
  keep their order and find what any thread compiled.

vk_vendor.h: worker_caches, on for VK_DRIVER_ID_MOLTENVK only, and only
with pipelineCreationCacheControl; RX_VK_WORKER_CACHES forces it either
way. Elsewhere nothing measured says a driver's cache is locked across
its compiles, and the caches would only add merges. The pipelines built,
the persisted cache's format and its stamp are the same either way.

tests/test_worker_caches.py runs vk_draw.cpp's own create_cached,
play_cache_make, worker_caches_on and worker_caches_merge and the real
header against a made-up driver whose caches lock across a compile:
four workers compile at once (without a set they cannot); pcache asked
first, nothing compiled with it; ask-only callers never compiled; caches
not externally synchronized; one merge in slot order whatever order the
slots were taken in, with no ask of pcache beside it; play's compiles
merged into the generation at once; runs and claims. The four-workers
part runs against the base's create_cached and fails there (1 of 4
compiles met the others). tests/test_pipeline_play_cache.py gets an
"alone" case, and both play-cache and cache-save harnesses the new
stand-ins (pcache_mu, worker_caches_on).

Windows and Linux: the same code paths as before (no set, one shared
cache), plus an uncontended shared lock around pcache's uses.

Assisted-by: Claude Opus 5.5
Co-Authored-By: Claude Opus 5.5 <[email protected]>
(cherry picked from commit 2174e33f14ccce8c1635565e734773a33bc32455)

fix(macos): the worker caches make no Vulkan call from a header b5f4f1febd

tests/test_recorder_sync.py holds every header to no recorder boundary
call: vk_work.h, included last by each renderer .cpp, turns
vkDestroyPipelineCache into a drain of RX_VK_RECORDER's packets and then
the call, and a header's calls would be wrapped or not depending on what
included it. worker_caches.h called vkCreatePipelineCache,
vkDestroyPipelineCache and vkMergePipelineCaches itself, from vk_draw.cpp
before vk_work.h.

worker_caches::Set and made_alone now take a Calls struct with the
three operations; vk_draw.cpp defines worker_cache_calls after
vk_work.h, so a set's cache destruction goes through the recorder's
boundary like every other vkDestroy* in the file. Behaviour is
otherwise the same: the same caches, made, merged and destroyed in the
same order.

tests/test_worker_caches.py and tests/pipeline_play_cache_test.cpp take
worker_cache_calls from the source and the model respectively.
test_recorder_sync now fails only as on the base (vk_vma.cpp's
vmaDestroyAllocator), and the four tests that touch the caches pass.

Assisted-by: Claude Opus 5.5
Co-Authored-By: Claude Opus 5.5 <[email protected]>
(cherry picked from commit aa8ecba546465f1ce65ac872ecf21d232b65cef0)

fix(macos): MoltenVK starts with an empty pipeline cache and writes none f59051783e

MoltenVK 1.4.2 keeps MSL, not GPU code, in a VkPipelineCache, and
vkCreatePipelineCache with saved data compiles every stored library one
after another on the calling thread (MVKPipeline.mm readData).
pcache_init hands it all of mkw_vk_pipelines.bin, and play_files_merge
every play file, at every start. The PrepFast GPU check on the M4 Max,
with Metal's per-user cache blocked as in the runner, ran three later
starts of the same 64 pipelines (80 libraries):
- reading the one file: 5.7 s of serial compiles, 6.6 s in all;
- 16 shards read in parallel: 1.4 s;
- nothing read, pipelines compiled from SPIR-V with the worker caches:
  1.4 s.
At the game's scale the read is tens of minutes per start.

vk_vendor.h gets pcache_no_load, on for VK_DRIVER_ID_MOLTENVK only and
logged in the vendor fixes' line with worker_caches. It is the device's
default for RX_VK_PCACHE_LOAD: pcache_load_off reads the switch first
(any value, as before) and falls back to the fix, so
RX_VK_PCACHE_LOAD=1 reads the files on MoltenVK as everywhere else. With
the fix on and the switch unset or off, pcache_files_unused holds:
- pcache_init reads neither the main file nor the play files and says
  so in the log; the cache lives in memory for the run;
- nothing is saved (pcache_no_save, as RX_VK_PCACHE_LOAD=0 already
  did), and play_cache_save writes no play file and keeps the
  generation in memory: unread play files would pile up, since they
  are compacted only where they are read;
- the stamp is left alone (pcache_drop_stale returns as for any
  start that does not read the files);
- precompile_run drops RX_VK_LOAD=create's longshot_pipelines.bin.built
  marker at each start, as it does when the files were dropped: no
  cache a later start reads holds the notes, so the warm builds them
  into the in-memory cache again before loads at create ask for them.

Saving does not continue on MoltenVK because nothing reads the file or
its stamp there:
- the prepared record (prepared_for) has its own file and stamp;
- --prepare-shaders' saved cache is useful only to a start that reads
  it;
- pcache_saved_now serves graphics pipeline libraries, which MoltenVK
  does not have;
- a start with RX_VK_PCACHE_LOAD=1 makes, saves and stamps its own.
Every other driver is unchanged, RX_VK_PCACHE_LOAD=0 there included
(main file unread and unsaved, play files still merged and written).

tests/test_vendor_fixes.py checks the two cache fixes on its 16 devices
and on MoltenVK: on for MoltenVK alone, RX_VK_WORKER_CACHES forcing
the first, vendor_fixes_env leaving pcache_no_load to the files'
switch. It also checks the log line and pcache_load_off's order.
tests/test_pcache_stamp.py runs pcache_load_off, pcache_files_unused,
pcache_drop_stale and pcache_init, cut from the source, in eight starts:
other drivers unset, 0, empty and 1 as before, and MoltenVK unset, 0,
off and 1. test_pipeline_play_cache gets an "unused" case. All three
fail on aa8ecba5. tests/test_warm_bounded.py's source check of the
marker's removal takes the new condition.

Assisted-by: Claude Opus 5.5
Co-Authored-By: Claude Opus 5.5 <[email protected]>
(cherry picked from commit 61bfbf1ae7c0966a9436ab0c556a9ee547bb3b5f)

fix(vk, macos): the renderer's threads start with a 16 MiB stack, not macOS's 512 KiB f0a62f7ba0

macOS starts a std::thread with 512 KiB of stack, where Linux starts one
with 8 MiB and Windows with longshot.exe's 8 MiB. On macOS MoltenVK
converts each shader with SPIRV-Cross on the thread that creates the
pipeline, recursing once per block of a program's control flow, and a
prepare_run worker overran its 512 KiB there (CFG::walk_from under
vkCreateComputePipelines from seed_compute, start #6, 2026-11-14).

gpu/vk/thread_stack.h: thread_stack::thread is std::thread itself off
macOS; on macOS a std::thread-like thread (construction with decayed
arguments, joinable, join, detach, native_handle, moves) started by
pthread_create with thread_stack::kBytes (16 MiB) of stack. Every
thread the renderer starts goes through it: the preparation's pools
(prepare_run, state_list_prepare, warm_libraries, precompile_at_create,
precompile_run, predict_run, warm_bounded::run), the job workers and
the serial builder, the replay, parallel-replay, presenter and
frame-generation threads, the program cache's loaders and the rest.
mkwvk_init logs the size once per start on macOS.

tests/test_thread_stack.py: no file under gpu/vk starts a thread any
other way; each kind of thread runs a recursion 2 MiB deep through the
helper, and on macOS each overflows as a std::thread (caught, no crash
report). Seven source checks follow the new spelling.

Assisted-by: Claude Opus 5.5
Co-Authored-By: Claude Opus 5.5 <[email protected]>
(cherry picked from commit 48b49054b1c64851b3e8c974e64dc0b757b2ced3)

test(macos): one overflow message from thread_stack_test, however many threads overflow at once ea4081d942

The first thread whose stack overflows says so and ends the process;
the others wait in their handlers, so the message is not interleaved.

Assisted-by: Claude Opus 5.5
Co-Authored-By: Claude Opus 5.5 <[email protected]>
(cherry picked from commit 62d0991e0cfb5402202f8f14e9eb799d5c20d262)

test: the checks that compile renderer code with its threads see thread_stack.h

CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Has been skipped

Details

CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Has been skipped

Details

CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Has been skipped

Details

Third-party parts / The same as upstream (pull_request) Has been skipped

Details

CI / Conventional Commit title (pull_request) Successful in 3s

Details

CI / Workflow lint (actionlint) (pull_request) Successful in 6s

Details

Third-party parts / As their notes say (pull_request) Successful in 9s

Details

PR description / Description follows the template (pull_request_target) Successful in 4s

Details

7707c63433

Eight checks cut renderer functions that start threads (or copy
vk_jobs.cpp and vk_recorder.h) into fixtures of their own; they now
build with gpu/vk/thread_stack.h beside them: test_capture_copy_helpers,
test_gpu_hang_watch, test_jobs_failure, test_jobs_wait_runs_queued,
test_pipeline_cache_save, test_prepare_stall, test_present_source_crop
and test_recorder_sync.

Assisted-by: Claude Opus 5.5
Co-Authored-By: Claude Opus 5.5 <[email protected]>
(cherry picked from commit fd46ff431f9f62b1125582a4a3434a8cc92c551e)

fix(macos): the worker caches follow main's fetch lock (#190 ) and Intel's gate (#252 )

CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Has been skipped

Details

CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Has been skipped

Details

CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Has been skipped

Details

Third-party parts / The same as upstream (pull_request) Has been skipped

Details

CI / Conventional Commit title (pull_request) Successful in 3s

Details

CI / Workflow lint (actionlint) (pull_request) Successful in 5s

Details

Third-party parts / As their notes say (pull_request) Successful in 8s

Details

PR description / Description follows the template (pull_request_target) Successful in 4s

Details

078c9a7a54

Main's #190 (f41a5c59) put a fetch lock (PcacheFetch, PcacheUse) on the pipeline cache's save and compile path, the
very path where the worker caches add the lock their merges need, and #252 (57f4eaec) put Intel's compiler gate
(IntelDriverGate) around the compiles. The worker caches now sit on top of both:

- pcache_mu, the merge lock, is taken only where the worker caches are on (pcache_use_held: MoltenVK by default).
  Elsewhere create_cached, the save's fetch and pcache_saved_now take exactly #190's locks, as on main: no new lock
  on any other driver. A play generation's lock is likewise taken only where play's compiles are merged into it
  (g->alone).
- The worker caches are never on with the fetch lock (worker_caches_on: not with pcache_fetch_lock_on, Intel's
  Windows driver or RX_VK_PCACHE_FETCH_LOCK=1): a merge reads its caches' data as a fetch does, which that lock keeps
  apart from the builds there, and the gate lets one compile at a time into Intel's compiler there anyway.
- The order of the locks: Intel's gate, then pcache_mu, then the fetch lock. The save's fetch and pcache_saved_now
  take pcache_mu before PcacheFetch; a merge takes pcache_mu alone.
- prepare_run's compute warm keeps #252's gate around its compile and, with the caches, goes through create_cached
  under it (cached_compute would take the gate a second time).
- tests: test_worker_caches checks the gating (with the caches off a pipeline is made while pcache_mu or the
  generation's lock is held alone elsewhere; with them on it waits for pcache_mu), worker_caches_on against the
  fetch lock, the warm's gate and pcache_mu ahead of PcacheFetch in the save. test_pipeline_play_cache's route case
  checks that no lock of theirs is taken off MoltenVK; pipeline_play_cache_test.cpp and pipeline_cache_save_test.cpp
  get pcache_use_held stand-ins. test_pcache_stamp's load fixture follows #321's cache_path and #190's PcacheFetch in
  pcache_init, and play_cache_save's new parameters.
- docs: #321's cache/longshot_vk_pipelines.bin in the RX_VK_WORKER_CACHES row and in vk_vendor.h, and a row for
  RX_VK_PCACHE_LOAD, whose default this PR changes on MoltenVK.

Assisted-by: Claude Opus 5.5
Co-Authored-By: Claude Opus 5.5 <[email protected]>

merge(macos): the renderer threads' 16 MiB stacks (mac/thread-stack 7707c634) under the precompile hand-off f9f166f7cb

The precompile hand-off starts its workers and its compile watcher through thread_stack::thread. One conflict file,
gpu/vk/vk_draw.cpp (4 blocks: the lines P31 had to resolve against main, the other way round): the include block keeps prepare_workers.h,
worker_caches.h and thread_stack.h; prepare_run's, precompile_at_create's and precompile_run's thread lines take
thread_stack::thread and keep the worker caches' merges beside them.

Assisted-by: Claude Opus 5.5
Co-Authored-By: Claude Opus 5.5 <[email protected]>

fix(vk): MoltenVK: the precompile goes on behind the game; slow compiles named; a failed compile tried once more 89606310bb

On the Mac's start 12 every precompile worker was held by a Metal compile of minutes while 37,600 of
42,234 items were still to hand out, so RX_VK_PRECOMPILE_STALL_S could not apply (it waits until every
item is handed out) and the title never came. With the new vendor fix compiler_service (MoltenVK only,
RX_VK_COMPILER_SERVICE):
- the precompile hands the rest to the game once nothing has finished for the stall time, or once all
  that is left is compiles running past RX_VK_SLOW_COMPILE_S; a quarter of the workers build the rest
  behind the game, every item still built, each straggler named when it ends;
- its order: noted pipelines first, then fallbacks and computes by SPIR-V words, what an earlier run
  found slow (longshot_pipelines.bin.slow) last;
- a compile over the slow time (10 s) is named with key, variant, kind, stage and SPIR-V words while it
  runs and when it ends, and written to the slow list;
- a pipeline whose compile failed is compiled once more with a new pipeline cache (MoltenVK keeps a
  failed library in the cache it was compiled with), and said made or not made.
Every other driver: the classic worker, wait and log line as they were.

Assisted-by: Claude Opus 5.5
Co-Authored-By: Claude Opus 5.5 <[email protected]>
(cherry picked from commit 77d4971cbdb6356d9064a44cdee923f4bde354f9)

test(vk): test_precompile_handoff: a tree without the change says each missing piece 306ffd28ca

Assisted-by: Claude Opus 5.5

Co-Authored-By: Claude Opus 5.5 <[email protected]>
(cherry picked from commit f9a179c08d3f9e102729ec8fe5fe923336f581b8)

docs: RX_VK_PRECOMPILE_STALL_S, RX_VK_COMPILER_SERVICE and RX_VK_SLOW_COMPILE_S 44f210178c

Assisted-by: Claude Opus 5.5

Co-Authored-By: Claude Opus 5.5 <[email protected]>
(cherry picked from commit 194a230dd959706797203501edad3b5a2331118a)

fix(vk): name a graphics compile at its precompile callers, not inside build_graphics ab2bf44680

test_fallback_pipeline_state compiles build_graphics cut from the source; the noted pipelines of behind_worker and precompile_warm name what they build instead.

Assisted-by: Claude Opus 5.5

Co-Authored-By: Claude Opus 5.5 <[email protected]>
(cherry picked from commit 528784dc5c0c293da1b692f6fd71da070a940d8b)

fix(vk): the precompile hand-off on main's Intel gate and quick builds (#252 ), fetch lock (#190 ) and game folder (#321 )

CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Has been skipped

Details

CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Has been skipped

Details

CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Has been skipped

Details

Third-party parts / The same as upstream (pull_request) Has been skipped

Details

CI / Conventional Commit title (pull_request) Successful in 3s

Details

CI / Workflow lint (actionlint) (pull_request) Successful in 5s

Details

Third-party parts / As their notes say (pull_request) Successful in 9s

Details

PR description / Description follows the template (pull_request_target) Successful in 5s

Details

7ef7cff977

The hand-off was written on the macOS integration branch, before main's #190 (f41a5c59), #252 (57f4eaec) and #321
(76699aad), and before the worker caches followed them (this branch's base). What it needs on main's code:

- cached_graphics and cached_compute keep #252's gate and main's `return create_cached(guarded_*_held, ci, pl);`, and go
  through create_watched only where compiles are watched (compiles_watched: compiler_service, MoltenVK's, or
  RX_VK_SLOW_COMPILE_S set). Every other driver makes its pipelines exactly as on main, and main's own source checks
  (test_intel_compiler_gate, test_pcache_fetch_lock) read the lines they read there.
- prepare_run's compute warm: with the worker caches and watched compiles through create_watched, under the gate it holds;
  otherwise P22's create_cached or main's guarded_compute_held, as they are.
- behind_worker, the precompile's worker under compiler_service, builds a compute pipeline as the classic worker now
  does (#252): on Intel's one-at-a-time builds (serial_builds_on) without the optimizer first, the optimized build owed
  to its first dispatch (precompile_compute_quick, optimize_owed). On MoltenVK, where compiler_service is on, that never
  applies unless RX_VK_COMPUTE_SERIAL=1; it keeps the two workers' builds the same.
- test_precompile_handoff's source checks follow main's spellings: the watched and plain lines in cached_* after the gate,
  compiles_watched, the warm, the dispatch's inline build named as main spells it (p, not the integration branch's plain),
  and behind_worker's quick build.
- docs: the slow list is cache/longshot_pipelines.bin.slow since #321 moved the notes into the game folder's cache/.

Assisted-by: Claude Opus 5.5
Co-Authored-By: Claude Opus 5.5 <[email protected]>

Rows
Columns

WIP: fix(macos): let MoltenVK's precompile go on behind the title and name slow compiles #652

Why

How to reproduce

What changed

Measurements

Flags and settings

Testing

Risks

Reviewer notes

Issues

Checklist

Everyone

When it applies

Pull request closed