WIP: feat(macos): RX_VK_METAL_ARCHIVE_DIR keeps MoltenVK's pipelines in Metal binary archives #667

Closed

mooki wants to merge 41 commits from mooki/Longshot:mac/metal-cache2 into main

pull from: mooki/Longshot:mac/metal-cache2

merge into: MagicBeanDev:main

MagicBeanDev:main

Conversation 1 Commits 41 Files changed 72 +9989 -138

mooki commented 2026-11-15 02:55:17 +00:00

Member

• Copy link
• Copy Markdown

Type: feat · Risk: low · Tested on: macOS 26.7.1, Apple M4 Max (MoltenVK 1.4.2), arm64 and x86_64 under Rosetta · Needs: macOS, Apple silicon (two starts with the switch on the rebased stack); Windows and Linux, any GPU (build, test and start)

Why

On a Mac, every start of the game compiles tens of thousands of Metal pipelines again, and the slowest of them (the large
compute programs, minutes each) on every start. Metal's own shader cache cannot keep them: each of its two files holds at
most 1 GiB, and once a start has touched 1 GiB it stores nothing more (read from macOS 26.7.1's Metal and libCoreFSCache;
runs #16-#18 filled both files in the first minute of the precompile). MoltenVK 1.4.2 has no way to keep compiled pipelines
itself (no MTLBinaryArchive use, no VK_KHR_pipeline_binary; MoltenVK issue 1765 is open since 2022). This PR adds an
opt-in switch that keeps MoltenVK's pipelines in Metal binary archives, files the game owns with no size limit, so that a
later start makes them without compiling, and saves those archives so that they cost the start nothing it can feel: once
each, off the compile threads, never while a giant compile or the game's play needs the machine, with Metal's staging removed.
It is part of the macOS port. See #420.

What changed

RX_VK_METAL_ARCHIVE_DIR=<folder> (macOS only, off by default): after the device is made, the renderer puts a hook under
the two calls MoltenVK 1.4.2 makes every pipeline with. Each pipeline is looked up in the archives earlier starts saved
(no compile), or, until the game shows its first frame, added to a run archive of this start (its one compile) that is saved
for the next. Anything that fails makes MoltenVK's own call unchanged, so no pipeline is lost. Unset or empty, nothing is
loaded, replaced or logged. All of it is inside #if defined(__APPLE__).

• the hook: gpu/vk/metal_archive.h (new). begin() replaces, on the class of Metal's device, MoltenVK's
  newComputePipelineStateWithDescriptor:options:completionHandler: and
  newRenderPipelineStateWithDescriptor:completionHandler: (MVKPipeline.mm). MoltenVK calls them inside
  @synchronized(device), so the replacement only copies the descriptor and returns; a thread_stack::thread of its own
  (at its requester's QoS class) then (1) makes the pipeline from earlier starts' archives with
  MTLPipelineOptionFailOnBinaryArchiveMiss, (2) otherwise, before the first frame, adds it to a run archive no other thread
  holds meanwhile and makes it from there, (3) otherwise makes MoltenVK's call with the untouched descriptor. MoltenVK's
  completion handler is called exactly once on every path. After 4 add-path failures with none added the start stops adding.
  Archives live in <folder>/<macOS build>-<GPU name>/ (an archive serves one pair). Render pipelines without a fragment
  function go to archives of their own. Everything is looked up with dlopen/dlsym (libobjc's C functions, the blocks
  ABI): no new link, no Objective-C source.
• the saving (467b90d4, from run #21's costs): a run archive takes pipelines until it is estimated at 64 MiB, then is saved
  once, by one saver thread at utility QoS, and let go; at most 16 take pipelines at once, and those not saved yet hold 1 GiB
  at most. The saver waits while a giant compile is in flight (compile_board().giants(), #666) or the game is in
  play, unless the archives not saved yet hold 512 MiB or more; the ones still open are saved at the close. A failed save is
  logged with Metal's reason, tried once more 5 s later and then let go. After each save the archive's own gpuarchiver-*
  staging folder (named by getArchiveIDWithError:) is removed. The close says the time spent adding, making, looking up and
  saving, and the staging removed.
• the renderer: gpu/vk/vk_main.cpp: mkwvk_init() calls metal_archive::begin(…, vkx::metal_archive_gate) right after the
  Metal cache check (after create_device, before any pipeline); the gate reads the compile board and whether the game has
  presented; mkwvk_close_asked() saves this start's archives after pipeline_cache_close(). The archives' state is made with
  new and never destroyed (95ce0915): a pipeline's job may still run while the process exits. gpu/vk/vk_draw.cpp:
  compile_board() moves out of the anonymous namespace, so vk_main.cpp can name it (0640d405).
• docs: docs/development.md, the RX_VK_METAL_ARCHIVE_DIR row.
• tests: tests/test_metal_archive.py runs the real header against made-up Metal calls on any OS (5ce09630: no -pthread
  on Windows), and on macOS also its Objective-C binding against made-up classes registered with libobjc (no Metal, no GPU).

Not kept by archives: the libraries (MSL to AIR) MoltenVK compiles before each pipeline call stay in Metal's library cache,
with its 1 GiB.

Measurements

Verdict: the mechanism works, and the saving rules this PR ships fix what its first version cost in the game; a warm game
start from archives is not measured yet, so the switch stays off. In the game-free probes every pipeline came back from the
archive in the second process: MC-01 (37 of 37), whose second process took 41% (compute), 52% (render) and 16% (large compute)
of the first's time per pipeline, and MC-02 (61 of 61, with this PR's saving: 3 run archives saved once each, 0 failed saves,
vertex-only render pipelines packed, 3 staging folders removed and 0 left, the saver waiting while a giant compiled). The add
path costs little (MC-02, archive over control per median: compute 1.003, render 1.144, depth-only 1.248, rasterizer-discard
1.174, large 0.985). The game's only start with archives on (run #21) used the first version's saving (81085744-5ce09630),
which cost about 91 s of graphics precompile, left 12 GB of Metal's staging and stopped the runner from carrying anything;
467b90d4 is the fix for each of those, and is in this PR for that reason. Run #21's lost device was not the archives: run
#22 lost it the same way without them.

Baseline: main does not start on macOS. The game columns are the same integration build 09f9a171 (which carries the first
version), arm64, MoltenVK 1.4.2, the field-dawn save, once without and once with the switch (runs #22 and #21); the probes
compare the switch off and on in standalone MoltenVK processes that call the renderer's own headers (MC-01 the first version's,
MC-02 this head's metal_archive.h).

Hardware: macOS 26.7.1 (25G241), Apple M4 Max (40-core GPU), MoltenVK 1.4.2, 16-core CPU, 64 GB
Scene: runs #21/#22: preparation and precompile from a seeded program cache, then the title and the field-dawn save's load,
windowless; the probes: 24 compute, 12 render and 1 large compute pipeline per process (MC-02 also 12 depth-only, 12 rasterizer
discard and a 2,200-pipeline route), distinct SPIR-V, no VkPipelineCache, a fresh empty Metal cache folder per process
Method: runs #21/#22: the precompile's time line and statistics line, the run's Metal folder and the runner's receipt, one run
each; probes MC-01 (reports/MetalCache2-files/probe/runs-01, run.json f017ef2d…, 2026-11-14 16:47 UTC) and MC-02
(reports/ArchiveFix-files/probe/runs-01, run.json 60572c7e…, 2026-11-14 19:29 UTC): processes A then B per route,
per-pipeline medians from each process's SUMMARY line
How to reproduce: the probes: reports/2026-11-14-metal-cache2.md §4 and reports/2026-11-14-archive-fix.md §6 (the exact
driver.py run commands); in the game, set RX_VK_METAL_ARCHIVE_DIR to an existing folder (with RX_VK_METAL_CACHE_DIR set
too, as the runner does) and read the vk: Metal binary archives lines

	Before: integration @ 09f9a171 (run #22, switch off)	After, RX_VK_METAL_ARCHIVE_DIR set (the first version's saving): integration @ 09f9a171 (run #21)
graphics precompile (30,330 pipelines)	about 50 s; summed compile 750,887 ms	about 141 s; summed compile 2,120,881 ms
compute precompile	about 368 s	about 553 s
archives written	none	42 files, 2,681,969,088 B (3 saves failed: "expecting 'fragment' stage in pipeline no. 62/76/65")
Metal's gpuarchiver-* staging left in the Metal cache folder	none	1,042 folders, 11,995,119,780 B
caches carried to the next start	published	refused ("Metal cache holds something other than its measured files")

MC-02 (this head's header)	archive: A (empty folder)	archive: B (A's folder)
close line	0 from the archives, 61 compiled and added; 3 run archives saved once each (3 MB), 0 saves failed	61 from the archives, 0 compiled and added
time line	4.4 s adding, 0.1 s saving; 3 staging folders removed (4 MB), 0 not named by Metal	0.0 s in all

MC-01 archive B: vk: Metal binary archives: 37 pipelines from the archives, 0 compiled and added, 0 compiled without them, 0 as MoltenVK asked; 0 saves of 0 run archives (0 failed) in …/archive-b/25G241-Apple_M4_Max; 0 still compiling at the close
MC-02 archive A: vk: Metal binary archives: 0 pipelines from the archives, 61 compiled and added, 0 compiled without them (0 past the bounds, 0 after a failure, 0 missed in play), 0 in play with nothing to look up, 0 as MoltenVK asked; 3 run archives saved once each (3 MB), 0 saves failed, 0 archives let go unsaved (0 pipelines) in …
MC-02 archive A: vk: Metal binary archives: time in all: 4.4 s adding, 0.0 s making from the run archives, 0.0 s looking up, 0.1 s saving; Metal's staging: 3 folders removed after the saves (4 MB), 0 not named by Metal
MC-02 archive B: vk: Metal binary archives: 61 pipelines from the archives, 0 compiled and added, 0 compiled without them (0 past the bounds, 0 after a failure, 0 missed in play), 0 in play with nothing to look up, 0 as MoltenVK asked; 0 run archives saved once each (0 MB), …
#21: graphics part about 141 s (30,970 at 141 s); 30330 pipelines (2120881 ms); 1,042 gpuarchiver-* folders, 139,350 files, 11,995,119,780 B

Reports: reports/2026-11-14-metal-cache2.md (Metal's cache limit, the design, MC-01), -archive-fix.md (run #21's costs,
where the staging comes from, the fixes, MC-02), -mac-startup-21.md, -mac-startup-22.md, -device-lost-21.md (the archive
hook ranked at 5-10% as #21's cause: the archiver was idle in the critical window).

Flags and settings

One new switch, macOS only, off by default.

Flag	What it is	Before this PR	After this PR	Effect	Helps / doesn't help / untested	Recommendation
RX_VK_METAL_ARCHIVE_DIR=<folder>	the folder MoltenVK's pipelines are kept in as Metal binary archives (an existing absolute path)	no switch: every start compiles what Metal's 1 GiB cache dropped	unset: unchanged; set: pipelines looked up in archives there, and added to run archives until the first frame, or a NOT applied line and unchanged	a later start makes archived pipelines without a compile	helps: the pipeline (back-end) compile, 48-84% of a cold compile in MC-01; doesn't help: the library (front-end) compile; untested: a warm game start from archives with this head's saving	off; for testing until a game start from archives is measured

Testing

Tested: on macOS 26.7.1, Apple M4 Max, at this head (main 6c9edffe plus #438, #653, #650, #651, #652, #664,
#665, #666 and this PR):

• tests/test_metal_archive.py with /usr/bin/clang++ and with clang++ -arch x86_64 (the fixture under Rosetta): pass on both;
  #653's test_metal_cache_dir (both), #651's test_thread_stack (arm64 and x86_64 in one run) and test_loading_stall
  (arm64) pass too (reports/PubWave-files/PubLate1/final-proof-03/, at 31fe0f1b, the same code; test_metal_archive again at
  this head, final-proof-04/mc2-*). At the first version's last commit (5ce09630) test_metal_archive passes as well
  (final-proof-03/mc2-mid-archive-arm64).
• Negatives (the test must fail): MetalCache2's negatives.py on the first version (base: the branch before the archives, and 7
  mutations that each break one rule: begin before the device, no close, the fallback given the copy, a lookup that may compile,
  the handler kept, never giving up, no save at the close): all 8 fail (mc2-neg-mid-487977a9/); ArchiveFix's negatives.py on
  this head (base: the first version; its header with the new wiring; 15 mutations that each put one of run #21's defects back):
  all 17 fail (mc2-neg-head-829fc066/). They ran on the branch as first built; the files they read (vk_main.cpp,
  metal_archive.h, thread_stack.h, docs/development.md, the test) are byte-identical at this head (blob ids checked).
• Every check that reads a renderer file changed by the branch, and the new ones (304 names), through tools/ls-suite.py at
  31fe0f1b: 284 pass, 20 fail: the same 19 as at #652's head (16 also fail on main 6c9edffe on macOS, 3 only because named),
  and main's test_intel_quick_compute, fixed in #666's 3cd25d0e and passing at this head
  (metal-cache2-related-02/, -compare*.txt; final-proof-04/mc2-iqc).
• gpu/vk/vk_main.cpp and gpu/vk/vk_draw.cpp compiled whole for arm64 and x86_64 with main's renderer flags (with #476's and
  #492's macOS compile pieces in a scratch tree): 4 objects made, exit 0 (reports/PubWave-files/PubLate1/compile/metal-cache2/,
  at 31fe0f1b; both files are byte-identical at this head).
• Off Apple (unifdef -U__APPLE__, base against head): vk_main.cpp the same; vk_draw.cpp differs only by compile_board()
  leaving the anonymous namespace; metal_archive.h is new and not included there
  (reports/PubWave-files/PubLate1/offapple-metal-cache2.txt).

Not tested: a game start with this head's saving, and a warm game start that makes its pipelines from archives (run #21's
archives were never carried: the runner refused publication); a Mac with two GPUs or Metal API validation on (the hook would
be on another class: then no request arrives and the close line counts zeros); macOS versions other than 26.7.1; Windows and
Linux (no behaviour changes there: the header is included only under __APPLE__, and compile_board() only changes linkage;
test_metal_archive.py runs there with made-up Metal): not built on Windows or Linux; its test request covers it.

Must be tested:

• macOS, Apple silicon, on the rebased macOS stack: two starts with RX_VK_METAL_ARCHIVE_DIR set to one folder, the second
  from the first's, expecting the close lines with saves once each, no gpuarchiver-* left, and in the second most pipelines
  from the archives and a shorter precompile (by hand: the parent's GPU slot).
• Windows and Linux, any GPU: build this PR (head 0640d405, it carries #653, #438, #650, #651, #652, #664,
  #665 and #666) and main 6c9edffe the same way, run tests/test_metal_archive.py and start each to the
  title, expecting the test to pass and no vk: Metal binary archives line, and every other line to match main's (by hand: test
  request T-XXXX).

Risks

What could break: only starts that set the switch on macOS. With it set, a start adds its pipelines to run archives until the
first frame and saves them off the compile threads; a save that fails twice is let go (its pipelines are compiled again next
time); a start that dies before its close leaves the staging of the archives still open. A Metal or MoltenVK update that changes
the two calls makes the hook see no request (the close line counts zeros) or logs NOT applied; every failure path makes
MoltenVK's own call.

Turning it off: leave RX_VK_METAL_ARCHIVE_DIR unset (the default), or revert the PR.

Compatibility: Nothing changes for players: the switch is off by default and Apple-only. vk_main.cpp, vk_draw.cpp's
compile_board() and metal_archive.h are not hashed into the shader stamp, so no cache is translated again; no
MKWVK_API_VERSION change.

Reviewer notes

• Review 81085744..0640d405 (5 commits; in git's notation 5f7dfa90..0640d405): the hook as first committed (81085744,
  95ce0915, 5ce09630), then its saving fixed for run #21's costs (467b90d4) and compile_board()'s linkage (0640d405).
  The first three alone would ship run #21's costs, so they open together with the fix (PubWave's decision).
• Depends on #653 (metal_cache::check, the switch's neighbour, and through it #438) and #666 (compile_board() and
  Board::giants(), which the saver waits on; and through it #665, #664, #652, #650 and #651, whose
  thread_stack.h the jobs start with). The branch is #653's head fc8ee75d with #666's head 3cd25d0e merged
  (5f7dfa90): one conflict, gpu/vk/vk_main.cpp's Apple block after create_device, resolved as metal_cache::check, then
  #651's stack line.
• The commits were cherry-picked with -x from macos/metal-cache2 (050961c0 426ffa55 f4cd09cb) and macos/archive-fix
  (ccebc6a6 54ec563e; 54ec563e is the integration branch's d6a89208), with no conflict: metal_archive::begin lands between
  metal_cache::check and the stack line, as the integration branch has it. Each pick's change equals its source commit's line
  for line.
• No semantic commit: the picks needed no change to follow main.
• Sibling macOS PRs: git merge-tree of this head: docs/development.md with #654 (rows next to each other: keep both);
  host/posix/win32_posix.c with #480 (inherited from #438, not this PR); gpu/vk/vk_draw.cpp with #658 at the top of
  dispatch (#666's); and #652's own (vk_main.cpp with #428 and #498; #646's four files)
  (reports/PubWave-files/PubLate1/merge-check-metal-cache2*.txt). With #669: no conflict, and the saver and that
  gate cannot wait on each other (Integration 30's probe: the gate defers a giant's start, never one in flight).
• Conflicts with main: none.

Issues

See #420

Checklist

Everyone

• One topic.
• The title is a Conventional Commit (type(scope): description) of the status line's type.
• Why gives the reason plainly and links the issues that fit (the tracker searched).
• Measurements has a verdict, or one line saying why there is nothing to measure.
• Testing names what was tested, what wasn't, and what must still be tested, where and how. (Test requests not open yet.)
• Rebased on current main (main e090cc48).
• python -I -B tests/run_all.py passes (any failure that also fails on main is named). (On the wave's merge bfe5c752 of this PR and giant-mem-gate: 544 passed, 32 failed, 17 skipped against main 6c9edffe's 531/32/17 on this Mac; the 32 failures are main's own, nothing that passes on main fails, 13 new tests pass; reports/PubWave-files/waveC1/.)
• No game files, dumps, build output or generated files.
• third_party/ untouched (a new folder with a PROVENANCE.md instead).
• The license is OK: GPL-3.0-or-later, nothing incompatible and nothing decompiled.

When it applies

• AI assistance recorded with Assisted-by: / Co-Authored-By: trailers.
• Switches documented (docs/development.md; the README for a menu row) and shipped at their recommended value.
• A game-free test added for the fix, failing on main and passing here.
• A check under tests/ that reads changed source updated in this PR, not weakened.
• Docs updated.

**Type:** feat · **Risk:** low · **Tested on:** macOS 26.7.1, Apple M4 Max (MoltenVK 1.4.2), arm64 and x86_64 under Rosetta · **Needs:** macOS, Apple silicon (two starts with the switch on the rebased stack); Windows and Linux, any GPU (build, test and start) ## Why On a Mac, every start of the game compiles tens of thousands of Metal pipelines again, and the slowest of them (the large compute programs, minutes each) on every start. Metal's own shader cache cannot keep them: each of its two files holds at most 1 GiB, and once a start has touched 1 GiB it stores nothing more (read from macOS 26.7.1's Metal and libCoreFSCache; runs #16-#18 filled both files in the first minute of the precompile). MoltenVK 1.4.2 has no way to keep compiled pipelines itself (no `MTLBinaryArchive` use, no `VK_KHR_pipeline_binary`; MoltenVK issue 1765 is open since 2022). This PR adds an opt-in switch that keeps MoltenVK's pipelines in Metal binary archives, files the game owns with no size limit, so that a later start makes them without compiling, and saves those archives so that they cost the start nothing it can feel: once each, off the compile threads, never while a giant compile or the game's play needs the machine, with Metal's staging removed. It is part of the macOS port. See #420. ## What changed `RX_VK_METAL_ARCHIVE_DIR=<folder>` (macOS only, off by default): after the device is made, the renderer puts a hook under the two calls MoltenVK 1.4.2 makes every pipeline with. Each pipeline is looked up in the archives earlier starts saved (no compile), or, until the game shows its first frame, added to a run archive of this start (its one compile) that is saved for the next. Anything that fails makes MoltenVK's own call unchanged, so no pipeline is lost. Unset or empty, nothing is loaded, replaced or logged. All of it is inside `#if defined(__APPLE__)`. - the hook: `gpu/vk/metal_archive.h` (new). `begin()` replaces, on the class of Metal's device, MoltenVK's `newComputePipelineStateWithDescriptor:options:completionHandler:` and `newRenderPipelineStateWithDescriptor:completionHandler:` (`MVKPipeline.mm`). MoltenVK calls them inside `@synchronized(device)`, so the replacement only copies the descriptor and returns; a `thread_stack::thread` of its own (at its requester's QoS class) then (1) makes the pipeline from earlier starts' archives with `MTLPipelineOptionFailOnBinaryArchiveMiss`, (2) otherwise, before the first frame, adds it to a run archive no other thread holds meanwhile and makes it from there, (3) otherwise makes MoltenVK's call with the untouched descriptor. MoltenVK's completion handler is called exactly once on every path. After 4 add-path failures with none added the start stops adding. Archives live in `<folder>/<macOS build>-<GPU name>/` (an archive serves one pair). Render pipelines without a fragment function go to archives of their own. Everything is looked up with `dlopen`/`dlsym` (libobjc's C functions, the blocks ABI): no new link, no Objective-C source. - the saving (`467b90d4`, from run #21's costs): a run archive takes pipelines until it is estimated at 64 MiB, then is saved once, by one saver thread at utility QoS, and let go; at most 16 take pipelines at once, and those not saved yet hold 1 GiB at most. The saver waits while a giant compile is in flight (`compile_board().giants()`, #666) or the game is in play, unless the archives not saved yet hold 512 MiB or more; the ones still open are saved at the close. A failed save is logged with Metal's reason, tried once more 5 s later and then let go. After each save the archive's own `gpuarchiver-*` staging folder (named by `getArchiveIDWithError:`) is removed. The close says the time spent adding, making, looking up and saving, and the staging removed. - the renderer: `gpu/vk/vk_main.cpp`: `mkwvk_init()` calls `metal_archive::begin(…, vkx::metal_archive_gate)` right after the Metal cache check (after `create_device`, before any pipeline); the gate reads the compile board and whether the game has presented; `mkwvk_close_asked()` saves this start's archives after `pipeline_cache_close()`. The archives' state is made with `new` and never destroyed (`95ce0915`): a pipeline's job may still run while the process exits. `gpu/vk/vk_draw.cpp`: `compile_board()` moves out of the anonymous namespace, so `vk_main.cpp` can name it (`0640d405`). - docs: `docs/development.md`, the `RX_VK_METAL_ARCHIVE_DIR` row. - tests: `tests/test_metal_archive.py` runs the real header against made-up Metal calls on any OS (`5ce09630`: no `-pthread` on Windows), and on macOS also its Objective-C binding against made-up classes registered with libobjc (no Metal, no GPU). Not kept by archives: the libraries (MSL to AIR) MoltenVK compiles before each pipeline call stay in Metal's library cache, with its 1 GiB. ## Measurements **Verdict:** the mechanism works, and the saving rules this PR ships fix what its first version cost in the game; a warm game start from archives is not measured yet, so the switch stays off. In the game-free probes every pipeline came back from the archive in the second process: MC-01 (37 of 37), whose second process took 41% (compute), 52% (render) and 16% (large compute) of the first's time per pipeline, and MC-02 (61 of 61, with this PR's saving: 3 run archives saved once each, 0 failed saves, vertex-only render pipelines packed, 3 staging folders removed and 0 left, the saver waiting while a giant compiled). The add path costs little (MC-02, archive over control per median: compute 1.003, render 1.144, depth-only 1.248, rasterizer-discard 1.174, large 0.985). The game's only start with archives on (run #21) used the first version's saving (`81085744`-`5ce09630`), which cost about 91 s of graphics precompile, left 12 GB of Metal's staging and stopped the runner from carrying anything; `467b90d4` is the fix for each of those, and is in this PR for that reason. Run #21's lost device was not the archives: run #22 lost it the same way without them. Baseline: main does not start on macOS. The game columns are the same integration build `09f9a171` (which carries the first version), arm64, MoltenVK 1.4.2, the field-dawn save, once without and once with the switch (runs #22 and #21); the probes compare the switch off and on in standalone MoltenVK processes that call the renderer's own headers (MC-01 the first version's, MC-02 this head's `metal_archive.h`). Hardware: macOS 26.7.1 (25G241), Apple M4 Max (40-core GPU), MoltenVK 1.4.2, 16-core CPU, 64 GB Scene: runs #21/#22: preparation and precompile from a seeded program cache, then the title and the field-dawn save's load, windowless; the probes: 24 compute, 12 render and 1 large compute pipeline per process (MC-02 also 12 depth-only, 12 rasterizer discard and a 2,200-pipeline route), distinct SPIR-V, no `VkPipelineCache`, a fresh empty Metal cache folder per process Method: runs #21/#22: the precompile's time line and statistics line, the run's Metal folder and the runner's receipt, one run each; probes MC-01 (`reports/MetalCache2-files/probe/runs-01`, `run.json` `f017ef2d…`, 2026-11-14 16:47 UTC) and MC-02 (`reports/ArchiveFix-files/probe/runs-01`, `run.json` `60572c7e…`, 2026-11-14 19:29 UTC): processes A then B per route, per-pipeline medians from each process's `SUMMARY` line How to reproduce: the probes: `reports/2026-11-14-metal-cache2.md` §4 and `reports/2026-11-14-archive-fix.md` §6 (the exact `driver.py run` commands); in the game, set `RX_VK_METAL_ARCHIVE_DIR` to an existing folder (with `RX_VK_METAL_CACHE_DIR` set too, as the runner does) and read the `vk: Metal binary archives` lines | | Before: integration @ 09f9a171 (run #22, switch off) | After, RX_VK_METAL_ARCHIVE_DIR set (the first version's saving): integration @ 09f9a171 (run #21) | |---|---|---| | graphics precompile (30,330 pipelines) | about 50 s; summed compile 750,887 ms | about 141 s; summed compile 2,120,881 ms | | compute precompile | about 368 s | about 553 s | | archives written | none | 42 files, 2,681,969,088 B (3 saves failed: "expecting 'fragment' stage in pipeline no. 62/76/65") | | Metal's `gpuarchiver-*` staging left in the Metal cache folder | none | 1,042 folders, 11,995,119,780 B | | caches carried to the next start | published | refused ("Metal cache holds something other than its measured files") | | MC-02 (this head's header) | archive: A (empty folder) | archive: B (A's folder) | |---|---|---| | close line | 0 from the archives, 61 compiled and added; 3 run archives saved once each (3 MB), 0 saves failed | 61 from the archives, 0 compiled and added | | time line | 4.4 s adding, 0.1 s saving; 3 staging folders removed (4 MB), 0 not named by Metal | 0.0 s in all | ``` MC-01 archive B: vk: Metal binary archives: 37 pipelines from the archives, 0 compiled and added, 0 compiled without them, 0 as MoltenVK asked; 0 saves of 0 run archives (0 failed) in …/archive-b/25G241-Apple_M4_Max; 0 still compiling at the close MC-02 archive A: vk: Metal binary archives: 0 pipelines from the archives, 61 compiled and added, 0 compiled without them (0 past the bounds, 0 after a failure, 0 missed in play), 0 in play with nothing to look up, 0 as MoltenVK asked; 3 run archives saved once each (3 MB), 0 saves failed, 0 archives let go unsaved (0 pipelines) in … MC-02 archive A: vk: Metal binary archives: time in all: 4.4 s adding, 0.0 s making from the run archives, 0.0 s looking up, 0.1 s saving; Metal's staging: 3 folders removed after the saves (4 MB), 0 not named by Metal MC-02 archive B: vk: Metal binary archives: 61 pipelines from the archives, 0 compiled and added, 0 compiled without them (0 past the bounds, 0 after a failure, 0 missed in play), 0 in play with nothing to look up, 0 as MoltenVK asked; 0 run archives saved once each (0 MB), … #21: graphics part about 141 s (30,970 at 141 s); 30330 pipelines (2120881 ms); 1,042 gpuarchiver-* folders, 139,350 files, 11,995,119,780 B ``` Reports: `reports/2026-11-14-metal-cache2.md` (Metal's cache limit, the design, MC-01), `-archive-fix.md` (run #21's costs, where the staging comes from, the fixes, MC-02), `-mac-startup-21.md`, `-mac-startup-22.md`, `-device-lost-21.md` (the archive hook ranked at 5-10% as #21's cause: the archiver was idle in the critical window). ### Flags and settings One new switch, macOS only, off by default. | Flag | What it is | Before this PR | After this PR | Effect | Helps / doesn't help / untested | Recommendation | |---|---|---|---|---|---|---| | `RX_VK_METAL_ARCHIVE_DIR=<folder>` | the folder MoltenVK's pipelines are kept in as Metal binary archives (an existing absolute path) | no switch: every start compiles what Metal's 1 GiB cache dropped | unset: unchanged; set: pipelines looked up in archives there, and added to run archives until the first frame, or a `NOT applied` line and unchanged | a later start makes archived pipelines without a compile | helps: the pipeline (back-end) compile, 48-84% of a cold compile in MC-01; doesn't help: the library (front-end) compile; untested: a warm game start from archives with this head's saving | off; for testing until a game start from archives is measured | ## Testing **Tested:** on macOS 26.7.1, Apple M4 Max, at this head (main 6c9edffe plus #438, #653, #650, #651, #652, #664, #665, #666 and this PR): - `tests/test_metal_archive.py` with `/usr/bin/clang++` and with `clang++ -arch x86_64` (the fixture under Rosetta): pass on both; #653's `test_metal_cache_dir` (both), #651's `test_thread_stack` (arm64 and x86_64 in one run) and `test_loading_stall` (arm64) pass too (`reports/PubWave-files/PubLate1/final-proof-03/`, at `31fe0f1b`, the same code; `test_metal_archive` again at this head, `final-proof-04/mc2-*`). At the first version's last commit (`5ce09630`) `test_metal_archive` passes as well (`final-proof-03/mc2-mid-archive-arm64`). - Negatives (the test must fail): MetalCache2's `negatives.py` on the first version (base: the branch before the archives, and 7 mutations that each break one rule: begin before the device, no close, the fallback given the copy, a lookup that may compile, the handler kept, never giving up, no save at the close): all 8 fail (`mc2-neg-mid-487977a9/`); ArchiveFix's `negatives.py` on this head (base: the first version; its header with the new wiring; 15 mutations that each put one of run #21's defects back): all 17 fail (`mc2-neg-head-829fc066/`). They ran on the branch as first built; the files they read (`vk_main.cpp`, `metal_archive.h`, `thread_stack.h`, `docs/development.md`, the test) are byte-identical at this head (blob ids checked). - Every check that reads a renderer file changed by the branch, and the new ones (304 names), through `tools/ls-suite.py` at `31fe0f1b`: 284 pass, 20 fail: the same 19 as at #652's head (16 also fail on main 6c9edffe on macOS, 3 only because named), and main's `test_intel_quick_compute`, fixed in #666's `3cd25d0e` and passing at this head (`metal-cache2-related-02/`, `-compare*.txt`; `final-proof-04/mc2-iqc`). - `gpu/vk/vk_main.cpp` and `gpu/vk/vk_draw.cpp` compiled whole for arm64 and x86_64 with main's renderer flags (with #476's and #492's macOS compile pieces in a scratch tree): 4 objects made, exit 0 (`reports/PubWave-files/PubLate1/compile/metal-cache2/`, at `31fe0f1b`; both files are byte-identical at this head). - Off Apple (`unifdef -U__APPLE__`, base against head): `vk_main.cpp` the same; `vk_draw.cpp` differs only by `compile_board()` leaving the anonymous namespace; `metal_archive.h` is new and not included there (`reports/PubWave-files/PubLate1/offapple-metal-cache2.txt`). **Not tested:** a game start with this head's saving, and a warm game start that makes its pipelines from archives (run #21's archives were never carried: the runner refused publication); a Mac with two GPUs or Metal API validation on (the hook would be on another class: then no request arrives and the close line counts zeros); macOS versions other than 26.7.1; Windows and Linux (no behaviour changes there: the header is included only under `__APPLE__`, and `compile_board()` only changes linkage; `test_metal_archive.py` runs there with made-up Metal): not built on Windows or Linux; its test request covers it. **Must be tested:** - macOS, Apple silicon, on the rebased macOS stack: two starts with `RX_VK_METAL_ARCHIVE_DIR` set to one folder, the second from the first's, expecting the close lines with saves once each, no `gpuarchiver-*` left, and in the second most pipelines `from the archives` and a shorter precompile (by hand: the parent's GPU slot). - Windows and Linux, any GPU: build this PR (head `0640d405`, it carries #653, #438, #650, #651, #652, #664, #665 and #666) and main 6c9edffe the same way, run `tests/test_metal_archive.py` and start each to the title, expecting the test to pass and no `vk: Metal binary archives` line, and every other line to match main's (by hand: test request T-XXXX). ## Risks **What could break:** only starts that set the switch on macOS. With it set, a start adds its pipelines to run archives until the first frame and saves them off the compile threads; a save that fails twice is let go (its pipelines are compiled again next time); a start that dies before its close leaves the staging of the archives still open. A Metal or MoltenVK update that changes the two calls makes the hook see no request (the close line counts zeros) or logs `NOT applied`; every failure path makes MoltenVK's own call. **Turning it off:** leave `RX_VK_METAL_ARCHIVE_DIR` unset (the default), or revert the PR. **Compatibility:** Nothing changes for players: the switch is off by default and Apple-only. `vk_main.cpp`, `vk_draw.cpp`'s `compile_board()` and `metal_archive.h` are not hashed into the shader stamp, so no cache is translated again; no `MKWVK_API_VERSION` change. ## Reviewer notes - Review `81085744..0640d405` (5 commits; in git's notation `5f7dfa90..0640d405`): the hook as first committed (`81085744`, `95ce0915`, `5ce09630`), then its saving fixed for run #21's costs (`467b90d4`) and `compile_board()`'s linkage (`0640d405`). The first three alone would ship run #21's costs, so they open together with the fix (PubWave's decision). - Depends on #653 (`metal_cache::check`, the switch's neighbour, and through it #438) and #666 (`compile_board()` and `Board::giants()`, which the saver waits on; and through it #665, #664, #652, #650 and #651, whose `thread_stack.h` the jobs start with). The branch is #653's head `fc8ee75d` with #666's head `3cd25d0e` merged (`5f7dfa90`): one conflict, `gpu/vk/vk_main.cpp`'s Apple block after `create_device`, resolved as `metal_cache::check`, then #651's stack line. - The commits were cherry-picked with `-x` from `macos/metal-cache2` (`050961c0 426ffa55 f4cd09cb`) and `macos/archive-fix` (`ccebc6a6 54ec563e`; `54ec563e` is the integration branch's `d6a89208`), with no conflict: `metal_archive::begin` lands between `metal_cache::check` and the stack line, as the integration branch has it. Each pick's change equals its source commit's line for line. - No semantic commit: the picks needed no change to follow main. - Sibling macOS PRs: `git merge-tree` of this head: `docs/development.md` with #654 (rows next to each other: keep both); `host/posix/win32_posix.c` with #480 (inherited from #438, not this PR); `gpu/vk/vk_draw.cpp` with #658 at the top of `dispatch` (#666's); and #652's own (`vk_main.cpp` with #428 and #498; #646's four files) (`reports/PubWave-files/PubLate1/merge-check-metal-cache2*.txt`). With #669: no conflict, and the saver and that gate cannot wait on each other (Integration 30's probe: the gate defers a giant's start, never one in flight). - Conflicts with main: none. ## Issues See #420 ## Checklist ### Everyone - [x] One topic. - [x] The title is a Conventional Commit (`type(scope): description`) of the status line's type. - [x] Why gives the reason plainly and links the issues that fit (the tracker searched). - [x] Measurements has a verdict, or one line saying why there is nothing to measure. - [ ] Testing names what was tested, what wasn't, and what must still be tested, where and how. (Test requests not open yet.) - [x] Rebased on current `main` (main e090cc48). - [x] `python -I -B tests/run_all.py` passes (any failure that also fails on `main` is named). (On the wave's merge `bfe5c752` of this PR and giant-mem-gate: 544 passed, 32 failed, 17 skipped against main 6c9edffe's 531/32/17 on this Mac; the 32 failures are main's own, nothing that passes on main fails, 13 new tests pass; `reports/PubWave-files/waveC1/`.) - [x] No game files, dumps, build output or generated files. - [x] `third_party/` untouched (a new folder with a `PROVENANCE.md` instead). - [x] The license is OK: GPL-3.0-or-later, nothing incompatible and nothing decompiled. ### When it applies - [x] AI assistance recorded with `Assisted-by:` / `Co-Authored-By:` trailers. - [x] Switches documented (docs/development.md; the README for a menu row) and shipped at their recommended value. - [x] A game-free test added for the fix, failing on `main` and passing here. - [x] A check under `tests/` that reads changed source updated in this PR, not weakened. - [x] Docs updated.

mooki added 40 commits 2026-11-15 02:55:18 +00:00

feat(vk): add macOS Vulkan loader and device admission ... b73888eb6f

Select a local Vulkan loader or MoltenVK with RX_VK_LIBRARY and keep SDL and
renderer startup on that same library. Capture SDL's instance extensions on
the window thread, enable advertised portability extensions, and name missing
admission requirements and VkResults without changing Windows or Linux paths.

Keep the non-Apple host API at 21 for existing prebuilt renderers; append the
two macOS callbacks only in Apple's version 22. Cover pure decisions, legacy
platform expectations, ABI layout, and the SDL2/SDL3 bridge without a GPU.

Assisted-by: GPT-6 Astra
Co-Authored-By: GPT-6 Astra <[email protected]>
Assisted-by: Claude Opus 5.5
Co-Authored-By: Claude Opus 5.5 <[email protected]>

test(vk): use semantic admission contracts ... 1f4abf7d2d

Remove the SDL source-string wiring and source-index ordering assertions while
keeping the sixteen compiled behavioral cases and surface adapter compile.
Replace the comment-sensitive API header hash with fixed baseline field names,
byte offsets, versions and sizes, retaining typed Apple callback checks and
both platforms' actual admission decisions and negative controls.

Production code is unchanged. Prior full-suite and device receipts remain
bound to 818e33b1; only the affected game-free tests were rerun.

Assisted-by: GPT-6 Astra
Co-Authored-By: GPT-6 Astra <[email protected]>
Assisted-by: Claude Opus 5.5
Co-Authored-By: Claude Opus 5.5 <[email protected]>

test(vk): follow main's LoadLibraryA and float-add choice in the admission fixture ... 1ac24df96f

The fixture cuts LoadLibraryA and create_device out of main's source. Since
#34, LoadLibraryA keeps dlerror()'s text in load_error, and since #191
native_fadd_wanted also takes the device ID and driver version, so the cut
code no longer compiled against the fixture's stand-ins on this base. Give
it load_error and a dlerror stand-in, and the four-argument stand-in. No
expectation changed.

Assisted-by: Claude Opus 5.5
Co-Authored-By: Claude Opus 5.5 <[email protected]>

test(vk): pin the admission ABI at main's version 22 and macOS 23 ... 0f2262a378

Main's custom output size (#51) took MKWVK_API_VERSION 22 on every
platform and appended take_text at offset 352. The macOS loader
callbacks now follow it: the rebased header says 23 on Apple and 22
elsewhere. The fixed contract follows: non-Apple version 22, size 360
and 45 named offsets (take_text added); Apple version 23, size 376,
vulkan_library and vulkan_extensions at 360 and 368.

Assisted-by: Claude Opus 5.5
Co-Authored-By: Claude Opus 5.5 <[email protected]>

fix(macos): the shader preparation runs on every usable CPU on macOS ... 1843f6f809

prepare_run's first part (every shader translated, and each compute
variant compiled for the pipeline cache) ran on half the CPUs
everywhere: prepare_cpus gives half_cpus unless the Windows process heap
is the segment heap. Half exists for the NT heap's lock that an
in-process driver compiler queued on, and for whole-PC freezes when
those compilers ran on every CPU at High priority. Neither holds on
macOS: MoltenVK converts SPIR-V to MSL in a few ms and hands every
compile to Metal's compiler service (MTLCompilerService, a launchd
service at nice 0), so the workers mostly wait on it. The Mac's first
start logged "8 workers for the shader programs (16 CPUs usable; half
the CPUs)".

prepare_workers::count takes an `every` argument (default false): each
usable CPU gets a worker, at least half_cpus and 1, at most kMaxWorkers.
kEveryCpu is true where __APPLE__ is defined, and prepare_cpus passes
it; the preparation's log line says "every one: macOS". In a synthetic
MoltenVK 1.4.2 benchmark on an M4 Max (16 CPUs, macOS 26.7.1) 16 threads
compiled 56-61 % more compute and 35-39 % more graphics pipelines a
second than 8, but only with a pipeline cache per thread; with the one
shared cache LS uses today the rate is flat from 4 to 16 threads. The
next commit gives each worker its own cache.

tests/test_prepare_every_cpu.py compiles the real prepare_workers.h and
prepare_cpus() cut from vk_progcache.cpp: on this compiler's target
prepare_cpus() gives 16 of 16 usable CPUs on macOS (8 elsewhere); count
with and without `every`; prepare_cpus with kEveryCpu forced either way.
On the base it fails: prepare_cpus() gives 8 of 16 on macOS.

Windows and Linux: unchanged (kEveryCpu false; the existing
test_prepare_workers and test_prepare_cpus cases are untouched).

Assisted-by: Claude Opus 5.5
Co-Authored-By: Claude Opus 5.5 <[email protected]>
(cherry picked from commit 3573e736e15078554543e4740c5e2d53c719fa89)

feat(macos): Metal's shader cache in a folder the caller names ... fc8ee75dc7

RX_VK_METAL_CACHE_DIR=<folder>, macOS only and off by default, puts
Metal's own shader cache (compiled functions, libraries, front-end
modules) in that folder instead of Metal's per-user folder under
/var/folders, which a sandbox that keeps every game file in one folder
(tools/ls-run.py) denies. With it denied, every start compiled all of
its pipelines again; the game run's 2,489 "fopen failed for list file:
errno = 1" lines came from that.

gpu/vk/metal_cache.h looks up Metal's exported MTLSetShaderCachePath
and MTLGetShaderCachePath (SPI: in the SDK's Metal.tbd, declared by
WebKit, in no public header) and CoreFoundation's string calls with
dlopen/dlsym, so the renderer links nothing new and has no Objective-C.
load_global asks for the folder before the host loads the Vulkan
library (Metal fixes its folder on first use and silently ignores a
later call), and mkwvk_init logs after create_device whether Metal took
it. A relative or over-long path, a folder that does not exist (the
game never makes it), Metal or a symbol missing, or a folder Metal had
already chosen is logged as "RX_VK_METAL_CACHE_DIR NOT applied" and the
start goes on as without the switch. Unset or empty, nothing is loaded,
called or logged. All of it is under #if defined(__APPLE__): vk_main.cpp's
Windows and Linux text is byte-identical to 5fd1c74b.

Measured with a game-free MoltenVK 1.4.2 probe on the M4 Max (macOS
26.7.1, reports/2026-11-14-metal-cache-dir.md): a synthetic compute
pipeline compiled in ~400 ms with Metal's folder denied, ~213 ms with
a writable empty folder, and 2.6 ms in a second process that found it
there, the folder carried by clonefile; a call after vkCreateDevice was
ignored. Not run in the game.

tests/test_metal_cache_dir.py compiles the real load_global and header
against made-up Metal, CoreFoundation and host loader calls: the order
(the folder is handed over before the library is loaded), the unset,
empty, refused, missing-symbol and mismatch paths, and the source (macOS
only, the check after create_device, the docs row). It fails on
5fd1c74b and on three mutations of the change.

Assisted-by: Claude Opus 5.5
Co-Authored-By: Claude Opus 5.5 <[email protected]>
(cherry picked from commit d7586df37f65b60e507cc3ffb79f38e0f7f7faf0)

fix(macos): a pipeline cache per preparation worker on MoltenVK ... 55749103ac

MoltenVK 1.4.2 holds one mutex per VkPipelineCache across the SPIR-V to
MSL conversion and Metal's library compile (MVKPipeline.mm
getShaderLibrary, _shaderCacheLock). Every worker of the start's
preparation compiled with the one start cache, pcache, so they compiled
one pipeline at a time: the Mac's first start (mac-arm64-a5f2d6f9-01,
M4 Max) logged strict pairs of Metal compiles every ~195 ms, 5.05
compute variants a second, an estimated 33 minutes for the compute
programs alone. A synthetic benchmark on that Mac (PrepSpeed, MoltenVK
1.4.2, macOS 26.7.1): one shared cache stays at 10.5 compute and 9.6
graphics pipelines a second from 4 to 16 threads; a cache per thread,
merged after, gives 31.5 / 29.7 at 8 threads and 49-51 / 40-41 at 16.

gpu/vk/worker_caches.h: a Set of empty caches, one per worker of a
phase, that a worker enters with Scope; create() asks the shared cache
first under VK_PIPELINE_CREATE_FAIL_ON_PIPELINE_COMPILE_REQUIRED_BIT
and compiles what it lacks with the worker's own; merge_into() merges
them into the start cache with one vkMergePipelineCaches in slot order
(MoltenVK retains each Metal library, no compile). The caches are not
externally synchronized, so a shared slot stays correct.

vk_draw.cpp:
- create_cached: before play, a worker in a phase with a set goes
  through worker_caches::create (pcache asked first, so a saved file's
  or an earlier phase's pipelines are found as before); every use of
  pcache holds pcache_mu shared, and worker_caches_merge holds it alone
  (vkMergePipelineCaches's dstCache is externally synchronized:
  vk.xml externsync without INTERNALLY_SYNCHRONIZED_MERGE). A save's
  reads of pcache's data hold it shared too.
- prepare_run (the compute warm), precompile_run, precompile_warm (both
  passes) and precompile_at_create make a set under worker_caches_on,
  enter it in their workers and merge it right after the join; the
  precompile's stragglers left running are merged as far as they got.
- precompile_run, with the caches on, sorts the notes by program
  variant and hands a variant's notes to one worker in runs of up to
  32 (worker_caches::runs/claim), so its shader libraries are compiled
  once, not by every worker that met one of its notes at the same time.
- In play, with the caches on, a compile goes into a new cache of its
  own merged into the generation's at once under the generation's lock
  (made_alone), which the asks of the generation hold shared: the asks
  keep their order and find what any thread compiled.

vk_vendor.h: worker_caches, on for VK_DRIVER_ID_MOLTENVK only, and only
with pipelineCreationCacheControl; RX_VK_WORKER_CACHES forces it either
way. Elsewhere nothing measured says a driver's cache is locked across
its compiles, and the caches would only add merges. The pipelines built,
the persisted cache's format and its stamp are the same either way.

tests/test_worker_caches.py runs vk_draw.cpp's own create_cached,
play_cache_make, worker_caches_on and worker_caches_merge and the real
header against a made-up driver whose caches lock across a compile:
four workers compile at once (without a set they cannot); pcache asked
first, nothing compiled with it; ask-only callers never compiled; caches
not externally synchronized; one merge in slot order whatever order the
slots were taken in, with no ask of pcache beside it; play's compiles
merged into the generation at once; runs and claims. The four-workers
part runs against the base's create_cached and fails there (1 of 4
compiles met the others). tests/test_pipeline_play_cache.py gets an
"alone" case, and both play-cache and cache-save harnesses the new
stand-ins (pcache_mu, worker_caches_on).

Windows and Linux: the same code paths as before (no set, one shared
cache), plus an uncontended shared lock around pcache's uses.

Assisted-by: Claude Opus 5.5
Co-Authored-By: Claude Opus 5.5 <[email protected]>
(cherry picked from commit 2174e33f14ccce8c1635565e734773a33bc32455)

fix(macos): the worker caches make no Vulkan call from a header ... b5f4f1febd

tests/test_recorder_sync.py holds every header to no recorder boundary
call: vk_work.h, included last by each renderer .cpp, turns
vkDestroyPipelineCache into a drain of RX_VK_RECORDER's packets and then
the call, and a header's calls would be wrapped or not depending on what
included it. worker_caches.h called vkCreatePipelineCache,
vkDestroyPipelineCache and vkMergePipelineCaches itself, from vk_draw.cpp
before vk_work.h.

worker_caches::Set and made_alone now take a Calls struct with the
three operations; vk_draw.cpp defines worker_cache_calls after
vk_work.h, so a set's cache destruction goes through the recorder's
boundary like every other vkDestroy* in the file. Behaviour is
otherwise the same: the same caches, made, merged and destroyed in the
same order.

tests/test_worker_caches.py and tests/pipeline_play_cache_test.cpp take
worker_cache_calls from the source and the model respectively.
test_recorder_sync now fails only as on the base (vk_vma.cpp's
vmaDestroyAllocator), and the four tests that touch the caches pass.

Assisted-by: Claude Opus 5.5
Co-Authored-By: Claude Opus 5.5 <[email protected]>
(cherry picked from commit aa8ecba546465f1ce65ac872ecf21d232b65cef0)

fix(macos): MoltenVK starts with an empty pipeline cache and writes none ... f59051783e

MoltenVK 1.4.2 keeps MSL, not GPU code, in a VkPipelineCache, and
vkCreatePipelineCache with saved data compiles every stored library one
after another on the calling thread (MVKPipeline.mm readData).
pcache_init hands it all of mkw_vk_pipelines.bin, and play_files_merge
every play file, at every start. The PrepFast GPU check on the M4 Max,
with Metal's per-user cache blocked as in the runner, ran three later
starts of the same 64 pipelines (80 libraries):
- reading the one file: 5.7 s of serial compiles, 6.6 s in all;
- 16 shards read in parallel: 1.4 s;
- nothing read, pipelines compiled from SPIR-V with the worker caches:
  1.4 s.
At the game's scale the read is tens of minutes per start.

vk_vendor.h gets pcache_no_load, on for VK_DRIVER_ID_MOLTENVK only and
logged in the vendor fixes' line with worker_caches. It is the device's
default for RX_VK_PCACHE_LOAD: pcache_load_off reads the switch first
(any value, as before) and falls back to the fix, so
RX_VK_PCACHE_LOAD=1 reads the files on MoltenVK as everywhere else. With
the fix on and the switch unset or off, pcache_files_unused holds:
- pcache_init reads neither the main file nor the play files and says
  so in the log; the cache lives in memory for the run;
- nothing is saved (pcache_no_save, as RX_VK_PCACHE_LOAD=0 already
  did), and play_cache_save writes no play file and keeps the
  generation in memory: unread play files would pile up, since they
  are compacted only where they are read;
- the stamp is left alone (pcache_drop_stale returns as for any
  start that does not read the files);
- precompile_run drops RX_VK_LOAD=create's longshot_pipelines.bin.built
  marker at each start, as it does when the files were dropped: no
  cache a later start reads holds the notes, so the warm builds them
  into the in-memory cache again before loads at create ask for them.

Saving does not continue on MoltenVK because nothing reads the file or
its stamp there:
- the prepared record (prepared_for) has its own file and stamp;
- --prepare-shaders' saved cache is useful only to a start that reads
  it;
- pcache_saved_now serves graphics pipeline libraries, which MoltenVK
  does not have;
- a start with RX_VK_PCACHE_LOAD=1 makes, saves and stamps its own.
Every other driver is unchanged, RX_VK_PCACHE_LOAD=0 there included
(main file unread and unsaved, play files still merged and written).

tests/test_vendor_fixes.py checks the two cache fixes on its 16 devices
and on MoltenVK: on for MoltenVK alone, RX_VK_WORKER_CACHES forcing
the first, vendor_fixes_env leaving pcache_no_load to the files'
switch. It also checks the log line and pcache_load_off's order.
tests/test_pcache_stamp.py runs pcache_load_off, pcache_files_unused,
pcache_drop_stale and pcache_init, cut from the source, in eight starts:
other drivers unset, 0, empty and 1 as before, and MoltenVK unset, 0,
off and 1. test_pipeline_play_cache gets an "unused" case. All three
fail on aa8ecba5. tests/test_warm_bounded.py's source check of the
marker's removal takes the new condition.

Assisted-by: Claude Opus 5.5
Co-Authored-By: Claude Opus 5.5 <[email protected]>
(cherry picked from commit 61bfbf1ae7c0966a9436ab0c556a9ee547bb3b5f)

fix(vk, macos): the renderer's threads start with a 16 MiB stack, not macOS's 512 KiB ... f0a62f7ba0

macOS starts a std::thread with 512 KiB of stack, where Linux starts one
with 8 MiB and Windows with longshot.exe's 8 MiB. On macOS MoltenVK
converts each shader with SPIRV-Cross on the thread that creates the
pipeline, recursing once per block of a program's control flow, and a
prepare_run worker overran its 512 KiB there (CFG::walk_from under
vkCreateComputePipelines from seed_compute, start #6, 2026-11-14).

gpu/vk/thread_stack.h: thread_stack::thread is std::thread itself off
macOS; on macOS a std::thread-like thread (construction with decayed
arguments, joinable, join, detach, native_handle, moves) started by
pthread_create with thread_stack::kBytes (16 MiB) of stack. Every
thread the renderer starts goes through it: the preparation's pools
(prepare_run, state_list_prepare, warm_libraries, precompile_at_create,
precompile_run, predict_run, warm_bounded::run), the job workers and
the serial builder, the replay, parallel-replay, presenter and
frame-generation threads, the program cache's loaders and the rest.
mkwvk_init logs the size once per start on macOS.

tests/test_thread_stack.py: no file under gpu/vk starts a thread any
other way; each kind of thread runs a recursion 2 MiB deep through the
helper, and on macOS each overflows as a std::thread (caught, no crash
report). Seven source checks follow the new spelling.

Assisted-by: Claude Opus 5.5
Co-Authored-By: Claude Opus 5.5 <[email protected]>
(cherry picked from commit 48b49054b1c64851b3e8c974e64dc0b757b2ced3)

test(macos): one overflow message from thread_stack_test, however many threads overflow at once ... ea4081d942

The first thread whose stack overflows says so and ends the process;
the others wait in their handlers, so the message is not interleaved.

Assisted-by: Claude Opus 5.5
Co-Authored-By: Claude Opus 5.5 <[email protected]>
(cherry picked from commit 62d0991e0cfb5402202f8f14e9eb799d5c20d262)

test: the checks that compile renderer code with its threads see thread_stack.h ... 7707c63433

Eight checks cut renderer functions that start threads (or copy
vk_jobs.cpp and vk_recorder.h) into fixtures of their own; they now
build with gpu/vk/thread_stack.h beside them: test_capture_copy_helpers,
test_gpu_hang_watch, test_jobs_failure, test_jobs_wait_runs_queued,
test_pipeline_cache_save, test_prepare_stall, test_present_source_crop
and test_recorder_sync.

Assisted-by: Claude Opus 5.5
Co-Authored-By: Claude Opus 5.5 <[email protected]>
(cherry picked from commit fd46ff431f9f62b1125582a4a3434a8cc92c551e)

fix(macos): the worker caches follow main's fetch lock (#190) and Intel's gate (#252) ... 078c9a7a54

Main's #190 (f41a5c59) put a fetch lock (PcacheFetch, PcacheUse) on the pipeline cache's save and compile path, the
very path where the worker caches add the lock their merges need, and #252 (57f4eaec) put Intel's compiler gate
(IntelDriverGate) around the compiles. The worker caches now sit on top of both:

- pcache_mu, the merge lock, is taken only where the worker caches are on (pcache_use_held: MoltenVK by default).
  Elsewhere create_cached, the save's fetch and pcache_saved_now take exactly #190's locks, as on main: no new lock
  on any other driver. A play generation's lock is likewise taken only where play's compiles are merged into it
  (g->alone).
- The worker caches are never on with the fetch lock (worker_caches_on: not with pcache_fetch_lock_on, Intel's
  Windows driver or RX_VK_PCACHE_FETCH_LOCK=1): a merge reads its caches' data as a fetch does, which that lock keeps
  apart from the builds there, and the gate lets one compile at a time into Intel's compiler there anyway.
- The order of the locks: Intel's gate, then pcache_mu, then the fetch lock. The save's fetch and pcache_saved_now
  take pcache_mu before PcacheFetch; a merge takes pcache_mu alone.
- prepare_run's compute warm keeps #252's gate around its compile and, with the caches, goes through create_cached
  under it (cached_compute would take the gate a second time).
- tests: test_worker_caches checks the gating (with the caches off a pipeline is made while pcache_mu or the
  generation's lock is held alone elsewhere; with them on it waits for pcache_mu), worker_caches_on against the
  fetch lock, the warm's gate and pcache_mu ahead of PcacheFetch in the save. test_pipeline_play_cache's route case
  checks that no lock of theirs is taken off MoltenVK; pipeline_play_cache_test.cpp and pipeline_cache_save_test.cpp
  get pcache_use_held stand-ins. test_pcache_stamp's load fixture follows #321's cache_path and #190's PcacheFetch in
  pcache_init, and play_cache_save's new parameters.
- docs: #321's cache/longshot_vk_pipelines.bin in the RX_VK_WORKER_CACHES row and in vk_vendor.h, and a row for
  RX_VK_PCACHE_LOAD, whose default this PR changes on MoltenVK.

Assisted-by: Claude Opus 5.5
Co-Authored-By: Claude Opus 5.5 <[email protected]>

merge(macos): the renderer threads' 16 MiB stacks (mac/thread-stack 7707c634) under the precompile hand-off ... f9f166f7cb

The precompile hand-off starts its workers and its compile watcher through thread_stack::thread. One conflict file,
gpu/vk/vk_draw.cpp (4 blocks: the lines P31 had to resolve against main, the other way round): the include block keeps prepare_workers.h,
worker_caches.h and thread_stack.h; prepare_run's, precompile_at_create's and precompile_run's thread lines take
thread_stack::thread and keep the worker caches' merges beside them.

Assisted-by: Claude Opus 5.5
Co-Authored-By: Claude Opus 5.5 <[email protected]>

fix(vk): MoltenVK: the precompile goes on behind the game; slow compiles named; a failed compile tried once more ... 89606310bb

On the Mac's start 12 every precompile worker was held by a Metal compile of minutes while 37,600 of
42,234 items were still to hand out, so RX_VK_PRECOMPILE_STALL_S could not apply (it waits until every
item is handed out) and the title never came. With the new vendor fix compiler_service (MoltenVK only,
RX_VK_COMPILER_SERVICE):
- the precompile hands the rest to the game once nothing has finished for the stall time, or once all
  that is left is compiles running past RX_VK_SLOW_COMPILE_S; a quarter of the workers build the rest
  behind the game, every item still built, each straggler named when it ends;
- its order: noted pipelines first, then fallbacks and computes by SPIR-V words, what an earlier run
  found slow (longshot_pipelines.bin.slow) last;
- a compile over the slow time (10 s) is named with key, variant, kind, stage and SPIR-V words while it
  runs and when it ends, and written to the slow list;
- a pipeline whose compile failed is compiled once more with a new pipeline cache (MoltenVK keeps a
  failed library in the cache it was compiled with), and said made or not made.
Every other driver: the classic worker, wait and log line as they were.

Assisted-by: Claude Opus 5.5
Co-Authored-By: Claude Opus 5.5 <[email protected]>
(cherry picked from commit 77d4971cbdb6356d9064a44cdee923f4bde354f9)

test(vk): test_precompile_handoff: a tree without the change says each missing piece ... 306ffd28ca

Assisted-by: Claude Opus 5.5

Co-Authored-By: Claude Opus 5.5 <[email protected]>
(cherry picked from commit f9a179c08d3f9e102729ec8fe5fe923336f581b8)

docs: RX_VK_PRECOMPILE_STALL_S, RX_VK_COMPILER_SERVICE and RX_VK_SLOW_COMPILE_S ... 44f210178c

Assisted-by: Claude Opus 5.5

Co-Authored-By: Claude Opus 5.5 <[email protected]>
(cherry picked from commit 194a230dd959706797203501edad3b5a2331118a)

fix(vk): name a graphics compile at its precompile callers, not inside build_graphics ... ab2bf44680

test_fallback_pipeline_state compiles build_graphics cut from the source; the noted pipelines of behind_worker and precompile_warm name what they build instead.

Assisted-by: Claude Opus 5.5

Co-Authored-By: Claude Opus 5.5 <[email protected]>
(cherry picked from commit 528784dc5c0c293da1b692f6fd71da070a940d8b)

fix(vk): the precompile hand-off on main's Intel gate and quick builds (#252), fetch lock (#190) and game folder (#321) ... 7ef7cff977

The hand-off was written on the macOS integration branch, before main's #190 (f41a5c59), #252 (57f4eaec) and #321
(76699aad), and before the worker caches followed them (this branch's base). What it needs on main's code:

- cached_graphics and cached_compute keep #252's gate and main's `return create_cached(guarded_*_held, ci, pl);`, and go
  through create_watched only where compiles are watched (compiles_watched: compiler_service, MoltenVK's, or
  RX_VK_SLOW_COMPILE_S set). Every other driver makes its pipelines exactly as on main, and main's own source checks
  (test_intel_compiler_gate, test_pcache_fetch_lock) read the lines they read there.
- prepare_run's compute warm: with the worker caches and watched compiles through create_watched, under the gate it holds;
  otherwise P22's create_cached or main's guarded_compute_held, as they are.
- behind_worker, the precompile's worker under compiler_service, builds a compute pipeline as the classic worker now
  does (#252): on Intel's one-at-a-time builds (serial_builds_on) without the optimizer first, the optimized build owed
  to its first dispatch (precompile_compute_quick, optimize_owed). On MoltenVK, where compiler_service is on, that never
  applies unless RX_VK_COMPUTE_SERIAL=1; it keeps the two workers' builds the same.
- test_precompile_handoff's source checks follow main's spellings: the watched and plain lines in cached_* after the gate,
  compiles_watched, the warm, the dispatch's inline build named as main spells it (p, not the integration branch's plain),
  and behind_worker's quick build.
- docs: the slow list is cache/longshot_pipelines.bin.slow since #321 moved the notes into the game folder's cache/.

Assisted-by: Claude Opus 5.5
Co-Authored-By: Claude Opus 5.5 <[email protected]>

fix(vk): under compiler_service the preparation leaves the compute pipelines to the precompile ... abfc05c8ea

On the Mac's cold start 14, prepare_run's compute warm kept 13 of its 16 workers on compute compiles of
122k-209k SPIR-V words for 205-274 s each. The preparation reached 278 of 38,809 programs in 430 s, and the
precompile, whose hand-off lets the title start while such compiles go on behind it, never began.

With compiler_service (MoltenVK only) the warm compiles nothing. Each compute variant is still translated and
kept by seed_compute (program_cache_save), and the state list and the prepared record follow as before. The
precompile compiles the pipelines: every kept compute program on a first start (computes_plan), in its order
and with its hand-off; one it does not take is compiled at its first dispatch. Every other driver's warm is
unchanged.

tests/test_warm_handoff.py runs prepare_run's own decision and warm_compute against stand-ins on both arches,
and pins the source chain that keeps each variant for the precompile.

Assisted-by: Claude Opus 5.5

Co-Authored-By: Claude Opus 5.5 <[email protected]>
(cherry picked from commit 84c32d68c536468b61af24230ea8c283c5f091ef)

fix(vk): at most four giant compiles at once in the precompile under compiler_service ... bd313bffae

Start 14 had 13 compute compiles of 122k-209k SPIR-V words in flight at once while the other programs went
at 0.1-0.3 a second, and Metal's compiler service aborted twice (LLVM bad_alloc).

precompile_handoff.h Giants: at most RX_VK_GIANT_COMPILES (4) compiles of more than RX_VK_GIANT_WORDS
(100,000) SPIR-V words in flight at once among behind_worker's workers, before the game and behind it. A
giant claimed while the bound is reached is deferred, never dropped, and named once with a count; its
worker takes the next item, and the deferred ones are taken first once a giant ends. A worker with nothing
else waits for them and leaves only when none is deferred. behind_worker claims with claim_below, which
never passes the end, and a worker adds its one "ran out" to next as it leaves, so the wait's
next >= total + threads still means every worker has left. The classic worker (every other driver) is
unchanged.

The precompile_handoff fixture runs the gate with blocking giants: at most 2 of 6 in flight, the small
items going on meanwhile, each deferral named once with its count, every item built once, before the game
and behind it with the hand-off.

Assisted-by: Claude Opus 5.5

Co-Authored-By: Claude Opus 5.5 <[email protected]>
(cherry picked from commit f1e028c8db05ef31383fa44c05fa156984c0f6c9)

docs: RX_VK_GIANT_COMPILES and RX_VK_GIANT_WORDS; the preparation's compute pipelines under compiler_service ... 1725c432f4

Assisted-by: Claude Opus 5.5

Co-Authored-By: Claude Opus 5.5 <[email protected]>
(cherry picked from commit 8aaa2f33d51b6b6f8db335cac69639137833537e)

fix(vk): giant compiles never hold every worker of a phase ... e10c0ad542

The bound on giant compiles in flight is now min(RX_VK_GIANT_COMPILES, workers in the phase - 1), and at
least 1. Before the hand-off the phase is the precompile's workers (16 on an M4 Max: 4); at the hand-off
precompile_run sets it to the share behind the game (4 workers: 3), so one worker there always stays on the
smaller items. Giants already in flight over the new bound finish; no more start until they are under it.
RX_VK_GIANT_COMPILES=0 is still no bound. The order line, the hand-off line and each deferral line give the
bound in use; docs/development.md says so.

The precompile_handoff fixture gains a hand-off by the stall time with three giants (4 workers: 3) and a
stuck small item holding every worker and most of the list still to hand out: behind the game, on a share
of 2, one giant at a time and the other worker on the small items, every item built once. Plus limit_for
and phase.

Assisted-by: Claude Opus 5.5

Co-Authored-By: Claude Opus 5.5 <[email protected]>
(cherry picked from commit acd3a71a2717b1e1bfc83c1952932fdcf1afd9de)

test(vk): test_warm_handoff follows main's warm and dispatch (#252, #190) ... 20367871b5

The fixture cuts prepare_run's warm out of vk_draw.cpp. On main the warm builds
under Intel's compiler gate (#252 IntelDriverGate) through guarded_compute_held,
with the worker caches through create_cached (#190, PrepFast) or create_watched
where compiles are watched (PrecompileStall's compiles_watched). The fixture now
has stand-ins for those; without worker caches, as there, the warm still goes to
guarded_compute_held into pcache and is counted the same way.

The source check for a dispatch's build of an unmade compute pipeline reads
main's name for the dispatch's program, p (the topic's tree had a later
rename to plain).

Assisted-by: Claude Opus 5.5

Co-Authored-By: Claude Opus 5.5 <[email protected]>

fix(vk): behind the game no giant compile starts before the first frame; the precompile's giants and its work behind the game below the game's QoS class ... c291d0c67d

Run 17 (MoltenVK 1.4.2, M4 Max): the game thread's first vkQueueSubmit2 waited about four minutes in
MVKCmdBufferImageCopy::encode. The AGX driver compiled a blit program variant for the copy synchronously
(findOrCreateBlitProgramVariant -> MTLCompilerScheduler::buildRequest -> MTLXPCCompilerConnection::tryBoost ->
dispatch_sync), and its request waited on one of Metal's 16 compiler connections behind a giant compute compile
that ended only when the compiler service, crashing under 16 of them, gave up on it.

Under compiler_service (MoltenVK) only:
- Giants::hold: from the hand-off, no compile of more than RX_VK_GIANT_WORDS starts until the game has shown
  RX_VK_GIANT_HOLD_PRESENTS frames (1), or RX_VK_GIANT_HOLD_S (120 s) at most; the ones in flight finish, the
  deferred ones go first after it. Nothing is dropped.
- RX_VK_COMPILE_QOS (utility): behind the game every run, and every giant before it, is built at QoS class
  utility; the rest, which the title waits for, at the thread's own (DEFAULT, the game threads' class).
  MoltenVK dispatches each Metal compile from the calling thread to the default global queue, whose QoS is a
  fallback only, so the request carries the worker's class.

The classic worker and every other driver are unchanged; compile_qos_apply does nothing off macOS.

Assisted-by: Claude Opus 5.5
Co-Authored-By: Claude Opus 5.5 <[email protected]>
(cherry picked from commit 4f319a0d9a658d4205d52fe25be645443764ed75)

docs: RX_VK_GIANT_HOLD_PRESENTS, RX_VK_GIANT_HOLD_S and RX_VK_COMPILE_QOS ... b0c650ef74

Assisted-by: Claude Opus 5.5
Co-Authored-By: Claude Opus 5.5 <[email protected]>
(cherry picked from commit 36e04c5eb85f15398ec3a82afdef7205bd37e304)

fix(vk): a giant compile that failed is tried once more alone, not at once among the other giants ... d429f0f08d

Runs 17 and 18 (MoltenVK 1.4.2, M4 Max): Metal's compiler service aborted 13 and 7 times on an LLVM
allocation failure while it held giant compute compiles, and each abort takes every compile of that
instance with it (up to 8). A giant that came back VK_ERROR_INITIALIZATION_FAILED was compiled again
at once, with a new pipeline cache, among the giants still in flight.

Under compiler_service (MoltenVK) only, in the precompile's behind_worker: a giant's failed compile is
only said (create_watched, Retry::Later); its worker gives its place back (Giants::solo_ask) and waits
until no other giant is in flight and none is held, while no giant starts (Giants::solo), then
compiles it once more with a new pipeline cache (Retry::Fresh) and says so. Its compute_state stays
claimed meanwhile; a stop cancels its turn. Every other compile, and every other driver, as before.

Assisted-by: Claude Opus 5.5
Co-Authored-By: Claude Opus 5.5 <[email protected]>
(cherry picked from commit b6069bbab0719980b61f26c6b1df469f989f930a)

fix(vk): a dispatch's compute pipeline that MoltenVK's compiler service failed is built, not left out for good ... 7a8ede6307

With RX_VK_COMPILER_SERVICE (MoltenVK only), the precompile goes on behind the game with three or four giant compiles in flight,
and Metal's compiler service ended under 13 and 8 compiles in starts 17 and 18. A compute pipeline a dispatch's build could not
make was failed for good: program_pick never gave its program again, and every dispatch of it was left out.

- precompile_handoff.h Board: a compile a dispatch waits for is wanted from its push until it ends; while one is, the precompile
  starts no giant (Giants::share), deferred ones included; play's own compiles never wait on the giant bound.
- A build that failed while another giant compile was in flight is handed to compute_retry_alone: tried again on a thread of its
  own, once no giant compile is in flight and with none beginning beside it, at most RX_VK_ALONE_TRIES times (2). Its
  compute_state stays 1 meanwhile, so its dispatches are left out and counted as still compiling; one not made alone either is
  failed as before.
- RX_VK_SKIP_LOG=1: each dispatch left out named with its program key and why, once per key and reason (skip_log.h).
- tests/test_loading_stall.py with tests/loading_stall_test.cpp; seg_gathers_test.cpp takes the named skips.

Every other driver: no board call (each behind compiler_service_on()); RX_VK_SKIP_LOG off unless set.

Assisted-by: Claude Opus 5.5
Co-Authored-By: Claude Opus 5.5 <[email protected]>
(cherry picked from commit 830c1a023812449db91b6436a3406ff58e5d24e6)

test: test_loading_stall: a tree without the build's retry is a failure said, not a stop ... a088234ae2

Assisted-by: Claude Opus 5.5
Co-Authored-By: Claude Opus 5.5 <[email protected]>
(cherry picked from commit 40683ae53e44ba5702f2229a1ddd62cb0cd92b90)

perf(vk): RX_VK_SKIP_LOG off works out nothing for a dispatch left out ... 76561a8ef0

SKIP_DISPATCH and the still-compiling skip ask skip_log_on() before they work out a key or reason (compute_code_key, the
retry's lock), so every driver's skip paths cost what they did before with the log off.

Assisted-by: Claude Opus 5.5
Co-Authored-By: Claude Opus 5.5 <[email protected]>
(cherry picked from commit 8885591d407ece67a9511a4f908d0f4bbcdfa94e)

fix(vk): a dispatch's build lets play's want go by its own scope, its failed() as it was ... f0bd773d27

The build's failed() captured board_on to let the want go on a throw; test_jobs_failure runs each [job]() noexcept recovery as
it stands and found one missing. BuildWant now lets the want go when the build ends, by return or throw (before failed()
runs), unless a retry alone took it over; failed() is back to what it was.

Assisted-by: Claude Opus 5.5
Co-Authored-By: Claude Opus 5.5 <[email protected]>
(cherry picked from commit 9e1a04021849d9acf945f35400cf87151ddeb0bd)

fix(vk): one place, one count and one wording for a failed compile's tries alone ... 6f62eddae3

The semantic merge fix for Integration 28's TitleStall (b6069bba) and
LoadingStall (8885591d) inputs. Each brought a retry alone of its own:
the precompile's failed giant queued in Giants (solo_), play's failed
dispatch build waited on the Board, with two bounds (once; and
RX_VK_ALONE_TRIES, 2) and two wordings. Merged as they were (2c4a9231),
a giant that gave its place back for its try alone (Giants::solo_ask)
stayed counted on the board it shares, so Board::giants() never came
back to 0 and no try alone of play's ever ran after it (shown on a
2c4a9231 archive: board giants 1 after the solo, play's alone_begin
not run within 2 s), and the two could run at once.

Now, with the parent's rules:
- One place: Board's ticket queue (alone_ask/alone_go/alone_again/
  alone_cancel, alone_begin for play) is the only alone slot. Giants'
  solo calls go to the board it shares (a board of its own when it
  shares none); solo_ask ends its giant on the board too. Play's tries
  go ahead of the precompile's; the precompile's also waits for TitleStall's
  first-present hold and for every compile play wants. While any try is
  queued or runs, no precompile giant starts; a giant of play's waits
  for a running try, a small compile does not.
- One count: RX_VK_ALONE_TRIES (2) tries alone per program, the
  precompile's and play's together (Board::alone_left/alone_spend by key
  and variant); used up, neither path tries again, and one line says so
  once with its key (alone_used_up). The precompile's then leaves it to
  its first dispatch, as before; 0 turns both off.
- Every try alone uses a new pipeline cache from the start
  (Retry::Fresh), play's too.
- One wording from three functions (alone_asked, alone_tried,
  alone_used_up); the deferral line names a try alone apart from play's
  want.

New tests/test_retry_alone.py and tests/retry_alone_test.cpp cover the
combined rules; test_loading_stall.py's two source needles that pinned
compute_alone_tries() as play's bound now pin alone_left. Both lanes'
fixtures pass unchanged.

Assisted-by: Claude Opus 5.5
Co-Authored-By: Claude Opus 5.5 <[email protected]>
(cherry picked from commit d89514202b92e65d2ef27eac9e62dce8441e4207)

fix(vk): the dispatch's want and LoadingStall's checks follow main's build (#252) ... fedbfd1e24

Main's dispatch build has Intel's worker path (#252: on_worker, made_how,
the quick first build and compute_optimize), which returns early when a
worker of Intel's compiler gate makes the pipeline. BuildWant now opens the
build, so play's want on the compile board is let go on those returns too,
not only on the ones after the compile. (On the Mac on_worker is never set:
the gate is Intel's; the scope makes the rule hold by construction.)

test_loading_stall follows main's code: the build's capture list, the serial
push with main's third argument, the dispatch's program named p (the topic's
tree had a later rename to plain), the order check against the build's own
last compute_state store (main's worker path stores above it), and the
named skips counted against main's dispatch (8; the segmented binding, not on
main, brings five more).

Assisted-by: Claude Opus 5.5

Co-Authored-By: Claude Opus 5.5 <[email protected]>

test(vk): test_intel_quick_compute reads the dispatch's build with the compile board's capture ... 3cd25d0ead

Main's check (#252) finds the dispatch's build by its capture list. The build
now also captures board_on (the compile board, compiler_service only), so the
check looks for that list; what it checks inside the build is unchanged.

Assisted-by: Claude Opus 5.5

Co-Authored-By: Claude Opus 5.5 <[email protected]>

merge(macos): the precompile hand-off through loading-stall (mac/loading-stall 3cd25d0e) under the Metal archives ... 5f7dfa900a

The Metal archives' code needs thread_stack.h (mac/thread-stack 7707c634, in
loading-stall through mac/precompile-stall) and the compile board's giants()
that its saver waits on (precompile_handoff.h Board, loading-stall). In
mkwvk_init's Apple block metal_cache::check stays first, then the renderer
threads' stack line.

Assisted-by: Claude Opus 5.5

Co-Authored-By: Claude Opus 5.5 <[email protected]>

feat(macos): MoltenVK's pipelines kept in Metal binary archives ... 81085744bb

RX_VK_METAL_ARCHIVE_DIR=<folder>, macOS only and off by default, keeps
every pipeline MoltenVK makes in Metal binary archives (MTLBinaryArchive)
in that folder, so that a start makes every pipeline an earlier start
compiled without compiling it again.

Metal's own shader cache cannot do that for this game. Each of its two
files (functions.data, libraries.data) holds at most 1 GiB:
MTL_SHADER_CACHE_SIZE and FS_CACHE_SIZE can only lower it. And
libCoreFSCache evicts only entries the running process has neither
looked up nor added, so once a start has touched 1 GiB, everything it
compiles after that is dropped (insert error 0xd3, ignored by Metal).
On runs #17 and #18 both files stopped growing in the first minute of
the precompile, and the large compute programs, compiled last, were
compiled again on every start (static reading of macOS 26.7.1's Metal
and libCoreFSCache: reports/2026-11-14-metal-cache2.md).

An archive is a file the app owns, with no size limit (WWDC20 10615).
MoltenVK 1.4.2 makes none (no newBinaryArchiveWithDescriptor: call, no
VK_KHR_pipeline_binary; MoltenVK issue 1765 is open), and rebuilding it
is not an option here, so gpu/vk/metal_archive.h adds them beneath it.
After the device is made it replaces, on the class of Metal's device,
the two calls MoltenVK 1.4.2 makes every pipeline with
(newComputePipelineStateWithDescriptor:options:completionHandler: and
newRenderPipelineStateWithDescriptor:completionHandler:, MVKPipeline.mm).
MoltenVK makes them inside @synchronized(device), so the replacement
only copies the descriptor and returns; a thread of its own then looks
the pipeline up in the archives earlier starts saved (with
MTLPipelineOptionFailOnBinaryArchiveMiss: no compile), or adds it to a
run archive that only that thread uses meanwhile (the one compile) and
makes it from there. Run archives are saved through a .tmp renamed in
place every 64 pipelines, at once after an add of 2 s or more, after
30 s, and when the window closes (mkwvk_close_asked); one saved at
64 MiB is let go with its memory. Archives live in
<folder>/<macOS build>-<GPU name>/, since an archive serves one pair.

MoltenVK's completion handler is called exactly once on every path:
anything that fails (no add, not found after the add, no run archive)
makes MoltenVK's own call, with the descriptor as MoltenVK made it, so
the pipeline is compiled as without the switch; after 4 such add-path
failures with none added a start stops adding. A descriptor with
archives of its own, a call that asked to fail on a miss, and every
call after the close go to MoltenVK's call unchanged. A refusal at the
start (bad path, no folder, a Metal without the calls) is logged as
"RX_VK_METAL_ARCHIVE_DIR NOT applied" and nothing is replaced. Unset or
empty: nothing is loaded, replaced or logged. Everything is looked up
with dlopen/dlsym (libobjc's C functions, CoreFoundation's NSURL and
NSArray types, the blocks ABI): no new link, no Objective-C source. All
of it is under #if defined(__APPLE__): vk_main.cpp's Windows and Linux
text is byte-identical to 1c53cff1. Not kept: the libraries (MSL to
AIR) MoltenVK compiles before the pipeline call stay in Metal's
library cache with its 1 GiB.

tests/test_metal_archive.py runs the real header against made-up Metal
calls on any OS, and on macOS also its Objective-C binding against
made-up classes registered with libobjc (no Metal, no GPU).

Assisted-by: Claude Opus 5.5
Co-Authored-By: Claude Opus 5.5 <[email protected]>
(cherry picked from commit 050961c01610e7fd2bb19d46d3e288330bbd21db)

fix(macos): the binary archives' state outlives the process's exit ... 95ce0915e1

A pipeline's job runs on its own thread and may still be compiling
when the process exits (the hand-off leaves giants compiling behind the
game, and the window's close saves only the run archives not in use).
As a static object, metal_archive::Archives would be destroyed by
exit() under such a job: its mutex and lists freed while the job uses
them. vk_main.cpp now makes it once with new and never destroys it.

Assisted-by: Claude Opus 5.5
Co-Authored-By: Claude Opus 5.5 <[email protected]>
(cherry picked from commit 426ffa5571c28620896a110d8a12bff19dd0e97d)

test(macos): test_metal_archive: no -pthread on Windows ... 5ce0963086

As tests/test_bug_report.py does: clang on Windows takes no -pthread,
and with -Werror its unused-argument warning would stop the build.

Assisted-by: Claude Opus 5.5
Co-Authored-By: Claude Opus 5.5 <[email protected]>
(cherry picked from commit f4cd09cb873cedd6ec3d05341ded530e9388e567)

fix(macos): Metal binary archives saved once each, by one gated saver, with Metal's staging removed ... 467b90d47a

Run #21 (reports/2026-11-14-device-lost-21.md) found the RX_VK_METAL_ARCHIVE_DIR hook rewriting every
run archive in full every 64 adds (1,042 saves of 42 archives), never letting go of an archive whose save
failed, leaving 12 GB of Metal's gpuarchiver-* staging in the shader cache folder, saving on the compile
jobs' own threads, and slowing the graphics precompile from about 30 s to about 140 s.

- A run archive takes pipelines until it is estimated at 64 MiB (each kind's bytes a pipeline as this
  start's saves measured them, 64 KiB before), is then saved once and let go; the ones still open when
  the game starts are saved at the close. At most 16 take pipelines at once and those not saved yet hold
  an estimated 1 GiB at most; past either a pipeline is compiled without them, and counted.
- One saver thread at utility QoS saves them one at a time; it waits while a giant compile is in flight
  (the compile board) or the game is in play (its first present), unless 512 MiB or more wait. From the
  game's first frame pipelines are only looked up.
- A failed save is logged with Metal's reason every time, tried once more 5 s later, and let go after
  the second failure. Render pipelines without a fragment function go to archives of their own; once
  one of those is let go for Metal's "stage in pipeline no." they are compiled without the archives.
- Metal stages each archive in gpuarchiver-XXXXXX under MTLGetGPUArchiverCachePath() and leaves it
  after a successful save (MTLCompiler generateMachO deletes it only on failure): the folder the archive
  names through getArchiveIDWithError: is removed after its save.
- A job compiles at its requester's QoS class; the close line adds the time spent adding, making from
  the run archives, looking up and saving, and the staging removed.

tests/test_metal_archive.py covers each defect; docs/development.md's row says the new rules.

Assisted-by: Claude Opus 5.5
Co-Authored-By: Claude Opus 5.5 <[email protected]>
(cherry picked from commit ccebc6a66fd9c3cfc8c1f8c0285c980eb5f71cd6)

fix(macos): compile_board has external linkage, for the Metal archive saver in vk_main.cpp ... 0640d405dd

vk_main.cpp declares mkwvk::compile_board() and calls it from the archive saver's gate, but vk_draw.cpp
defined it inside its anonymous namespace, so the renderer dylib's link failed with an undefined symbol
(found by Integration 30's build). The anonymous namespace now closes before the definition and reopens
after it; nothing else moves.

Assisted-by: Claude Opus 5.5
Co-Authored-By: Claude Opus 5.5 <[email protected]>
(cherry picked from commit 54ec563e5ef1674a1c7b3221fffcc835ed9419e1)

mooki referenced this pull request 2026-11-15 02:55:20 +00:00

feat(macos): RX_VK_METAL_CACHE_DIR keeps Metal's shader cache in a chosen folder #653

mooki referenced this pull request 2026-11-15 02:55:23 +00:00

WIP: fix(macos): put a dispatch's compute pipeline ahead of MoltenVK's giant compiles and retry it alone, not fail it for good #666

mooki referenced this pull request 2026-11-15 03:11:18 +00:00

WIP: fix(macos): start MoltenVK's giant compiles behind the game only while memory allows, and say a lost device as one #669

mooki referenced this pull request 2026-11-15 04:33:59 +00:00

WIP: fix(macos): Rosetta host imports at whole hardware pages, and a refused import asked again only when its backing changes #686

mooki referenced this pull request 2026-11-15 05:05:20 +00:00

WIP: fix(vk): the MoltenVK nesting bound counts only the reads SPIRV-Cross certainly makes #692

mooki referenced this pull request 2026-11-15 05:05:32 +00:00

WIP: fix(vk): MoltenVK's vertex stages run the guest's warp operations as one-lane warps #694

mooki referenced this pull request 2026-11-15 05:05:39 +00:00

WIP: feat(vk): RX_VK_SPV_PROFILE, what the big SPIR-V modules are made of #695

mooki referenced this pull request 2026-11-15 05:05:47 +00:00

WIP: perf(vk): search storage buffer tables in loops in every global memory fallback on MoltenVK and KosmicKrisp #696

mooki referenced this pull request 2026-11-15 05:05:54 +00:00

WIP: perf(vk): zero local memory at computed offsets by a loop on MoltenVK, not a written-out initializer #697

mooki referenced this pull request 2026-11-15 05:05:59 +00:00

WIP: fix(macos): keep storage image reads in program order on KosmicKrisp too #698

mooki force-pushed mac/metal-cache2 from 0640d405dd to bf5be68150 2026-11-15 07:25:48 +00:00 Compare

mooki referenced this pull request 2026-11-15 11:56:02 +00:00

WIP: fix(vk): heavy compiles run alone under Metal's compiler service; the first giant behind the game no longer waits for 16 GB #784

mooki referenced this pull request 2026-11-15 11:56:08 +00:00

WIP: fix(macos): KosmicKrisp's compiles go through the compiler service's precompile, as MoltenVK's do #785

mooki referenced this pull request 2026-11-15 11:56:12 +00:00

WIP: fix(macos): Metal's shader cache folder asked at the host's start, before SDL's window #786

mooki force-pushed mac/metal-cache2 from bf5be68150 to bf988f6226 2026-11-15 13:54:59 +00:00 Compare

mooki added the

area/renderer

kind

feature

platform/macos

priority

low

labels 2026-11-15 13:57:41 +00:00

mooki referenced this pull request 2026-11-15 17:40:00 +00:00

WIP: fix(macos): MoltenVK's precompile and pipelines on Metal #849

mooki commented 2026-11-15 18:08:38 +00:00

Author

Member

• Copy link
• Copy Markdown

Folded into #849 (fix(macos): MoltenVK's precompile and pipelines on Metal)

Folded into #849 (fix(macos): MoltenVK's precompile and pipelines on Metal)

mooki closed this pull request 2026-11-15 18:08:38 +00:00

Some checks failed

Hide checks

CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Has been skipped

Details

CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Has been skipped

Details

CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Has been skipped

Details

Third-party parts / The same as upstream (pull_request) Has been skipped

Details

Website / Website on Cloudflare (pull_request) Has been skipped

Details

PR description / Description follows the template (pull_request_target) Successful in 6s

Details

Website / Website on Cloudflare (pull_request_target) Has been skipped

Details

CI / Conventional Commit title (pull_request) Successful in 4s

Details

CI / Workflow lint (actionlint) (pull_request) Successful in 7s

Details

Guide screenshots / Guide images and local UI source review (pull_request) Failing after 6s

Details

Third-party parts / As their notes say (pull_request) Successful in 10s

Details

Pull request closed

Please reopen this pull request to perform a merge.

Sign in to join this conversation.

Reviewers

No reviewers

Labels

Clear labels   

area/accessibility

Barriers for people with disabilities

  

area/audio

Sound and music

  

area/input

Controllers, keyboard, mouse

  

area/install

Installing, updating, building from your game

  

area/menu

In-game menu and settings

  

area/renderer

Vulkan renderer, shaders, effects

  

area/saves

Save files

  

area/upscaling

DLSS, FSR, frame generation

  

beanstalk: approved @ 1697603

Set by Beanstalk for show; the farm's database is what counts

  

beanstalk: approved @ 2c5a62e

Set by Beanstalk for show; the farm's database is what counts

  

beanstalk: approved @ 4fd82b7

Set by Beanstalk for show; the farm's database is what counts

  

beanstalk: approved @ 64d6d09

Set by Beanstalk for show; the farm's database is what counts

  

beanstalk: approved @ 6f91fae

Set by Beanstalk for show; the farm's database is what counts

  

beanstalk: approved @ 70a78f9

Set by Beanstalk for show; the farm's database is what counts

  

beanstalk: approved @ 8d361dc

Set by Beanstalk for show; the farm's database is what counts

  

beanstalk: approved @ 9d8356e

Set by Beanstalk for show; the farm's database is what counts

  

beanstalk: approved @ abf2227

Set by Beanstalk for show; the farm's database is what counts

  

beanstalk: approved @ d82f202

Set by Beanstalk for show; the farm's database is what counts

  

beanstalk: approved @ e5f1e40

Set by Beanstalk for show; the farm's database is what counts

  

beanstalk: approved @ ee8ab5f

Set by Beanstalk for show; the farm's database is what counts

  

ci: approved

A maintainer approved CircleCI for this fork PR (sticky; removed after 14 quiet days)

  

good first issue

A good start for a new contributor

  

gpu/amd

AMD GPUs

  

gpu/igpu

Integrated GPUs

  

gpu/intel

Intel GPUs

  

gpu/nvidia

NVIDIA GPUs

  

help wanted

Extra hands or knowledge wanted

  

kind

bug

Something doesn't work as it should

  

kind

chore

Build, CI, dependencies, tidying

  

kind

docs

Documentation

  

kind

feature

A new feature or an improvement

  

kind

performance

Frame rate, frame time, loading, memory

  

kind

question

A question rather than a change

  

needs measurements

Waiting for before/after measurements

  

platform/linux

Affects Linux (a change for every system carries both Windows and Linux)

  

platform/macos

macOS (Apple silicon or Intel Macs)

  

platform/steam-deck

Steam Deck

  

platform/windows

Affects Windows (a change for every system carries both Windows and Linux)

  

preview

Maintainers: deploy a preview of this PR's website (read its changes first). Removed after the run.

  

priority

critical

Crashes, data loss, or blocks most players

  

priority

high

Important: next in line

  

priority

low

Nice to have

  

regression

Worked in an earlier build, broken in a newer one: highest priority, above other bugs

  

status

blocked

Waits on something else first

  

status

confirmed

Reproduced or agreed on

  

status

duplicate

Already reported or proposed

  

status

invalid

Not a problem in this project

  

status

needs-info

Waiting for more information from the reporter

  

status

wontfix

Will not be worked on

  

testing wanted

Wanted: a test on other hardware (posts a test request in #test-requests)

  

ci: approved

No labels

area/accessibility

area/audio

area/input

area/install

area/menu

area/renderer

area/saves

area/upscaling

beanstalk: approved @ 1697603

beanstalk: approved @ 2c5a62e

beanstalk: approved @ 4fd82b7

beanstalk: approved @ 64d6d09

beanstalk: approved @ 6f91fae

beanstalk: approved @ 70a78f9

beanstalk: approved @ 8d361dc

beanstalk: approved @ 9d8356e

beanstalk: approved @ abf2227

beanstalk: approved @ d82f202

beanstalk: approved @ e5f1e40

beanstalk: approved @ ee8ab5f

ci: approved

good first issue

gpu/amd

gpu/igpu

gpu/intel

gpu/nvidia

help wanted

kind

bug

kind

chore

kind

docs

kind

feature

kind

performance

kind

question

needs measurements

platform/linux

platform/macos

platform/steam-deck

platform/windows

preview

priority

critical

priority

high

priority

low

regression

status

blocked

status

confirmed

status

duplicate

status

invalid

status

needs-info

status

wontfix

testing wanted

ci: approved

Milestone

Clear milestone

No items

No milestone

Projects

Clear projects

No items

No project

Assignees

Clear assignees

No assignees

1 participant

Notifications

Subscribe

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference

MagicBeanDev/Longshot!667

No description provided.