MagicBeanDev/Longshot

Watch

Fork 99

WIP: fix(macos): MoltenVK's precompile and pipelines on Metal #849

Draft

mooki wants to merge 68 commits from mooki/Longshot:mac/topic-precompile-metal into main

mooki commented

2026-11-15 17:39:59 +00:00

Member

Type: fix · Risk: medium · Tested on: macOS 26.7.1, Apple M4 Max (arm64; x86_64 under Rosetta) · Needs: Windows and Linux, any GPU

Why

On a Mac, Metal compiles every pipeline in its compiler service, a few processes the whole game shares, and the game asks it for
about 42,000 pipelines and compute programs, a handful of them giants that take minutes and gigabytes each. Started as on any
other GPU, the game crawled or never reached the title: the shader preparation compiled one pipeline at a time behind MoltenVK's
one pipeline-cache lock (39 programs in its first 180 s); a renderer thread's 512 KiB stack overflowed in SPIRV-Cross and the
game crashed; the precompile never let the title come while giants held every worker (run #12); the game's first frame waited
minutes behind 16 giants on the compiler's connections; giants compiled together ran the service out of memory and died, so
three of the four giant mesh programs were never made (run #31); a compile that failed beside a giant was lost for the session;
and every start compiled again what Metal's own shader cache did not keep. This PR folds the macOS port's twelve precompile PRs
into one: the preparation and precompile shaped for Metal's compiler service, on MoltenVK and KosmicKrisp only. It is part of
the macOS port. See #420.

How to reproduce

Seen on: macOS 26.7.1, Apple M4 Max, MoltenVK 1.4.2
Settings and switches: defaults (the macOS port's stack without this PR: main does not start on macOS)
Steps: start the game on topic 4 (shader-backend, this PR's base) from an empty program cache, then again from the one it left; watch longshot_log.txt's vk: preparation: and vk: precompile lines and the first present
Expected: the preparation and precompile finish within minutes, the title comes, the first frame follows within seconds, every program is made
Actual: 39 of 38,809 programs prepared in 180 s; a SIGILL at about 93 s in SPIRV-Cross (start #6); the precompile stalls with no title (run #12: 4,600 items for 160 s, 7,698 for 80 s); the first frame about 3 minutes after the hand-off (start 17); service processes ended under the giants and three giant mesh programs not made (run #31)

What changed

Under the vendor fix compiler_service (on by default on MoltenVK and KosmicKrisp, off on every other driver), the preparation
translates and keeps every program but compiles no compute pipeline, each worker with a pipeline cache of its own on every CPU;
the precompile orders the work for Metal's compiler service, lets the title come while the last giants build behind the game,
keeps at most four giants and one heavy compile in flight, holds giants until the first frame and while memory is short, tries a
failed compile again alone, and puts a dispatch's compile ahead of them. An opt-in switch keeps MoltenVK's pipelines in Metal
binary archives, and two log-only switches name each module's private memory and each slow compile. Every other driver runs the
code it ran before (the off-Apple text changes are listed under Testing). One commit per original PR, in order, then the ports
and the merge fixes:

perf(vk): a pipeline cache per preparation worker, every CPU, and no saved pipeline cache on MoltenVK
(gpu/vk/worker_caches.h, gpu/vk/prepare_workers.h, vk_vendor.h worker_caches/pcache_no_load; RX_VK_WORKER_CACHES,
RX_VK_PCACHE_LOAD's MoltenVK default). Was: #650
fix(macos): renderer threads start with a 16 MiB stack on macOS (gpu/vk/thread_stack.h, every thread the renderer starts;
std::thread itself elsewhere). Was: #651
fix(macos): the precompile goes on behind the title on MoltenVK, slow compiles named, a compile the service lost tried once
more (gpu/vk/precompile_handoff.h, the precompile in vk_draw.cpp, vk_vendor.h compiler_service;
RX_VK_COMPILER_SERVICE, RX_VK_SLOW_COMPILE_S, RX_VK_PRECOMPILE_STALL_S). Was:
#652
fix(macos): the preparation leaves MoltenVK's compute pipelines to the precompile, which keeps at most 4 giants in flight
(seed_compute in vk_shader.cpp, the giant bound in precompile_handoff.h; RX_VK_GIANT_COMPILES, RX_VK_GIANT_WORDS).
Was: #664
fix(macos): giants held behind the game until its first frame, at QoS class utility; a failed giant retried alone
(RX_VK_GIANT_HOLD_PRESENTS, RX_VK_GIANT_HOLD_S, RX_VK_COMPILE_QOS). Was:
#665
fix(macos): a dispatch's compute pipeline goes ahead of the giants and is retried alone, not failed for good (the compile
board in precompile_handoff.h; RX_VK_ALONE_TRIES, RX_VK_SKIP_LOG, gpu/vk/skip_log.h). Was:
#666
feat(macos): RX_VK_METAL_ARCHIVE_DIR keeps MoltenVK's pipelines in Metal binary archives, macOS only, off by default
(gpu/vk/metal_archive.h, its hook in vk_main.cpp, all inside #if defined(__APPLE__)). Was:
#667
fix(macos): giants start behind the game only while memory allows; a lost device said as one, with the system's memory
(gpu/vk/system_memory.h, RX_VK_GIANT_ROOM_MB). Was: #669
feat(macos): RX_VK_LMEM_LOG names each module's private memory and each pipeline's first bind, log-only, macOS only
(gpu/vk/private_memory.h, vk_progcache.cpp; RX_VK_LMEM_LOG_MIN). Was:
#681
fix(vk): heavy compiles run alone under Metal's compiler service; the first giant behind the game no longer waits for 16 GB
(behind_worker in vk_draw.cpp, heavy_compile in precompile_handoff.h, the local memory weight in vk_progcache.cpp;
RX_VK_HEAVY_WEIGHT, RX_VK_HEAVY_WORDS). Was: #784
fix(macos): KosmicKrisp's compiles go through the compiler service's precompile, as MoltenVK's do (vk_vendor.h). Was:
#785
fix(macos): Metal's shader cache folder (RX_VK_METAL_CACHE_DIR) asked at the host's start, before SDL's window
(host/posix/darwin_metal_cache.h, hos_main.c, rx_backend.c). Was:
#786
Port, fix(vk): a compile that never ends keeps the heavy compiles and the giants back for RX_VK_STALE_COMPILE_S (240 s) at
most (Board and Giants in precompile_handoff.h). From macos/giant-starve 5292596d (reports/2026-11-15-giant-starve.md),
placed in main's dispatch build (#252).
Port, fix(vk): what an earlier run found slow for minutes compiles first, and is play's from the hand-off
(RX_VK_SLOW_FIRST_S, Giants::hand_over). From macos/slow-first 40a86bae (reports/2026-11-15-slow-first.md).
Port, docs(vk): metal_archive.h's notes on Metal's cache and staging say only what runs and public notes showed. From
macos/integrated 63fc3d2c; no code change.
Merge fixes (tests the members each passed alone, failing where two met here; each follows I41's fix for the same pair):
test_giant_alone cuts main's dispatch build (the giant-starve port); test_warm_handoff finds seed_compute's save with what
#695's line adds (I41 1cbb594a); test_vendor_fixes runs its checks once, after every check is defined; and the merge of topic
4's fixture fix (test_vendor_fixes, test_subgroup_vertex).

Measurements

Verdict: fixes the Mac's start: with these changes the macOS stack reaches the title and plays, every program made. On I41
78ec1c47 (every member here, before Sweep3's rebase, and the giant-starve port), E9a handed off 8 m 14 s after the start,
made all four giant mesh programs and the four other heavy compiles, made 44511d69 (the tree leaves' program) 887 s into the
hold, and held 1,197 s with 12 of 12 must-shows, 0 skipped draws and 0 crash reports. Each member measured its own part on the
integration build of its day:

#650: 4.4× the compute and 4.6× the graphics pipelines a second in the preparation (16 workers, a cache each, against one shared
cache: 45.41 against 10.35 compute a second), a later start's pipelines in 1.41 s compiled anew against 6.65 s through the
saved cache, and the first start at 230 programs in 180 s against 39 (integration b7a7c95f run #2 against aefe6b0a run #4).
#651: start #6 died with SIGILL at about 93 s; starts #7-#9 with the 16 MiB stack ended cleanly, #8 at 1,011 programs.
#652: the title came (run #17: hand-off at 1,308 s with 42,043 of 42,135 done) where run #12 stalled with none.
#664: the preparation 118.3 s with 0 compute pipelines compiled in it (run #18) against 1,550.4 s compiling 1,381 (run #16).
#665: the first frame 5.0 s after the hand-off (start 21) against about 188-198 s (start 17); 13 service crashes against 6.
#666 and #669: nothing to compare in their starts (no compile failed beside a giant; the memory gate was never asked); their
game-free tests show the rules, and #669's lost-device line named the memory at start 23's loss.
#667: in probe MC-02, 61 of 61 pipelines from the archives in the second process; the switch stays off (no warm game start
from archives measured).
#681: two starts showed no program asks more than 9,359 B a thread, which ruled local memory out of the memory growth.
#784: in probe GC-02 the 464k giants' twin was made alone (1.07 GB of one service process); the giants are made in E9a.
#785: probe KK-03 measured KosmicKrisp's compiles at 1.0-5.8× MoltenVK's, growing with size (no KosmicKrisp game start yet).
#786: a windowed start with RX_VK_METAL_CACHE_DIR from start to hand-off in 6 m 53 s (run #36) against 24 m 28 s (run #35).
Giant-starve: E9a's 7 stale-compile begin lines, all eight heavy compiles made beside 44511d69 (I41 carries it).
Slow-first: not in a game run yet (its prediction: 44511d69 made about 280-660 s after the hand-off in a warm start instead of
932-1,031 s; reports/2026-11-15-slow-first.md).

Hardware: macOS 26.7.1 (25G241), Apple M4 Max, 64 GB; MoltenVK 1.4.2, and MoltenVK main 5189f8de with the argbuf-slots patch from run #31 on
Scene: the title and the field-dawn save's first area, windowed; LS's default settings
Method: each run's longshot_log.txt (the vk: preparation:, vk: precompile, hand-off, slow-compile and close lines, the FPS counter's Dev level in the hold); one start each, so the counts and times are what each start reached, not an interleaved median
How to reproduce: the macOS runner (tools/ls-run.py) on an integration build, a seeded or empty program cache as named, START and A at the title

Baseline: main does not start on a Mac, so the Before column is integration a7711f3c (run #12), the macOS stack with #650 and #651 and none of #652-#786; the After column is I41 78ec1c47 (E9a), which carries every member (as before Sweep3's rebase onto aa95d35f) and the giant-starve port, not slow-first's port or the comment rewording. The two starts' program caches were not the same.

	Before: integration @ a7711f3c (run #12)	After: integration @ 78ec1c47 (E9a)
precompile → title	stalled (4,600 items for about 160 s, 7,698 for about 80 s); no title at the quit (412 s)	hand-off 8 m 14 s after the start; play 48 s later
the four giant mesh programs and four other heavies	not reached	all eight made (5096bcc7 50.2 s, 6dbfda5a 29.9 s, 5c945415 34.1 s, 55d1da3f 34.1 s)
44511d69 (the tree leaves)	not reached	made in 895.6 s, 887 s into the hold
hold	none	1,197 s, 12/12 must-shows, 0 skipped draws, 0 crash reports, 37.0 fps

E9a (mac-arm64-78ec1c47-01): precompile 479.0 s, hand-off 07:23:28.6 (8 m 14 s), play 07:24:16, hold 07:24:25.3-07:44:22.6
giants: 5096bcc7 50.2 s (began alone at the hand-off), 6dbfda5a 29.9, 5c945415 34.1, 55d1da3f 34.1; 44511d69 895.6 s
must-shows 12/12; 7 stale-compile begin lines; 0 XPC errors; first 600 s 37.0 fps / 27.03 ms / GPU 19.4
run #12 (a7711f3c): precompile stalled at 4,600 for about 160 s and at 7,698 for about 80 s; 10,043 of 42,234 at the quit (412 s)

Flags and settings

Eighteen new switches and two new defaults, every one read on MoltenVK or KosmicKrisp only (most through compiler_service), or
macOS only; nothing changes on another driver. HOS_HEADLESS, RX_VK_NO_WARM, RX_VK_PCACHE_FETCH_LOCK and RX_VK_SKIP_PROG
appear in the diff as main's switches, read as before.

Flag	What it is	Before this PR	After this PR	Effect	Helps / doesn't help / untested	Recommendation
`RX_VK_WORKER_CACHES`	a pipeline cache per preparation worker (`=0` one shared, `=1` on any driver with pipelineCreationCacheControl)	no switch	on for MoltenVK only	the preparation's compiles run side by side	helps: MoltenVK 4.4×-4.6×	as it ships
`RX_VK_PCACHE_LOAD`	the pipeline cache file read and saved (`=0` neither, `=1` both)	read and saved everywhere	on MoltenVK neither, elsewhere as before	a later MoltenVK start compiles anew, faster than reading	helps: MoltenVK (1.41 s against 6.65 s)	as it ships
`RX_VK_COMPILER_SERVICE`	the compiler-service path (`=0` as on other drivers, `=1` on any)	no switch	on for MoltenVK and KosmicKrisp	preparation without compute pipelines, the precompile's hand-off, order, bounds and retries	helps: MoltenVK (runs #17-#36, E9a)	as it ships; `=0` only to compare
`RX_VK_SLOW_COMPILE_S`	seconds after which a compile is named and noted slow (`0` none)	no switch	10 with the compiler service	log lines, the slow list	helps: naming the giants	as it ships
`RX_VK_PRECOMPILE_STALL_S`	seconds with nothing finished before the game starts	30, once every item is handed out	30, whatever is left with the compiler service	the title is not held by a few giants	helps: MoltenVK	as it ships
`RX_VK_GIANT_COMPILES`	giant compiles in flight at once (`0` no bound)	no switch	4 with the compiler service	small items finish while giants compile	helps: MoltenVK (runs #18, #19)	as it ships
`RX_VK_GIANT_WORDS`	SPIR-V words above which a compile is a giant	no switch	100000	which compiles the bound holds	helps: MoltenVK; other values untested	as it ships
`RX_VK_GIANT_HOLD_PRESENTS`	frames after the hand-off before a giant starts behind the game (`0` no hold)	no switch	1	the first submit finds the compiler free of new giants	helps: MoltenVK (starts 21-24)	as it ships
`RX_VK_GIANT_HOLD_S`	the hold's longest time	no switch	120	bounds the hold	untested beyond the default	as it ships
`RX_VK_COMPILE_QOS`	QoS class of giants and work behind the game (`utility`, `background`, `default`)	no switch	utility	Metal serves the game's own compiles first	helps: shown on a real thread; untested against `default` in the game	as it ships
`RX_VK_ALONE_TRIES`	tries alone per program, precompile's and play's (`0` none)	no switch	2 with the compiler service	a compile failed beside a giant is made, not lost	untested in the game (no failure in starts 21-24)	as it ships
`RX_VK_SKIP_LOG`	each dispatch left out named once per key and reason	no switch	off	log lines	helps: diagnosis	off; on for diagnosis
`RX_VK_METAL_ARCHIVE_DIR`	folder for MoltenVK's pipelines as Metal binary archives (macOS)	no switch	unset: unchanged	a start makes earlier starts' pipelines without compiling	helps: MC-01, MC-02; untested in a warm game start	off until measured in the game
`RX_VK_GIANT_ROOM_MB`	memory one giant compile is given behind the game (`0` no gate)	no switch	a quarter of memory, 16384 at most; the first giant starts unless the pressure is critical	giants wait for memory	untested in the game (never asked)	as it ships
`RX_VK_LMEM_LOG`	each module's private memory and each first bind logged (macOS)	no switch	off	log lines only	helps: ruled local memory out (runs #23, #24)	off; on for diagnosis
`RX_VK_LMEM_LOG_MIN`	the bytes a thread from which `RX_VK_LMEM_LOG` names a module	no switch	1024	which modules get a line	as above	1024
`RX_VK_HEAVY_WEIGHT`	local memory weight from which a compile is heavy and runs alone (`0` none by weight)	no switch	10000000	the four giant mesh programs compile alone	helps: GC-02, E9a (all made)	as it ships
`RX_VK_HEAVY_WORDS`	SPIR-V words from which a compile is heavy too (`0` none by size)	no switch	1000000	the 1.09M-word programs compile alone	untested: whether they fit alone	as it ships
`RX_VK_STALE_COMPILE_S`	seconds after which a non-heavy compile in flight no longer holds heavy ones and giants (`0` never)	no switch	240	a never-ending compile cannot starve the precompile	helps: E9a (7 stale-rule begins)	as it ships
`RX_VK_SLOW_FIRST_S`	seconds of an earlier run's compile from which it goes first (`0` none first)	no switch	240	44511d69 begins at the precompile's start	untested in the game	as it ships
`RX_VK_METAL_CACHE_DIR`	Metal's shader cache folder (macOS, from #653)	asked by the renderer, after SDL's window: lost in a windowed start	asked in `main()` before SDL	the switch applies in windowed starts too	helps: run #36	as it ships (off by default)

Testing

Tested: on macOS 26.7.1, Apple M4 Max, at this PR's head e3f55e1b on topic 4:

The members' own checks (their Sweep3 commands, 67, deduplicated) at the head, on arm64 and on x86_64 under Rosetta where the
member ran it (eight arm64-only): every one passes. Three failed at the pre-fix tip where two members met (test_vendor_fixes,
test_warm_handoff, test_giant_alone) and pass after the merge fixes above, both arches; test_metal_cache_dir's #667 form
(--headers) is #786's form now, which passes both arches. The ports' own checks: giant-starve's ten at 310fbb8e and
slow-first's test_slow_first at the head, both arches, pass (test_giant_alone after its fix). Logs:
reports/Consolidate-files/proof/c5-*, c5gs-*, c5sf-*, c5f-*, c5m-*.
The ports' whole files: vk_draw.cpp and vk_main.cpp compiled whole on arm64 and x86_64 with nx2/build.py's own macOS
commands at the pre-port tip, at 310fbb8e and at 7e1d702f: 0 failures; no undefined symbol added against the pre-port tip.
python -I -B tests/run_all.py at the head against main aa95d35f's 543/33/17 on this Mac: 615 passed, 15 failed, 17
skipped; by name nothing fails here that passes on main; 17 of main's failures pass (the macOS host's and caches' fixes from
topics 1-4), 54 new checks pass, test_appimage_package_retry is skipped on macOS (#454) and test_dav1d_av1 runs (#454);
the 15 failures fail on main too: test_appimage_player_build, test_cpm_discovery, test_crash_dialog, test_env_on,
test_game_layout, test_lift_cleanup_error, test_linux_preflight, test_make_game_cache, test_options_budget_once,
test_options_path, test_pipeline_tour, test_readback_late, test_split_giants, test_vk_pgo, test_walk_key
(reports/Consolidate-files/checks/compare-precompile-metal.txt).
The renderer's cross-file link check at the head (every mkwvk:: name declared in one gpu/vk file and defined in another,
each defining file compiled on both arches): 0 not exported, 0 undefined across the 33 objects of each arch.
Off Apple (unifdef -U__APPLE__ of every changed non-test C/C++ file, against topic 4): 5 files the same, 18 differ, 7 new,
1 gone, all of them the members' runtime-gated text, none a behaviour change off the Mac: #651's thread_stack.h include in
the files that start threads (thread_stack::thread is std::thread off macOS); the compiler-service, worker-cache and giant
code in vk_draw.cpp, vk_vendor.h, vk_shader.h, prepare_workers.h (#650: half the CPUs off macOS, as before) and the new
precompile_handoff.h, worker_caches.h and skip_log.h, behind vendor fixes that are false on every driver but MoltenVK
and KosmicKrisp (RX_VK_SKIP_LOG off by default everywhere); vk_progcache.cpp's private-memory and weight notes (#681,
#784, read only on macOS); the new metal_archive.h and private_memory.h; host/posix/darwin_metal_cache.h, macOS's own,
replacing metal_cache.h (#786).
In the game: E9a on I41 78ec1c47 (arm64, 1,197 s hold, 12/12 must-shows, clean end, 0 skipped draws), an integration build
that carries every member here and the giant-starve port; run #34 on I38 (x86_64 under Rosetta, 16 m 52 s, every draw drawn,
19/19, clean exit), which carried #650-#681 as they were then (not #784-#786 or the ports).

Not tested: Windows and Linux (no such machine here): every change is behind compiler_service (MoltenVK and KosmicKrisp)
or __APPLE__, but vk_draw.cpp, vk_progcache.cpp and the headers listed above compile there with new text. Slow-first's
port in a game run. A warm game start from Metal binary archives (RX_VK_METAL_ARCHIVE_DIR stays off). KosmicKrisp in a game
start.

Must be tested:

Windows and Linux, any GPU: build this PR (head e3f55e1b) and main aa95d35f the same way, run python -I -B tests/run_all.py on each and start each to the title, expecting the build to succeed, the new tests to pass with no other check changing status, the vk: vendor fixes line without compiler_service, and no new vk: precompile or vk: private memory line (Test Farm: @Beanstalk windows linux, build, run_all and start unchanged)

Risks

What could break: on MoltenVK and KosmicKrisp, the start's order: which compiles run when, how many giants run at once and
when the title comes; a rule held wrong could delay a program behind the game (none is dropped: each is built later or by its
first dispatch). On Windows and Linux nothing should change; the code is compiled there, gated off at run time.

Turning it off: RX_VK_COMPILER_SERVICE=0 (the precompile and preparation as on every other driver), RX_VK_WORKER_CACHES=0,
RX_VK_PCACHE_LOAD=1, the giant switches at 0, RX_VK_SLOW_FIRST_S=0, RX_VK_STALE_COMPILE_S=0; or revert the commits.

Compatibility: on MoltenVK the pipeline cache files are no longer read or written (RX_VK_PCACHE_LOAD=1 reads them); a new
file longshot_pipelines.bin.slow beside the program cache lists slow compiles. The vk ABI on macOS: MKWVK_API_VERSION 26 →
27 (#786's metal_cache_check host slot), so the host and the renderer of one macOS build go together; Windows and Linux stay at
23. No change to saves, longshot_options.ini or the program cache's format. Nothing changes for a Windows or Linux player.

Issues

See #420. The macOS topic PRs, merged in this order; "topic N" in each description means these: 1 #845 (macos-host), 2 #846 (vulkan-on-metal, on 1), 3 #847 (mvk-descriptor-limits), 4 #848 (shader-backend, on 2, 3 and #431), 5 #849 (precompile-metal, on 4), 6 #850 (draw-path, on 4), 7 #851 (draw-path-quads-mesh, on 6, #705 and #707), 8 #852 (mac-perf, on 7).

Checklist

Everyone

One topic.
The title is a Conventional Commit (type(scope): description) of the status line's type.
Why gives the reason plainly and links the issues that fit (the tracker searched).
Measurements has a verdict, or one line saying why there is nothing to measure.
Testing names what was tested, what wasn't, and what must still be tested, where and how. (Windows and Linux untested; the test request is not open yet.)
Rebased on current main: on aa95d35f, through topic 4 (shader-backend).
python -I -B tests/run_all.py passes (any failure that also fails on main is named). (On this Mac at the head: 615/15/17 against main aa95d35f's 543/33/17; the 15 failures, named under Testing, fail on main too; no check fails here that passes on main.)
No game files, dumps, build output or generated files.
third_party/ untouched (a new folder with a PROVENANCE.md instead).
The license is OK: GPL-3.0-or-later, nothing incompatible and nothing decompiled.

When it applies

AI assistance recorded with Assisted-by: / Co-Authored-By: trailers.
Switches documented (docs/development.md; the README for a menu row) and shipped at their recommended value.
MKWVK_API_VERSION bumped (gpu/vk/vk_api.h's MkwVkHost or MkwVkRenderer changed). (Apple 26 → 27 for the new host slot; Windows and Linux unchanged at 23.)
Host code builds on both Windows (llvm-mingw) and Linux, with UTF-8 paths. (Not built here: the Windows/Linux test request asks for it.)
A game-free test added for the fix, failing on main and passing here.
A check under tests/ that reads changed source updated in this PR, not weakened.
Docs updated.

**Type:** fix · **Risk:** medium · **Tested on:** macOS 26.7.1, Apple M4 Max (arm64; x86_64 under Rosetta) · **Needs:** Windows and Linux, any GPU ## Why On a Mac, Metal compiles every pipeline in its compiler service, a few processes the whole game shares, and the game asks it for about 42,000 pipelines and compute programs, a handful of them giants that take minutes and gigabytes each. Started as on any other GPU, the game crawled or never reached the title: the shader preparation compiled one pipeline at a time behind MoltenVK's one pipeline-cache lock (39 programs in its first 180 s); a renderer thread's 512 KiB stack overflowed in SPIRV-Cross and the game crashed; the precompile never let the title come while giants held every worker (run #12); the game's first frame waited minutes behind 16 giants on the compiler's connections; giants compiled together ran the service out of memory and died, so three of the four giant mesh programs were never made (run #31); a compile that failed beside a giant was lost for the session; and every start compiled again what Metal's own shader cache did not keep. This PR folds the macOS port's twelve precompile PRs into one: the preparation and precompile shaped for Metal's compiler service, on MoltenVK and KosmicKrisp only. It is part of the macOS port. See #420. ## How to reproduce **Seen on:** macOS 26.7.1, Apple M4 Max, MoltenVK 1.4.2 **Settings and switches:** defaults (the macOS port's stack without this PR: main does not start on macOS) **Steps:** start the game on topic 4 (`shader-backend`, this PR's base) from an empty program cache, then again from the one it left; watch `longshot_log.txt`'s `vk: preparation:` and `vk: precompile` lines and the first present **Expected:** the preparation and precompile finish within minutes, the title comes, the first frame follows within seconds, every program is made **Actual:** 39 of 38,809 programs prepared in 180 s; a SIGILL at about 93 s in SPIRV-Cross (start #6); the precompile stalls with no title (run #12: 4,600 items for 160 s, 7,698 for 80 s); the first frame about 3 minutes after the hand-off (start 17); service processes ended under the giants and three giant mesh programs not made (run #31) ## What changed Under the vendor fix `compiler_service` (on by default on MoltenVK and KosmicKrisp, off on every other driver), the preparation translates and keeps every program but compiles no compute pipeline, each worker with a pipeline cache of its own on every CPU; the precompile orders the work for Metal's compiler service, lets the title come while the last giants build behind the game, keeps at most four giants and one heavy compile in flight, holds giants until the first frame and while memory is short, tries a failed compile again alone, and puts a dispatch's compile ahead of them. An opt-in switch keeps MoltenVK's pipelines in Metal binary archives, and two log-only switches name each module's private memory and each slow compile. Every other driver runs the code it ran before (the off-Apple text changes are listed under Testing). One commit per original PR, in order, then the ports and the merge fixes: - `perf(vk)`: a pipeline cache per preparation worker, every CPU, and no saved pipeline cache on MoltenVK (`gpu/vk/worker_caches.h`, `gpu/vk/prepare_workers.h`, `vk_vendor.h` `worker_caches`/`pcache_no_load`; `RX_VK_WORKER_CACHES`, `RX_VK_PCACHE_LOAD`'s MoltenVK default). Was: [#650](https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/650) - `fix(macos)`: renderer threads start with a 16 MiB stack on macOS (`gpu/vk/thread_stack.h`, every thread the renderer starts; `std::thread` itself elsewhere). Was: [#651](https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/651) - `fix(macos)`: the precompile goes on behind the title on MoltenVK, slow compiles named, a compile the service lost tried once more (`gpu/vk/precompile_handoff.h`, the precompile in `vk_draw.cpp`, `vk_vendor.h` `compiler_service`; `RX_VK_COMPILER_SERVICE`, `RX_VK_SLOW_COMPILE_S`, `RX_VK_PRECOMPILE_STALL_S`). Was: [#652](https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/652) - `fix(macos)`: the preparation leaves MoltenVK's compute pipelines to the precompile, which keeps at most 4 giants in flight (`seed_compute` in `vk_shader.cpp`, the giant bound in `precompile_handoff.h`; `RX_VK_GIANT_COMPILES`, `RX_VK_GIANT_WORDS`). Was: [#664](https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/664) - `fix(macos)`: giants held behind the game until its first frame, at QoS class utility; a failed giant retried alone (`RX_VK_GIANT_HOLD_PRESENTS`, `RX_VK_GIANT_HOLD_S`, `RX_VK_COMPILE_QOS`). Was: [#665](https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/665) - `fix(macos)`: a dispatch's compute pipeline goes ahead of the giants and is retried alone, not failed for good (the compile board in `precompile_handoff.h`; `RX_VK_ALONE_TRIES`, `RX_VK_SKIP_LOG`, `gpu/vk/skip_log.h`). Was: [#666](https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/666) - `feat(macos)`: `RX_VK_METAL_ARCHIVE_DIR` keeps MoltenVK's pipelines in Metal binary archives, macOS only, off by default (`gpu/vk/metal_archive.h`, its hook in `vk_main.cpp`, all inside `#if defined(__APPLE__)`). Was: [#667](https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/667) - `fix(macos)`: giants start behind the game only while memory allows; a lost device said as one, with the system's memory (`gpu/vk/system_memory.h`, `RX_VK_GIANT_ROOM_MB`). Was: [#669](https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/669) - `feat(macos)`: `RX_VK_LMEM_LOG` names each module's private memory and each pipeline's first bind, log-only, macOS only (`gpu/vk/private_memory.h`, `vk_progcache.cpp`; `RX_VK_LMEM_LOG_MIN`). Was: [#681](https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/681) - `fix(vk)`: heavy compiles run alone under Metal's compiler service; the first giant behind the game no longer waits for 16 GB (`behind_worker` in `vk_draw.cpp`, `heavy_compile` in `precompile_handoff.h`, the local memory weight in `vk_progcache.cpp`; `RX_VK_HEAVY_WEIGHT`, `RX_VK_HEAVY_WORDS`). Was: [#784](https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/784) - `fix(macos)`: KosmicKrisp's compiles go through the compiler service's precompile, as MoltenVK's do (`vk_vendor.h`). Was: [#785](https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/785) - `fix(macos)`: Metal's shader cache folder (`RX_VK_METAL_CACHE_DIR`) asked at the host's start, before SDL's window (`host/posix/darwin_metal_cache.h`, `hos_main.c`, `rx_backend.c`). Was: [#786](https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/786) - Port, `fix(vk)`: a compile that never ends keeps the heavy compiles and the giants back for `RX_VK_STALE_COMPILE_S` (240 s) at most (`Board` and `Giants` in `precompile_handoff.h`). From `macos/giant-starve` 5292596d (reports/2026-11-15-giant-starve.md), placed in main's dispatch build (#252). - Port, `fix(vk)`: what an earlier run found slow for minutes compiles first, and is play's from the hand-off (`RX_VK_SLOW_FIRST_S`, `Giants::hand_over`). From `macos/slow-first` 40a86bae (reports/2026-11-15-slow-first.md). - Port, `docs(vk)`: `metal_archive.h`'s notes on Metal's cache and staging say only what runs and public notes showed. From `macos/integrated` 63fc3d2c; no code change. - Merge fixes (tests the members each passed alone, failing where two met here; each follows I41's fix for the same pair): test_giant_alone cuts main's dispatch build (the giant-starve port); test_warm_handoff finds `seed_compute`'s save with what #695's line adds (I41 1cbb594a); test_vendor_fixes runs its checks once, after every check is defined; and the merge of topic 4's fixture fix (test_vendor_fixes, test_subgroup_vertex). ## Measurements **Verdict:** fixes the Mac's start: with these changes the macOS stack reaches the title and plays, every program made. On I41 `78ec1c47` (every member here, before Sweep3's rebase, and the giant-starve port), E9a handed off 8 m 14 s after the start, made all four giant mesh programs and the four other heavy compiles, made 44511d69 (the tree leaves' program) 887 s into the hold, and held 1,197 s with 12 of 12 must-shows, 0 skipped draws and 0 crash reports. Each member measured its own part on the integration build of its day: - #650: 4.4× the compute and 4.6× the graphics pipelines a second in the preparation (16 workers, a cache each, against one shared cache: 45.41 against 10.35 compute a second), a later start's pipelines in 1.41 s compiled anew against 6.65 s through the saved cache, and the first start at 230 programs in 180 s against 39 (integration b7a7c95f run #2 against aefe6b0a run #4). - #651: start #6 died with SIGILL at about 93 s; starts #7-#9 with the 16 MiB stack ended cleanly, #8 at 1,011 programs. - #652: the title came (run #17: hand-off at 1,308 s with 42,043 of 42,135 done) where run #12 stalled with none. - #664: the preparation 118.3 s with 0 compute pipelines compiled in it (run #18) against 1,550.4 s compiling 1,381 (run #16). - #665: the first frame 5.0 s after the hand-off (start 21) against about 188-198 s (start 17); 13 service crashes against 6. - #666 and #669: nothing to compare in their starts (no compile failed beside a giant; the memory gate was never asked); their game-free tests show the rules, and #669's lost-device line named the memory at start 23's loss. - #667: in probe MC-02, 61 of 61 pipelines from the archives in the second process; the switch stays off (no warm game start from archives measured). - #681: two starts showed no program asks more than 9,359 B a thread, which ruled local memory out of the memory growth. - #784: in probe GC-02 the 464k giants' twin was made alone (1.07 GB of one service process); the giants are made in E9a. - #785: probe KK-03 measured KosmicKrisp's compiles at 1.0-5.8× MoltenVK's, growing with size (no KosmicKrisp game start yet). - #786: a windowed start with `RX_VK_METAL_CACHE_DIR` from start to hand-off in 6 m 53 s (run #36) against 24 m 28 s (run #35). - Giant-starve: E9a's 7 stale-compile begin lines, all eight heavy compiles made beside 44511d69 (I41 carries it). - Slow-first: not in a game run yet (its prediction: 44511d69 made about 280-660 s after the hand-off in a warm start instead of 932-1,031 s; reports/2026-11-15-slow-first.md). Hardware: macOS 26.7.1 (25G241), Apple M4 Max, 64 GB; MoltenVK 1.4.2, and MoltenVK main 5189f8de with the argbuf-slots patch from run #31 on Scene: the title and the field-dawn save's first area, windowed; LS's default settings Method: each run's `longshot_log.txt` (the `vk: preparation:`, `vk: precompile`, hand-off, slow-compile and close lines, the FPS counter's Dev level in the hold); one start each, so the counts and times are what each start reached, not an interleaved median How to reproduce: the macOS runner (`tools/ls-run.py`) on an integration build, a seeded or empty program cache as named, START and A at the title Baseline: main does not start on a Mac, so the Before column is integration a7711f3c (run #12), the macOS stack with #650 and #651 and none of #652-#786; the After column is I41 78ec1c47 (E9a), which carries every member (as before Sweep3's rebase onto aa95d35f) and the giant-starve port, not slow-first's port or the comment rewording. The two starts' program caches were not the same. | | Before: integration @ a7711f3c (run #12) | After: integration @ 78ec1c47 (E9a) | |---|---|---| | precompile → title | stalled (4,600 items for about 160 s, 7,698 for about 80 s); no title at the quit (412 s) | hand-off 8 m 14 s after the start; play 48 s later | | the four giant mesh programs and four other heavies | not reached | all eight made (5096bcc7 50.2 s, 6dbfda5a 29.9 s, 5c945415 34.1 s, 55d1da3f 34.1 s) | | 44511d69 (the tree leaves) | not reached | made in 895.6 s, 887 s into the hold | | hold | none | 1,197 s, 12/12 must-shows, 0 skipped draws, 0 crash reports, 37.0 fps | ``` E9a (mac-arm64-78ec1c47-01): precompile 479.0 s, hand-off 07:23:28.6 (8 m 14 s), play 07:24:16, hold 07:24:25.3-07:44:22.6 giants: 5096bcc7 50.2 s (began alone at the hand-off), 6dbfda5a 29.9, 5c945415 34.1, 55d1da3f 34.1; 44511d69 895.6 s must-shows 12/12; 7 stale-compile begin lines; 0 XPC errors; first 600 s 37.0 fps / 27.03 ms / GPU 19.4 run #12 (a7711f3c): precompile stalled at 4,600 for about 160 s and at 7,698 for about 80 s; 10,043 of 42,234 at the quit (412 s) ``` ### Flags and settings Eighteen new switches and two new defaults, every one read on MoltenVK or KosmicKrisp only (most through `compiler_service`), or macOS only; nothing changes on another driver. `HOS_HEADLESS`, `RX_VK_NO_WARM`, `RX_VK_PCACHE_FETCH_LOCK` and `RX_VK_SKIP_PROG` appear in the diff as main's switches, read as before. | Flag | What it is | Before this PR | After this PR | Effect | Helps / doesn't help / untested | Recommendation | |---|---|---|---|---|---|---| | `RX_VK_WORKER_CACHES` | a pipeline cache per preparation worker (`=0` one shared, `=1` on any driver with pipelineCreationCacheControl) | no switch | on for MoltenVK only | the preparation's compiles run side by side | helps: MoltenVK 4.4×-4.6× | as it ships | | `RX_VK_PCACHE_LOAD` | the pipeline cache file read and saved (`=0` neither, `=1` both) | read and saved everywhere | on MoltenVK neither, elsewhere as before | a later MoltenVK start compiles anew, faster than reading | helps: MoltenVK (1.41 s against 6.65 s) | as it ships | | `RX_VK_COMPILER_SERVICE` | the compiler-service path (`=0` as on other drivers, `=1` on any) | no switch | on for MoltenVK and KosmicKrisp | preparation without compute pipelines, the precompile's hand-off, order, bounds and retries | helps: MoltenVK (runs #17-#36, E9a) | as it ships; `=0` only to compare | | `RX_VK_SLOW_COMPILE_S` | seconds after which a compile is named and noted slow (`0` none) | no switch | 10 with the compiler service | log lines, the slow list | helps: naming the giants | as it ships | | `RX_VK_PRECOMPILE_STALL_S` | seconds with nothing finished before the game starts | 30, once every item is handed out | 30, whatever is left with the compiler service | the title is not held by a few giants | helps: MoltenVK | as it ships | | `RX_VK_GIANT_COMPILES` | giant compiles in flight at once (`0` no bound) | no switch | 4 with the compiler service | small items finish while giants compile | helps: MoltenVK (runs #18, #19) | as it ships | | `RX_VK_GIANT_WORDS` | SPIR-V words above which a compile is a giant | no switch | 100000 | which compiles the bound holds | helps: MoltenVK; other values untested | as it ships | | `RX_VK_GIANT_HOLD_PRESENTS` | frames after the hand-off before a giant starts behind the game (`0` no hold) | no switch | 1 | the first submit finds the compiler free of new giants | helps: MoltenVK (starts 21-24) | as it ships | | `RX_VK_GIANT_HOLD_S` | the hold's longest time | no switch | 120 | bounds the hold | untested beyond the default | as it ships | | `RX_VK_COMPILE_QOS` | QoS class of giants and work behind the game (`utility`, `background`, `default`) | no switch | utility | Metal serves the game's own compiles first | helps: shown on a real thread; untested against `default` in the game | as it ships | | `RX_VK_ALONE_TRIES` | tries alone per program, precompile's and play's (`0` none) | no switch | 2 with the compiler service | a compile failed beside a giant is made, not lost | untested in the game (no failure in starts 21-24) | as it ships | | `RX_VK_SKIP_LOG` | each dispatch left out named once per key and reason | no switch | off | log lines | helps: diagnosis | off; on for diagnosis | | `RX_VK_METAL_ARCHIVE_DIR` | folder for MoltenVK's pipelines as Metal binary archives (macOS) | no switch | unset: unchanged | a start makes earlier starts' pipelines without compiling | helps: MC-01, MC-02; untested in a warm game start | off until measured in the game | | `RX_VK_GIANT_ROOM_MB` | memory one giant compile is given behind the game (`0` no gate) | no switch | a quarter of memory, 16384 at most; the first giant starts unless the pressure is critical | giants wait for memory | untested in the game (never asked) | as it ships | | `RX_VK_LMEM_LOG` | each module's private memory and each first bind logged (macOS) | no switch | off | log lines only | helps: ruled local memory out (runs #23, #24) | off; on for diagnosis | | `RX_VK_LMEM_LOG_MIN` | the bytes a thread from which `RX_VK_LMEM_LOG` names a module | no switch | 1024 | which modules get a line | as above | 1024 | | `RX_VK_HEAVY_WEIGHT` | local memory weight from which a compile is heavy and runs alone (`0` none by weight) | no switch | 10000000 | the four giant mesh programs compile alone | helps: GC-02, E9a (all made) | as it ships | | `RX_VK_HEAVY_WORDS` | SPIR-V words from which a compile is heavy too (`0` none by size) | no switch | 1000000 | the 1.09M-word programs compile alone | untested: whether they fit alone | as it ships | | `RX_VK_STALE_COMPILE_S` | seconds after which a non-heavy compile in flight no longer holds heavy ones and giants (`0` never) | no switch | 240 | a never-ending compile cannot starve the precompile | helps: E9a (7 stale-rule begins) | as it ships | | `RX_VK_SLOW_FIRST_S` | seconds of an earlier run's compile from which it goes first (`0` none first) | no switch | 240 | 44511d69 begins at the precompile's start | untested in the game | as it ships | | `RX_VK_METAL_CACHE_DIR` | Metal's shader cache folder (macOS, from #653) | asked by the renderer, after SDL's window: lost in a windowed start | asked in `main()` before SDL | the switch applies in windowed starts too | helps: run #36 | as it ships (off by default) | ## Testing **Tested:** on macOS 26.7.1, Apple M4 Max, at this PR's head `e3f55e1b` on topic 4: - The members' own checks (their Sweep3 commands, 67, deduplicated) at the head, on arm64 and on x86_64 under Rosetta where the member ran it (eight arm64-only): every one passes. Three failed at the pre-fix tip where two members met (test_vendor_fixes, test_warm_handoff, test_giant_alone) and pass after the merge fixes above, both arches; test_metal_cache_dir's #667 form (`--headers`) is #786's form now, which passes both arches. The ports' own checks: giant-starve's ten at `310fbb8e` and slow-first's test_slow_first at the head, both arches, pass (test_giant_alone after its fix). Logs: `reports/Consolidate-files/proof/c5-*`, `c5gs-*`, `c5sf-*`, `c5f-*`, `c5m-*`. - The ports' whole files: `vk_draw.cpp` and `vk_main.cpp` compiled whole on arm64 and x86_64 with nx2/build.py's own macOS commands at the pre-port tip, at `310fbb8e` and at `7e1d702f`: 0 failures; no undefined symbol added against the pre-port tip. - `python -I -B tests/run_all.py` at the head against main aa95d35f's 543/33/17 on this Mac: 615 passed, 15 failed, 17 skipped; by name nothing fails here that passes on main; 17 of main's failures pass (the macOS host's and caches' fixes from topics 1-4), 54 new checks pass, `test_appimage_package_retry` is skipped on macOS (#454) and `test_dav1d_av1` runs (#454); the 15 failures fail on main too: test_appimage_player_build, test_cpm_discovery, test_crash_dialog, test_env_on, test_game_layout, test_lift_cleanup_error, test_linux_preflight, test_make_game_cache, test_options_budget_once, test_options_path, test_pipeline_tour, test_readback_late, test_split_giants, test_vk_pgo, test_walk_key (`reports/Consolidate-files/checks/compare-precompile-metal.txt`). - The renderer's cross-file link check at the head (every `mkwvk::` name declared in one gpu/vk file and defined in another, each defining file compiled on both arches): 0 not exported, 0 undefined across the 33 objects of each arch. - Off Apple (`unifdef -U__APPLE__` of every changed non-test C/C++ file, against topic 4): 5 files the same, 18 differ, 7 new, 1 gone, all of them the members' runtime-gated text, none a behaviour change off the Mac: #651's `thread_stack.h` include in the files that start threads (`thread_stack::thread` is `std::thread` off macOS); the compiler-service, worker-cache and giant code in `vk_draw.cpp`, `vk_vendor.h`, `vk_shader.h`, `prepare_workers.h` (#650: half the CPUs off macOS, as before) and the new `precompile_handoff.h`, `worker_caches.h` and `skip_log.h`, behind vendor fixes that are false on every driver but MoltenVK and KosmicKrisp (`RX_VK_SKIP_LOG` off by default everywhere); `vk_progcache.cpp`'s private-memory and weight notes (#681, #784, read only on macOS); the new `metal_archive.h` and `private_memory.h`; `host/posix/darwin_metal_cache.h`, macOS's own, replacing `metal_cache.h` (#786). - In the game: E9a on I41 `78ec1c47` (arm64, 1,197 s hold, 12/12 must-shows, clean end, 0 skipped draws), an integration build that carries every member here and the giant-starve port; run #34 on I38 (x86_64 under Rosetta, 16 m 52 s, every draw drawn, 19/19, clean exit), which carried #650-#681 as they were then (not #784-#786 or the ports). **Not tested:** Windows and Linux (no such machine here): every change is behind `compiler_service` (MoltenVK and KosmicKrisp) or `__APPLE__`, but `vk_draw.cpp`, `vk_progcache.cpp` and the headers listed above compile there with new text. Slow-first's port in a game run. A warm game start from Metal binary archives (`RX_VK_METAL_ARCHIVE_DIR` stays off). KosmicKrisp in a game start. **Must be tested:** - Windows and Linux, any GPU: build this PR (head `e3f55e1b`) and main aa95d35f the same way, run `python -I -B tests/run_all.py` on each and start each to the title, expecting the build to succeed, the new tests to pass with no other check changing status, the `vk: vendor fixes` line without `compiler_service`, and no new `vk: precompile` or `vk: private memory` line (Test Farm: @Beanstalk windows linux, build, run_all and start unchanged) ## Risks **What could break:** on MoltenVK and KosmicKrisp, the start's order: which compiles run when, how many giants run at once and when the title comes; a rule held wrong could delay a program behind the game (none is dropped: each is built later or by its first dispatch). On Windows and Linux nothing should change; the code is compiled there, gated off at run time. **Turning it off:** `RX_VK_COMPILER_SERVICE=0` (the precompile and preparation as on every other driver), `RX_VK_WORKER_CACHES=0`, `RX_VK_PCACHE_LOAD=1`, the giant switches at `0`, `RX_VK_SLOW_FIRST_S=0`, `RX_VK_STALE_COMPILE_S=0`; or revert the commits. **Compatibility:** on MoltenVK the pipeline cache files are no longer read or written (`RX_VK_PCACHE_LOAD=1` reads them); a new file `longshot_pipelines.bin.slow` beside the program cache lists slow compiles. The vk ABI on macOS: `MKWVK_API_VERSION` 26 → 27 (#786's `metal_cache_check` host slot), so the host and the renderer of one macOS build go together; Windows and Linux stay at 23. No change to saves, `longshot_options.ini` or the program cache's format. Nothing changes for a Windows or Linux player. ## Issues See #420. The macOS topic PRs, merged in this order; "topic N" in each description means these: 1 #845 (macos-host), 2 #846 (vulkan-on-metal, on 1), 3 #847 (mvk-descriptor-limits), 4 #848 (shader-backend, on 2, 3 and #431), 5 #849 (precompile-metal, on 4), 6 #850 (draw-path, on 4), 7 #851 (draw-path-quads-mesh, on 6, #705 and #707), 8 #852 (mac-perf, on 7). ## Checklist ### Everyone - [x] One topic. - [x] The title is a Conventional Commit (`type(scope): description`) of the status line's type. - [x] Why gives the reason plainly and links the issues that fit (the tracker searched). - [x] Measurements has a verdict, or one line saying why there is nothing to measure. - [ ] Testing names what was tested, what wasn't, and what must still be tested, where and how. (Windows and Linux untested; the test request is not open yet.) - [x] Rebased on current `main`: on aa95d35f, through topic 4 (`shader-backend`). - [x] `python -I -B tests/run_all.py` passes (any failure that also fails on `main` is named). (On this Mac at the head: 615/15/17 against main aa95d35f's 543/33/17; the 15 failures, named under Testing, fail on main too; no check fails here that passes on main.) - [x] No game files, dumps, build output or generated files. - [x] `third_party/` untouched (a new folder with a `PROVENANCE.md` instead). - [x] The license is OK: GPL-3.0-or-later, nothing incompatible and nothing decompiled. ### When it applies - [x] AI assistance recorded with `Assisted-by:` / `Co-Authored-By:` trailers. - [x] Switches documented (docs/development.md; the README for a menu row) and shipped at their recommended value. - [x] `MKWVK_API_VERSION` bumped (`gpu/vk/vk_api.h`'s `MkwVkHost` or `MkwVkRenderer` changed). (Apple 26 → 27 for the new host slot; Windows and Linux unchanged at 23.) - [ ] Host code builds on both Windows (llvm-mingw) and Linux, with UTF-8 paths. (Not built here: the Windows/Linux test request asks for it.) - [x] A game-free test added for the fix, failing on `main` and passing here. - [x] A check under `tests/` that reads changed source updated in this PR, not weakened. - [x] Docs updated.

mooki added 68 commits

2026-11-15 17:40:02 +00:00

fix(vk): scan the import table as far back as its largest live import e968b6d5c0

Every scan down vk_memory.cpp's import table by start (pool_lookup's
covering import and merge plan, bg_drain, prefetch_chunks,
import_in_background, memory_note_gpu_write and memory_guest_imported)
stopped once an import ended more than 1 GiB below its address, on the
belief that no import is larger than that. pool_lookup's own merge escape
leaves an import above 1 GiB over the smaller ones inside it ("left
overlapping"), and the device allows imports up to max_import_size: a scan
stopped at a small import inside a large one, missed the large one, and
the same bytes were imported again, two memories over the same bytes.

import_reach.h counts the live imports by span class; its bound is above
every live span, within twice the largest. imported_add and imported_take,
now the table's only ways in and out, keep the count. A scan stops only
where no import further down can reach its address, so it also stops
sooner than before when the imports are small. The covering-import checks
in pool_lookup and import_in_background look past the last import at or
below the range for the same reason.

Tests: test_import_reach.py runs the actual scans with a 4 GiB import over
a 16 MiB chunk, left by the merge escape (imports are recorded ranges: no
payload, no Vulkan, no game). Its lookup, query, note, drain, background
and prefetch cases fail on main 5e072fc8 and pass here, and a source check
keeps every change to the table in the two helpers. test_import_windows
runs pool_lookup with the helpers; test_mirror_gpu_only names the new
publication point.

Assisted-by: Claude Opus 5.5
Co-Authored-By: Claude Opus 5.5 <[email protected]>

test(vk): the readback refresh fixture runs memory_note_gpu_write with the import table's helpers

CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Has been skipped

Details

CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Has been skipped

Details

CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Has been skipped

Details

Third-party parts / The same as upstream (pull_request) Has been skipped

Details

Website / Website on Cloudflare (pull_request) Has been skipped

Details

CI / Conventional Commit title (pull_request) Successful in 4s

Details

CI / Workflow lint (actionlint) (pull_request) Successful in 7s

Details

Guide screenshots / Guide images and local UI source review (pull_request) Failing after 6s

Details

Third-party parts / As their notes say (pull_request) Successful in 10s

Details

PR description / Description follows the template (pull_request_target) Successful in 5s

Details

497be2d480

test_parallel_rb_refresh cuts memory_note_gpu_write out of vk_memory.cpp,
whose scan now stops at the live imports' reach: the fixture includes
import_reach.h, cuts live_reach, imported_add and imported_take beside it,
and puts its pool in the table through imported_add. It failed to build
at 11e9ee44 ("use of undeclared identifier 'import_reach'").

Assisted-by: Claude Opus 5.5
Co-Authored-By: Claude Opus 5.5 <[email protected]>

test: teach the game-free runner about macOS e81d4af47a

Was: #454 (#454), branch mac/run-all at 264f826a7c (Sweep3, on main aa95d35f).

Its own commits, squashed:
- test: teach the game-free runner about macOS (45c54bc6f)
- test: skip main's new Windows-only guard check on macOS, and undo the SDK's vsnprintf macro in a fixture (264f826a7)

Assisted-by: Claude Opus 5.5
Co-Authored-By: Claude Opus 5.5 <[email protected]>

fix(runtime): compile host jumps and pause hints on macOS arm64 441182e2c2

Was: #477 (#477), branch mac/arm64-host at 0f021f29c1 (Sweep3, on main aa95d35f).

Its own commits, squashed:
- fix(runtime): compile host jumps and pause hints on macOS arm64 (0f021f29c)

Assisted-by: Claude Opus 5.5
Co-Authored-By: Claude Opus 5.5 <[email protected]>

fix(runtime): preserve guest pages when discarding on macOS 10e52402b7

Was: #478 (#478), branch mac/guest-memory at dfb71ec850 (Sweep3, on main aa95d35f).

Its own commits, squashed:
- fix(runtime): preserve guest pages when discarding on macOS (b668404c8)
- fix(runtime): retain guest-page rounding in Darwin memory wrappers (e9c77d9f3)
- test(runtime): cut main's int hos_commit in the Darwin guest-memory test (7e737ce0f)
- test(runtime): follow main's HLE call counts in the Darwin guest-memory fixture (dfb71ec85)

Assisted-by: Claude Opus 5.5
Co-Authored-By: Claude Opus 5.5 <[email protected]>

fix(runtime): translate macOS fault contexts on both CPU routes 9784b1b5a6

Was: #436 (#436), branch mac/fault-contexts at bccfd1c2e9 (Sweep3, on main aa95d35f).

Its own commits, squashed:
- fix(runtime): translate macOS fault contexts on both CPU routes (08940fb3d)
- test(runtime): keep Darwin fault checks native by default (7d7f99dcb)
- test(runtime): classify the fault bridge's fatal _exit in main's exit inventory (bccfd1c2e)

Assisted-by: Claude Opus 5.5
Co-Authored-By: Claude Opus 5.5 <[email protected]>

fix(runtime): add macOS waits and deadlines, shared by host and renderer a492f0ac52

Was: #437 (#437), branch mac/waits at beb8a2881a (Sweep3, on main aa95d35f).

Its own commits, squashed:
- fix(runtime): port Win32 waits and deadlines to macOS (2f05d007f)
- test(runtime): link Darwin waits into the pointer fixture (ef081688d)
- fix(macos): share address wait state through the executable (186a461c5)
- docs(runtime): describe the executable-owned macOS address-wait state (beb8a2881)

Assisted-by: Claude Opus 5.5
Co-Authored-By: Claude Opus 5.5 <[email protected]>

feat(vk): add macOS Vulkan loader and device admission 58bdbdc027

Was: #438 (#438), branch mac/vk-admission at 681e37081d (Sweep3, on main aa95d35f).

Its own commits, squashed:
- feat(vk): add macOS Vulkan loader and device admission (6f34d00bc)
- test(vk): use semantic admission contracts (fdd161190)
- test(vk): follow main's LoadLibraryA and float-add choice in the admission fixture (3e088644f)
- test(vk): pin the admission ABI at main's version 23 and macOS 24 (4f4ca3c73)
- fix(macos): a window without Vulkan extensions ends as a clean exit, as main's exit inventory asks (681e37081)

Assisted-by: Claude Opus 5.5
Co-Authored-By: Claude Opus 5.5 <[email protected]>

feat(macos): native Darwin threads, fibers, restart, process I/O and profiler 6e36141173

Was: #490 (#490), branch mac/darwin-host at 162a39dcd3 (Sweep3, on main aa95d35f).

It was stacked on #436, #437, #438, #454, #477, #478; their changes are the earlier commits here.

Its own commits, squashed:
- feat(macos): complete Darwin host lifecycle and diagnostics (71a9ed0bb)
- fix(macos): integrate profile consumers and joined-thread shutdown (f420761ab)
- docs(macos): describe Darwin profile symbol loading (04e40191a)
- test(macos): integration fix: cut the tagged HThread in the waits fixture (9c18e64c4)
- test(macos): integration fix: stub the kept startup in the display fixture (5f8d511e6)
- test(macos): integration fix: prof symbols run on macOS, not a skip (a1e73d31e)
- test(macos): the memory-notice fixture runs the Darwin query (f4d7aaf76)
- fix(macos): a failed swap query still shows the low-RAM notice (28ed288a2)
- test(macos): the Darwin LoadLibraryA keeps dlerror()'s text for hos_load_error (52faf1d3d)
- fix(macos): integration fix: main's process affinity for new threads is Linux-only (61ec2411b)
- fix(macos): the POSIX main's end after the game's thread marks the clean end, as main's exit inventory asks (162a39dcd)

Assisted-by: Claude Opus 5.5
Co-Authored-By: Claude Opus 5.5 <[email protected]>

build(macos): discover dual-architecture toolchains and form Mach-O commands f68637f873

Was: #453 (#453), branch mac/build-flags at 8ca4bff7a4 (Sweep3, on main aa95d35f).

Its own commits, squashed:
- build(macos): discover dual-architecture toolchains and form Mach-O commands (499e3853a)
- test(macos): default real link fixtures to the process architecture (f6f6f042c)
- test(macos): isolate synthetic command paths from the test host (7c846ae91)
- build(macos): keep third_party/a64-kit as it is; nx2/darwin_toolchain.py makes its commands Mach-O (88054c966)
- build(macos): follow main's one-way renderer build and CPU target in the moved build code (a5b02633d)
- test(macos): the frozen Windows/Linux build arrays are main e090cc48's (c3009e01c)
- test: main's third-party objects test reads build.py with this PR's macOS names (8ca4bff7a)

Assisted-by: Claude Opus 5.5
Co-Authored-By: Claude Opus 5.5 <[email protected]>

build(macos): classify Darwin helpers and package a relocatable macOS 14.4 dylib set 9a1094dac0

Was: #479 (#479), branch mac/build-complete at 1bd269c7e2 (Sweep3, on main aa95d35f).

It was stacked on #437, #453; their changes are the earlier commits here.

Its own commits, squashed:
- build(macos): complete strict standalone dependency packaging (9424b4886)
- fix(macos): bind SDL3, imports and archive members to their real formats (8ad0e3265)
- fix(macos): keep the lazy-bind library ordinal across DONE (48f6ec02b)
- build(macos): take the architecture and SDK from darwin_toolchain when packaging (5ce85548c)
- build(macos): no prebuilt renderer on macOS either, as main has none (1bd269c7e)

Assisted-by: Claude Opus 5.5
Co-Authored-By: Claude Opus 5.5 <[email protected]>

fix(builder): account for available memory and compiler children on macOS 2540028349

Was: #455 (#455), branch mac/mem-gate at 82c2e10341 (Sweep3, on main aa95d35f).

Its own commits, squashed:
- fix(builder): account for available memory and compiler children on macOS (8f8e03b7f)
- fix(builder): avoid stale Darwin available-memory snapshots (5240e431a)
- fix(builder): keep Darwin RSS behind the shared tree backend (82c2e1034)

Assisted-by: Claude Opus 5.5
Co-Authored-By: Claude Opus 5.5 <[email protected]>

fix(runtime): support control bridge Unix sockets on macOS 1b3354a0e7

Was: #456 (#456), branch mac/control-sockets at fc2f104050 (Sweep3, on main aa95d35f).

Its own commits, squashed:
- fix(runtime): support control bridge Unix sockets on macOS (5fc3fe390)
- test(control): link the bridge's macOS socket helpers into test_oot_state_tap's fixture (fc2f10405)

Assisted-by: Claude Opus 5.5
Co-Authored-By: Claude Opus 5.5 <[email protected]>

test(control): wait until the bridge listens, and reap it when a quit check fails 68d940cae0

Was: #457 (#457), branch mac/control-quit at fe3e8c9178 (Sweep3, on main aa95d35f).

It was stacked on #456; their changes are the earlier commits here.

Its own commits, squashed:
- test(control): quit's parent waits for its bridge to listen, and a failed check ends the bridge it forked (fe3e8c917)

Assisted-by: Claude Opus 5.5
Co-Authored-By: Claude Opus 5.5 <[email protected]>

fix(vk): preserve cache file reservations on macOS eb8d5573f6

Was: #476 (#476), branch mac/cache-files at 7e561fbb53 (Sweep3, on main aa95d35f).

Its own commits, squashed:
- fix(vk): preserve cache file reservations on macOS (7e561fbb5)

Assisted-by: Claude Opus 5.5
Co-Authored-By: Claude Opus 5.5 <[email protected]>

fix(macos): serialize guest memory ownership on 16 KiB pages and pin imported backing 570a527793

Was: #480 (#480), branch mac/guest-16k at 657b780359 (Sweep3, on main aa95d35f).

It was stacked on #436, #478; their changes are the earlier commits here.

Merged with the topic's earlier commits by hand where they met (Consolidate-files/hand-resolutions.md).

Its own commits, squashed:
- fix(macos): serialize guest backing and native-page watch lifetimes (d7d999b92)
- fix(macos): retain pending watch faults across protection changes (a6a3e9e98)
- fix(macos): make the guest-memory changes Apple-only; Windows and Linux keep main's code (126d7bd67)
- docs(macos): describe the guest window's one owner, import pins and native-page watches (657b78035)

Assisted-by: Claude Opus 5.5
Co-Authored-By: Claude Opus 5.5 <[email protected]>

fix(macos): build vk_programs, vk_shader and the debug tools on Darwin 98c070dcfd

Was: #492 (#492), branch mac/renderer-compile at 59f750aa38 (Sweep3, on main aa95d35f).

Its own commits, squashed:
- fix(macos): the renderer's last two files compile on Darwin (7ee5fd584)
- fix(macos): the debug tools are compiled on Darwin too (82b2d6606)
- test(macos): the debug tools check reads hos_oot_player_seen's body too (59f750aa3)

Assisted-by: Claude Opus 5.5
Co-Authored-By: Claude Opus 5.5 <[email protected]>

fix(runtime): build and keep crash records on macOS e5a1034360

Was: #511 (#511), branch mac/crash-records at c4d95017aa (Sweep3, on main aa95d35f).

It was stacked on #436; their changes are the earlier commits here.

Its own commits, squashed:
- fix(runtime): take the macOS report run id from getentropy (d3e86ddd7)
- fix(runtime): read crash-record memory and modules without /proc on macOS (aa039e142)
- test(runtime): build the crash-record fixture on macOS (adebf9dbb)
- test(runtime): link the macOS fault bridge into the crash-record fixture (c4d95017a)

Assisted-by: Claude Opus 5.5
Co-Authored-By: Claude Opus 5.5 <[email protected]>

fix(control): keep command-line sessions alive on macOS

CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Has been skipped

Details

CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Has been skipped

Details

CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Has been skipped

Details

Third-party parts / The same as upstream (pull_request) Has been skipped

Details

Website / Website on Cloudflare (pull_request_target) Has been skipped

Details

CI / Conventional Commit title (pull_request) Successful in 14s

Details

CI / Workflow lint (actionlint) (pull_request) Successful in 19s

Details

Guide screenshots / Guide images and local UI source review (pull_request) Failing after 19s

Details

Third-party parts / As their notes say (pull_request) Successful in 30s

Details

PR description / Description follows the template (pull_request_target) Successful in 19s

Details

40c1c5f5f8

Was: #516 (#516), branch mac/longshot-ctl-darwin at 3b9d3cfa2a (Sweep3, on main aa95d35f).

Its own commits, squashed:
- fix(control): tell macOS processes apart by their sysctl start time (56f4ca876)
- test(control): compare launch's data folder with the resolved run folder (3b9d3cfa2)

Assisted-by: Claude Opus 5.5
Co-Authored-By: Claude Opus 5.5 <[email protected]>

fix(macos): pin Vulkan host imports until their last owner, and unwind failed startups c1fa25a4b2

Was: #491 (#491), branch mac/renderer-correctness at 695f6859dc (Sweep3, on main aa95d35f).

It was stacked on #438, #480; their changes are the earlier commits here.

Its own commits, squashed:
- fix(macos): pin renderer imports through final resource ownership (db82a0f23)
- test(macos): cut the non-Apple Pool and import route in three more fixtures (6cee76eb1)
- fix(vk): drain the recorder before startup rollback destroys the allocator (18c05be52)
- test(macos): follow main's #324 and #250 in the Darwin renderer fixture (c099f5da6)
- test(vk): follow the moved Pool in test_mirror_gpu_only's cut (165bf7e61)
- fix(macos): the import pin callbacks are macOS host API version 25 (fa691b55b)
- fix(macos): vk_memory.cpp compiles on macOS: #324's evicted_pools declaration is not Apple's (695f6859d)

Assisted-by: Claude Opus 5.5
Co-Authored-By: Claude Opus 5.5 <[email protected]>

fix(macos): on unified memory the GPU's allowance is what the Mac's RAM leaves, not Metal's working set 45e1d66607

Was: #642 (#642), branch mac/gpu-budget at 09978824a0 (Sweep3, on main aa95d35f).

Its own commits, squashed:
- feat(vk): read the Mac's system memory for the renderer (gpu/vk/system_memory.h) (ba1952096)
- fix(macos): on unified memory the GPU's allowance is what the Mac's RAM leaves, not Metal's working set (632636ee0)
- fix(macos): vk_work.h stays vk_vma.cpp's last include (0e63a1dc1)
- fix(macos): the unified budget reads the system's memory at its first ask (5240eade7)
- docs(macos): RX_VK_UNIFIED_BUDGET in the switch table (09978824a)

Assisted-by: Claude Opus 5.5
Co-Authored-By: Claude Opus 5.5 <[email protected]>

feat(macos): RX_VK_METAL_CACHE_DIR keeps Metal's shader cache in a chosen folder 01630d4b6e

Was: #653 (#653), branch mac/metal-cache at db50cf3d26 (Sweep3, on main aa95d35f).

It was stacked on #438; their changes are the earlier commits here.

Its own commits, squashed:
- feat(macos): Metal's shader cache in a folder the caller names (db50cf3d2)

Assisted-by: Claude Opus 5.5
Co-Authored-By: Claude Opus 5.5 <[email protected]>

fix(vk): give storage buffer tables descriptor sets instead of pushes on MoltenVK 063d87c431

Was: #643 (#643), branch mac/mvk-push at ca17df3be6 (Sweep3, on main aa95d35f).

Its own commits, squashed:
- fix(macos): storage buffer tables get descriptor sets on MoltenVK (ca17df3be)

Assisted-by: Claude Opus 5.5
Co-Authored-By: Claude Opus 5.5 <[email protected]>

feat(macos): ship the Vulkan loader with KosmicKrisp and MoltenVK, MoltenVK the default

CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Has been skipped

Details

CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Has been skipped

Details

CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Has been skipped

Details

Third-party parts / The same as upstream (pull_request) Has been skipped

Details

Website / Website on Cloudflare (pull_request_target) Has been skipped

Details

CI / Conventional Commit title (pull_request) Successful in 14s

Details

CI / Workflow lint (actionlint) (pull_request) Successful in 19s

Details

Guide screenshots / Guide images and local UI source review (pull_request) Failing after 17s

Details

Third-party parts / As their notes say (pull_request) Successful in 31s

Details

PR description / Description follows the template (pull_request_target) Successful in 22s

Details

7df74b124d

Was: #654 (#654), branch mac/kosmickrisp at 9d34f9a1c8 (Sweep3, on main aa95d35f).

It was stacked on #438, #479; their changes are the earlier commits here.

Merged with the topic's earlier commits by hand where they met (Consolidate-files/hand-resolutions.md).

Its own commits, squashed:
- feat(macos): ship the Vulkan loader with KosmicKrisp and MoltenVK, chosen at startup (fbd562421)
- test(macos): the driver package fixture takes the architecture and SDK from darwin_toolchain (3472d670c)
- fix(macos): type the driver manifest's ICD record for Pyright (296b5347b)
- fix(macos): MoltenVK is the default Vulkan driver on every macOS (86c017e42)
- test(macos): the display fixture counts longshot_env.txt's loads on Apple only (9d34f9a1c)

Assisted-by: Claude Opus 5.5
Co-Authored-By: Claude Opus 5.5 <[email protected]>

fix(vk): give programs that read storage buffer lengths descriptor sets on MoltenVK 02ad928d81

Was: #644 (#644), branch mac/mvk-sizes at f08eb70206 (Sweep3, on main aa95d35f).

It was stacked on #643; their changes are the earlier commits here.

Its own commits, squashed:
- fix(macos): programs reading storage buffer lengths get descriptor sets on MoltenVK (f08eb7020)

Assisted-by: Claude Opus 5.5
Co-Authored-By: Claude Opus 5.5 <[email protected]>

fix(vk): use descriptor sets for stages with more than 8 read_write textures on MoltenVK 5b523fef19

Was: #645 (#645), branch mac/mvk-rwtex at b8b1ba998a (Sweep3, on main aa95d35f).

It was stacked on #644; their changes are the earlier commits here.

Its own commits, squashed:
- fix(macos): stages with more than 8 read_write textures get descriptor sets on MoltenVK (b8b1ba998)

Assisted-by: Claude Opus 5.5
Co-Authored-By: Claude Opus 5.5 <[email protected]>

fix(vk): use update-after-bind sets past Metal's per-stage limits on MoltenVK

CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Has been skipped

Details

CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Has been skipped

Details

CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Has been skipped

Details

Third-party parts / The same as upstream (pull_request) Has been skipped

Details

Website / Website on Cloudflare (pull_request_target) Has been skipped

Details

CI / Conventional Commit title (pull_request) Successful in 15s

Details

CI / Workflow lint (actionlint) (pull_request) Successful in 16s

Details

Guide screenshots / Guide images and local UI source review (pull_request) Failing after 12s

Details

Third-party parts / As their notes say (pull_request) Successful in 14s

Details

PR description / Description follows the template (pull_request_target) Successful in 16s

Details

792a7d8a32

Was: #646 (#646), branch mac/mvk-pushlimits at 374c55a031 (Sweep3, on main aa95d35f).

It was stacked on #645; their changes are the earlier commits here.

Its own commits, squashed:
- fix(macos): stages past a Metal function's buffer, texture or sampler table get descriptor sets on MoltenVK (6c9e3c9fa)
- fix(macos): MoltenVK's descriptor sets are update-after-bind layouts, within Vulkan's limits for them (3e7d8d3d3)
- test(vk): recording_admission's device stub has the fixes make_pool now reads (374c55a03)

Assisted-by: Claude Opus 5.5
Co-Authored-By: Claude Opus 5.5 <[email protected]>

Merge topic/mvk-descriptor-limits into topic/shader-backend c8f7f435b1

The topic needs it (as the stacks merged it before).

Merge #431 fix/import-lookback into topic/shader-backend c1229ac20f

The topic needs it (as the stacks merged it before).

feat(vk): add a segment planner for guest spans larger than one device allocation a0afe1219a

Was: #647 (#647), branch mac/seg-plan at 52def6832c (Sweep3, on main aa95d35f).

Its own commits, squashed:
- feat(vk): plan where an oversize guest span is cut into import segments (601c00ccd)
- feat(vk): let the segment planner add segments that pay for themselves (52def6832)

Assisted-by: Claude Opus 5.5
Co-Authored-By: Claude Opus 5.5 <[email protected]>

feat(vk): bind storage and texel descriptors as piece arrays in the SPIR-V back end ad8133a32f

Was: #648 (#648), branch mac/seg-shader at 9b73e82942 (Sweep3, on main aa95d35f).

Its own commits, squashed:
- feat(vk): descriptors bound as pieces in the SPIR-V back end and program variants (1f9183c2a)
- fix(vk): MslArrayPtr: no pointer into a storage buffer array is passed to a function (9b73e8294)

Assisted-by: Claude Opus 5.5
Co-Authored-By: Claude Opus 5.5 <[email protected]>

feat(vk): read texel formats MoltenVK cannot view through R8/R16/R32 carrier codecs ff007b3ddd

Was: #649 (#649), branch mac/texel-codec at c5cbd39d26 (Sweep3, on main aa95d35f).

It was stacked on #648; their changes are the earlier commits here.

Its own commits, squashed:
- feat(vk): texel codecs in the SPIR-V back end and program variants (c5cbd39d2)

Assisted-by: Claude Opus 5.5
Co-Authored-By: Claude Opus 5.5 <[email protected]>

feat(vk): admit guest buffers per operation and import over-cap spans as segments on macOS cfb6702019

Was: #658 (#658), branch mac/renderer-core at f8d4e51ad5 (Sweep3, on main aa95d35f).

It was stacked on #431, #491, #647; their changes are the earlier commits here.

Merged with the topic's earlier commits by hand where they met (Consolidate-files/hand-resolutions.md).

Its own commits, squashed:
- fix(vk): join #431's import table helpers with #491's macOS Pool (a1d08d92a)
- fix(macos): import NOACCESS guest pages and name a missing audio service (e6e92a90d)
- feat(vk): admit an operation's guest buffers before it binds them (alias isolation, macOS) (a27bee681)
- feat(vk): import a span past one device allocation as segments (SegMem, alias isolation) (dd4f13047)
- test(vk): follow the renderer-core merge's moved code in three extractions (6e77a383f)
- fix(vk, macos): admit a parallel batch's uniform updates in order, and keep its own updates out of its handoff (3fbfca08d)
- fix(vk, macos): keep every byte the page table reached mapped when an admission retires imports (616eacb83)
- fix(vk, macos): one view per byte of an Apple spare backing, and no endless retry of a failed view (04e00832b)
- fix(vk, macos): bg_drain publishes no background import over this epoch's retired memory or across a domain (a3913a646)
- fix(vk, macos): admission merges no import past 1 GiB again, so mirror_make can still mirror it (b8d16a20b)
- fix(vk, macos): a segment keeps a crossing import's bytes as leftovers clipped at the cuts, not as whole segments (aab3fd8a6)
- fix(vk, macos): each new import over a retired import's bytes inherits its share of that import's VRAM mirror (73bbb89ad)
- test(vk): the admission fixtures follow main's #250 sparse mirrors (6fa302cce)
- fix(vk, macos): the actor motion's resolver follows #59: a pose read at its draw is not copied (e0941ccc0)
- test(vk): test_split_ub_flush follows flush_ub_updates' end parameter (f8d4e51ad)

Assisted-by: Claude Opus 5.5
Co-Authored-By: Claude Opus 5.5 <[email protected]>

feat(vk): add the inert contract for segmented binding consumers 39a916d6e3

Was: #662 (#662), branch mac/seg-contract at 0401eeb237 (Sweep3, on main aa95d35f).

It was stacked on #649, #658; their changes are the earlier commits here.

Its own commits, squashed:
- feat(vk): the segmented consumers' frozen contract (C0) (0401eeb23)

Assisted-by: Claude Opus 5.5
Co-Authored-By: Claude Opus 5.5 <[email protected]>

feat(vk): bind segment-straddling buffers and texel views as piece arrays, and split transfers at segment cuts, on macOS 7eb53a2db8

Was: #663 (#663), branch mac/seg-consumers at 5400f1a0b6 (Sweep3, on main aa95d35f).

It was stacked on #645, #662; their changes are the earlier commits here.

Its own commits, squashed:
- feat(vk): texel codecs' views, texel piece views, null-descriptor stand-ins and presence words (TexelDesc, L3) (da741bdb2)
- test(vk): the texel ownership fixture names the case that failed (39d6490dd)
- feat(vk): SegBind's descriptor core, gathers and arena (L1) (7d5d51276)
- fix(vk): a draw reusing its descriptors drops an earlier build's refusal (SegBind) (7104d8324)
- test(vk): SegBind's gathers and notes fixtures, its parts of three checks, and its docs section (31d52778f)
- test(vk): test_set_for_tables reads its cut from --source (4462202ff)
- test(vk): test_mesh_fallback's topology anchor follows draw()'s `plain` (0f743f0b9)
- test(vk): test_set_for_lengths' layout fixture takes SegBind's seg_layout_count and presence_words (3b5f3b477)
- feat(vk): SegXfer, the transfer consumers of a range across a segment's cut (L2) (3200ff464)
- fix(vk): dispatch()'s compute-variant path (#252) names SegBind's pre-seam `plain` (c13540fc5)
- test(vk): the seg_gathers fixture stubs #162's rec::OwnSlot (8089dc409)
- test(vk): SegXfer's fixture and resolver sites follow #59's actor motion (1ab6beb2b)
- test(vk): SegXfer's gpu_write checks follow #329's ordered inline writes (5400f1a0b)

Assisted-by: Claude Opus 5.5
Co-Authored-By: Claude Opus 5.5 <[email protected]>

feat(vk): emulate null descriptors in the SPIR-V back end on MoltenVK 3ccdc8afe5

Was: #674 (#674), branch mac/null-shader at b7683077e6 (Sweep3, on main aa95d35f).

It was stacked on #663; their changes are the earlier commits here.

Its own commits, squashed:
- feat(vk): null-descriptor emulation in the SPIR-V back end (NullShader, L4) (71003696f)
- docs(vk): NullShader: image and image buffer arrays share a known gap (17667bea3)
- fix(vk): NullShader: two source-cutting checks the wip commit broke (f53eda6cd)
- fix(vk): MslArrayPtr's NullShader half: a descriptor bound whole calls its loop function, and the loops' CPU runs (b7683077e)

Assisted-by: Claude Opus 5.5
Co-Authored-By: Claude Opus 5.5 <[email protected]>

fix(vk): keep storage image reads in program order on MoltenVK 2843f77e9e

Was: #675 (#675), branch mac/tex-read-order at c5c6a853a8 (Sweep3, on main aa95d35f).

It was stacked on #674; their changes are the earlier commits here.

Its own commits, squashed:
- fix(vk): TexOrder: store each image read in place on MoltenVK (fb51f7807)
- test: test_amd_subgroup_profiles: the MoltenVK driver id in make_profile's stand-in (8d5fc5373)
- fix(vk): TexOrder: an image memory barrier before each image write and atomic on MoltenVK (c5c6a853a)

Assisted-by: Claude Opus 5.5
Co-Authored-By: Claude Opus 5.5 <[email protected]>

fix(vk): bound SPIRV-Cross expression nesting on MoltenVK 85b977e302

Was: #676 (#676), branch mac/msl-depth at f8e18c953b (Sweep3, on main aa95d35f).

It was stacked on #675; their changes are the earlier commits here.

Its own commits, squashed:
- fix(macos): MoltenVK's Metal source nests no deeper than 32 forwarded instructions (71d2498c4)
- test(vk): the subgroup profile fixture knows MoltenVK's driver id (f8e18c953)

Assisted-by: Claude Opus 5.5
Co-Authored-By: Claude Opus 5.5 <[email protected]>

fix(macos): Rosetta host imports at whole hardware pages, and a refused import asked again only when its backing changes 48af6a5453

Was: #686 (#686), branch mac/rosetta-import at 6ec9629e59 (Sweep3, on main aa95d35f).

It was stacked on #658; their changes are the earlier commits here.

Its own commits, squashed:
- fix(macos): Rosetta host imports: whole hardware pages, refused imports asked again only when their backing changes (ae91338ae)
- fix(macos): the backing generation is Apple host API version 26, after #491's import pins at 25 (6ec9629e5)

Assisted-by: Claude Opus 5.5
Co-Authored-By: Claude Opus 5.5 <[email protected]>

fix(vk): the MoltenVK nesting bound counts only the reads SPIRV-Cross certainly makes 6ada36089b

Was: #692 (#692), branch mac/msl-depth2 at a67e1934ec (Sweep3, on main aa95d35f).

It was stacked on #676; their changes are the earlier commits here.

Its own commits, squashed:
- fix(macos): the nesting bound counts only the reads SPIRV-Cross certainly makes (a67e1934e)

Assisted-by: Claude Opus 5.5
Co-Authored-By: Claude Opus 5.5 <[email protected]>

fix(vk): bound the control-flow nesting of SPIRV-Cross's Metal source on MoltenVK 4c8dcc77d2

Was: #693 (#693), branch mac/compile-gaps at 5ceed495b2 (Sweep3, on main aa95d35f).

It was stacked on #692; their changes are the earlier commits here.

Its own commits, squashed:
- fix(vk): bound the control-flow nesting of SPIRV-Cross's Metal source on MoltenVK (46b5810a9)
- perf(vk): parse the module in place when the scope bound has nothing to move (5ceed495b)

Assisted-by: Claude Opus 5.5
Co-Authored-By: Claude Opus 5.5 <[email protected]>

fix(vk): MoltenVK's vertex stages run the guest's warp operations as one-lane warps 1e976d5faf

Was: #694 (#694), branch mac/subgroup-vertex at a3e8a51faf (Sweep3, on main aa95d35f).

It was stacked on #676; their changes are the earlier commits here.

Merged with the topic's earlier commits by hand where they met (Consolidate-files/hand-resolutions.md).

Its own commits, squashed:
- fix(vk): MoltenVK's vertex stages run the guest's warp operations as one-lane warps (a2b3b0578)
- test(vk): test_subgroup_vertex: the base tree's own checks unprinted, and one failure message per cause (e20482fa9)
- test(vk): test_subgroup_vertex: SPIRV-Cross drops a subgroup barrier in a vertex function (a3e8a51fa)

Assisted-by: Claude Opus 5.5
Co-Authored-By: Claude Opus 5.5 <[email protected]>

feat(vk): RX_VK_SPV_PROFILE, what the big SPIR-V modules are made of d87bcd25fe

Was: #695 (#695), branch mac/spv-size at 43d1a23420 (Sweep3, on main aa95d35f).

It was stacked on #676; their changes are the earlier commits here.

Its own commits, squashed:
- feat(vk): RX_VK_SPV_PROFILE, what the big SPIR-V modules are made of (3d7b1f5f6)
- docs(vk): spirv_profile.h's store names where the line is printed (ebf1854b4)
- docs(vk): RX_VK_SPV_PROFILE's text names the slow compile line as the precompile hand-off's (43d1a2342)

Assisted-by: Claude Opus 5.5
Co-Authored-By: Claude Opus 5.5 <[email protected]>

perf(vk): search storage buffer tables in loops in every global memory fallback on MoltenVK and KosmicKrisp d58c00e2b2

Was: #696 (#696), branch mac/gm-loop at 231115f14c (Sweep3, on main aa95d35f).

It was stacked on #694, #695; their changes are the earlier commits here.

Merged with the topic's earlier commits by hand where they met (Consolidate-files/hand-resolutions.md).

Its own commits, squashed:
- feat(vk): MoltenVK and KosmicKrisp search storage buffer tables in loops in every global memory fallback (39a40fed6)
- test(vk): gm_loop_ir --gpu: the loop forms first, then the written-out ones, smaller first (99616b7ab)
- test(vk): stub table_loops_on where make_profile is compiled cut out (231115f14)

Assisted-by: Claude Opus 5.5
Co-Authored-By: Claude Opus 5.5 <[email protected]>

perf(vk): zero local memory at computed offsets by a loop on MoltenVK, not a written-out initializer 51da27ed65

Was: #697 (#697), branch mac/local-mem-buffer at a526c34c9f (Sweep3, on main aa95d35f).

It was stacked on #696; their changes are the earlier commits here.

Its own commits, squashed:
- feat(vk): MoltenVK zeroes local memory at computed offsets by a loop, not a written-out initializer (a526c34c9)

Assisted-by: Claude Opus 5.5
Co-Authored-By: Claude Opus 5.5 <[email protected]>

fix(macos): keep storage image reads in program order on KosmicKrisp too 234bb376db

Was: #698 (#698), branch mac/kk-tex-order at 8a3d7db0ce (Sweep3, on main aa95d35f).

It was stacked on #654, #675; their changes are the earlier commits here.

Its own commits, squashed:
- fix(macos): TexOrder's stored image reads on KosmicKrisp too (27d566b45)
- test(macos): the Apple drivers' check names the fixes this tree makes (8a3d7db0c)

Assisted-by: Claude Opus 5.5
Co-Authored-By: Claude Opus 5.5 <[email protected]>

test: giant mesh fallbacks' shape through the real SPIR-V back end (tests/giant_shape_ir.cpp) 1bf322f4ac

Was: #787 (#787), branch mac/giant-shape at 96b51b55b5 (Sweep3, on main aa95d35f).

It was stacked on #697; their changes are the earlier commits here.

Its own commits, squashed:
- test: giant mesh fallbacks' shape through the real SPIR-V back end (tests/giant_shape_ir.cpp) (96b51b55b)

Assisted-by: Claude Opus 5.5
Co-Authored-By: Claude Opus 5.5 <[email protected]>

fix(vk): the null descriptors' stand-in clear not done in 5 s is waited for before its fence and pool go 55cfa8870d

The Mac-only part of fix/async-safe-frees-i40 (c151fcea, MvkAsyncFix's F5) that #788 does not carry, because
null_dummies_make (#663, macOS: MoltenVK has no nullDescriptor) exists only on the macOS stack. The stand-in images' clear
waits 5 s for its fence; when that wait ran out, the fence and the command pool were destroyed while the GPU might still be
clearing (a destroyed pool under a pending command buffer). Now a clear submitted and not seen done in 5 s is waited for
without a limit first, as #788 does for the read-back. The clear is still reported as not done in time (the "may read
garbage" line), as before.

tests/test_null_dummies.py: case make-slow-clear (the fixture's fence wait runs out once): one unbounded wait after the 5 s
one, before the fence and the pool are destroyed. Fails before this commit (one wait, none unbounded), passes after, on arm64
and x86_64 under Rosetta.

Assisted-by: Claude Opus 5.5
Co-Authored-By: Claude Opus 5.5 <[email protected]>

perf(vk): per-worker pipeline caches, every CPU and no saved pipeline cache for shader preparation on MoltenVK 4f012a4c30

Was: #650 (#650), branch mac/prep-fast at 6bc44ad83b (Sweep3, on main aa95d35f).

Merged with the topic's earlier commits by hand where they met (Consolidate-files/hand-resolutions.md).

Its own commits, squashed:
- fix(macos): the shader preparation runs on every usable CPU on macOS (928d6afc0)
- fix(macos): a pipeline cache per preparation worker on MoltenVK (71ddd7247)
- fix(macos): the worker caches make no Vulkan call from a header (7b957f8ec)
- fix(macos): MoltenVK starts with an empty pipeline cache and writes none (4c8c82548)
- fix(macos): the worker caches follow main's fetch lock (#190) and Intel's gate (#252) (6bc44ad83)

Assisted-by: Claude Opus 5.5
Co-Authored-By: Claude Opus 5.5 <[email protected]>

fix(macos): start renderer threads with a 16 MiB stack 5c76bed272

Was: #651 (#651), branch mac/thread-stack at a7c1f493de (Sweep3, on main aa95d35f).

Its own commits, squashed:
- fix(vk, macos): the renderer's threads start with a 16 MiB stack, not macOS's 512 KiB (16f8f3d2a)
- test(macos): one overflow message from thread_stack_test, however many threads overflow at once (b355d677f)
- test: the checks that compile renderer code with its threads see thread_stack.h (a7c1f493d)

Assisted-by: Claude Opus 5.5
Co-Authored-By: Claude Opus 5.5 <[email protected]>

fix(macos): let MoltenVK's precompile go on behind the title and name slow compiles d6ab6c38c8

Was: #652 (#652), branch mac/precompile-stall at ba5aa17684 (Sweep3, on main aa95d35f).

It was stacked on #650, #651; their changes are the earlier commits here.

Merged with the topic's earlier commits by hand where they met (Consolidate-files/hand-resolutions.md).

Its own commits, squashed:
- fix(vk): MoltenVK: the precompile goes on behind the game; slow compiles named; a failed compile tried once more (5ceb62a44)
- test(vk): test_precompile_handoff: a tree without the change says each missing piece (576970be4)
- docs: RX_VK_PRECOMPILE_STALL_S, RX_VK_COMPILER_SERVICE and RX_VK_SLOW_COMPILE_S (3c8381113)
- fix(vk): name a graphics compile at its precompile callers, not inside build_graphics (fe6efa98d)
- fix(vk): the precompile hand-off on main's Intel gate and quick builds (#252), fetch lock (#190) and game folder (#321) (ba5aa1768)

Assisted-by: Claude Opus 5.5
Co-Authored-By: Claude Opus 5.5 <[email protected]>

fix(macos): leave MoltenVK's compute pipelines to the precompile and bound its giant compiles 5915733d97

Was: #664 (#664), branch mac/warm-handoff at 1f7065181d (Sweep3, on main aa95d35f).

It was stacked on #652; their changes are the earlier commits here.

Merged with the topic's earlier commits by hand where they met (Consolidate-files/hand-resolutions.md).

Its own commits, squashed:
- fix(vk): under compiler_service the preparation leaves the compute pipelines to the precompile (22e75176e)
- fix(vk): at most four giant compiles at once in the precompile under compiler_service (748c4970f)
- docs: RX_VK_GIANT_COMPILES and RX_VK_GIANT_WORDS; the preparation's compute pipelines under compiler_service (225b7b0be)
- fix(vk): giant compiles never hold every worker of a phase (23d8d276b)
- test(vk): test_warm_handoff follows main's warm and dispatch (#252, #190) (1f7065181)

Assisted-by: Claude Opus 5.5
Co-Authored-By: Claude Opus 5.5 <[email protected]>

fix(macos): hold MoltenVK's giant compiles behind the game until its first frame, at a lower QoS class 202c6c572f

Was: #665 (#665), branch mac/title-stall at a7e966ec87 (Sweep3, on main aa95d35f).

It was stacked on #664; their changes are the earlier commits here.

Merged with the topic's earlier commits by hand where they met (Consolidate-files/hand-resolutions.md).

Its own commits, squashed:
- fix(vk): behind the game no giant compile starts before the first frame; the precompile's giants and its work behind the game below the game's QoS class (1e92c87bb)
- docs: RX_VK_GIANT_HOLD_PRESENTS, RX_VK_GIANT_HOLD_S and RX_VK_COMPILE_QOS (5fb67e4fc)
- fix(vk): a giant compile that failed is tried once more alone, not at once among the other giants (a7e966ec8)

Assisted-by: Claude Opus 5.5
Co-Authored-By: Claude Opus 5.5 <[email protected]>

fix(macos): put a dispatch's compute pipeline ahead of MoltenVK's giant compiles and retry it alone, not fail it for good b4d9cb2823

Was: #666 (#666), branch mac/loading-stall at 91b67ad468 (Sweep3, on main aa95d35f).

It was stacked on #665; their changes are the earlier commits here.

Merged with the topic's earlier commits by hand where they met (Consolidate-files/hand-resolutions.md).

Its own commits, squashed:
- fix(vk): a dispatch's compute pipeline that MoltenVK's compiler service failed is built, not left out for good (1cb6be613)
- test: test_loading_stall: a tree without the build's retry is a failure said, not a stop (2869b762e)
- perf(vk): RX_VK_SKIP_LOG off works out nothing for a dispatch left out (f9ca934d8)
- fix(vk): a dispatch's build lets play's want go by its own scope, its failed() as it was (daf95bfb7)
- fix(vk): one place, one count and one wording for a failed compile's tries alone (324b0f0fe)
- fix(vk): the dispatch's want and LoadingStall's checks follow main's build (#252) (f0b82bc57)
- test(vk): test_intel_quick_compute reads the dispatch's build with the compile board's capture (91b67ad46)

Assisted-by: Claude Opus 5.5
Co-Authored-By: Claude Opus 5.5 <[email protected]>

feat(macos): RX_VK_METAL_ARCHIVE_DIR keeps MoltenVK's pipelines in Metal binary archives 6ae2e663cc

Was: #667 (#667), branch mac/metal-cache2 at bf988f6226 (Sweep3, on main aa95d35f).

It was stacked on #653, #666; their changes are the earlier commits here.

Its own commits, squashed:
- feat(macos): MoltenVK's pipelines kept in Metal binary archives (6cdd4ddb0)
- fix(macos): the binary archives' state outlives the process's exit (69d3197c6)
- test(macos): test_metal_archive: no -pthread on Windows (eef27adbc)
- fix(macos): Metal binary archives saved once each, by one gated saver, with Metal's staging removed (584acb7a8)
- fix(macos): compile_board has external linkage, for the Metal archive saver in vk_main.cpp (bf988f622)

Comment reworded here (gpu/vk/metal_archive.h's Why, lines 5-10): it states what the runs observed of Metal's
shader cache and no longer cites how Apple's library works.

Assisted-by: Claude Opus 5.5
Co-Authored-By: Claude Opus 5.5 <[email protected]>

fix(macos): start MoltenVK's giant compiles behind the game only while memory allows, and say a lost device as one 573633ff48

Was: #669 (#669), branch mac/giant-mem-gate at 143345156d (Sweep3, on main aa95d35f).

It was stacked on #642, #666; their changes are the earlier commits here.

Its own commits, squashed:
- fix(vk): behind the game, giant compiles wait for memory; a lost device on the Mac is said as one (415eb9a45)
- docs: RX_VK_GIANT_ROOM_MB in the switch table (cb5eef04c)
- fix(macos): the lost wait's line and end on the Mac only; vk_replay.cpp as it was elsewhere (143345156)

Assisted-by: Claude Opus 5.5
Co-Authored-By: Claude Opus 5.5 <[email protected]>

feat(macos): RX_VK_LMEM_LOG names each module's private memory and each pipeline's first bind 7e5cc8ac23

Was: #681 (#681), branch mac/metal-scratch at e0400e839c (Sweep3, on main aa95d35f).

It was stacked on #663; their changes are the earlier commits here.

Merged with the topic's earlier commits by hand where they met (Consolidate-files/hand-resolutions.md).

Its own commits, squashed:
- diag(vk): RX_VK_LMEM_LOG names each module's private memory and each pipeline's first bind (0f7820dc3)
- test: the excerpt fixtures know RX_VK_LMEM_LOG's hooks (83ae4962d)
- fix(vk): RX_VK_LMEM_LOG on main's dispatch and program cache: the bound pipeline named, the notes beside array_length (6f4512064)
- fix(macos): RX_VK_LMEM_LOG logs on macOS only (e0400e839)

Assisted-by: Claude Opus 5.5
Co-Authored-By: Claude Opus 5.5 <[email protected]>

fix(vk): heavy compiles run alone under Metal's compiler service; the first giant behind the game no longer waits for 16 GB d8917d8de4

Was: #784 (#784), branch mac/giant-compile-2 at 99e43bd2f3 (Sweep3, on main aa95d35f).

It was stacked on #669, #681; their changes are the earlier commits here.

Its own commits, squashed:
- fix(vk): SegBind's left-out dispatches named, and the precompile's checks read the renderer's dispatch names, after #669 met #681 (05ab99e91)
- test: test_private_memory looks for note_private_memory after a stage's words, as the program cache notes them once #669 met #681 (c7cca440f)
- fix(vk): heavy compiles run alone under Metal's compiler service; the first giant behind the game no longer waits for 16 GB (8b4538f58)
- fix(vk): the local memory weight lives in private_memory.h, so vk_progcache.cpp's cut-out fixtures build (f4be3d899)
- docs(vk): heavy_weight's comment says what probe GC-02 found (9a4f8a6d0)
- docs: RX_VK_GIANT_ROOM_MB's row says the first giant behind the game starts unless the pressure is critical (a70cb0f37)
- fix(vk): the local memory weight is noted on macOS only, where the precompile reads it (99e43bd2f)

Assisted-by: Claude Opus 5.5
Co-Authored-By: Claude Opus 5.5 <[email protected]>

fix(macos): KosmicKrisp's compiles go through the compiler service's precompile, as MoltenVK's do c9c588b9ba

Was: #785 (#785), branch mac/kk-compiler-service at 588d5f3851 (Sweep3, on main aa95d35f).

It was stacked on #665; their changes are the earlier commits here.

Its own commits, squashed:
- fix(macos): KosmicKrisp's compiles go through the compiler service's precompile, as MoltenVK's do (588d5f385)

Assisted-by: Claude Opus 5.5
Co-Authored-By: Claude Opus 5.5 <[email protected]>

fix(macos): Metal's shader cache folder asked at the host's start, before SDL's window 04a466172c

Was: #786 (#786), branch mac/windowed-metal-cache at 976bb0ac6d (Sweep3, on main aa95d35f).

It was stacked on #654, #667, #686; their changes are the earlier commits here.

Merged with the topic's earlier commits by hand where they met (Consolidate-files/hand-resolutions.md).

Its own commits, squashed:
- fix(macos): Metal's shader cache folder asked at the host's start, before SDL's window (42ae6d37d)
- fix(macos): the Metal cache check is Apple host API 27, after the Rosetta imports' backing generation at 26 (976bb0ac6)

Assisted-by: Claude Opus 5.5
Co-Authored-By: Claude Opus 5.5 <[email protected]>

fix(vk): a compile that never ends keeps the heavy compiles and the giants back for the stale time at most 310fbb8ec5

Run 36 (I40) began one of its eight heavy compiles in a 15-minute hold. Play's dispatch asked for and
began compute 44511d69 (126,010 SPIR-V words, not heavy) at the load into play; it takes about 15
minutes (889 s in run 25, 965.6 s in run 34, the only runs that held it that long), and while it was
wanted and in flight giant-compile-2's alone rule held every heavy compile back (Board::holds, a giant
of play's on the board), and play's want held the 204 deferred giants back too. A run of the
precompile's that never ended would have stopped the whole precompile behind the game (the drain).

Now a compile in flight for RX_VK_STALE_COMPILE_S (240 s; 0: never, as before) no longer counts,
unless it is heavy: a run of the precompile's (by its first item, from when next() gave it), play's
giant compile (Span, from play_begin) and a compile play waits for (by its program, from want). A
heavy compile still never begins beside another heavy one, play's included whatever its age (play's
compile is heavy by the precompile's measure: heavy_program), nor while memory refuses or the hold
lasts; a young compile still keeps it back until it ends. Tries alone keep their rule. The begin line
says what a heavy compile began beside. 240 s: of the 234 non-heavy compiles of 10 s or more that ended
in runs 32 and 34-36 and I39's eight, all but three took at most 182.7 s, one 429.8 s.

MoltenVK and KosmicKrisp only (compiler_service); every other driver as before.
reports/2026-11-15-giant-starve.md.

Ported onto topic/precompile-metal (main aa95d35f's dispatch build, #252): there the build's want is its
first line, ahead of Intel's on_worker returns, and its serial push passes true; the want names its
program at that first line and at the push, as on I40, and test_loading_stall.py checks both.

Assisted-by: Claude Opus 5.5
Co-Authored-By: Claude Opus 5.5 <[email protected]>
(cherry picked from commit 5292596d69ccf321d0d12f73e39e65753f7ca2be)

fix(vk): what an earlier run found slow for minutes compiles first, and is play's from the hand-off 604d13062c

Compute 44511d69 (126,010 SPIR-V words, a giant but not heavy) takes about 15 minutes (889 s in run 25,
965.6 s in run 34). In every run read (runs 32 and 34-36, I39's eight, E8a) the precompile claimed it in
its last minutes, among the largest computes or last of the slow list, with four giants in flight: it was
deferred at the giant bound, still deferred at the hand-off, and play's dispatch began it 43-65 s after
the hand-off, at the load into play. Its dispatches were left out for about the first 15 minutes of play
in every start, and no arm64 run held it that long, so Metal's cache never had it.

Now the compute and mesh fallback programs the slow list says took RX_VK_SLOW_FIRST_S (240 s; 0: none)
or more, none heavy, go first in the precompile's order, the longest first, at most the giant limit less
one (slow_first): each compiles beside the rest from the start, at the thread's own QoS class as play's
dispatch asks. Before the game each is a giant like any other: the bound stays four. At the hand-off the
ones still in flight are handed to play (Giants::hand_over): behind the game they keep no place of the
limit and the memory gate does not count them (Giants::counted), as play's own compile of the program was
not counted in every run before; a try alone still waits for them and, while young, a heavy compile
(GiantStarve's rule). Not handed, the slow one would keep every other giant and heavy compile back until
it ends whenever the memory is short of a room, which it was behind the game in runs 32 and 34-36 and E8a.
A first one begun after the hand-off is never handed. behind_worker leaves a run before giving its place
back, and its try alone names its run (solo_ask(at)), so a handed run leaves the counts exactly once.

MoltenVK and KosmicKrisp only (compiler_service); every other driver as before.
reports/2026-11-15-slow-first.md.

Ported onto topic/precompile-metal after giant-starve's port: docs/development.md's RX_VK_COMPILE_QOS row as this
commit words it, followed by the topic's RX_VK_GIANT_ROOM_MB row (#784), which I40 did not have there; every other
change line for line as 40a86bae made it on 5292596d.

Assisted-by: Claude Opus 5.5
Co-Authored-By: Claude Opus 5.5 <[email protected]>
(cherry picked from commit 40a86baedd6ba1651dc2e62e74af64eccbc18e5c)

docs(vk): metal_archive.h's notes on Metal's cache and staging say only what runs showed 7e1d702fb5

The header's comment, two of its code comments and three comments in test_metal_archive.py's fixture
named Apple's internal libraries, functions and return codes (libCoreFSCache, its 0xd3, the archiver's
and packer's internals). They came from reading Apple's binaries, which is not ours to do; they are
reworded to what runs and public notes show (reports/2026-11-15-metal-cache-cap.md):
- the cache: two files capped at about 1 GiB each (public notes), unchanged in size since run #17,
  written by a warm start within a minute of the hand-off at the latest, nothing compiled later kept;
- the staging: one gpuarchiver-* folder per archive, left behind after a save (run #21: 1,042, 12 GB),
  named by the archive and removed by the saver;
- render pipelines without a fragment function: archives of their own, so that a save Metal refuses
  with "expecting 'fragment' stage in pipeline no. N" (run #21) costs none of the others.
No code change: both files are line for line the same without their // comments.

Ported onto topic/precompile-metal: the header's Why (lines 5-10), which #667's commit here already reworded, now
reads as this commit words it; both files are byte for byte 63fc3d2c's.

Assisted-by: Claude Opus 5.5
Co-Authored-By: Claude Opus 5.5 <[email protected]>
(cherry picked from commit 63fc3d2cecd006a97726f9451c8bdfc4176e83a6)

test: test_giant_alone cuts main's dispatch build, as the giant-starve port's test_loading_stall does 89db091cde

The giant-starve port (310fbb8e) moved play's want into main's dispatch build (#252), the lambda
"auto build = [job, on_worker, made_how, board_on]", and changed test_loading_stall.py's cut to it;
test_giant_alone.py still cut I40's "auto build = [job, board_on]", which the topic does not have, and
failed with "a function not found". Its cut now names main's build; the checks on what the build holds
(play's giant compile begun, heavy or not, as it begins) are unchanged.

Assisted-by: Claude Opus 5.5
Co-Authored-By: Claude Opus 5.5 <[email protected]>

test: test_warm_handoff: seed_compute's save found with what its line adds 974bc32046

In this topic #664's check (test_warm_handoff) meets #695's RX_VK_SPV_PROFILE line in seed_compute (topic 4), as
Integration 25 met them; the check failed at the topic's tip (it passes at #664's Sweep3 head), so this takes I41's fix:

A second semantic merge fix for Integration 25. test_warm_handoff
(WarmHandoff, acd3a71a) checks in vk_shader.cpp's seed_compute that a
kept variant is saved (program_cache_save) on the line before the warm
is given it, by matching the two lines verbatim. SpvSize (1a50b01b)
adds spv_profile_lines before the save on that line and a comment after
it, so on the merged tree the check failed though the order is
unchanged. Each branch passes alone. The check now strips comments and
asks for a line ending in the save, followed by `if (made) made(*t.p);`
(checked: it still fails with the two lines swapped or without the
save).

Assisted-by: Claude Opus 5.5
Co-Authored-By: Claude Opus 5.5 <[email protected]>
(cherry picked from commit 1cbb594a6ab716a864ef54b327339470f33a0f8b)

test: test_vendor_fixes runs its checks once, after every check is defined a6aea9452c

The chain's resolution of #650 (tests/test_vendor_fixes.py: its pipeline-cache check placed whole, before the
Apple drivers' check that #698 added in topic 4, hand-resolutions.md) brought #650's own end of the file with it:
"if __name__ == '__main__': main()" ran before check_apple_drivers was defined, and the test failed with a
NameError at the topic's tip (it passes at #650's and #785's Sweep3 heads). That first call is removed; the
file's own call at its end runs every check, as on I41 78ec1c47.

Assisted-by: Claude Opus 5.5
Co-Authored-By: Claude Opus 5.5 <[email protected]>

test: make_profile's cut-out fixtures know KosmicKrisp, SubgroupVertex, GmLoop and LocalMemBuffer

CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Has been skipped

Details

CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Has been skipped

Details

CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Has been skipped

Details

Third-party parts / The same as upstream (pull_request) Has been skipped

Details

Website / Website on Cloudflare (pull_request_target) Has been skipped

Details

CI / Conventional Commit title (pull_request) Successful in 7s

Details

CI / Workflow lint (actionlint) (pull_request) Successful in 11s

Details

Guide screenshots / Guide images and local UI source review (pull_request) Failing after 9s

Details

Third-party parts / As their notes say (pull_request) Successful in 11s

Details

PR description / Description follows the template (pull_request_target) Successful in 16s

Details

70cc5ba1cb

In this topic #698 (KosmicKrisp's TexOrder, its Apple drivers' fixture in test_vendor_fixes) meets #694/#696
(SubgroupVertex: make_profile reads dev.subgroup_stages and Shader::Stage) and #697's two switches (GmLoop's
table_loops_on, LocalMemBuffer's local_zero_loop_on), and test_subgroup_vertex's stub meets KosmicKrisp's
VK_DRIVER_ID_MESA_KOSMICKRISP in make_profile. Each member passes at its own Sweep3 head; at the topic's tip the
fixtures did not build. This takes I41's fixes 02b10861 and 560923891's stub line:
- test_subgroup_vertex: the KosmicKrisp driver constant (28, the Vulkan headers' value);
- test_vendor_fixes' Apple drivers' fixture: subgroup_stages (all stages), stage.h, and table_loops_on and
  local_zero_loop_on stubs; its columns are unchanged.

Assisted-by: Claude Opus 5.5
Co-Authored-By: Claude Opus 5.5 <[email protected]>
(cherry picked from commit 02b108611 and the local_zero_loop_on line of 560923891)

Merge topic/shader-backend into topic/precompile-metal

CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Has been skipped

Details

CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Has been skipped

Details

CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Has been skipped

Details

Third-party parts / The same as upstream (pull_request) Has been skipped

Details

Website / Website on Cloudflare (pull_request_target) Has been skipped

Details

CI / Conventional Commit title (pull_request) Successful in 5s

Details

CI / Workflow lint (actionlint) (pull_request) Successful in 7s

Details

Guide screenshots / Guide images and local UI source review (pull_request) Failing after 7s

Details

Third-party parts / As their notes say (pull_request) Successful in 14s

Details

PR description / Description follows the template (pull_request_target) Successful in 17s

Details

e3f55e1bd2

The base topic's resolution fix 70cc5ba1 (test_vendor_fixes' Apple drivers' fixture and test_subgroup_vertex's stub
know KosmicKrisp, SubgroupVertex, GmLoop and LocalMemBuffer): its checks run at this topic's tip too.

Assisted-by: Claude Opus 5.5
Co-Authored-By: Claude Opus 5.5 <[email protected]>

BeanIssues added the

labels

2026-11-15 17:40:10 +00:00

mooki referenced this pull request

2026-11-15 17:41:20 +00:00

WIP: feat(macos): run the host on macOS: build, runtime, control, crash records and the Vulkan loader #845

mooki referenced this pull request

2026-11-15 17:41:24 +00:00

WIP: feat(macos): Vulkan on Metal: pinned host imports, the unified-memory budget, Metal's cache folder and KosmicKrisp #846

mooki referenced this pull request

2026-11-15 17:41:28 +00:00

WIP: fix(vk): MoltenVK's descriptor limits in descriptor sets #847

mooki referenced this pull request

2026-11-15 17:41:31 +00:00

WIP: feat(vk): the SPIR-V back end on MoltenVK and guest spans past one device allocation #848

mooki referenced this pull request

2026-11-15 17:41:40 +00:00

WIP: fix(macos): draw-path emulation on MoltenVK #850

mooki referenced this pull request

2026-11-15 17:41:44 +00:00

WIP: fix(vk): draw indirect quads and mesh count draws on every driver, with their Mac halves #851

mooki referenced this pull request

2026-11-15 17:41:48 +00:00

WIP: perf(macos): Mac performance on MoltenVK #852

mooki referenced this pull request

2026-11-15 18:08:22 +00:00

WIP: perf(vk): per-worker pipeline caches, every CPU and no saved pipeline cache for shader preparation on MoltenVK #650

mooki referenced this pull request

2026-11-15 18:08:24 +00:00

WIP: fix(macos): start renderer threads with a 16 MiB stack #651

mooki referenced this pull request

2026-11-15 18:08:28 +00:00

WIP: fix(macos): let MoltenVK's precompile go on behind the title and name slow compiles #652

mooki referenced this pull request

2026-11-15 18:08:31 +00:00

WIP: fix(macos): leave MoltenVK's compute pipelines to the precompile and bound its giant compiles #664

mooki referenced this pull request

2026-11-15 18:08:33 +00:00

WIP: fix(macos): hold MoltenVK's giant compiles behind the game until its first frame, at a lower QoS class #665

mooki referenced this pull request

2026-11-15 18:08:35 +00:00

WIP: fix(macos): put a dispatch's compute pipeline ahead of MoltenVK's giant compiles and retry it alone, not fail it for good #666

mooki referenced this pull request

2026-11-15 18:08:38 +00:00

WIP: feat(macos): RX_VK_METAL_ARCHIVE_DIR keeps MoltenVK's pipelines in Metal binary archives #667

mooki referenced this pull request

2026-11-15 18:08:40 +00:00

WIP: fix(macos): start MoltenVK's giant compiles behind the game only while memory allows, and say a lost device as one #669

mooki referenced this pull request

2026-11-15 18:08:42 +00:00

WIP: feat(macos): RX_VK_LMEM_LOG names each module's private memory and each pipeline's first bind #681

mooki referenced this pull request

2026-11-15 18:08:44 +00:00

WIP: fix(vk): heavy compiles run alone under Metal's compiler service; the first giant behind the game no longer waits for 16 GB #784

mooki referenced this pull request

2026-11-15 18:08:48 +00:00

WIP: fix(macos): KosmicKrisp's compiles go through the compiler service's precompile, as MoltenVK's do #785

mooki referenced this pull request

2026-11-15 18:08:51 +00:00

WIP: fix(macos): Metal's shader cache folder asked at the host's start, before SDL's window #786

mooki referenced this pull request

2026-11-15 18:14:44 +00:00

Idea: a native macOS build (Apple silicon, x86-64 under Rosetta) with Vulkan on Metal through MoltenVK #420

HumanBean commented

2026-11-15 18:49:16 +00:00

Owner

This conflicts with main and is built on top of the macOS series (it carries the commits of #845 and others below it). nx2/build.py, nx2/make_game.py and docs/development.md conflict with #577 (the WebGPU renderer removed from the Windows build). Please rebase onto current main once its base PR #845 lands or is rebased; I didn't touch it because a rebase here would have to be redone after that.

This conflicts with `main` and is built on top of the macOS series (it carries the commits of https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/845 and others below it). `nx2/build.py`, `nx2/make_game.py` and `docs/development.md` conflict with https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/577 (the WebGPU renderer removed from the Windows build). Please rebase onto current `main` once its base PR https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/845 lands or is rebased; I didn't touch it because a rebase here would have to be redone after that.

CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Has been skipped

Details

CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Has been skipped

Details

CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Has been skipped

Details

Third-party parts / The same as upstream (pull_request) Has been skipped

Details

Website / Website on Cloudflare (pull_request_target) Has been skipped

Details

CI / Conventional Commit title (pull_request) Successful in 5s

Details

CI / Workflow lint (actionlint) (pull_request) Successful in 7s

Details

Guide screenshots / Guide images and local UI source review (pull_request) Failing after 7s

Details

Third-party parts / As their notes say (pull_request) Successful in 14s

Details

PR description / Description follows the template (pull_request_target) Successful in 17s

Details

This pull request has changes conflicting with the target branch.

docs/development.md
gpu/vk/eden/shader_recompiler/backend/spirv/emit_spirv_image_atomic.cpp
gpu/vk/eden/shader_recompiler/backend/spirv/spirv_emit_context.cpp
gpu/vk/vk_draw.cpp
gpu/vk/vk_memory.cpp
gpu/vk/vk_replay.cpp
gpu/vk/vk_shader.cpp
gpu/vk/vk_shader.h
gpu/vk/vk_texture.cpp
host/posix/win32_posix.c
nx2/build.py
nx2/make_game.py
tests/test_amd_subgroup_profiles.py

View command line instructions

Manual merge helper

Use this merge commit message when completing the merge manually.

Merge commit title

WIP: fix(macos): MoltenVK's precompile and pipelines on Metal (#849)

Merge commit body

Reviewed-on: https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/849

WIP: fix(macos): MoltenVK's precompile and pipelines on Metal (#849)

Reviewed-on: https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/849

Checkout

From your project repository, check out a new branch and test the changes.

git fetch -u 'mac/topic-precompile-metal:mooki-mac/topic-precompile-metal'

git switch 'mooki-mac/topic-precompile-metal'

No reviewers

No labels

beanstalk: approved @ 1697603

beanstalk: approved @ 2c5a62e

beanstalk: approved @ 4fd82b7

beanstalk: approved @ 64d6d09

beanstalk: approved @ 6f91fae

beanstalk: approved @ 70a78f9

beanstalk: approved @ 8015c9a

beanstalk: approved @ 8d361dc

beanstalk: approved @ 9d8356e

beanstalk: approved @ abf2227

beanstalk: approved @ d82f202

beanstalk: approved @ e5f1e40

beanstalk: approved @ ee8ab5f

No milestone

No project

No assignees

2 participants

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference

MagicBeanDev/Longshot!849

No description provided.

Rows
Columns