WIP: fix(vk): preserve cache file reservations on macOS #476

Draft
mooki wants to merge 1 commit from mooki/Longshot:mac/cache-files into main
Member

Why

The renderer's two cache files do not compile on macOS. gpu/vk/pipeline_cache_file.h reserves the pipeline cache's
disk space with Linux's posix_fallocate, and gpu/vk/vk_progcache.cpp asks for read-ahead with POSIX_FADV_WILLNEED;
macOS has neither. So six of main's game-free cache checks fail on a Mac, and no Mac renderer can build. The reservation
itself has to stay: the driver writes the pipeline cache through a shared mapping, and on a sparse file a full disk is a
SIGBUS in the middle of a write instead of a save that fails cleanly.

What changed

  • vk: PipelineCacheFile::writable in gpu/vk/pipeline_cache_file.h.
    • On macOS the new, empty temporary reserves its whole capacity with fcntl(F_PREALLOCATE) (F_ALLOCATEALL, all or
      nothing; F_PEOFPOSMODE). Only then does ftruncate set its length and is it mapped.
    • Either call failing returns the empty result that both callers (vk_draw.cpp:839 and :10352) already handle: the
      temporary is removed and the published cache is left as it was. There is no sparse-file fallback.
  • vk: program_cache_load in gpu/vk/vk_progcache.cpp. On macOS each record's two read hints (its first 4 KiB, its last
    8 KiB) use fcntl(F_RDADVISE) for the same ranges as Linux's posix_fadvise. Best effort, as before: the reads that
    follow do the I/O and check it either way. F_RDAHEAD was not used, because it changes the file's read-ahead policy
    rather than asking for these ranges.
  • Windows and Linux: unchanged. Both Apple branches sit in #if defined(__APPLE__), and #else keeps main's lines
    as they are.
  • tests: tests/pipeline_cache_io_fault_test.cpp. On macOS its injected ENOSPC reaches the F_PREALLOCATE call
    (through an fcntl stand-in), so the existing native failure stages run there too. Linux's posix_fallocate
    stand-in is unchanged.
  • tests: tests/test_pipeline_cache_allocation.py (new; macOS only, skipped elsewhere). It compiles
    pipeline_cache_file.h (--source-root selects another tree) and checks, on a real file:
    • an 8,388,645-byte temporary has every block allocated before anything is written through its mapping (16,392
      blocks of 512 bytes, against 0 for a control file sized with ftruncate alone);
    • a child process with a 4,096-byte file-size limit gets an empty result, the old 64-byte cache is kept, the
      temporaries are removed, and a retry succeeds.
  • docs: docs/pipeline-cache-storage.md says what macOS reserves and hints, and what the new test checks.

Flags and settings

None.

Measurements

main: 5e072fc8c2adadba5ea610148be1c47800ba0cca (2026-11-14)
this PR: mac/cache-files @ 587afb479e2005b0ba29bb64e6bed2059d6f680a
Hardware: Apple M4 Max (16 CPUs, 64 GiB), APFS, macOS 26.7.1 (25G241), Apple clang 21.0.0. Game-free: no GPU, no game
Scene: none (a portability fix; its proof is the tests, as CONTRIBUTING's fix row allows)
Method: the six cache checks and the new test, compiled for arm64 and for x86_64 (run under Rosetta), against main's
source (git archive 5e072fc8) and against this PR. Two interleaved rounds per route: PR, main, PR, main. Then the full
game-free suite on this PR, compared by test name with main's on the same Mac.
How to reproduce: python -I -B tests/run_all.py pipeline_cache pipeline_play_cache pcache_header program_cache_cleanup program_cache_one_variant on each side; python -I -B tests/test_pipeline_cache_allocation.py --cxx clang++ --source-root <a tree of main> for the new test against main's header. For Rosetta, pass a compiler that adds
-arch x86_64.

main @ 5e072fc8 this PR @ 587afb47
test_pipeline_cache_file.py compile error: posix_fallocate pass
test_pipeline_cache_save.py compile error: posix_fallocate pass: 14 cases, the optional comparison with an original source skipped as on every platform
test_pipeline_play_cache.py compile error: posix_fallocate pass: all 12 cases
test_pcache_header_read.py compile error: posix_fallocate pass
test_program_cache_cleanup.py compile error: POSIX_FADV_WILLNEED pass: 9 cases
test_program_cache_one_variant.py compile error: POSIX_FADV_WILLNEED pass: the same variant for each of 9 picks
test_pipeline_cache_allocation.py (new) compile error: posix_fallocate pass: 2 cases

Each row is the same on arm64 and under Rosetta, in both rounds: 7 of 7 fail on main and 7 of 7 pass here, 28 runs in
all.

main @ 5e072fc8 (arm64 and x86_64, both rounds):
gpu/vk/pipeline_cache_file.h:274:13: error: use of undeclared identifier 'posix_fallocate'
cache_functions.inc:469:61: error: use of undeclared identifier 'POSIX_FADV_WILLNEED'
BATTERY ... failed=7 of 7

this PR @ 587afb47 (arm64 and x86_64, both rounds):
{"filesystem":"apfs","capacity":8388645,"emptyBlocks":0,"sparseBlocks":0,"reservedBlocksBeforeWrite":16392,"publishedBlocks":16392,"passed":true}
{"limitBytes":4096,"refusedEmpty":true,"oldOutputPreserved":true,"retryPassed":true}
{"nativeFailureStages":13,"oldOutputPreserved":true,"retryPassed":true,"ownedTemporariesCleaned":true,"readIoFailureNotIncompatible":true}
BATTERY ... failed=0 of 7

The new test also catches a missing reservation, not only main's compile error. With this PR's header and only the
F_PREALLOCATE call removed (a throwaway copy, not committed), it compiles and fails on both routes:

AssertionError: 1 != 0 : FAIL line 49: reserved.st_blocks > empty.st_blocks

Testing

Tested:

  • On an Apple M4 Max with macOS 26.7.1, game-free only, on arm64 and under Rosetta (x86_64), as above.

  • The full suite, tests/run_all.py, 6 at once, on this PR, compared by test name with main on the same Mac (the
    runner labels macOS linux):

    main @ 5e072fc8:     525 passed, 19 failed, 17 skipped in 177 s (linux, 6 at once)
    this PR @ 587afb47:  532 passed, 13 failed, 17 skipped in 173 s (linux, 6 at once)
    
    • By test name, seven checks changed status, all to pass: the six cache checks that fail on main
      (test_pcache_header_read, test_pipeline_cache_file, test_pipeline_cache_save, test_pipeline_play_cache,
      test_program_cache_cleanup, test_program_cache_one_variant) and the new test_pipeline_cache_allocation. No check
      that passes on main fails here, and the same 17 are skipped.
    • The other 13 failures are main's own on a Mac, the same on both sides; the macOS port's other pull requests
      address them: test_appimage_package_retry, test_collect_wide, test_control, test_framegen_resources,
      test_game_layout, test_low_memory_notice, test_mem_gate, test_mem_gate_children, test_mirror_trim,
      test_module_ids, test_oot_state_tap, test_pad_layout, test_prof_symbols.
    • No new crash report during this PR's run (main's run left one, from test_recorder_sync's fixture, which crashes
      on purpose).

Not tested:

  • Windows and Linux. Nothing was built or run there. Their lines are main's text, in the #else branches.
  • A renderer or game run on macOS. main's renderer does not build on macOS yet (the macOS port's other pull requests,
    #420); this PR is the cache-file part.
  • A really full disk, a quota, or a filesystem other than APFS. ENOSPC is injected at the F_PREALLOCATE call, and
    the file-size limit is real. A filesystem that refuses F_PREALLOCATE fails the save and keeps the old cache, by
    design.
  • macOS 14.4, the oldest the port supports. These calls are older than 14.4 and the test builds with
    -mmacosx-version-min=14.4, but it ran on 26.7.1 only.
  • Speed. F_RDADVISE is a hint, as posix_fadvise is; no timing is claimed.

Needs other testers:

  • Windows (llvm-mingw) and Linux: python -I -B tests/run_all.py pipeline_cache pipeline_play_cache pcache_header program_cache, expecting the same results as on main (request to open).
  • macOS 14.4 on Apple silicon: the same checks plus tests/test_pipeline_cache_allocation.py, expecting all to pass
    (request to open).

Screenshots

Nothing on screen.

Risk and rollback

  • Only macOS takes the new branches; the Windows and Linux paths are main's.
  • On macOS, a filesystem that refuses F_PREALLOCATE cannot save the pipeline cache: the save fails cleanly and the old
    cache stays. That is deliberate. A sparse mapping there could SIGBUS mid-write on a full disk.
  • Neither file is hashed into LONGSHOT_SHADER_STAMP (nx2/build.py:1176-1182), so no shader cache rebuilds. The cache
    formats are unchanged.
  • Reverting the commit is enough. There is no switch.

Reviewer notes

  • Read PipelineCacheFile::writable's order: create the empty temporary, reserve all of it or nothing, set its length,
    then map it. The two callers' handling of an empty result is unchanged.
  • F_RDADVISE's two ranges match the posix_fadvise ranges line for line.
  • main's #321 moved the caches into the game folder's cache/. The temporary is created beside its destination
    (create_temporary), so the reservation goes where the cache goes and nothing else here changes.
  • Review 587afb47 (one commit). It is macos/cache-files' 98b6d7cf cherry-picked onto main without a conflict
    (the same patch id), with the Assisted-by and Co-Authored-By trailers of this rebase added.

Issues

See #420 (the macOS port).

Checklist

For everyone

  • One topic.
  • The title is a Conventional Commit (type(scope): description).
  • Rebased on current main. Branched from 5e072fc8; main has moved since, and it is rebased in one sweep before ready.
  • python -I -B tests/run_all.py run and passing (any failure that also fails on main is named above). On macOS:
    532 passed, 17 skipped; the 13 failures are among main's 19 on this Mac, named under Testing. None is new.
  • Tested in the game, or what wasn't is named under Testing. No game run; named under Testing.
  • Testing filled in: what was tested, what wasn't, and what needs other testers (with test requests). The test
    requests are still to open.
  • Measured: main and this PR at named commits, on the same PC, scene and settings, at least 2 interleaved runs each,
    the median reported. For a fix: the tests fail on main and pass here, 2 interleaved rounds on each route.
  • No game files, dumps, build output or generated files.
  • third_party/ untouched (a new folder with a PROVENANCE.md instead).
  • The license is OK: GPL-3.0-or-later, nothing incompatible and nothing decompiled. Written from Apple's fcntl(2)
    and truncate(2) manuals and sys/fcntl.h; nothing copied.

If applicable

  • AI assistance recorded with Assisted-by: / Co-Authored-By: trailers.
  • A game-free test added for the fix, failing on main and passing here.
  • A check under tests/ that reads changed source updated in this PR, not weakened. (pipeline_cache_io_fault_test.cpp
    injects its failure at the new call.)
  • Docs updated.
## Why The renderer's two cache files do not compile on macOS. `gpu/vk/pipeline_cache_file.h` reserves the pipeline cache's disk space with Linux's `posix_fallocate`, and `gpu/vk/vk_progcache.cpp` asks for read-ahead with `POSIX_FADV_WILLNEED`; macOS has neither. So six of `main`'s game-free cache checks fail on a Mac, and no Mac renderer can build. The reservation itself has to stay: the driver writes the pipeline cache through a shared mapping, and on a sparse file a full disk is a `SIGBUS` in the middle of a write instead of a save that fails cleanly. ## What changed - vk: `PipelineCacheFile::writable` in `gpu/vk/pipeline_cache_file.h`. - On macOS the new, empty temporary reserves its whole capacity with `fcntl(F_PREALLOCATE)` (`F_ALLOCATEALL`, all or nothing; `F_PEOFPOSMODE`). Only then does `ftruncate` set its length and is it mapped. - Either call failing returns the empty result that both callers (`vk_draw.cpp:839` and `:10352`) already handle: the temporary is removed and the published cache is left as it was. There is no sparse-file fallback. - vk: `program_cache_load` in `gpu/vk/vk_progcache.cpp`. On macOS each record's two read hints (its first 4 KiB, its last 8 KiB) use `fcntl(F_RDADVISE)` for the same ranges as Linux's `posix_fadvise`. Best effort, as before: the reads that follow do the I/O and check it either way. `F_RDAHEAD` was not used, because it changes the file's read-ahead policy rather than asking for these ranges. - Windows and Linux: unchanged. Both Apple branches sit in `#if defined(__APPLE__)`, and `#else` keeps `main`'s lines as they are. - tests: `tests/pipeline_cache_io_fault_test.cpp`. On macOS its injected `ENOSPC` reaches the `F_PREALLOCATE` call (through an `fcntl` stand-in), so the existing native failure stages run there too. Linux's `posix_fallocate` stand-in is unchanged. - tests: `tests/test_pipeline_cache_allocation.py` (new; macOS only, skipped elsewhere). It compiles `pipeline_cache_file.h` (`--source-root` selects another tree) and checks, on a real file: - an 8,388,645-byte temporary has every block allocated before anything is written through its mapping (16,392 blocks of 512 bytes, against 0 for a control file sized with `ftruncate` alone); - a child process with a 4,096-byte file-size limit gets an empty result, the old 64-byte cache is kept, the temporaries are removed, and a retry succeeds. - docs: `docs/pipeline-cache-storage.md` says what macOS reserves and hints, and what the new test checks. ## Flags and settings None. ## Measurements main: `5e072fc8c2adadba5ea610148be1c47800ba0cca` (2026-11-14) this PR: `mac/cache-files` @ `587afb479e2005b0ba29bb64e6bed2059d6f680a` Hardware: Apple M4 Max (16 CPUs, 64 GiB), APFS, macOS 26.7.1 (25G241), Apple clang 21.0.0. Game-free: no GPU, no game Scene: none (a portability fix; its proof is the tests, as CONTRIBUTING's `fix` row allows) Method: the six cache checks and the new test, compiled for arm64 and for x86_64 (run under Rosetta), against `main`'s source (`git archive 5e072fc8`) and against this PR. Two interleaved rounds per route: PR, main, PR, main. Then the full game-free suite on this PR, compared by test name with `main`'s on the same Mac. How to reproduce: `python -I -B tests/run_all.py pipeline_cache pipeline_play_cache pcache_header program_cache_cleanup program_cache_one_variant` on each side; `python -I -B tests/test_pipeline_cache_allocation.py --cxx clang++ --source-root <a tree of main>` for the new test against `main`'s header. For Rosetta, pass a compiler that adds `-arch x86_64`. | | main @ 5e072fc8 | this PR @ 587afb47 | |---|---|---| | `test_pipeline_cache_file.py` | compile error: `posix_fallocate` | pass | | `test_pipeline_cache_save.py` | compile error: `posix_fallocate` | pass: 14 cases, the optional comparison with an original source skipped as on every platform | | `test_pipeline_play_cache.py` | compile error: `posix_fallocate` | pass: all 12 cases | | `test_pcache_header_read.py` | compile error: `posix_fallocate` | pass | | `test_program_cache_cleanup.py` | compile error: `POSIX_FADV_WILLNEED` | pass: 9 cases | | `test_program_cache_one_variant.py` | compile error: `POSIX_FADV_WILLNEED` | pass: the same variant for each of 9 picks | | `test_pipeline_cache_allocation.py` (new) | compile error: `posix_fallocate` | pass: 2 cases | Each row is the same on arm64 and under Rosetta, in both rounds: 7 of 7 fail on `main` and 7 of 7 pass here, 28 runs in all. ``` main @ 5e072fc8 (arm64 and x86_64, both rounds): gpu/vk/pipeline_cache_file.h:274:13: error: use of undeclared identifier 'posix_fallocate' cache_functions.inc:469:61: error: use of undeclared identifier 'POSIX_FADV_WILLNEED' BATTERY ... failed=7 of 7 this PR @ 587afb47 (arm64 and x86_64, both rounds): {"filesystem":"apfs","capacity":8388645,"emptyBlocks":0,"sparseBlocks":0,"reservedBlocksBeforeWrite":16392,"publishedBlocks":16392,"passed":true} {"limitBytes":4096,"refusedEmpty":true,"oldOutputPreserved":true,"retryPassed":true} {"nativeFailureStages":13,"oldOutputPreserved":true,"retryPassed":true,"ownedTemporariesCleaned":true,"readIoFailureNotIncompatible":true} BATTERY ... failed=0 of 7 ``` The new test also catches a missing reservation, not only `main`'s compile error. With this PR's header and only the `F_PREALLOCATE` call removed (a throwaway copy, not committed), it compiles and fails on both routes: ``` AssertionError: 1 != 0 : FAIL line 49: reserved.st_blocks > empty.st_blocks ``` ## Testing **Tested:** - On an Apple M4 Max with macOS 26.7.1, game-free only, on arm64 and under Rosetta (x86_64), as above. - The full suite, `tests/run_all.py`, 6 at once, on this PR, compared by test name with `main` on the same Mac (the runner labels macOS `linux`): ``` main @ 5e072fc8: 525 passed, 19 failed, 17 skipped in 177 s (linux, 6 at once) this PR @ 587afb47: 532 passed, 13 failed, 17 skipped in 173 s (linux, 6 at once) ``` - **By test name**, seven checks changed status, all to pass: the six cache checks that fail on `main` (`test_pcache_header_read`, `test_pipeline_cache_file`, `test_pipeline_cache_save`, `test_pipeline_play_cache`, `test_program_cache_cleanup`, `test_program_cache_one_variant`) and the new `test_pipeline_cache_allocation`. No check that passes on `main` fails here, and the same 17 are skipped. - **The other 13 failures are `main`'s own on a Mac**, the same on both sides; the macOS port's other pull requests address them: `test_appimage_package_retry`, `test_collect_wide`, `test_control`, `test_framegen_resources`, `test_game_layout`, `test_low_memory_notice`, `test_mem_gate`, `test_mem_gate_children`, `test_mirror_trim`, `test_module_ids`, `test_oot_state_tap`, `test_pad_layout`, `test_prof_symbols`. - No new crash report during this PR's run (`main`'s run left one, from `test_recorder_sync`'s fixture, which crashes on purpose). **Not tested:** - **Windows and Linux.** Nothing was built or run there. Their lines are `main`'s text, in the `#else` branches. - **A renderer or game run on macOS.** `main`'s renderer does not build on macOS yet (the macOS port's other pull requests, #420); this PR is the cache-file part. - **A really full disk, a quota, or a filesystem other than APFS.** `ENOSPC` is injected at the `F_PREALLOCATE` call, and the file-size limit is real. A filesystem that refuses `F_PREALLOCATE` fails the save and keeps the old cache, by design. - **macOS 14.4**, the oldest the port supports. These calls are older than 14.4 and the test builds with `-mmacosx-version-min=14.4`, but it ran on 26.7.1 only. - **Speed.** `F_RDADVISE` is a hint, as `posix_fadvise` is; no timing is claimed. **Needs other testers:** - Windows (llvm-mingw) and Linux: `python -I -B tests/run_all.py pipeline_cache pipeline_play_cache pcache_header program_cache`, expecting the same results as on `main` (request to open). - macOS 14.4 on Apple silicon: the same checks plus `tests/test_pipeline_cache_allocation.py`, expecting all to pass (request to open). ## Screenshots Nothing on screen. ## Risk and rollback - Only macOS takes the new branches; the Windows and Linux paths are `main`'s. - On macOS, a filesystem that refuses `F_PREALLOCATE` cannot save the pipeline cache: the save fails cleanly and the old cache stays. That is deliberate. A sparse mapping there could `SIGBUS` mid-write on a full disk. - Neither file is hashed into `LONGSHOT_SHADER_STAMP` (`nx2/build.py:1176-1182`), so no shader cache rebuilds. The cache formats are unchanged. - Reverting the commit is enough. There is no switch. ## Reviewer notes - Read `PipelineCacheFile::writable`'s order: create the empty temporary, reserve all of it or nothing, set its length, then map it. The two callers' handling of an empty result is unchanged. - `F_RDADVISE`'s two ranges match the `posix_fadvise` ranges line for line. - `main`'s #321 moved the caches into the game folder's `cache/`. The temporary is created beside its destination (`create_temporary`), so the reservation goes where the cache goes and nothing else here changes. - Review `587afb47` (one commit). It is `macos/cache-files`' `98b6d7cf` cherry-picked onto `main` without a conflict (the same patch id), with the `Assisted-by` and `Co-Authored-By` trailers of this rebase added. ## Issues See #420 (the macOS port). ## Checklist ### For everyone - [x] One topic. - [x] The title is a Conventional Commit (`type(scope): description`). - [ ] Rebased on current `main`. Branched from 5e072fc8; main has moved since, and it is rebased in one sweep before ready. - [x] `python -I -B tests/run_all.py` run and passing (any failure that also fails on `main` is named above). On macOS: 532 passed, 17 skipped; the 13 failures are among `main`'s 19 on this Mac, named under Testing. None is new. - [x] Tested in the game, or what wasn't is named under Testing. No game run; named under Testing. - [ ] Testing filled in: what was tested, what wasn't, and what needs other testers (with test requests). The test requests are still to open. - [x] Measured: main and this PR at named commits, on the same PC, scene and settings, at least 2 interleaved runs each, the median reported. For a `fix`: the tests fail on `main` and pass here, 2 interleaved rounds on each route. - [x] No game files, dumps, build output or generated files. - [x] `third_party/` untouched (a new folder with a `PROVENANCE.md` instead). - [x] The license is OK: GPL-3.0-or-later, nothing incompatible and nothing decompiled. Written from Apple's `fcntl(2)` and `truncate(2)` manuals and `sys/fcntl.h`; nothing copied. ### If applicable - [x] AI assistance recorded with `Assisted-by:` / `Co-Authored-By:` trailers. - [x] A game-free test added for the fix, failing on `main` and passing here. - [x] A check under `tests/` that reads changed source updated in this PR, not weakened. (`pipeline_cache_io_fault_test.cpp` injects its failure at the new call.) - [x] Docs updated.
fix(vk): preserve cache file reservations on macOS
All checks were successful
CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Has been skipped
CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Has been skipped
CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Has been skipped
Third-party parts / The same as upstream (pull_request) Has been skipped
PR description / Description follows the template (pull_request_target) Successful in 5s
CI / Conventional Commit title (pull_request) Successful in 7s
CI / Workflow lint (actionlint) (pull_request) Successful in 7s
Third-party parts / As their notes say (pull_request) Successful in 11s
Website / Website on Cloudflare (pull_request_target) Has been skipped
587afb479e
Reserve each empty pipeline-cache temporary with F_PREALLOCATE and
F_ALLOCATEALL before setting its length and exposing a writable mapping.
Keep failed reservations on the existing empty-result and cleanup path.
Use best-effort F_RDADVISE for the program cache's existing read ranges.

Leave the Windows and Linux implementation lines unchanged. Cover real
allocated blocks and a child-only file-size limit, and retain native
allocation-failure injection through the Darwin fcntl call.
Document the file guarantees and the incomplete renderer-port boundary.

Assisted-by: GPT-6 Astra
Co-Authored-By: GPT-6 Astra <[email protected]>
Assisted-by: Claude Opus 5.5
Co-Authored-By: Claude Opus 5.5 <[email protected]>
All checks were successful
CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Has been skipped
CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Has been skipped
CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Has been skipped
Third-party parts / The same as upstream (pull_request) Has been skipped
PR description / Description follows the template (pull_request_target) Successful in 5s
CI / Conventional Commit title (pull_request) Successful in 7s
CI / Workflow lint (actionlint) (pull_request) Successful in 7s
Third-party parts / As their notes say (pull_request) Successful in 11s
Website / Website on Cloudflare (pull_request_target) Has been skipped
This pull request is marked as a work in progress.
This branch is out-of-date with the base branch
View command line instructions

Checkout

From your project repository, check out a new branch and test the changes.
git fetch -u 'mac/cache-files:mooki-mac/cache-files'
git switch 'mooki-mac/cache-files'
Sign in to join this conversation.
No reviewers
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
MagicBeanDev/Longshot!476
No description provided.