fix(vk): preserve cache file reservations on macOS

mooki commented

2026-11-14 16:11:03 +00:00

Member

Type: fix · Risk: low · Tested on: macOS 26.7.1, Apple M4 Max (game-free checks, arm64 and Rosetta x86_64; no game run) · Needs: none

Why

The renderer's two cache files do not compile on macOS. gpu/vk/pipeline_cache_file.h reserves the pipeline cache's
disk space with Linux's posix_fallocate, and gpu/vk/vk_progcache.cpp asks for read-ahead with POSIX_FADV_WILLNEED;
macOS has neither. So six of main's game-free cache checks fail on a Mac, and no Mac renderer can build. The reservation
itself has to stay: the driver writes the pipeline cache through a shared mapping, and on a sparse file a full disk is a
SIGBUS in the middle of a write instead of a save that fails cleanly. Part of the macOS port (#420).

How to reproduce

Seen on: macOS 26.7.1, Apple M4 Max, APFS, Apple clang 21.0.0, arm64 and x86_64 under Rosetta (main's game-free
cache checks). Only macOS is affected: Windows and Linux have their own calls.
Settings and switches: defaults.
Steps: on a Mac, on main, run python -I -B tests/run_all.py pipeline_cache pipeline_play_cache pcache_header program_cache_cleanup program_cache_one_variant.
Expected: the six cache checks compile and pass.
Actual: at main @ 6c9edffe, on both routes: gpu/vk/pipeline_cache_file.h:274:13: error: use of undeclared identifier 'posix_fallocate' and cache_functions.inc:469:61: error: use of undeclared identifier 'POSIX_FADV_WILLNEED'; all six FAIL.

What changed

On macOS the pipeline cache's temporary reserves its whole capacity with fcntl(F_PREALLOCATE) before it gets its
length and is mapped, and the program cache's read hints use fcntl(F_RDADVISE) for the same ranges. Both are Apple-only
branches with main's lines in #else; a failed reservation is the empty result the callers already handle.

Read first: PipelineCacheFile::writable's order: create the empty temporary, reserve all of it or nothing, set its
length, then map it. The two callers' handling of an empty result is unchanged.
vk: PipelineCacheFile::writable in gpu/vk/pipeline_cache_file.h.
- On macOS the new, empty temporary reserves its whole capacity with fcntl(F_PREALLOCATE) (F_ALLOCATEALL, all or
  nothing; F_PEOFPOSMODE). Only then does ftruncate set its length and is it mapped.
- Either call failing returns the empty result that both callers (vk_draw.cpp:839 and :10352) already handle: the
  temporary is removed and the published cache is left as it was. There is no sparse-file fallback.
vk: program_cache_load in gpu/vk/vk_progcache.cpp. On macOS each record's two read hints (its first 4 KiB, its last
8 KiB) use fcntl(F_RDADVISE) for the same ranges as Linux's posix_fadvise, line for line. Best effort, as before:
the reads that follow do the I/O and check it either way. F_RDAHEAD was not used, because it changes the file's
read-ahead policy rather than asking for these ranges.
Windows and Linux: unchanged. Both Apple branches sit in #if defined(__APPLE__), and #else keeps main's lines
as they are.
tests: tests/pipeline_cache_io_fault_test.cpp. On macOS its injected ENOSPC reaches the F_PREALLOCATE call
(through an fcntl stand-in), so the existing native failure stages run there too. Linux's posix_fallocate
stand-in is unchanged.
tests: tests/test_pipeline_cache_allocation.py (new; macOS only: its test class is skipped elsewhere with
unittest.skipUnless(sys.platform == "darwin", …)). It compiles pipeline_cache_file.h (--source-root selects
another tree) and checks, on a real file:
- an 8,388,645-byte temporary has every block allocated before anything is written through its mapping (16,392
  blocks of 512 bytes, against 0 for a control file sized with ftruncate alone);
- a child process with a 4,096-byte file-size limit gets an empty result, the old 64-byte cache is kept, the
  temporaries are removed, and a retry succeeds.
docs: docs/pipeline-cache-storage.md says what macOS reserves and hints, and what the new test checks.
Design: main's #321 moved the caches into the game folder's cache/. The temporary is created beside its
destination (create_temporary), so the reservation goes where the cache goes and nothing else here changes.
Commits: review 5b929e7f (one commit). It is macos/cache-files' 98b6d7cf cherry-picked onto main without a
conflict (the same patch id), with the Assisted-by and Co-Authored-By trailers of that first rebase added. Rebased
again on main 6c9edffe (2026-11-14) without a conflict; main had changed vk_progcache.cpp (#370) away from its
lines.
Written from Apple's fcntl(2) and truncate(2) manuals and sys/fcntl.h; nothing copied.

Measurements

Verdict: fixed, a game-free proof (no timing claimed): on both routes 7 of 7 cache checks fail on main and 7 of 7
pass here, after the rebase as before it, and the full suite changes only by those seven (538 passed against main's
531, 26 failed against 32).

Commits: main 6c9edffe5d (2026-11-14); this PR mac/cache-files @
5b929e7fc1. Before the rebase: main 5e072fc8, this PR 587afb47.
Hardware: Apple M4 Max (16 CPUs, 64 GiB), APFS, macOS 26.7.1 (25G241), Apple clang 21.0.0. Game-free: no GPU, no game
Scene: none (a portability fix; its proof is the tests, as CONTRIBUTING's fix row allows)
Method: the six cache checks and the new test, compiled for arm64 and for x86_64 (run under Rosetta), against main's
source and against this PR. Before the rebase (main 5e072fc8, this PR 587afb47): two interleaved rounds per route, PR,
main, PR, main. After it (main 6c9edffe, this PR 5b929e7f): one round per route, with the same results; main's arm64 six
from its baseline suite run. Then the full game-free suite on this PR, compared by test name with main's on the same
Mac.
How to reproduce: python -I -B tests/run_all.py pipeline_cache pipeline_play_cache pcache_header program_cache_cleanup program_cache_one_variant on each side; python -I -B tests/test_pipeline_cache_allocation.py --cxx clang++ --source-root <a tree of main> for the new test against main's header. For Rosetta, pass a compiler that adds
-arch x86_64.

	Before: main @ `6c9edffe`	After: this PR @ `5b929e7f`
`test_pipeline_cache_file.py`	compile error: `posix_fallocate`	pass
`test_pipeline_cache_save.py`	compile error: `posix_fallocate`	pass: 14 cases, the optional comparison with an original source skipped as on every platform
`test_pipeline_play_cache.py`	compile error: `posix_fallocate`	pass: all 12 cases
`test_pcache_header_read.py`	compile error: `posix_fallocate`	pass
`test_program_cache_cleanup.py`	compile error: `POSIX_FADV_WILLNEED`	pass: 9 cases
`test_program_cache_one_variant.py`	compile error: `POSIX_FADV_WILLNEED`	pass: the same variant for each of 9 picks
`test_pipeline_cache_allocation.py` (new)	compile error: `posix_fallocate`	pass: 2 cases

Each row is the same on arm64 and under Rosetta, in every round: 7 of 7 fail on main and 7 of 7 pass here (28 runs
before the rebase, 14 after it, plus main's arm64 six in its suite run).

main @ 6c9edffe (arm64 and x86_64):
gpu/vk/pipeline_cache_file.h:274:13: error: use of undeclared identifier 'posix_fallocate'
cache_functions.inc:469:61: error: use of undeclared identifier 'POSIX_FADV_WILLNEED'
BATTERY ... failed=7 of 7

this PR @ 5b929e7f (arm64 and x86_64):
{"filesystem":"apfs","capacity":8388645,"emptyBlocks":0,"sparseBlocks":0,"reservedBlocksBeforeWrite":16392,"publishedBlocks":16392,"passed":true}
{"limitBytes":4096,"refusedEmpty":true,"oldOutputPreserved":true,"retryPassed":true}
{"nativeFailureStages":13,"oldOutputPreserved":true,"retryPassed":true,"ownedTemporariesCleaned":true,"readIoFailureNotIncompatible":true}
BATTERY ... failed=0 of 7

The new test also catches a missing reservation, not only main's compile error. Before the rebase, with this PR's
header and only the F_PREALLOCATE call removed (a throwaway copy, not committed), it compiled and failed on both
routes:

AssertionError: 1 != 0 : FAIL line 49: reserved.st_blocks > empty.st_blocks

Screenshots: none; nothing on screen.

Flags and settings

None.

Testing

Tested:

On an Apple M4 Max with macOS 26.7.1, game-free only, on arm64 and under Rosetta (x86_64), as above.
The full suite, tests/run_all.py, 6 at once, on this PR, compared by test name with main on the same Mac (the
runner labels macOS linux):
```
main @ 6c9edffe:     531 passed, 32 failed, 17 skipped in 181 s (linux, 6 at once)
this PR @ 5b929e7f:  538 passed, 26 failed, 17 skipped in 296 s (linux, 6 at once)
```
- By test name, against main 6c9edffe's run on the same Mac: 7 statuses changed: test_pipeline_cache_allocation,
  absent on main, passes here; test_pcache_header_read: FAIL on main, pass here; test_pipeline_cache_file: FAIL on
  main, pass here; test_pipeline_cache_save: FAIL on main, pass here; test_pipeline_play_cache: FAIL on main, pass
  here; test_program_cache_cleanup: FAIL on main, pass here; test_program_cache_one_variant: FAIL on main, pass
  here.
- These 26 checks fail here and on main on this Mac, by name (not triaged here: some need the macOS port's other pull
  requests, some are checks main added or changed since 5e072fc8): test_appimage_package_retry,
  test_appimage_player_build, test_collect_wide, test_control, test_cpm_discovery, test_crash_dialog,
  test_crash_record, test_env_on, test_game_layout, test_lift_cleanup_error, test_linux_preflight,
  test_longshot_ctl, test_low_memory_notice, test_make_game_cache, test_mem_gate, test_mem_gate_children,
  test_mirror_trim, test_oot_state_tap, test_pipeline_tour, test_prof_symbols, test_readback_late,
  test_report_run, test_split_giants, test_vk_pgo, test_vram_budget_row, test_walk_key.
- The same 17 checks skipped as on main.
- Crash reports during the run: none (main's run: fault-arm64-2026-11-14-180704.ips,
  recorder_sync_test.exe-2026-11-14-180744.ips).
The Windows and Linux text after the rebase, by unifdef -U__APPLE__ of each changed C++ file against main @
6c9edffe: gpu/vk/pipeline_cache_file.h, gpu/vk/vk_progcache.cpp and tests/pipeline_cache_io_fault_test.cpp
identical.
After the rebase onto main aa95d35f, at 7e561fbb (5b929e7f before the rebases onto e090cc48 and aa95d35f):
Sweep3's checks of this head (the C/C++ it changes compiled, and its own tests run, on arm64 and x86_64 where they
apply) gave the same results as before; python -I -B tests/run_all.py on wave S1's merge 11a5b4e7, which carries
this head: 568/22/17 (passed/failed/skipped) against main aa95d35f's 543/33/17 on this Mac, no new failure
(reports/2026-11-15-rebase-sweep-3.md).

Not tested:

Windows and Linux. Nothing was built or run there. Their lines are main's text, in the #else branches.
A renderer or game run on macOS. main's renderer does not build on macOS yet (the macOS port's other pull requests,
#420); this PR is the cache-file part.
A really full disk, a quota, or a filesystem other than this Mac's (Hardware line). ENOSPC is injected at the F_PREALLOCATE call, and
the file-size limit is real. A filesystem that refuses F_PREALLOCATE fails the save and keeps the old cache, by
design.
macOS 14.4, the oldest the port supports. These calls are older than 14.4 and the test builds with
-mmacosx-version-min=14.4, but it ran on 26.7.1 only; on macOS 14.4 on Apple silicon the check would be the six
cache checks and the new test, expecting all to pass.
Speed. F_RDADVISE is a hint, as posix_fadvise is; no timing is claimed.

Must be tested: None: the unifdef comparison shows Windows and Linux compile main's text byte for byte (unifdef -U__APPLE__ of pipeline_cache_file.h, vk_progcache.cpp and pipeline_cache_io_fault_test.cpp is identical to
main's), and the new test's class is skipped there. The macOS side is covered under Tested. The same comparison at the
rebased head 7e561fbb against main aa95d35f gives the same result.

Risks

What could break:

Only macOS takes the new branches; the Windows and Linux paths are main's.
On macOS, a filesystem that refuses F_PREALLOCATE cannot save the pipeline cache: the save fails cleanly and the old
cache stays. That is deliberate. A sparse mapping there could SIGBUS mid-write on a full disk.
Landing order: it depends on no other PR.

Turning it off: revert the commit. There is no switch.

Compatibility: Nothing changes. Neither file is one of the translator stamp's (nx2/translator_files.py), so no
shader cache rebuilds. The cache formats are unchanged; saves, the options file and the vk ABI too.

Issues

See #420 (the macOS port).

Checklist

Everyone

One topic.
The title is a Conventional Commit (type(scope): description) of the status line's type.
Why gives the reason plainly and links the issues that fit (the tracker searched).
Measurements has a verdict, or one line saying why there is nothing to measure.
Testing names what was tested, what wasn't, and what must still be tested, where and how. (nothing to test elsewhere: Must be tested is None)
Rebased on current main: on aa95d35f (#761, 2026-11-15), head 7e561fbb, without a conflict; this PR's own
commits, old → new: 5b929e7f → 7e561fbb; the commit names elsewhere in this description are from before the
rebase.
python -I -B tests/run_all.py passes (any failure that also fails on main is named). On macOS: 538 passed, 17
skipped, and 26 failures that also fail on main 6c9edffe on this Mac, named under Testing. After the rebase onto main
aa95d35f (head 7e561fbb): wave S1's merge 11a5b4e7, 568/22/17 against main aa95d35f's 543/33/17, no new failure.
No game files, dumps, build output or generated files.
third_party/ untouched (a new folder with a PROVENANCE.md instead).
The license is OK: GPL-3.0-or-later, nothing incompatible and nothing decompiled. Written from Apple's fcntl(2)
and truncate(2) manuals and sys/fcntl.h; nothing copied.

When it applies

AI assistance recorded with Assisted-by: / Co-Authored-By: trailers.
A game-free test added for the fix, failing on main and passing here.
A check under tests/ that reads changed source updated in this PR, not weakened. (pipeline_cache_io_fault_test.cpp
injects its failure at the new call.)
Docs updated.

**Type:** fix · **Risk:** low · **Tested on:** macOS 26.7.1, Apple M4 Max (game-free checks, arm64 and Rosetta x86_64; no game run) · **Needs:** none ## Why The renderer's two cache files do not compile on macOS. `gpu/vk/pipeline_cache_file.h` reserves the pipeline cache's disk space with Linux's `posix_fallocate`, and `gpu/vk/vk_progcache.cpp` asks for read-ahead with `POSIX_FADV_WILLNEED`; macOS has neither. So six of `main`'s game-free cache checks fail on a Mac, and no Mac renderer can build. The reservation itself has to stay: the driver writes the pipeline cache through a shared mapping, and on a sparse file a full disk is a `SIGBUS` in the middle of a write instead of a save that fails cleanly. Part of the macOS port (#420). ## How to reproduce **Seen on:** macOS 26.7.1, Apple M4 Max, APFS, Apple clang 21.0.0, arm64 and x86_64 under Rosetta (`main`'s game-free cache checks). Only macOS is affected: Windows and Linux have their own calls. **Settings and switches:** defaults. **Steps:** on a Mac, on `main`, run `python -I -B tests/run_all.py pipeline_cache pipeline_play_cache pcache_header program_cache_cleanup program_cache_one_variant`. **Expected:** the six cache checks compile and pass. **Actual:** at `main` @ `6c9edffe`, on both routes: `gpu/vk/pipeline_cache_file.h:274:13: error: use of undeclared identifier 'posix_fallocate'` and `cache_functions.inc:469:61: error: use of undeclared identifier 'POSIX_FADV_WILLNEED'`; all six FAIL. ## What changed On macOS the pipeline cache's temporary reserves its whole capacity with `fcntl(F_PREALLOCATE)` before it gets its length and is mapped, and the program cache's read hints use `fcntl(F_RDADVISE)` for the same ranges. Both are Apple-only branches with `main`'s lines in `#else`; a failed reservation is the empty result the callers already handle. - Read first: `PipelineCacheFile::writable`'s order: create the empty temporary, reserve all of it or nothing, set its length, then map it. The two callers' handling of an empty result is unchanged. - vk: `PipelineCacheFile::writable` in `gpu/vk/pipeline_cache_file.h`. - On macOS the new, empty temporary reserves its whole capacity with `fcntl(F_PREALLOCATE)` (`F_ALLOCATEALL`, all or nothing; `F_PEOFPOSMODE`). Only then does `ftruncate` set its length and is it mapped. - Either call failing returns the empty result that both callers (`vk_draw.cpp:839` and `:10352`) already handle: the temporary is removed and the published cache is left as it was. There is no sparse-file fallback. - vk: `program_cache_load` in `gpu/vk/vk_progcache.cpp`. On macOS each record's two read hints (its first 4 KiB, its last 8 KiB) use `fcntl(F_RDADVISE)` for the same ranges as Linux's `posix_fadvise`, line for line. Best effort, as before: the reads that follow do the I/O and check it either way. `F_RDAHEAD` was not used, because it changes the file's read-ahead policy rather than asking for these ranges. - Windows and Linux: unchanged. Both Apple branches sit in `#if defined(__APPLE__)`, and `#else` keeps `main`'s lines as they are. - tests: `tests/pipeline_cache_io_fault_test.cpp`. On macOS its injected `ENOSPC` reaches the `F_PREALLOCATE` call (through an `fcntl` stand-in), so the existing native failure stages run there too. Linux's `posix_fallocate` stand-in is unchanged. - tests: `tests/test_pipeline_cache_allocation.py` (new; macOS only: its test class is skipped elsewhere with `unittest.skipUnless(sys.platform == "darwin", …)`). It compiles `pipeline_cache_file.h` (`--source-root` selects another tree) and checks, on a real file: - an 8,388,645-byte temporary has every block allocated before anything is written through its mapping (16,392 blocks of 512 bytes, against 0 for a control file sized with `ftruncate` alone); - a child process with a 4,096-byte file-size limit gets an empty result, the old 64-byte cache is kept, the temporaries are removed, and a retry succeeds. - docs: `docs/pipeline-cache-storage.md` says what macOS reserves and hints, and what the new test checks. - Design: `main`'s #321 moved the caches into the game folder's `cache/`. The temporary is created beside its destination (`create_temporary`), so the reservation goes where the cache goes and nothing else here changes. - Commits: review `5b929e7f` (one commit). It is `macos/cache-files`' `98b6d7cf` cherry-picked onto `main` without a conflict (the same patch id), with the `Assisted-by` and `Co-Authored-By` trailers of that first rebase added. Rebased again on `main` 6c9edffe (2026-11-14) without a conflict; main had changed `vk_progcache.cpp` (#370) away from its lines. - Written from Apple's `fcntl(2)` and `truncate(2)` manuals and `sys/fcntl.h`; nothing copied. ## Measurements **Verdict:** fixed, a game-free proof (no timing claimed): on both routes 7 of 7 cache checks fail on `main` and 7 of 7 pass here, after the rebase as before it, and the full suite changes only by those seven (538 passed against `main`'s 531, 26 failed against 32). Commits: main 6c9edffe5da1f5b5383977aeab3f544c0de25d4c (2026-11-14); this PR `mac/cache-files` @ 5b929e7fc1fb150f33360b36f5adbb6fc136e736. Before the rebase: main 5e072fc8, this PR 587afb47. Hardware: Apple M4 Max (16 CPUs, 64 GiB), APFS, macOS 26.7.1 (25G241), Apple clang 21.0.0. Game-free: no GPU, no game Scene: none (a portability fix; its proof is the tests, as CONTRIBUTING's `fix` row allows) Method: the six cache checks and the new test, compiled for arm64 and for x86_64 (run under Rosetta), against `main`'s source and against this PR. Before the rebase (main 5e072fc8, this PR 587afb47): two interleaved rounds per route, PR, main, PR, main. After it (main 6c9edffe, this PR 5b929e7f): one round per route, with the same results; main's arm64 six from its baseline suite run. Then the full game-free suite on this PR, compared by test name with `main`'s on the same Mac. How to reproduce: `python -I -B tests/run_all.py pipeline_cache pipeline_play_cache pcache_header program_cache_cleanup program_cache_one_variant` on each side; `python -I -B tests/test_pipeline_cache_allocation.py --cxx clang++ --source-root <a tree of main>` for the new test against `main`'s header. For Rosetta, pass a compiler that adds `-arch x86_64`. | | Before: main @ 6c9edffe | After: this PR @ 5b929e7f | |---|---|---| | `test_pipeline_cache_file.py` | compile error: `posix_fallocate` | pass | | `test_pipeline_cache_save.py` | compile error: `posix_fallocate` | pass: 14 cases, the optional comparison with an original source skipped as on every platform | | `test_pipeline_play_cache.py` | compile error: `posix_fallocate` | pass: all 12 cases | | `test_pcache_header_read.py` | compile error: `posix_fallocate` | pass | | `test_program_cache_cleanup.py` | compile error: `POSIX_FADV_WILLNEED` | pass: 9 cases | | `test_program_cache_one_variant.py` | compile error: `POSIX_FADV_WILLNEED` | pass: the same variant for each of 9 picks | | `test_pipeline_cache_allocation.py` (new) | compile error: `posix_fallocate` | pass: 2 cases | Each row is the same on arm64 and under Rosetta, in every round: 7 of 7 fail on `main` and 7 of 7 pass here (28 runs before the rebase, 14 after it, plus main's arm64 six in its suite run). ``` main @ 6c9edffe (arm64 and x86_64): gpu/vk/pipeline_cache_file.h:274:13: error: use of undeclared identifier 'posix_fallocate' cache_functions.inc:469:61: error: use of undeclared identifier 'POSIX_FADV_WILLNEED' BATTERY ... failed=7 of 7 this PR @ 5b929e7f (arm64 and x86_64): {"filesystem":"apfs","capacity":8388645,"emptyBlocks":0,"sparseBlocks":0,"reservedBlocksBeforeWrite":16392,"publishedBlocks":16392,"passed":true} {"limitBytes":4096,"refusedEmpty":true,"oldOutputPreserved":true,"retryPassed":true} {"nativeFailureStages":13,"oldOutputPreserved":true,"retryPassed":true,"ownedTemporariesCleaned":true,"readIoFailureNotIncompatible":true} BATTERY ... failed=0 of 7 ``` The new test also catches a missing reservation, not only `main`'s compile error. Before the rebase, with this PR's header and only the `F_PREALLOCATE` call removed (a throwaway copy, not committed), it compiled and failed on both routes: ``` AssertionError: 1 != 0 : FAIL line 49: reserved.st_blocks > empty.st_blocks ``` Screenshots: none; nothing on screen. ### Flags and settings None. ## Testing **Tested:** - On an Apple M4 Max with macOS 26.7.1, game-free only, on arm64 and under Rosetta (x86_64), as above. - The full suite, `tests/run_all.py`, 6 at once, on this PR, compared by test name with `main` on the same Mac (the runner labels macOS `linux`): ``` main @ 6c9edffe: 531 passed, 32 failed, 17 skipped in 181 s (linux, 6 at once) this PR @ 5b929e7f: 538 passed, 26 failed, 17 skipped in 296 s (linux, 6 at once) ``` - By test name, against main 6c9edffe's run on the same Mac: 7 statuses changed: `test_pipeline_cache_allocation`, absent on main, passes here; `test_pcache_header_read`: FAIL on main, pass here; `test_pipeline_cache_file`: FAIL on main, pass here; `test_pipeline_cache_save`: FAIL on main, pass here; `test_pipeline_play_cache`: FAIL on main, pass here; `test_program_cache_cleanup`: FAIL on main, pass here; `test_program_cache_one_variant`: FAIL on main, pass here. - These 26 checks fail here and on main on this Mac, by name (not triaged here: some need the macOS port's other pull requests, some are checks main added or changed since 5e072fc8): `test_appimage_package_retry`, `test_appimage_player_build`, `test_collect_wide`, `test_control`, `test_cpm_discovery`, `test_crash_dialog`, `test_crash_record`, `test_env_on`, `test_game_layout`, `test_lift_cleanup_error`, `test_linux_preflight`, `test_longshot_ctl`, `test_low_memory_notice`, `test_make_game_cache`, `test_mem_gate`, `test_mem_gate_children`, `test_mirror_trim`, `test_oot_state_tap`, `test_pipeline_tour`, `test_prof_symbols`, `test_readback_late`, `test_report_run`, `test_split_giants`, `test_vk_pgo`, `test_vram_budget_row`, `test_walk_key`. - The same 17 checks skipped as on main. - Crash reports during the run: none (main's run: fault-arm64-2026-11-14-180704.ips, recorder_sync_test.exe-2026-11-14-180744.ips). - The Windows and Linux text after the rebase, by `unifdef -U__APPLE__` of each changed C++ file against `main` @ `6c9edffe`: `gpu/vk/pipeline_cache_file.h`, `gpu/vk/vk_progcache.cpp` and `tests/pipeline_cache_io_fault_test.cpp` identical. - **After the rebase onto main aa95d35f**, at `7e561fbb` (`5b929e7f` before the rebases onto e090cc48 and aa95d35f): Sweep3's checks of this head (the C/C++ it changes compiled, and its own tests run, on arm64 and x86_64 where they apply) gave the same results as before; `python -I -B tests/run_all.py` on wave S1's merge `11a5b4e7`, which carries this head: 568/22/17 (passed/failed/skipped) against main aa95d35f's 543/33/17 on this Mac, no new failure (`reports/2026-11-15-rebase-sweep-3.md`). **Not tested:** - **Windows and Linux.** Nothing was built or run there. Their lines are `main`'s text, in the `#else` branches. - **A renderer or game run on macOS.** `main`'s renderer does not build on macOS yet (the macOS port's other pull requests, #420); this PR is the cache-file part. - **A really full disk, a quota, or a filesystem other than this Mac's (Hardware line).** `ENOSPC` is injected at the `F_PREALLOCATE` call, and the file-size limit is real. A filesystem that refuses `F_PREALLOCATE` fails the save and keeps the old cache, by design. - **macOS 14.4**, the oldest the port supports. These calls are older than 14.4 and the test builds with `-mmacosx-version-min=14.4`, but it ran on 26.7.1 only; on macOS 14.4 on Apple silicon the check would be the six cache checks and the new test, expecting all to pass. - **Speed.** `F_RDADVISE` is a hint, as `posix_fadvise` is; no timing is claimed. **Must be tested:** None: the unifdef comparison shows Windows and Linux compile `main`'s text byte for byte (`unifdef -U__APPLE__` of `pipeline_cache_file.h`, `vk_progcache.cpp` and `pipeline_cache_io_fault_test.cpp` is identical to `main`'s), and the new test's class is skipped there. The macOS side is covered under Tested. The same comparison at the rebased head `7e561fbb` against main aa95d35f gives the same result. ## Risks **What could break:** - Only macOS takes the new branches; the Windows and Linux paths are `main`'s. - On macOS, a filesystem that refuses `F_PREALLOCATE` cannot save the pipeline cache: the save fails cleanly and the old cache stays. That is deliberate. A sparse mapping there could `SIGBUS` mid-write on a full disk. - Landing order: it depends on no other PR. **Turning it off:** revert the commit. There is no switch. **Compatibility:** Nothing changes. Neither file is one of the translator stamp's (`nx2/translator_files.py`), so no shader cache rebuilds. The cache formats are unchanged; saves, the options file and the vk ABI too. ## Issues See #420 (the macOS port). ## Checklist ### Everyone - [x] One topic. - [x] The title is a Conventional Commit (`type(scope): description`) of the status line's type. - [x] Why gives the reason plainly and links the issues that fit (the tracker searched). - [x] Measurements has a verdict, or one line saying why there is nothing to measure. - [x] Testing names what was tested, what wasn't, and what must still be tested, where and how. (nothing to test elsewhere: Must be tested is None) - [x] Rebased on current `main`: on aa95d35f (#761, 2026-11-15), head `7e561fbb`, without a conflict; this PR's own commits, old → new: `5b929e7f` → `7e561fbb`; the commit names elsewhere in this description are from before the rebase. - [x] `python -I -B tests/run_all.py` passes (any failure that also fails on `main` is named). On macOS: 538 passed, 17 skipped, and 26 failures that also fail on main 6c9edffe on this Mac, named under Testing. After the rebase onto main aa95d35f (head `7e561fbb`): wave S1's merge `11a5b4e7`, 568/22/17 against main aa95d35f's 543/33/17, no new failure. - [x] No game files, dumps, build output or generated files. - [x] `third_party/` untouched (a new folder with a `PROVENANCE.md` instead). - [x] The license is OK: GPL-3.0-or-later, nothing incompatible and nothing decompiled. Written from Apple's `fcntl(2)` and `truncate(2)` manuals and `sys/fcntl.h`; nothing copied. ### When it applies - [x] AI assistance recorded with `Assisted-by:` / `Co-Authored-By:` trailers. - [x] A game-free test added for the fix, failing on `main` and passing here. - [x] A check under `tests/` that reads changed source updated in this PR, not weakened. (`pipeline_cache_io_fault_test.cpp` injects its failure at the new call.) - [x] Docs updated.

mooki added 1 commit

2026-11-14 16:11:04 +00:00

fix(vk): preserve cache file reservations on macOS

CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Has been skipped

Details

CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Has been skipped

Details

CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Has been skipped

Details

Third-party parts / The same as upstream (pull_request) Has been skipped

Details

PR description / Description follows the template (pull_request_target) Successful in 5s

Details

CI / Conventional Commit title (pull_request) Successful in 7s

Details

CI / Workflow lint (actionlint) (pull_request) Successful in 7s

Details

Third-party parts / As their notes say (pull_request) Successful in 11s

Details

Website / Website on Cloudflare (pull_request_target) Has been skipped

Details

587afb479e

Reserve each empty pipeline-cache temporary with F_PREALLOCATE and
F_ALLOCATEALL before setting its length and exposing a writable mapping.
Keep failed reservations on the existing empty-result and cleanup path.
Use best-effort F_RDADVISE for the program cache's existing read ranges.

Leave the Windows and Linux implementation lines unchanged. Cover real
allocated blocks and a child-only file-size limit, and retain native
allocation-failure injection through the Darwin fcntl call.
Document the file guarantees and the incomplete renderer-port boundary.

Assisted-by: GPT-6 Astra
Co-Authored-By: GPT-6 Astra <[email protected]>
Assisted-by: Claude Opus 5.5
Co-Authored-By: Claude Opus 5.5 <[email protected]>

mooki added the

labels

2026-11-14 16:28:16 +00:00

mooki added the

platform/macos

label

2026-11-14 18:24:17 +00:00

mooki force-pushed mac/cache-files from 587afb479e

CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Has been skipped

Details

CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Has been skipped

Details

CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Has been skipped

Details

Third-party parts / The same as upstream (pull_request) Has been skipped

Details

PR description / Description follows the template (pull_request_target) Successful in 5s

Details

CI / Conventional Commit title (pull_request) Successful in 7s

Details

CI / Workflow lint (actionlint) (pull_request) Successful in 7s

Details

Third-party parts / As their notes say (pull_request) Successful in 11s

Details

Website / Website on Cloudflare (pull_request_target) Has been skipped

Details

to 5b929e7fc1

CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Has been skipped

Details

CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Has been skipped

Details

CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Has been skipped

Details

Third-party parts / The same as upstream (pull_request) Has been skipped

Details

Website / Website on Cloudflare (pull_request) Has been skipped

Details

CI / Conventional Commit title (pull_request) Successful in 3s

Details

CI / Workflow lint (actionlint) (pull_request) Successful in 6s

Details

Third-party parts / As their notes say (pull_request) Successful in 8s

Details

PR description / Description follows the template (pull_request_target) Successful in 3s

Details

2026-11-15 01:52:20 +00:00

Compare

Rows
Columns

fix(vk): preserve cache file reservations on macOS #476

Why

How to reproduce

What changed

Measurements

Flags and settings

Testing

Risks

Issues

Checklist

Everyone

When it applies

Pull request closed