perf(vk): skip the pipeline cache file on Linux NVIDIA PCs with 20 GB of RAM or less (0.7-0.8 GB less) #416

Open

tonesi wants to merge 2 commits from tonesi/Longshot:perf/388-pcache-low-ram into main

pull from: tonesi/Longshot:perf/388-pcache-low-ram

merge into: MagicBeanDev:main

MagicBeanDev:main

Conversation 3 Commits 2 Files changed 8 +225 -3

tonesi commented 2026-11-14 11:49:47 +00:00

Contributor

• Copy link
• Copy Markdown

Why

On a 16 GB Linux PC the game's memory outgrows RAM and swap fills as areas change
(#388). About a gigabyte of it is the pipeline cache's file, which is read
whole at every start and kept by the driver in the game's private memory, although NVIDIA's own shader cache already holds the
same pipelines. Not reading it there takes about 0.7-0.8 GB off the game's footprint on that PC.

What changed

• pipeline cache: pcache_load_off() in gpu/vk/vk_draw.cpp decides through gpu/vk/pcache_load.h (new, no Vulkan in it).
  Where RX_VK_PCACHE_LOAD is unset, the file is not read only when all of these hold:

  • programs are loaded at their create calls (programs_load_at_create());
  • Linux with NVIDIA's driver;
  • 20 GB of RAM or less;
  • the driver's own cache folders hold 512 MB or more (driver_cache_bytes(), walked only when the rest already holds);
  • not a --prepare-shaders run.

  Everywhere else the file is read as before. One log line gives the decision and its reason.

• pipeline cache: with the file not read (this rule or RX_VK_PCACHE_LOAD=0), play_files_merge() no longer merges the play
  files into the cache. Before, that cache started empty, and at 16 play files the merge wrote it over the file: a session
  measured here wrote 38 MB over a 1.2 GB file, which a start loading programs before the title then lacked. The play files
  stay on disk for a start that reads the file. play_cache_save() still frees play's generations at each save but writes
  none, because the driver's own cache keeps those pipelines. This is what the switch's description already said: "the cache
  starts empty and is not saved".

• tests: tests/test_pcache_load.py (new) compiles and runs the rule's table and checks that vk_draw.cpp decides through it.
  tests/pipeline_play_cache_test.cpp and tests/test_pipeline_play_cache.py gain a no-save case. It fails on main (the
  play files are merged, and folded into the file at 16) and passes here.

• docs: an RX_VK_PCACHE_LOAD row in docs/development.md's GPU switches.

Flags and settings

Flag	What it does	Default now	Recommendation	Helps on	Doesn't help on	Untested on
LONGSHOT_PREPARE_ONLY	Unchanged: the internal flag --prepare-shaders sets for its own process. Now also read by pcache_load_off(), so a preparation reads and saves the file whatever the rule says	Set only by --prepare-shaders	As it is	-	-	-
RX_VK_PCACHE_LOAD=0 or 1	The pipeline cache file not read, not saved and its play files left alone (0), or read as before (1)	Unset: not read on Linux + NVIDIA + 20 GB of RAM or less + programs loaded at create + a driver cache of 512 MB or more; read everywhere else	This default. 1 to read it on the small PC, for an A/B	16 GB Linux PCs with NVIDIA (0.7-0.8 GB less of the game in RAM and swap)	Programs loaded before the title (the precompile builds into the cache object anyway: measured no saving, and it took twice as long); PCs with more RAM; Windows	Linux NVIDIA with 8 or 12 GB of RAM; AMD and Intel on Linux (not switched: Mesa's own cache is untested)

Measurements

main: 0caf0618 (2026-11-14)
this PR: perf/388-pcache-low-ram @ 999bc37f (on 0caf0618; the branch is now 73035089, the same change rebased onto main 5e072fc8, whose two new commits change the temporal pass and a shader pass, nothing the pipeline cache, program loads or memory go through, so these numbers still hold)
Hardware: NVIDIA GeForce RTX 3070 Ti 8 GB (also drives the desktop), driver 595.104.02; Intel Core i7-12700K; 16 GB RAM (15,757 MB usable), swap 15.4 GB zram + 4 GB partition; Pop!_OS 24.04 (COSMIC), kernel 7.2.15
Scene: a teleport tour from a child save at Zora's River: Zora's Domain, Hyrule Field, Kakariko, the Market, Kokiri Forest, Lake Hylia, Death Mountain Trail, Goron City, 40 s each with a scripted walk and camera turns (about 7 minutes). Windowless (headless renderer), Quality preset Auto (this PC), which picks tier 3 here and so the reporter's memory_budget=3; output 2560x1440; no DLSS (a source build without it); programs loaded at create (the default with mesh shaders)
Method: a full build of main and an incremental build of this branch from the same work folder, each in its own game folder; main prepared, then its caches, NVIDIA cache and options handed to the PR's folder (same shader stamp: this PR leaves vk_api.h alone), as an update keeps them. Three interleaved runs per side (main, PR, main, PR, main, PR), each after RAM had settled to 9 GB+ available. The game's memory every 2 s from /proc/<pid>/status and /proc/<pid>/smaps_rollup (resident + swapped, file pages left out), the system's from /proc/meminfo; loads and waits from the log's close census; frames from the renderer's frame log (fps from 60 s on, an observation on the desktop's card, not a timing run)
How to reproduce: on a 16 GB Linux PC with NVIDIA, a prepared folder, the same tour (the debug teleport through scripts/longshot_ctl.py teleport), with and without RX_VK_PCACHE_LOAD=1; compare the game's RssAnon + RssShmem + smaps Swap and MemAvailable

	main @ 0caf0618 (3 runs, median)	this PR @ 999bc37f (3 runs, median)	Δ
peak game total (resident + swapped)	12,703 MB (12,820 / 12,602 / 12,703)	11,872 MB (12,014 / 11,872 / 11,867)	-831 MB
game total at the end	12,471 MB	11,739 MB	-732 MB
the game's private memory + its swap at the end	5,153 MB	4,420 MB	-733 MB
peak private memory resident	4,011 MB	3,296 MB	-715 MB
guest window peak (resident + swapped)	7,546 MB	7,553 MB	+7 MB
lowest MemAvailable	445 MB (445 / 507 / 271)	774 MB (930 / 774 / 352)	+329 MB
system swap in use, peak	6,390 MB	5,104 MB	-1,286 MB
fps from 60 s (observation)	53.6 (53.6 / 54.5 / 52.6)	53.9 (53.4 / 53.9 / 55.7)	+0.3
frames over 100 ms from 60 s	26 (24 / 26 / 26)	29 (29 / 30 / 20)	+3
program loads at create, median (on the load workers)	1.8 ms	5.0 ms	+3.2 ms
first uses that waited for a load / their wait in all	2 / 52 ms	10 / 206 ms	+8 / +154 ms
listed pipelines in no cache (compiled in play)	0	0	0

With NVIDIA's cache emptied (a fresh __GL_SHADER_DISK_CACHE_PATH, one run each), the PR reads the file as main does. On
NVIDIA the file then answers almost nothing: main had 10,143 graphics pipelines in no cache, the PR 10,901. A cold driver
cache is as bad on both sides, and this PR doesn't change that path.

Not this PR's numbers, measured on the same PC and tour for the issue: with programs loaded before the title (the
reporter's install), the game's total peaks at 16.2 GB and MemAvailable reaches 0, because the shader modules of about 19,400
programs take 8.5 GB of private memory. Not reading the file there saves about 0.2 GB and doubles the precompile (29 s against
15 s), so the rule leaves start mode alone. #223 is the change for that.

vk: pipeline cache file not read: 20 GB of RAM or less, programs loaded at create: the driver's own cache answers them (15757 MB of RAM, the driver's own cache 1199 MB; RX_VK_PCACHE_LOAD=1 reads it)
vk: pipeline cache /home/.../cache/longshot_vk_pipelines.bin (0 KB)
vk: pipeline cache file read: the driver's own cache is nearly empty (15757 MB of RAM, the driver's own cache 0 MB; RX_VK_PCACHE_LOAD=0 would leave it)
main (p1-a):    vk: program loads at create (close): 7286 queued, 7286 loaded (median 1.69 ms, 99th percentile 15.11 ms, worst 87.75 ms, 20536 ms and 1350.1 MB read in all); ... a first use ran 17 itself and waited for 2 (79.4 ms)
this PR (p1-c): vk: program loads at create (close): 7360 queued, 7360 loaded (median 5.08 ms, 99th percentile 30.64 ms, worst 80.13 ms, 49937 ms and 1359.7 MB read in all); ... a first use ran 36 itself and waited for 13 (275.6 ms)

Testing

Tested:

• In the game on the PC above, windowless, the tour described under Measurements: three interleaved runs per side, and one
  run each with NVIDIA's cache emptied. Every run reached all eight areas, with no device lost and 0 kernel or NVRM lines.
• The decision line as above in every PR run; in the cold run, the file read.
• The PR's folder kept its 1.2 GB longshot_vk_pipelines.bin through all its sessions, with no play file written or folded.
  Main's folded its play files into it at 16, as before.
• Before this PR, on the same PC: RX_VK_PCACHE_LOAD=0 with programs loaded before the title wrote a 38 MB cache over the
  1.2 GB file (the play-file fold this PR stops).
• Game-free: tests/test_pcache_load.py, tests/test_pipeline_play_cache.py (13 cases, no-save failing on main's
  vk_draw.cpp) and tests/test_pcache_stamp.py, with g++ and clang.
• The whole suite on 4fd82b75 (this branch rebased on 5e072fc8; Linux, python -I -B tests/run_all.py --jobs 8): 529 passed,
  3 failed, 30 skipped, as before the rebase. All three fail the same way on main 0caf0618: tests/test_framegen_resources.py and tests/test_module_ids.py (the fixtures
  #350 restores), and tests/test_pad_layout.py.
• The Windows back end (longshot_vk.dll) of 4fd82b75 compiled and linked on this PC as CI's build-windows.yml builds it:
  RX_TARGET=windows RX_VK_ONLY=1 RX_RELEASE=1 nx2/build.py --standalone with the pinned llvm-mingw 20261104 (Linux build,
  scripts/builder_package.py llvm-mingw --host linux): 0 failed. The change adds no path handling.

Not tested:

• Windows in the game: the rule's Windows branch reads the file, as main does. The Windows back end was compiled, not run (above).
• Linux with 8 or 12 GB of RAM, and other NVIDIA cards and drivers.
• AMD and Intel on Linux: not switched by this PR. Mesa keeps its own disk cache, which driver_cache_room raises to 10 GiB
  like NVIDIA's, so the same default may be right there, but it is unmeasured.
• A session hours long. The tours were 7 minutes.

Needs other testers:

• Linux, NVIDIA, 16 GB of RAM, a played session of 20+ minutes across several areas: expecting the pipeline cache file not read line, no listed pipelines ... in no cache at the close, and no new stutter against RX_VK_PCACHE_LOAD=1 (T-XTVW)

Screenshots

Nothing on screen.

Risk and rollback

• On the PCs the rule covers, a program loaded at create takes its pipelines from NVIDIA's disk cache instead of the file's
  in-memory copy. Each load is slower (median 5.0 against 1.8 ms, on the load workers), and draws waited for a load a little
  more (10 against 2 per tour, 206 against 52 ms in all). Frames over 100 ms were 29 against 26; fps was the same.
  RX_VK_PCACHE_LOAD=1 reads the file again.
• If NVIDIA's cache is cleared while it still holds 512 MB or more of other programs' shaders, the file stays unread. On
  NVIDIA the file answered almost nothing with the driver's cache cold, so this costs nothing beyond what main costs there.
• Reverting the commit is enough.

Reviewer notes

• Start with gpu/vk/pcache_load.h, then pcache_load_off() and the two early returns in play_files_merge() and
  play_cache_save().
• Tried on top of this and not taken: more load workers to win back the waits. With 4 while the game loads (2 runs), draws
  waited 6 and 5 times (251 and 247 ms); with 4 throughout (2 runs), 4 and 8 times (209 and 218 ms), each load slower (7.0 ms
  median). Neither beat the 197-276 ms of the default 2.
• Measured and not taken (in the issue's hill-climb): an import cap (imports halved, the game's total the same),
  MALLOC_ARENA_MAX=2 and more frequent malloc_trim, and giving back freed guest pages around imports. That last one was a
  host change; the game re-mapped what it freed at once.
• Left for follow-ups: the shader modules kept for the session (8.5 GB with programs loaded before the title, which
  #223 works on; 1.4 GB over the tour with programs loaded at create); and
  the load-start marker's switch to start mode costing 16 GB PCs about 5 GB for good.

Issues

See #388
See #223
See https://github.com/NotRexed/Hookshot/pull/2206

Checklist

For everyone

• One topic.
• The title is a Conventional Commit (type(scope): description).
• Rebased on current main.
• python -I -B tests/run_all.py run and passing (any failure that also fails on main is named above).
• Tested in the game, or what wasn't is named under Testing.
• Testing filled in: what was tested, what wasn't, and what needs other testers (with test requests).
• Measured: main and this PR at named commits, on the same PC, scene and settings, at least 2 interleaved runs each, the median reported.
• No game files, dumps, build output or generated files.
• third_party/ untouched (a new folder with a PROVENANCE.md instead).
• The license is OK: GPL-3.0-or-later, nothing incompatible and nothing decompiled.

If applicable

• AI assistance recorded with Assisted-by: / Co-Authored-By: trailers.
• Flags noted, and the switch reference updated (docs/development.md, the README).
• A file in docs/tested/ for each new measured run (format in docs/tested-configurations.md; no appended row or assembled view).
• A game-free test added for the fix, failing on main and passing here.
• A check under tests/ that reads changed source updated in this PR, not weakened.
• Host code builds on both Windows (llvm-mingw) and Linux, with UTF-8 paths.
• Docs updated.

🤖 Generated with Claude Code

## Why On a 16 GB Linux PC the game's memory outgrows RAM and swap fills as areas change (https://git.magicbean.dev/MagicBeanDev/Longshot/issues/388). About a gigabyte of it is the pipeline cache's file, which is read whole at every start and kept by the driver in the game's private memory, although NVIDIA's own shader cache already holds the same pipelines. Not reading it there takes about 0.7-0.8 GB off the game's footprint on that PC. ## What changed - pipeline cache: `pcache_load_off()` in `gpu/vk/vk_draw.cpp` decides through `gpu/vk/pcache_load.h` (new, no Vulkan in it). Where `RX_VK_PCACHE_LOAD` is unset, the file is not read only when all of these hold: - programs are loaded at their create calls (`programs_load_at_create()`); - Linux with NVIDIA's driver; - 20 GB of RAM or less; - the driver's own cache folders hold 512 MB or more (`driver_cache_bytes()`, walked only when the rest already holds); - not a `--prepare-shaders` run. Everywhere else the file is read as before. One log line gives the decision and its reason. - pipeline cache: with the file not read (this rule or `RX_VK_PCACHE_LOAD=0`), `play_files_merge()` no longer merges the play files into the cache. Before, that cache started empty, and at 16 play files the merge wrote it over the file: a session measured here wrote 38 MB over a 1.2 GB file, which a start loading programs before the title then lacked. The play files stay on disk for a start that reads the file. `play_cache_save()` still frees play's generations at each save but writes none, because the driver's own cache keeps those pipelines. This is what the switch's description already said: "the cache starts empty and is not saved". - tests: `tests/test_pcache_load.py` (new) compiles and runs the rule's table and checks that `vk_draw.cpp` decides through it. `tests/pipeline_play_cache_test.cpp` and `tests/test_pipeline_play_cache.py` gain a `no-save` case. It fails on `main` (the play files are merged, and folded into the file at 16) and passes here. - docs: an `RX_VK_PCACHE_LOAD` row in docs/development.md's GPU switches. ## Flags and settings | Flag | What it does | Default now | Recommendation | Helps on | Doesn't help on | Untested on | |---|---|---|---|---|---|---| | `LONGSHOT_PREPARE_ONLY` | Unchanged: the internal flag `--prepare-shaders` sets for its own process. Now also read by `pcache_load_off()`, so a preparation reads and saves the file whatever the rule says | Set only by `--prepare-shaders` | As it is | - | - | - | | `RX_VK_PCACHE_LOAD=0` or `1` | The pipeline cache file not read, not saved and its play files left alone (`0`), or read as before (`1`) | Unset: not read on Linux + NVIDIA + 20 GB of RAM or less + programs loaded at create + a driver cache of 512 MB or more; read everywhere else | This default. `1` to read it on the small PC, for an A/B | 16 GB Linux PCs with NVIDIA (0.7-0.8 GB less of the game in RAM and swap) | Programs loaded before the title (the precompile builds into the cache object anyway: measured no saving, and it took twice as long); PCs with more RAM; Windows | Linux NVIDIA with 8 or 12 GB of RAM; AMD and Intel on Linux (not switched: Mesa's own cache is untested) | ## Measurements main: 0caf0618 (2026-11-14) this PR: perf/388-pcache-low-ram @ 999bc37f (on 0caf0618; the branch is now 73035089, the same change rebased onto main 5e072fc8, whose two new commits change the temporal pass and a shader pass, nothing the pipeline cache, program loads or memory go through, so these numbers still hold) Hardware: NVIDIA GeForce RTX 3070 Ti 8 GB (also drives the desktop), driver 595.104.02; Intel Core i7-12700K; 16 GB RAM (15,757 MB usable), swap 15.4 GB zram + 4 GB partition; Pop!_OS 24.04 (COSMIC), kernel 7.2.15 Scene: a teleport tour from a child save at Zora's River: Zora's Domain, Hyrule Field, Kakariko, the Market, Kokiri Forest, Lake Hylia, Death Mountain Trail, Goron City, 40 s each with a scripted walk and camera turns (about 7 minutes). Windowless (headless renderer), Quality preset Auto (this PC), which picks tier 3 here and so the reporter's `memory_budget=3`; output 2560x1440; no DLSS (a source build without it); programs loaded at create (the default with mesh shaders) Method: a full build of main and an incremental build of this branch from the same work folder, each in its own game folder; main prepared, then its caches, NVIDIA cache and options handed to the PR's folder (same shader stamp: this PR leaves `vk_api.h` alone), as an update keeps them. Three interleaved runs per side (main, PR, main, PR, main, PR), each after RAM had settled to 9 GB+ available. The game's memory every 2 s from `/proc/<pid>/status` and `/proc/<pid>/smaps_rollup` (resident + swapped, file pages left out), the system's from `/proc/meminfo`; loads and waits from the log's close census; frames from the renderer's frame log (fps from 60 s on, an observation on the desktop's card, not a timing run) How to reproduce: on a 16 GB Linux PC with NVIDIA, a prepared folder, the same tour (the debug teleport through `scripts/longshot_ctl.py teleport`), with and without `RX_VK_PCACHE_LOAD=1`; compare the game's `RssAnon + RssShmem + smaps Swap` and `MemAvailable` | | main @ 0caf0618 (3 runs, median) | this PR @ 999bc37f (3 runs, median) | Δ | |---|---:|---:|---:| | peak game total (resident + swapped) | 12,703 MB (12,820 / 12,602 / 12,703) | **11,872 MB** (12,014 / 11,872 / 11,867) | -831 MB | | game total at the end | 12,471 MB | **11,739 MB** | -732 MB | | the game's private memory + its swap at the end | 5,153 MB | **4,420 MB** | -733 MB | | peak private memory resident | 4,011 MB | 3,296 MB | -715 MB | | guest window peak (resident + swapped) | 7,546 MB | 7,553 MB | +7 MB | | lowest MemAvailable | 445 MB (445 / 507 / 271) | **774 MB** (930 / 774 / 352) | +329 MB | | system swap in use, peak | 6,390 MB | **5,104 MB** | -1,286 MB | | fps from 60 s (observation) | 53.6 (53.6 / 54.5 / 52.6) | 53.9 (53.4 / 53.9 / 55.7) | +0.3 | | frames over 100 ms from 60 s | 26 (24 / 26 / 26) | 29 (29 / 30 / 20) | +3 | | program loads at create, median (on the load workers) | 1.8 ms | 5.0 ms | +3.2 ms | | first uses that waited for a load / their wait in all | 2 / 52 ms | 10 / 206 ms | +8 / +154 ms | | listed pipelines in no cache (compiled in play) | 0 | 0 | 0 | With NVIDIA's cache emptied (a fresh `__GL_SHADER_DISK_CACHE_PATH`, one run each), the PR reads the file as main does. On NVIDIA the file then answers almost nothing: main had 10,143 graphics pipelines in no cache, the PR 10,901. A cold driver cache is as bad on both sides, and this PR doesn't change that path. Not this PR's numbers, measured on the same PC and tour for the issue: with programs loaded before the title (the reporter's install), the game's total peaks at 16.2 GB and MemAvailable reaches 0, because the shader modules of about 19,400 programs take 8.5 GB of private memory. Not reading the file there saves about 0.2 GB and doubles the precompile (29 s against 15 s), so the rule leaves start mode alone. https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/223 is the change for that. ``` vk: pipeline cache file not read: 20 GB of RAM or less, programs loaded at create: the driver's own cache answers them (15757 MB of RAM, the driver's own cache 1199 MB; RX_VK_PCACHE_LOAD=1 reads it) vk: pipeline cache /home/.../cache/longshot_vk_pipelines.bin (0 KB) vk: pipeline cache file read: the driver's own cache is nearly empty (15757 MB of RAM, the driver's own cache 0 MB; RX_VK_PCACHE_LOAD=0 would leave it) main (p1-a): vk: program loads at create (close): 7286 queued, 7286 loaded (median 1.69 ms, 99th percentile 15.11 ms, worst 87.75 ms, 20536 ms and 1350.1 MB read in all); ... a first use ran 17 itself and waited for 2 (79.4 ms) this PR (p1-c): vk: program loads at create (close): 7360 queued, 7360 loaded (median 5.08 ms, 99th percentile 30.64 ms, worst 80.13 ms, 49937 ms and 1359.7 MB read in all); ... a first use ran 36 itself and waited for 13 (275.6 ms) ``` ## Testing **Tested:** - In the game on the PC above, windowless, the tour described under Measurements: three interleaved runs per side, and one run each with NVIDIA's cache emptied. Every run reached all eight areas, with no device lost and 0 kernel or NVRM lines. - The decision line as above in every PR run; in the cold run, the file read. - The PR's folder kept its 1.2 GB `longshot_vk_pipelines.bin` through all its sessions, with no play file written or folded. Main's folded its play files into it at 16, as before. - Before this PR, on the same PC: `RX_VK_PCACHE_LOAD=0` with programs loaded before the title wrote a 38 MB cache over the 1.2 GB file (the play-file fold this PR stops). - Game-free: `tests/test_pcache_load.py`, `tests/test_pipeline_play_cache.py` (13 cases, `no-save` failing on main's `vk_draw.cpp`) and `tests/test_pcache_stamp.py`, with g++ and clang. - The whole suite on 4fd82b75 (this branch rebased on 5e072fc8; Linux, `python -I -B tests/run_all.py --jobs 8`): 529 passed, 3 failed, 30 skipped, as before the rebase. All three fail the same way on main 0caf0618: `tests/test_framegen_resources.py` and `tests/test_module_ids.py` (the fixtures https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/350 restores), and `tests/test_pad_layout.py`. - The Windows back end (`longshot_vk.dll`) of 4fd82b75 compiled and linked on this PC as CI's build-windows.yml builds it: `RX_TARGET=windows RX_VK_ONLY=1 RX_RELEASE=1 nx2/build.py --standalone` with the pinned llvm-mingw 20261104 (Linux build, `scripts/builder_package.py llvm-mingw --host linux`): 0 failed. The change adds no path handling. **Not tested:** - Windows in the game: the rule's Windows branch reads the file, as main does. The Windows back end was compiled, not run (above). - Linux with 8 or 12 GB of RAM, and other NVIDIA cards and drivers. - AMD and Intel on Linux: not switched by this PR. Mesa keeps its own disk cache, which `driver_cache_room` raises to 10 GiB like NVIDIA's, so the same default may be right there, but it is unmeasured. - A session hours long. The tours were 7 minutes. **Needs other testers:** - Linux, NVIDIA, 16 GB of RAM, a played session of 20+ minutes across several areas: expecting the `pipeline cache file not read` line, no `listed pipelines ... in no cache` at the close, and no new stutter against `RX_VK_PCACHE_LOAD=1` (T-XTVW) ## Screenshots Nothing on screen. ## Risk and rollback - On the PCs the rule covers, a program loaded at create takes its pipelines from NVIDIA's disk cache instead of the file's in-memory copy. Each load is slower (median 5.0 against 1.8 ms, on the load workers), and draws waited for a load a little more (10 against 2 per tour, 206 against 52 ms in all). Frames over 100 ms were 29 against 26; fps was the same. `RX_VK_PCACHE_LOAD=1` reads the file again. - If NVIDIA's cache is cleared while it still holds 512 MB or more of other programs' shaders, the file stays unread. On NVIDIA the file answered almost nothing with the driver's cache cold, so this costs nothing beyond what main costs there. - Reverting the commit is enough. ## Reviewer notes - Start with `gpu/vk/pcache_load.h`, then `pcache_load_off()` and the two early returns in `play_files_merge()` and `play_cache_save()`. - Tried on top of this and not taken: more load workers to win back the waits. With 4 while the game loads (2 runs), draws waited 6 and 5 times (251 and 247 ms); with 4 throughout (2 runs), 4 and 8 times (209 and 218 ms), each load slower (7.0 ms median). Neither beat the 197-276 ms of the default 2. - Measured and not taken (in the issue's hill-climb): an import cap (imports halved, the game's total the same), `MALLOC_ARENA_MAX=2` and more frequent `malloc_trim`, and giving back freed guest pages around imports. That last one was a host change; the game re-mapped what it freed at once. - Left for follow-ups: the shader modules kept for the session (8.5 GB with programs loaded before the title, which https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/223 works on; 1.4 GB over the tour with programs loaded at create); and the load-start marker's switch to start mode costing 16 GB PCs about 5 GB for good. ## Issues See https://git.magicbean.dev/MagicBeanDev/Longshot/issues/388 See https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/223 See https://github.com/NotRexed/Hookshot/pull/2206 ## Checklist ### For everyone - [x] One topic. - [x] The title is a Conventional Commit (`type(scope): description`). - [x] Rebased on current `main`. - [x] `python -I -B tests/run_all.py` run and passing (any failure that also fails on `main` is named above). - [x] Tested in the game, or what wasn't is named under Testing. - [x] Testing filled in: what was tested, what wasn't, and what needs other testers (with test requests). - [x] Measured: main and this PR at named commits, on the same PC, scene and settings, at least 2 interleaved runs each, the median reported. - [x] No game files, dumps, build output or generated files. - [x] `third_party/` untouched (a new folder with a `PROVENANCE.md` instead). - [x] The license is OK: GPL-3.0-or-later, nothing incompatible and nothing decompiled. ### If applicable - [x] AI assistance recorded with `Assisted-by:` / `Co-Authored-By:` trailers. - [x] Flags noted, and the switch reference updated (docs/development.md, the README). - [x] A file in `docs/tested/` for each new measured run (format in `docs/tested-configurations.md`; no appended row or assembled view). - [x] A game-free test added for the fix, failing on `main` and passing here. - [x] A check under `tests/` that reads changed source updated in this PR, not weakened. - [x] Host code builds on both Windows (llvm-mingw) and Linux, with UTF-8 paths. - [x] Docs updated. 🤖 Generated with [Claude Code](https://claude.com/claude-code)

tonesi added 2 commits 2026-11-14 11:49:47 +00:00

perf(vk): on a 16 GB Linux PC with NVIDIA, programs loaded at create leave the pipeline cache file unread (issue 388) ... 73035089fa

The pipeline cache file (1.2 GB after a preparation) is read whole at every start and the driver keeps its copy in the
process's private memory for the session. On Linux the drivers keep their own shader caches with room for it
(driver_cache_room), and a program loaded at its create call finds its pipelines there too. Where all of these hold the
file is now not read: programs loaded at create, Linux with NVIDIA's driver, 20 GB of RAM or less, the driver's own cache
not nearly empty, not a --prepare-shaders run (gpu/vk/pcache_load.h). RX_VK_PCACHE_LOAD=1 reads it, =0 never.

With the file not read (this or RX_VK_PCACHE_LOAD=0), the start no longer merges the play files into the empty cache and
writes it over the file at 16 of them (a 38 MB cache over a 1.2 GB file), and play's generations are not written.

Assisted-by: Claude Opus 5.5
Co-Authored-By: Claude Opus 5.5 <[email protected]>

docs(tested): issue 388's memory tour, main and this branch, three runs each ... 4fd82b7559

Assisted-by: Claude Opus 5.5
Co-Authored-By: Claude Opus 5.5 <[email protected]>

tonesi commented 2026-11-14 11:53:14 +00:00

Author

Contributor

• Copy link
• Copy Markdown

Description completed: the measured commit named on the 'this PR' line (999bc37f, rebased since onto 5e072fc8 unchanged, and why the numbers still hold), LONGSHOT_PREPARE_ONLY in the flags table, the Windows back end compiled and linked with the pinned llvm-mingw (Linux build, as CI does), and test request T-XTVW for a longer session on another 16 GB Linux NVIDIA PC.

Description completed: the measured commit named on the 'this PR' line (999bc37f, rebased since onto 5e072fc8 unchanged, and why the numbers still hold), LONGSHOT_PREPARE_ONLY in the flags table, the Windows back end compiled and linked with the pinned llvm-mingw (Linux build, as CI does), and test request T-XTVW for a longer session on another 16 GB Linux NVIDIA PC.

tonesi changed title from WIP: perf(vk): skip the pipeline cache file on Linux NVIDIA PCs with 20 GB of RAM or less (0.7-0.8 GB less) to perf(vk): skip the pipeline cache file on Linux NVIDIA PCs with 20 GB of RAM or less (0.7-0.8 GB less) 2026-11-14 11:53:25 +00:00

tonesi referenced this pull request 2026-11-14 11:53:59 +00:00

Game: on 16 GB of RAM the game's memory outgrows RAM within a minute of loading a save, then swap fills on area changes until RAM is at 0 #388

mooki added the

kind

performance

priority

high

labels 2026-11-14 12:24:54 +00:00

mooki requested changes 2026-11-14 12:39:53 +00:00

mooki left a comment

• Copy link
• Copy Markdown

Read head 4fd82b75 against main 5e072fc8: the complete eight-file diff, description, sole comment, empty reviews, issue 388 and both comments, cache readers/writers, program-load dispatch and the relevant changes in 415 and 370; ran nine game-free check invocations across five files, all passed, with GCC and Clang for the compiled checks. No game, full build or timing run.

What's good

• The cost is real. Main maps the complete base file and hands it to vkCreatePipelineCache; file-backed input avoids a large temporary byte vector, not the driver's retained cache. The reported memory reduction is consistent with removing that retained input.
• The explicit-off overwrite fix is separate and useful. The new guard in play_files_merge() prevents an empty base cache plus 16 rolling files from replacing the larger base file. The new no-save case passes with both compilers and preserves the authored base and rolling files.
• The decision is latched once, keeps an explicit override, and excludes ordinary preparation and load-before-title starts from the automatic skip. It does not change the translated-program format or shader-stamp inputs.

Must change

1. Reach generation retirement from the ordinary save entry point. In gpu/vk/vk_draw.cpp, lines 945-949, the new no-save branch destroys retired play caches, but the periodic caller cannot reach it: vk_replay.cpp, line 4099, calls pipeline_cache_save(false), whose first condition at vk_draw.cpp, line 10343, returns when pcache_no_save is true. play_cache_begin() still creates a generation, and create_cached() keeps adding play pipelines to it. On the PCs newly selected by this default, those additions therefore stay in the same generation until close, instead of being retired at the periodic saves as on main. Route generation retirement around the file-write prohibition while keeping the current user-count synchronization. Extend the fixture through pipeline_cache_save(false), after enough new pipelines to trigger a save, rather than calling play_cache_save() directly. It should prove that the old cache is destroyed, new work still succeeds, and no file is written. Traced through the source; the passing no-save case does not exercise this caller. No long-session memory run.

2. Use the selected driver's cache for the cold-cache guard. In gpu/vk/vk_draw.cpp, lines 1055-1057, the new decision treats driver_cache_bytes(nullptr) as NVIDIA's cache size. That helper, lines 9619-9633, adds NVIDIA's and Mesa's folders together. A fresh, empty __GL_SHADER_DISK_CACHE_PATH with at least 512 MiB left in Mesa's folder therefore selects “file not read”, even though NVIDIA's cache is empty. This is not just the disclosed case of other programs filling NVIDIA's cache: an unrelated driver's bytes defeat the explicit cold-cache fallback. The base and rolling files are then ignored, and listed pipelines which neither live cache can answer are deferred to their first draw or dispatch (vk_programs.cpp, lines 240-255). Count only the selected driver's cache for this decision and treat an unavailable scan as unknown/cold. Add an adapter-level case with an empty NVIDIA folder and a populated Mesa folder, requiring the file to be read. Traced through the source, not reproduced in a game; the new table test supplies a number directly and never exercises folder selection.

What the condition actually selects

• The platform branch is #ifndef _WIN32, and the GPU check is vendorID == 0x10DE. On the supported platforms this means Linux with an NVIDIA GPU, not specifically NVIDIA's proprietary driver. Mesa NVK is not excluded by that check; dev.driver_id is already available if proprietary-driver-only is the intended scope.
• RAM is sysconf(_SC_PHYS_PAGES) * sysconf(_SC_PAGE_SIZE), shifted right by 20. It is total RAM reported by the OS, not current free RAM, VRAM or a process memory allowance. Unknown RAM falls back to reading the file. The inclusive limit is 20,480 MiB, or 20 GiB, with the byte count rounded down to MiB. A typical 16 GiB PC qualifies, including the reported 15,757 MiB machine. Typical 24 and 32 GiB PCs do not; unusually large hardware reservations can change the reported total.
• programs_load_decide() runs after device creation and before this choice. The existing load mode matters: native mesh normally selects create, while the fallback, compressed program cache or an existing load-start marker can select start. This PR does not help the issue's original load-before-title configuration by default.
• Nonempty RX_VK_PCACHE_LOAD=0, off, false or no forces the skip; 1 or another non-off value forces loading. Empty is treated as automatic. The switch wins over preparation too: choose(0, true, ...) skips the file, and the new test explicitly requires that. The documentation's “always by --prepare-shaders” needs that exception.
• The reason line is emitted only for small-PC automatic candidates or an explicit override. Other automatic paths, including Windows, larger RAM and load-before-title, do not get that reason line. An explicit override also prints a driver-cache size of 0 without scanning it, so that number is not an observation of an empty cache. The later path/size line remains.

Cold, disabled or unwritable driver caches

A genuinely empty aggregate scan falls back to main's file-loading path. A first start without a valid warm marker also has the existing up-front preparation/warm pass, so it is not correct to say every cold first start necessarily compiles everything in play.

The size check does not establish that this game's entries are present, compatible or usable. It does not check __GL_SHADER_DISK_CACHE=0, and it does not establish usability on a read-only home. Read-only storage alone does not prove existing entries cannot be read. NVIDIA also documents that its cached binaries are driver-version-specific: https://download.nvidia.com/XFree86/Linux-x86_64/570.124.04/README/openglenvvariables.html. These cases need separate evidence, not a directory-size inference.

With an existing warm marker, a skipped file and driver misses, loads at create only probe the caches. Missing pipelines are compiled when first needed in play. More than 500 listed misses can write longshot_load_start.txt, making the next run load before the title and disabling this automatic memory saving. The description discloses unrelated NVIDIA entries, but “this costs nothing beyond main” is not established for disabled caches, read-only storage or other drivers.

Files retained for the next run

When the file is skipped, neither the base file nor new rolling files are written. Existing published files and their stamp are left alone; old temporary-file cleanup still runs. A later run that reads the file can reuse the retained base and rolling files if their translator stamp and Vulkan vendor/device/cache UUID still match. Otherwise the normal stamp/header checks reject them. This preserves an existing valid cache, but does not keep it updated with the skipped run's new pipelines or make an old-driver file valid for a new driver.

The explicit RX_VK_PCACHE_LOAD=0 overwrite protection does work independently of the new default. The new retirement branch also stops the close path from writing a rolling file. The periodic-retirement problem in finding 1 is about retained memory, not a remaining overwrite path.

Measurements

The reported warm-cache comparison meets the important paired-memory conditions: named builds, the same 16 GB PC and settings, and three alternating tours per side with medians. Arithmetic from the committed run rows gives:

Published measure	Main range	PR range	Difference of medians
Peak game resident + swapped	12,602-12,820 MB	11,867-12,014 MB	-831 MB
Private memory + swap at the end	5,108-5,280 MB	4,301-4,481 MB	-733 MB
Lowest system MemAvailable	271-507 MB	352-930 MB	+329 MB

The first two ranges are disjoint and their median gaps exceed their within-side spreads. That supports the claimed memory reduction on this PC and tour. MemAvailable has overlapping ranges and substantial variation; it does not establish that memory pressure is solved. The reported fps difference is smaller than the spread and was explicitly not collected as a controlled timing comparison. There is no demonstrated speed gain. The reported program-load median worsens from 1.8 to 5.0 ms, and total first-use waits from 52 to 206 ms; retain all three runs' load and wait rows to show their spread, not just one excerpt and the medians. No overall startup-time improvement is shown.

A cold-driver-cache run was measured, once per side, with 10,143 and 10,901 graphics misses. However, the candidate read the file in that run. It tests the cold fallback, not the behavior of skipping the file while the relevant driver cache is cold or disabled. It also lacks repeated cold pairs.

Correct the rebase explanation. Main's intervening change includes gpu/vk/eden/shader_recompiler/ir_opt/scene_tonemap_pass.h. nx2/build.py, lines 1178-1182, hashes every file under eden into the shader stamp. That change therefore does go through cache invalidation, even though the pipeline-loading code is unchanged. Keep the numbers labelled as measured on 999bc37f; explain the changed cache identity and distinguish warmed steady-state results from the first start after updating. The measured commit was not available from the fetched refs or the public commit endpoints, so its claimed identical replay was not independently established here. Full per-run logs and memory samples are not linked in the thread.

Overlaps and remaining checks

• #415 fixes the same explicit-off persistence problem differently: it prevents play generations from starting and guards the merge call in pcache_init(). This PR deliberately keeps generations. Do not combine those approaches unchanged: 415's guard would prevent the retirement design here from running at all and send new pipelines into the base in-memory cache. Their switch documentation and fixture cases also need one combined definition. No combined build or test was run.
• #370 changes which sources define the translated-program stamp, not this load policy. Its exclusion of vk_api.h does not exclude the changed Eden header above. Installing its new formula causes a one-time invalidation. While this PR skips loading, the old pipeline files and stamp remain untouched; on a later loading run, normal stamp rejection still applies. Do not relabel old warm-cache evidence as a combined-stack update test.
• Passed here: test_pcache_load, test_pipeline_play_cache (13 cases), test_pcache_stamp and test_pcache_fetch_lock, each with GCC and Clang, plus test_gpl_library_cache. No test was weakened. These are rule, file-model and source checks, not a native NVIDIA-cache reproduction.
• Not run: the full suite, Windows execution, a full renderer build, NVK/AMD/Intel, a real disabled/read-only driver cache, a driver update, or extended play. The useful remaining evidence is an integrated periodic-retirement test, a real cache-selection fixture, and repeated longer warm/cold/disabled runs on the target low-RAM machine with the selected path, cache state, load-start marker, memory and first-use waits retained.

Read head `4fd82b75` against main `5e072fc8`: the complete eight-file diff, description, sole comment, empty reviews, issue 388 and both comments, cache readers/writers, program-load dispatch and the relevant changes in 415 and 370; ran nine game-free check invocations across five files, all passed, with GCC and Clang for the compiled checks. No game, full build or timing run. ### What's good - The cost is real. Main maps the complete base file and hands it to `vkCreatePipelineCache`; file-backed input avoids a large temporary byte vector, not the driver's retained cache. The reported memory reduction is consistent with removing that retained input. - The explicit-off overwrite fix is separate and useful. The new guard in `play_files_merge()` prevents an empty base cache plus 16 rolling files from replacing the larger base file. The new `no-save` case passes with both compilers and preserves the authored base and rolling files. - The decision is latched once, keeps an explicit override, and excludes ordinary preparation and load-before-title starts from the automatic skip. It does not change the translated-program format or shader-stamp inputs. ### Must change 1. **Reach generation retirement from the ordinary save entry point.** In `gpu/vk/vk_draw.cpp`, lines 945-949, the new no-save branch destroys retired play caches, but the periodic caller cannot reach it: `vk_replay.cpp`, line 4099, calls `pipeline_cache_save(false)`, whose first condition at `vk_draw.cpp`, line 10343, returns when `pcache_no_save` is true. `play_cache_begin()` still creates a generation, and `create_cached()` keeps adding play pipelines to it. On the PCs newly selected by this default, those additions therefore stay in the same generation until close, instead of being retired at the periodic saves as on main. Route generation retirement around the file-write prohibition while keeping the current user-count synchronization. Extend the fixture through `pipeline_cache_save(false)`, after enough new pipelines to trigger a save, rather than calling `play_cache_save()` directly. It should prove that the old cache is destroyed, new work still succeeds, and no file is written. **Traced through the source; the passing `no-save` case does not exercise this caller. No long-session memory run.** 2. **Use the selected driver's cache for the cold-cache guard.** In `gpu/vk/vk_draw.cpp`, lines 1055-1057, the new decision treats `driver_cache_bytes(nullptr)` as NVIDIA's cache size. That helper, lines 9619-9633, adds NVIDIA's and Mesa's folders together. A fresh, empty `__GL_SHADER_DISK_CACHE_PATH` with at least 512 MiB left in Mesa's folder therefore selects “file not read”, even though NVIDIA's cache is empty. This is not just the disclosed case of other programs filling NVIDIA's cache: an unrelated driver's bytes defeat the explicit cold-cache fallback. The base and rolling files are then ignored, and listed pipelines which neither live cache can answer are deferred to their first draw or dispatch (`vk_programs.cpp`, lines 240-255). Count only the selected driver's cache for this decision and treat an unavailable scan as unknown/cold. Add an adapter-level case with an empty NVIDIA folder and a populated Mesa folder, requiring the file to be read. **Traced through the source, not reproduced in a game; the new table test supplies a number directly and never exercises folder selection.** ### What the condition actually selects - The platform branch is `#ifndef _WIN32`, and the GPU check is `vendorID == 0x10DE`. On the supported platforms this means Linux with an NVIDIA GPU, not specifically NVIDIA's proprietary driver. Mesa NVK is not excluded by that check; `dev.driver_id` is already available if proprietary-driver-only is the intended scope. - RAM is `sysconf(_SC_PHYS_PAGES) * sysconf(_SC_PAGE_SIZE)`, shifted right by 20. It is total RAM reported by the OS, not current free RAM, VRAM or a process memory allowance. Unknown RAM falls back to reading the file. The inclusive limit is **20,480 MiB, or 20 GiB**, with the byte count rounded down to MiB. A typical 16 GiB PC qualifies, including the reported 15,757 MiB machine. Typical 24 and 32 GiB PCs do not; unusually large hardware reservations can change the reported total. - `programs_load_decide()` runs after device creation and before this choice. The existing load mode matters: native mesh normally selects create, while the fallback, compressed program cache or an existing load-start marker can select start. This PR does not help the issue's original load-before-title configuration by default. - Nonempty `RX_VK_PCACHE_LOAD=0`, `off`, `false` or `no` forces the skip; `1` or another non-off value forces loading. Empty is treated as automatic. The switch wins over preparation too: `choose(0, true, ...)` skips the file, and the new test explicitly requires that. The documentation's “always by --prepare-shaders” needs that exception. - The reason line is emitted only for small-PC automatic candidates or an explicit override. Other automatic paths, including Windows, larger RAM and load-before-title, do not get that reason line. An explicit override also prints a driver-cache size of 0 without scanning it, so that number is not an observation of an empty cache. The later path/size line remains. ### Cold, disabled or unwritable driver caches A genuinely empty aggregate scan falls back to main's file-loading path. A first start without a valid warm marker also has the existing up-front preparation/warm pass, so it is not correct to say every cold first start necessarily compiles everything in play. The size check does not establish that this game's entries are present, compatible or usable. It does not check `__GL_SHADER_DISK_CACHE=0`, and it does not establish usability on a read-only home. Read-only storage alone does not prove existing entries cannot be read. NVIDIA also documents that its cached binaries are driver-version-specific: https://download.nvidia.com/XFree86/Linux-x86_64/570.124.04/README/openglenvvariables.html. These cases need separate evidence, not a directory-size inference. With an existing warm marker, a skipped file and driver misses, loads at create only probe the caches. Missing pipelines are compiled when first needed in play. More than 500 listed misses can write `longshot_load_start.txt`, making the next run load before the title and disabling this automatic memory saving. The description discloses unrelated NVIDIA entries, but “this costs nothing beyond main” is not established for disabled caches, read-only storage or other drivers. ### Files retained for the next run When the file is skipped, **neither the base file nor new rolling files are written**. Existing published files and their stamp are left alone; old temporary-file cleanup still runs. A later run that reads the file can reuse the retained base and rolling files if their translator stamp and Vulkan vendor/device/cache UUID still match. Otherwise the normal stamp/header checks reject them. This preserves an existing valid cache, but does not keep it updated with the skipped run's new pipelines or make an old-driver file valid for a new driver. The explicit `RX_VK_PCACHE_LOAD=0` overwrite protection does work independently of the new default. The new retirement branch also stops the close path from writing a rolling file. The periodic-retirement problem in finding 1 is about retained memory, not a remaining overwrite path. ### Measurements The reported warm-cache comparison meets the important paired-memory conditions: named builds, the same 16 GB PC and settings, and three alternating tours per side with medians. Arithmetic from the committed run rows gives: | Published measure | Main range | PR range | Difference of medians | |---|---:|---:|---:| | Peak game resident + swapped | 12,602-12,820 MB | 11,867-12,014 MB | -831 MB | | Private memory + swap at the end | 5,108-5,280 MB | 4,301-4,481 MB | -733 MB | | Lowest system MemAvailable | 271-507 MB | 352-930 MB | +329 MB | The first two ranges are disjoint and their median gaps exceed their within-side spreads. That supports the claimed memory reduction on this PC and tour. MemAvailable has overlapping ranges and substantial variation; it does not establish that memory pressure is solved. The reported fps difference is smaller than the spread and was explicitly not collected as a controlled timing comparison. There is no demonstrated speed gain. The reported program-load median worsens from 1.8 to 5.0 ms, and total first-use waits from 52 to 206 ms; retain all three runs' load and wait rows to show their spread, not just one excerpt and the medians. No overall startup-time improvement is shown. **A cold-driver-cache run was measured**, once per side, with 10,143 and 10,901 graphics misses. However, the candidate read the file in that run. It tests the cold fallback, not the behavior of skipping the file while the relevant driver cache is cold or disabled. It also lacks repeated cold pairs. **Correct the rebase explanation.** Main's intervening change includes `gpu/vk/eden/shader_recompiler/ir_opt/scene_tonemap_pass.h`. `nx2/build.py`, lines 1178-1182, hashes every file under `eden` into the shader stamp. That change therefore does go through cache invalidation, even though the pipeline-loading code is unchanged. Keep the numbers labelled as measured on `999bc37f`; explain the changed cache identity and distinguish warmed steady-state results from the first start after updating. The measured commit was not available from the fetched refs or the public commit endpoints, so its claimed identical replay was not independently established here. Full per-run logs and memory samples are not linked in the thread. ### Overlaps and remaining checks - https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/415 fixes the same explicit-off persistence problem differently: it prevents play generations from starting and guards the merge call in `pcache_init()`. This PR deliberately keeps generations. Do not combine those approaches unchanged: 415's guard would prevent the retirement design here from running at all and send new pipelines into the base in-memory cache. Their switch documentation and fixture cases also need one combined definition. No combined build or test was run. - https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/370 changes which sources define the translated-program stamp, not this load policy. Its exclusion of `vk_api.h` does not exclude the changed Eden header above. Installing its new formula causes a one-time invalidation. While this PR skips loading, the old pipeline files and stamp remain untouched; on a later loading run, normal stamp rejection still applies. Do not relabel old warm-cache evidence as a combined-stack update test. - Passed here: `test_pcache_load`, `test_pipeline_play_cache` (13 cases), `test_pcache_stamp` and `test_pcache_fetch_lock`, each with GCC and Clang, plus `test_gpl_library_cache`. No test was weakened. These are rule, file-model and source checks, not a native NVIDIA-cache reproduction. - Not run: the full suite, Windows execution, a full renderer build, NVK/AMD/Intel, a real disabled/read-only driver cache, a driver update, or extended play. The useful remaining evidence is an integrated periodic-retirement test, a real cache-selection fixture, and repeated longer warm/cold/disabled runs on the target low-RAM machine with the selected path, cache state, load-start marker, memory and first-use waits retained.

p0lux commented 2026-11-14 14:34:04 +00:00

Contributor

• Copy link
• Copy Markdown

The same idea measured on AMD with RADV, a Steam Deck OLED, in case the rule goes past NVIDIA. Not this PR's code: the switch main already has, RX_VK_PCACHE_LOAD=0, against unset.

Setup. Steam Deck OLED (AMD Custom APU 0932, RADV VANGOGH, Mesa 26.2.0-devel), 16 GB shared with the GPU, SteamOS 3.9.5, kernel 7.2.7-valve2. A renderer built from main 514591a3 with renderer-only changes of other pull requests (#238, #384, #387, #391, #223's commit), programs loaded at create (RX_VK_LOAD=create), Minimum preset, handheld profile, no screen (headless). The pipeline cache file is 307 MB there. Each run starts the game and loads a save by script; memory is read from /proc/<pid>/status and /proc/meminfo at the end of the measured span, KiB divided by 1,024.

What it gives back (RssAnon, MiB, file read then not read):

	read	not read
heavy view, 2 alternated pairs, Mesa's cache in ~/.cache (1,489 MB)	1,698 / 1,707	1,488 / 1,464
Hyrule Field, Mesa on a copy of the cache Steam gives the game (187 MB, below)	1,717 / 1,719 / 1,818 / 1,722	1,558 / 1,518 / 1,522 / 1,543 / 1,539
Kokiri Forest, same cache	1,890	1,474 / 1,565
heavy view, same cache	1,737	1,499

About 200 to 300 MiB for a 307 MB file. MemAvailable follows (heavy view 2,488 / 2,652 against 2,762 / 2,802 MiB). GPU memory (drm-memory-gtt) does not move, and standing still the frame rate is the same (29.28 / 29.57 against 29.46 / 29.36 fps).

What it costs here.

• Nothing drawn late: no draw left out and no wait for a shader or a pipeline in any run without the file; at most 3 first uses ran their own load (6 ms in all).
• The loads are slower, on the workers: a program load's median goes from 0.58-0.69 ms to 1.07-1.56 ms, and the log's cumulated pipeline creation from 0.4-0.5 s to 2.1-7.1 s a run.
• The arrival in an area, the 600 frames after a save loads (RX_VK_FRAME_TRACE=1, time spent in frames over 60 ms): Kokiri Forest 5.5 s read against 3.1 and 6.0 s; heavy view 3.2 against 3.4 s; Hyrule Field 2.5 / 2.8 / 3.5 / 2.6 s read against 3.0 / 3.4 / 3.8 / 3.5 / 2.6 s. In Hyrule Field the longest frame is about 1 s in 4 of the 5 runs without the file (514 to 1,011 ms) against 1 of the 4 with it (529 to 1,118 ms). Small samples, but on this 4-core machine the workers building pipelines seem to cost the replay while it uploads the area's textures.

For the driver cache guard (the review's point 2). On a Deck in Game Mode, Steam points Mesa at a cache of the game's own, steamapps/shadercache/<the shortcut's id>/mesa_shader_cache_sf (a single-file cache, through MESA_SHADER_CACHE_DIR), not at ~/.cache/mesa_shader_cache. On this Deck the first holds 187 MB and the second 1,489 MB (filled by headless runs started over SSH). The 187 MB one answered every pipeline in the runs above: the copy grew by 3 MB in its first run and not after. So a 512 MB threshold on a folder's size would look at the wrong folder here, and would turn down a cache that is complete. If the rule goes to Mesa, the folder to read is the one MESA_SHADER_CACHE_DIR names when it is set.

Not tested: this PR's code; a start from Steam itself (its cache was a copy, read through MESA_SHADER_CACHE_DIR and MESA_DISK_CACHE_SINGLE_FILE=1 in a headless run); a cold Mesa cache; a long session. The game folder's cache files were copied aside and put back after each series.

The same idea measured on AMD with RADV, a Steam Deck OLED, in case the rule goes past NVIDIA. Not this PR's code: the switch main already has, `RX_VK_PCACHE_LOAD=0`, against unset. **Setup.** Steam Deck OLED (AMD Custom APU 0932, RADV VANGOGH, Mesa 26.2.0-devel), 16 GB shared with the GPU, SteamOS 3.9.5, kernel 7.2.7-valve2. A renderer built from main 514591a3 with renderer-only changes of other pull requests (#238, #384, #387, #391, #223's commit), programs loaded at create (`RX_VK_LOAD=create`), Minimum preset, handheld profile, no screen (headless). The pipeline cache file is 307 MB there. Each run starts the game and loads a save by script; memory is read from `/proc/<pid>/status` and `/proc/meminfo` at the end of the measured span, KiB divided by 1,024. **What it gives back** (`RssAnon`, MiB, file read then not read): | | read | not read | |---|---|---| | heavy view, 2 alternated pairs, Mesa's cache in `~/.cache` (1,489 MB) | 1,698 / 1,707 | 1,488 / 1,464 | | Hyrule Field, Mesa on a copy of the cache Steam gives the game (187 MB, below) | 1,717 / 1,719 / 1,818 / 1,722 | 1,558 / 1,518 / 1,522 / 1,543 / 1,539 | | Kokiri Forest, same cache | 1,890 | 1,474 / 1,565 | | heavy view, same cache | 1,737 | 1,499 | About 200 to 300 MiB for a 307 MB file. `MemAvailable` follows (heavy view 2,488 / 2,652 against 2,762 / 2,802 MiB). GPU memory (`drm-memory-gtt`) does not move, and standing still the frame rate is the same (29.28 / 29.57 against 29.46 / 29.36 fps). **What it costs here.** - Nothing drawn late: no draw left out and no wait for a shader or a pipeline in any run without the file; at most 3 first uses ran their own load (6 ms in all). - The loads are slower, on the workers: a program load's median goes from 0.58-0.69 ms to 1.07-1.56 ms, and the log's cumulated pipeline creation from 0.4-0.5 s to 2.1-7.1 s a run. - The arrival in an area, the 600 frames after a save loads (`RX_VK_FRAME_TRACE=1`, time spent in frames over 60 ms): Kokiri Forest 5.5 s read against 3.1 and 6.0 s; heavy view 3.2 against 3.4 s; Hyrule Field 2.5 / 2.8 / 3.5 / 2.6 s read against 3.0 / 3.4 / 3.8 / 3.5 / 2.6 s. In Hyrule Field the longest frame is about 1 s in 4 of the 5 runs without the file (514 to 1,011 ms) against 1 of the 4 with it (529 to 1,118 ms). Small samples, but on this 4-core machine the workers building pipelines seem to cost the replay while it uploads the area's textures. **For the driver cache guard** (the review's point 2). On a Deck in Game Mode, Steam points Mesa at a cache of the game's own, `steamapps/shadercache/<the shortcut's id>/mesa_shader_cache_sf` (a single-file cache, through `MESA_SHADER_CACHE_DIR`), not at `~/.cache/mesa_shader_cache`. On this Deck the first holds 187 MB and the second 1,489 MB (filled by headless runs started over SSH). The 187 MB one answered every pipeline in the runs above: the copy grew by 3 MB in its first run and not after. So a 512 MB threshold on a folder's size would look at the wrong folder here, and would turn down a cache that is complete. If the rule goes to Mesa, the folder to read is the one `MESA_SHADER_CACHE_DIR` names when it is set. **Not tested:** this PR's code; a start from Steam itself (its cache was a copy, read through `MESA_SHADER_CACHE_DIR` and `MESA_DISK_CACHE_SINGLE_FILE=1` in a headless run); a cold Mesa cache; a long session. The game folder's cache files were copied aside and put back after each series.

mooki added the

area/renderer

gpu/nvidia

platform/linux

labels 2026-11-14 16:27:53 +00:00

Some checks are pending

Hide checks

CI / Conventional Commit title (pull_request) Blocked by required conditions

Details

CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Blocked by required conditions

Details

CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Blocked by required conditions

Details

CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Blocked by required conditions

Details

CI / Workflow lint (actionlint) (pull_request) Blocked by required conditions

Details

Third-party parts / As their notes say (pull_request) Blocked by required conditions

Details

Third-party parts / The same as upstream (pull_request) Blocked by required conditions

Details

Website / Website on Cloudflare (pull_request) Blocked by required conditions

Details

PR description / Description follows the template (pull_request_target) Successful in 3s

Details

Website / Website on Cloudflare (pull_request_target) Has been skipped

Details

This pull request doesn't have enough approvals yet. 0 of 1 approvals granted.

Some workflows are waiting to be reviewed.

This branch is out-of-date with the base branch

You are not authorized to merge this pull request.

View command line instructions

Checkout

From your project repository, check out a new branch and test the changes.

git fetch -u 'perf/388-pcache-low-ram:tonesi-perf/388-pcache-low-ram'

git switch 'tonesi-perf/388-pcache-low-ram'

Sign in to join this conversation.

Reviewers

No reviewers

mooki

Labels

Clear labels   

area/accessibility

Barriers for people with disabilities

  

area/audio

Sound and music

  

area/input

Controllers, keyboard, mouse

  

area/install

Installing, updating, building from your game

  

area/menu

In-game menu and settings

  

area/renderer

Vulkan renderer, shaders, effects

  

area/saves

Save files

  

area/upscaling

DLSS, FSR, frame generation

  

good first issue

A good start for a new contributor

  

gpu/amd

AMD GPUs

  

gpu/igpu

Integrated GPUs

  

gpu/intel

Intel GPUs

  

gpu/nvidia

NVIDIA GPUs

  

help wanted

Extra hands or knowledge wanted

  

kind

bug

Something doesn't work as it should

  

kind

chore

Build, CI, dependencies, tidying

  

kind

docs

Documentation

  

kind

feature

A new feature or an improvement

  

kind

performance

Frame rate, frame time, loading, memory

  

kind

question

A question rather than a change

  

needs measurements

Waiting for before/after measurements

  

platform/linux

Linux only

  

platform/steam-deck

Steam Deck

  

platform/windows

Windows only

  

preview

Maintainers: deploy a preview of this PR's website (read its changes first). Removed after the run.

  

priority

critical

Crashes, data loss, or blocks most players

  

priority

high

Important: next in line

  

priority

low

Nice to have

  

status

blocked

Waits on something else first

  

status

confirmed

Reproduced or agreed on

  

status

duplicate

Already reported or proposed

  

status

invalid

Not a problem in this project

  

status

needs-info

Waiting for more information from the reporter

  

status

wontfix

Will not be worked on

  

testing wanted

Wanted: a test on other hardware (posts a test request in #test-requests)

No labels

area/accessibility

area/audio

area/input

area/install

area/menu

area/renderer

area/saves

area/upscaling

good first issue

gpu/amd

gpu/igpu

gpu/intel

gpu/nvidia

help wanted

kind

bug

kind

chore

kind

docs

kind

feature

kind

performance

kind

question

needs measurements

platform/linux

platform/steam-deck

platform/windows

preview

priority

critical

priority

high

priority

low

status

blocked

status

confirmed

status

duplicate

status

invalid

status

needs-info

status

wontfix

testing wanted

Milestone

Clear milestone

No items

No milestone

Projects

Clear projects

No items

No project

Assignees

Clear assignees

No assignees

3 participants

Notifications

Subscribe

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference

MagicBeanDev/Longshot!416

No description provided.