fix(vk): honor cache-off for rolling pipeline files #415

Merged

HumanBean merged 4 commits from klimovich008/Longshot:fix/pipeline-cache-off into main 2026-11-15 07:46:47 +00:00

Conversation 5 Commits 4 Files changed 6 +119 -12

klimovich008 commented 2026-11-14 11:49:23 +00:00

Contributor

• Copy link
• Copy Markdown

Why

RX_VK_PCACHE_LOAD=0 still imported and saved rolling pipeline-cache files.

What changed

vk_draw.cpp: guard imports/generations. Extend existing fixtures/docs.

Flags and settings

None added. Existing RX_VK_PCACHE_LOAD=0 skips application persistence; defaults unchanged.

Measurements

main: 5e072fc8 (2026-11-14)
this PR: fix/pipeline-cache-off @ ddcf10f6

Fixture	main @ 5e072fc8	this PR @ ddcf10f6
OFF failures	4/19	0/19

Game, RX 9070 XT, AMD 26.10.2, Ryzen 7 5700X3D, Windows 11: the standard test (Hyrule, 16:9, 20 s, unlocked), defaults (the switch unset; the fix only acts with it off), both runtimes from the same cache seed; this fix (backport 57f06de8) on base 839e85ff against 839e85ff. 3 alternating pairs, no warm-ups, all valid; diagnostic:

	839e85ff	+ this PR
fps (median)	61.74, 62.54, 63.09 (62.54)	63.19, 63.44, 64.79 (63.44)
CPU ms a frame (median)	65.21	65.44

Runs in docs/tested/. No gain claimed (the controls' stalls). With RX_VK_PCACHE_LOAD=0 main merged three play files; this PR merged none and rebuilt pipelines at start-up, too long for a timed run.

Testing

Tested: Windows: python -I -B tests/test_pipeline_play_cache.py --cxx clang++; fails on old code, passes now. Full suites: 516/9/36 pass/fail/skip each. Shared failures: test_check_third_party, test_fpx_sticky, test_framegen_resources, test_module_ids, test_oot_state_tap, test_pad_layout, test_release, test_spawn_emit, test_tested_configurations. Linux (WSL Ubuntu): vk_draw.cpp as an object with clang 21 and the build's Linux flags; the 19-case fixture passes there.

Not tested: current-main gameplay; linked Linux game.

Needs other testers: Linux OFF behavior (T-7NCG).

Screenshots

Nothing on screen.

Risk and rollback

Temporary cleanup and driver caching remain. Revert.

Reviewer notes

None.

Issues

None.

Checklist

For everyone

• One topic.
• The title is a Conventional Commit.
• Rebased on current main.
• python -I -B tests/run_all.py run; shared-main failures named above.
• Tested in the game, or untested scope named.
• Testing filled in: what ran, gaps and test requests.
• Measured: main and this PR at named commits, same PC/scene/settings, at least two interleaved runs each; medians.
• No game files, dumps, build output or generated files.
• third_party/ untouched (a new folder needs PROVENANCE.md).
• The license is OK: GPL-3.0-or-later; compatible source, nothing decompiled.

If applicable

• AI assistance recorded with Assisted-by:/Co-Authored-By: trailers.
• Flags noted, and the switch reference updated.
• A file in docs/tested/ for each measured run, per guide.
• A game-free test added: fails on main, passes here.
• A check under tests/ that reads changed source updated without weakening.
• Host code builds on Windows/Linux with UTF-8 paths. Linux: object and fixture only; full build T-7NCG.
• Docs updated.

## Why `RX_VK_PCACHE_LOAD=0` still imported and saved rolling pipeline-cache files. ## What changed `vk_draw.cpp`: guard imports/generations. Extend existing fixtures/docs. ## Flags and settings None added. Existing `RX_VK_PCACHE_LOAD=0` skips application persistence; defaults unchanged. ## Measurements main: 5e072fc8 (2026-11-14) this PR: fix/pipeline-cache-off @ ddcf10f6 | Fixture | main @ 5e072fc8 | this PR @ ddcf10f6 | |---|---|---| | OFF failures | 4/19 | 0/19 | Game, RX 9070 XT, AMD 26.10.2, Ryzen 7 5700X3D, Windows 11: the standard test (Hyrule, 16:9, 20 s, unlocked), defaults (the switch unset; the fix only acts with it off), both runtimes from the same cache seed; this fix (backport 57f06de8) on base 839e85ff against 839e85ff. 3 alternating pairs, no warm-ups, all valid; diagnostic: | | 839e85ff | + this PR | |---|---|---| | fps (median) | 61.74, 62.54, 63.09 (62.54) | 63.19, 63.44, 64.79 (63.44) | | CPU ms a frame (median) | 65.21 | 65.44 | Runs in `docs/tested/`. No gain claimed (the controls' stalls). With `RX_VK_PCACHE_LOAD=0` main merged three play files; this PR merged none and rebuilt pipelines at start-up, too long for a timed run. ## Testing **Tested:** Windows: `python -I -B tests/test_pipeline_play_cache.py --cxx clang++`; fails on old code, passes now. Full suites: 516/9/36 pass/fail/skip each. Shared failures: test_check_third_party, test_fpx_sticky, test_framegen_resources, test_module_ids, test_oot_state_tap, test_pad_layout, test_release, test_spawn_emit, test_tested_configurations. Linux (WSL Ubuntu): `vk_draw.cpp` as an object with clang 21 and the build's Linux flags; the 19-case fixture passes there. **Not tested:** current-main gameplay; linked Linux game. **Needs other testers:** Linux OFF behavior (T-7NCG). ## Screenshots Nothing on screen. ## Risk and rollback Temporary cleanup and driver caching remain. Revert. ## Reviewer notes None. ## Issues None. ## Checklist ### For everyone - [x] One topic. - [x] The title is a Conventional Commit. - [x] Rebased on current `main`. - [x] `python -I -B tests/run_all.py` run; shared-main failures named above. - [x] Tested in the game, or untested scope named. - [x] Testing filled in: what ran, gaps and test requests. - [x] Measured: main and this PR at named commits, same PC/scene/settings, at least two interleaved runs each; medians. - [x] No game files, dumps, build output or generated files. - [x] `third_party/` untouched (a new folder needs `PROVENANCE.md`). - [x] The license is OK: GPL-3.0-or-later; compatible source, nothing decompiled. ### If applicable - [x] AI assistance recorded with `Assisted-by:`/`Co-Authored-By:` trailers. - [x] Flags noted, and the switch reference updated. - [x] A file in `docs/tested/` for each measured run, per guide. - [x] A game-free test added: fails on main, passes here. - [x] A check under `tests/` that reads changed source updated without weakening. - [x] Host code builds on Windows/Linux with UTF-8 paths. Linux: object and fixture only; full build T-7NCG. - [x] Docs updated.

klimovich008 added 1 commit 2026-11-14 11:49:24 +00:00

fix(vk): honor cache-off for rolling pipeline files ... 24dfba29e6

Skip rolling imports and play-generation persistence with RX_VK_PCACHE_LOAD=0.
Required pipelines still build in memory; default cache handling is unchanged.
Extend the existing real-function fixture to cover startup and OFF routing.

Assisted-by: OpenAI Codex

klimovich008 commented 2026-11-14 11:50:07 +00:00

Author

Contributor

• Copy link
• Copy Markdown

Description only: the Linux test request is open as T-7NCG (claimable once a maintainer adds the testing wanted label). Main has moved two commits since 0caf0618 (5e072fc8: #59, #400); this head still merges cleanly and was not rebased, so the fixture and suite results above stay those of 0caf0618. Still WIP.

Description only: the Linux test request is open as T-7NCG (claimable once a maintainer adds the testing wanted label). Main has moved two commits since 0caf0618 (5e072fc8: #59, #400); this head still merges cleanly and was not rebased, so the fixture and suite results above stay those of 0caf0618. Still WIP.

mooki added the

kind

bug

priority

high

labels 2026-11-14 12:24:54 +00:00

klimovich008 commented 2026-11-14 12:44:30 +00:00

Author

Contributor

• Copy link
• Copy Markdown

Description only: the Rebased on current main box is unticked. This head is on 0caf0618, main is 5e072fc8; it merges cleanly, but it has not been rebased. Still WIP.

Description only: the Rebased on current main box is unticked. This head is on 0caf0618, main is 5e072fc8; it merges cleanly, but it has not been rebased. Still WIP.

klimovich008 force-pushed fix/pipeline-cache-off from 24dfba29e6 to ddcf10f609 2026-11-14 15:25:04 +00:00 Compare

klimovich008 commented 2026-11-14 15:26:07 +00:00

Author

Contributor

• Copy link
• Copy Markdown

Rebased onto main 5e072fc8 (Codex: a03dc0ad, same four files) plus the run records (ddcf10f6). The fixture fails its OFF cases on main and passes here (19/19, also on Linux); full suite 516/9/36 with the same 9 failures as clean main. Measured: the standard test with defaults on both arms from the same cache seed, 3 valid pairs, no gain claimed. With RX_VK_PCACHE_LOAD=0, main merged three play files and this PR merged none; an OFF-arm timed run cannot finish here because the start-up rebuilds the pipelines. Linux: object and fixture with clang 21; T-7NCG for the full build. Taking it out of WIP.

Rebased onto main 5e072fc8 (Codex: a03dc0ad, same four files) plus the run records (ddcf10f6). The fixture fails its OFF cases on main and passes here (19/19, also on Linux); full suite 516/9/36 with the same 9 failures as clean main. Measured: the standard test with defaults on both arms from the same cache seed, 3 valid pairs, no gain claimed. With RX_VK_PCACHE_LOAD=0, main merged three play files and this PR merged none; an OFF-arm timed run cannot finish here because the start-up rebuilds the pipelines. Linux: object and fixture with clang 21; T-7NCG for the full build. Taking it out of WIP.

klimovich008 changed title from WIP: fix(vk): honor cache-off for rolling pipeline files to fix(vk): honor cache-off for rolling pipeline files 2026-11-14 15:26:27 +00:00

mooki added the

area/renderer

label 2026-11-14 16:27:52 +00:00

mooki added the

platform/linux

platform/windows

labels 2026-11-14 18:23:45 +00:00

tonesi referenced this pull request 2026-11-14 22:56:48 +00:00

perf(vk): skip the pipeline cache file on Linux NVIDIA PCs with 20 GB of RAM or less (0.8 GB less) #416

tonesi referenced this pull request 2026-11-14 22:56:49 +00:00

perf(vk): skip the pipeline cache file on Linux NVIDIA PCs with 20 GB of RAM or less (0.8 GB less) #416

HumanBean commented 2026-11-15 07:18:22 +00:00

Owner

• Copy link
• Copy Markdown

Review notes (fixture tests built and run on a scratch copy of main plus this PR):

1. The bug is worse than the description says. On main, with RX_VK_PCACHE_LOAD=0 and 16 or more play files, play_files_merge writes the never-loaded in-memory cache over longshot_vk_pipelines.bin and then deletes the play files: the real pipeline cache is lost. The new fixture fails 4 of its load-off cases on main's vk_draw.cpp and passes 19/19 with the PR. Worth saying in the description.
2. No new bug in the diff. The default path is unchanged (pcache && !off is pcache, pcache_no_save is false), moving pcache_no_save earlier is safe, and the close and save paths were already guarded.
3. Optional, and it predates this PR: under cache-off, pcache_saved_now compares an old base file on disk with the near-empty in-memory cache and returns true, so prepare_run writes prepared_for.warmed. After a driver update, one cache-off run can mark the stage libraries warmed when nothing was saved. Either return false there when pcache_no_save is set, or soften the doc line "neither ... loaded or saved".
4. The "Rebased on current main" checkbox isn't accurate: vk_draw.cpp has changed about 255 lines since the merge base. The diff still applies cleanly.

The shader stamp is correctly unchanged: vk_draw.cpp isn't in translator_files.py. Looks good to merge.

🤖 Review drafted with Claude Code

Review notes (fixture tests built and run on a scratch copy of main plus this PR): 1. **The bug is worse than the description says.** On main, with `RX_VK_PCACHE_LOAD=0` and 16 or more play files, `play_files_merge` writes the never-loaded in-memory cache over `longshot_vk_pipelines.bin` and then deletes the play files: the real pipeline cache is lost. The new fixture fails 4 of its load-off cases on main's `vk_draw.cpp` and passes 19/19 with the PR. Worth saying in the description. 2. **No new bug in the diff.** The default path is unchanged (`pcache && !off` is `pcache`, `pcache_no_save` is false), moving `pcache_no_save` earlier is safe, and the close and save paths were already guarded. 3. **Optional, and it predates this PR:** under cache-off, `pcache_saved_now` compares an old base file on disk with the near-empty in-memory cache and returns true, so `prepare_run` writes `prepared_for.warmed`. After a driver update, one cache-off run can mark the stage libraries warmed when nothing was saved. Either return false there when `pcache_no_save` is set, or soften the doc line "neither ... loaded or saved". 4. **The "Rebased on current main" checkbox isn't accurate:** `vk_draw.cpp` has changed about 255 lines since the merge base. The diff still applies cleanly. The shader stamp is correctly unchanged: `vk_draw.cpp` isn't in `translator_files.py`. Looks good to merge. 🤖 Review drafted with [Claude Code](https://claude.com/claude-code)

HumanBean approved these changes 2026-11-15 07:40:15 +00:00

Dismissed

HumanBean left a comment

• Copy link
• Copy Markdown

Approving: the fix is correct and the default path is unchanged.

One optional follow-up, on HumanBean's fork as review/pr415 (one commit on this PR's head, 5ff0d427; cherry-pick it if you want it):

• pcache_saved_now() returns false when pcache_no_save is set, so with RX_VK_PCACHE_LOAD=0 the prepare record is no longer written as warmed when nothing was saved. The docs line "neither … loaded or saved" is then accurate.
• The fixture now extracts pcache_saved_now and checks it in the load-off cases (the old code fails) and in the normal ones.
• Not built or run on my side; checked by reading.

Not changed, but the same kind of problem: the precompile path writes its "built" marker after a pipeline_cache_save(true) that does nothing under cache-off.

Suggested addition to the description:

This also fixes data loss on main. With RX_VK_PCACHE_LOAD=0, pcache_init starts from an empty cache but still calls play_files_merge(). When 16 or more play files are on disk, that merge writes the empty cache plus the play files over longshot_vk_pipelines.bin and then deletes the play files. So an A/B run with cache loading off destroyed the real base pipeline cache. With this PR, cache-off reads, merges and writes none of the pipeline cache files.

(The "Rebased on current main" box is ticked, but the fork point is 5e072fc8; the diff still applies cleanly.)

🤖 Drafted with Claude Code

Approving: the fix is correct and the default path is unchanged. One optional follow-up, on HumanBean's fork as [`review/pr415`](https://git.magicbean.dev/HumanBean/Longshot/compare/main...review/pr415) (one commit on this PR's head, `5ff0d427`; cherry-pick it if you want it): - `pcache_saved_now()` returns false when `pcache_no_save` is set, so with `RX_VK_PCACHE_LOAD=0` the prepare record is no longer written as warmed when nothing was saved. The docs line "neither … loaded or saved" is then accurate. - The fixture now extracts `pcache_saved_now` and checks it in the load-off cases (the old code fails) and in the normal ones. - Not built or run on my side; checked by reading. Not changed, but the same kind of problem: the precompile path writes its "built" marker after a `pipeline_cache_save(true)` that does nothing under cache-off. Suggested addition to the description: > This also fixes data loss on `main`. With `RX_VK_PCACHE_LOAD=0`, `pcache_init` starts from an empty cache but still calls `play_files_merge()`. When 16 or more play files are on disk, that merge writes the empty cache plus the play files over `longshot_vk_pipelines.bin` and then deletes the play files. So an A/B run with cache loading off destroyed the real base pipeline cache. With this PR, cache-off reads, merges and writes none of the pipeline cache files. (The "Rebased on current main" box is ticked, but the fork point is 5e072fc8; the diff still applies cleanly.) 🤖 Drafted with [Claude Code](https://claude.com/claude-code)

klimovich008 added 2 commits 2026-11-15 07:46:19 +00:00

fix(vk): under RX_VK_PCACHE_LOAD=0 the warm pass's save says not saved, so the prepare record is not written as done ... 5ff0d42733

pcache_saved_now() forced a save that does nothing under pcache_no_save, then
compared the base file left from before against the cache that started
empty: the old file passed for it, and prepare_run wrote prepared_for.warmed
although nothing was saved. It now returns false under cache-off, so
warm_libraries reports the libraries as not saved and the record stays as it
was. The fixture extracts pcache_saved_now and checks it in the load cases.

Assisted-by: Claude Opus 5.5
Co-Authored-By: Claude Opus 5.5 <[email protected]>

Merge main a4821d56 into fix/pipeline-cache-off ... bf02a8fb2d

Co-Authored-By: Claude Opus 5.5 <[email protected]>

klimovich008 dismissed HumanBean's review 2026-11-15 07:46:19 +00:00

Reason:

New commits pushed, approval review dismissed automatically according to repository settings

HumanBean merged commit 35465d12ac into main 2026-11-15 07:46:47 +00:00

HumanBean deleted branch fix/pipeline-cache-off 2026-11-15 07:46:47 +00:00

HumanBean referenced this pull request from a commit 2026-11-15 07:46:48 +00:00

fix(vk): honor cache-off for rolling pipeline files (#415)

zeldaootr referenced this pull request 2026-11-15 16:41:53 +00:00

fix(vk): a recording without a draw stream is submitted instead of crashing #552

Some checks failed

Hide checks

CI / Workflow lint (actionlint) (pull_request) Blocked by required conditions

Details

CI / Conventional Commit title (pull_request) Blocked by required conditions

Details

CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Blocked by required conditions

Details

CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Blocked by required conditions

Details

CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Blocked by required conditions

Details

Guide screenshots / Guide images and local UI source review (pull_request) Blocked by required conditions

Details

Third-party parts / As their notes say (pull_request) Blocked by required conditions

Details

Third-party parts / The same as upstream (pull_request) Blocked by required conditions

Details

Website / Website on Cloudflare (pull_request) Blocked by required conditions

Details

PR description / Description follows the template (pull_request_target) Failing after 7s

Details

Some workflows are waiting to be reviewed.

Sign in to join this conversation.

Reviewers

No reviewers

Labels

Clear labels   

area/accessibility

Barriers for people with disabilities

  

area/audio

Sound and music

  

area/input

Controllers, keyboard, mouse

  

area/install

Installing, updating, building from your game

  

area/menu

In-game menu and settings

  

area/renderer

Vulkan renderer, shaders, effects

  

area/saves

Save files

  

area/upscaling

DLSS, FSR, frame generation

  

beanstalk: approved @ 1697603

Set by Beanstalk for show; the farm's database is what counts

  

beanstalk: approved @ 2c5a62e

Set by Beanstalk for show; the farm's database is what counts

  

beanstalk: approved @ 4fd82b7

Set by Beanstalk for show; the farm's database is what counts

  

beanstalk: approved @ 64d6d09

Set by Beanstalk for show; the farm's database is what counts

  

beanstalk: approved @ 6f91fae

Set by Beanstalk for show; the farm's database is what counts

  

beanstalk: approved @ 70a78f9

Set by Beanstalk for show; the farm's database is what counts

  

beanstalk: approved @ 8d361dc

Set by Beanstalk for show; the farm's database is what counts

  

beanstalk: approved @ 9d8356e

Set by Beanstalk for show; the farm's database is what counts

  

beanstalk: approved @ abf2227

Set by Beanstalk for show; the farm's database is what counts

  

beanstalk: approved @ d82f202

Set by Beanstalk for show; the farm's database is what counts

  

beanstalk: approved @ e5f1e40

Set by Beanstalk for show; the farm's database is what counts

  

beanstalk: approved @ ee8ab5f

Set by Beanstalk for show; the farm's database is what counts

  

ci: approved

A maintainer approved CircleCI for this fork PR (sticky; removed after 14 quiet days)

  

good first issue

A good start for a new contributor

  

gpu/amd

AMD GPUs

  

gpu/igpu

Integrated GPUs

  

gpu/intel

Intel GPUs

  

gpu/nvidia

NVIDIA GPUs

  

help wanted

Extra hands or knowledge wanted

  

kind

bug

Something doesn't work as it should

  

kind

chore

Build, CI, dependencies, tidying

  

kind

docs

Documentation

  

kind

feature

A new feature or an improvement

  

kind

performance

Frame rate, frame time, loading, memory

  

kind

question

A question rather than a change

  

needs measurements

Waiting for before/after measurements

  

platform/linux

Affects Linux (a change for every system carries both Windows and Linux)

  

platform/macos

macOS (Apple silicon or Intel Macs)

  

platform/steam-deck

Steam Deck

  

platform/windows

Affects Windows (a change for every system carries both Windows and Linux)

  

preview

Maintainers: deploy a preview of this PR's website (read its changes first). Removed after the run.

  

priority

critical

Crashes, data loss, or blocks most players

  

priority

high

Important: next in line

  

priority

low

Nice to have

  

regression

Worked in an earlier build, broken in a newer one: highest priority, above other bugs

  

status

blocked

Waits on something else first

  

status

confirmed

Reproduced or agreed on

  

status

duplicate

Already reported or proposed

  

status

invalid

Not a problem in this project

  

status

needs-info

Waiting for more information from the reporter

  

status

wontfix

Will not be worked on

  

testing wanted

Wanted: a test on other hardware (posts a test request in #test-requests)

  

ci: approved

No labels

area/accessibility

area/audio

area/input

area/install

area/menu

area/renderer

area/saves

area/upscaling

beanstalk: approved @ 1697603

beanstalk: approved @ 2c5a62e

beanstalk: approved @ 4fd82b7

beanstalk: approved @ 64d6d09

beanstalk: approved @ 6f91fae

beanstalk: approved @ 70a78f9

beanstalk: approved @ 8d361dc

beanstalk: approved @ 9d8356e

beanstalk: approved @ abf2227

beanstalk: approved @ d82f202

beanstalk: approved @ e5f1e40

beanstalk: approved @ ee8ab5f

ci: approved

good first issue

gpu/amd

gpu/igpu

gpu/intel

gpu/nvidia

help wanted

kind

bug

kind

chore

kind

docs

kind

feature

kind

performance

kind

question

needs measurements

platform/linux

platform/macos

platform/steam-deck

platform/windows

preview

priority

critical

priority

high

priority

low

regression

status

blocked

status

confirmed

status

duplicate

status

invalid

status

needs-info

status

wontfix

testing wanted

ci: approved

Milestone

Clear milestone

No items

No milestone

Projects

Clear projects

No items

No project

Assignees

Clear assignees

No assignees

2 participants

Notifications

Subscribe

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference

MagicBeanDev/Longshot!415

No description provided.