fix(vk): stop the VRAM compaction's hold from evicting textures in use #2198
No reviewers
Labels
No labels
accessibility
autorelease: pending
autorelease: tagged
bug
dependencies
documentation
duplicate
enhancement
github_actions
good first issue
help wanted
invalid
question
testing wanted
wontfix
No milestone
No assignees
1 participant
No due date set.
Dependencies
No dependencies set
Reference
MagicBeanDev/Hookshot!2198
Loading…
Reference in a new issue
No description provided.
Delete branch "fix/vram-hold-cold"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
With
RX_VK_COMPACT=1on a card with little room, textures that were still in use were thrown out and loaded again every time the camera turned, and at some places the game could freeze for a few seconds. Both came from the rule that holds VRAM at 85%; this fixes that rule and gives it a small reserve. With that fixed, the compaction is now on by default (RX_VK_COMPACT=0turns it off), as hckrman101 suggested in Discord.Tracking: #995. Follow-up to #1821 and to @hckrman101's 6 GB result there (4-5x the frames over 33 ms with the switch on).
What went wrong
The hold sets the texture budget to "what the textures hold, less the excess over 85% of the allowance". Whenever the count stays over that line (a small card, a burst of new textures, another program taking VRAM), every pass is a pass over the budget, and two things follow:
The change
vram_compact::hold_met,hold_keeps,kHoldUnused).reserve_pct,hold_target,kReserveUnused). With one line the cache sat at it and whatever came in next went over.RX_VK_COMPACT=0(vram_compact::enabled: not set is on,0or empty is off). It was opt-in while the hold could let go of textures in use.Numbers
Built from this branch (v2.0.11-782-gf0c6e73) and from main (v2.0.11-780-g3932421). Unattended runs with scripted input, DLAA 2560x1440, RTX 4070 SUPER (617.42), Ryzen 9 9950X3D, Windows 11, nothing else running, two runs each. Frames from
RX_VK_FRAME_TRACE.RX_VK_TEST_ALLOWED_MBstands in for a smaller card or for other programs.Camera turning in place in Hyrule Field,
RX_VK_COMPACT=1 RX_VK_TEST_ALLOWED_MB=7000, 50 seconds:Five-minute walk across Hyrule Field,
RX_VK_COMPACT=1 RX_VK_ASTC_BC_ALL=1:No validation, device-lost or "not made" line in any run.
Those runs were on main as it was yesterday (
3932421). After rebasing onto today's main (4d68bc8) I ran each once more, built from this branch (v2.1.0-rc.3-50-g3acf477a) and from main (v2.1.0-rc.3-48-g4d68bc83), counted from the end of the load:In that spin run the branch's count passed the allowance once while the save loaded (7,016 of 7,000 MB), which is what gave the shadows back; main's run did not, and neither did the four spin runs on the earlier base. One run, so I can't say whether it is the branch or chance.
On by default against off
Built from this branch after the default changed (v2.1.0-rc.3-58-g06ce5785, main
49f75dcmerged in). No switch set againstRX_VK_COMPACT=0, DLAA 2560x1440, the same card, nothing else running, counted from the end of the load.Five-minute walk across Hyrule Field, two runs each:
RX_VK_COMPACT=0Camera turning in place with
RX_VK_TEST_ALLOWED_MB=7000, one run each: on 60.0 fps, 1,203 MB uploaded, 1 frame over 25 ms; off 59.9 fps, 2,660 MB, 8 frames. No validation, device-lost or "not made" line in any of these.On my local build (the same two commits on an older base, with a measuring hook that holds 768 MB of the card), the reserve alone took the share of a walk spent over 85% of the allowance from 47-68% to 2-5%, and over 87% from 7-26% to 0-1%, with uploads and frame times unchanged.
Limits
RX_VK_COMPACT=1.RX_VK_COMPACT=0is the way back there.Test
python -I -B tests/test_vram_compact.py: the rules' arithmetic (hold_keeps,hold_met,reserve_pct,hold_target), and source checks that the hold's pass counts the dead targets before it chooses, does not stop at the excess, judges a live texture only once it knows whether a target drew over it, and that the driver's limit clears both lines; andenabled: not set is on,0or empty is off. Fails on main.🤖 Generated with Claude Code
Tested on Linux, which the PR lists as untested. It holds up there too.
Setup:
hookshot_vk.soonly (v2.0.11-795-g8baccd6bhost). The docs commit was left out.RX_VK_COMPACT=1, a player's own daytime Hyrule Field save. 50 seconds of the camera turning in place after the load, frames fromRX_VK_FRAME_TRACE. A and B differ only in the renderer, and the runs alternated between them.RX_VK_TEST_ALLOWED_MB=7000RX_VK_TEST_ALLOWED_MB=7000RX_VK_TEST_ALLOWED_MB=7000RX_VK_TEST_ALLOWED_MB=7000Results:
tests/test_vram_compact.pyand the fulltests/run_all.pypass (330 passed, 0 failed, 10 skipped).🤖 Generated with Claude Code
I tested this on Windows with an RTX 5080 (16 GB), using the same test as Joey-Magnus above. It holds up here too.
Setup:
18a5be0) built with the Builder's llvm-mingw. The "before" renderer is main4d68bc8with the two code files reverted, built into the same folder. Onlyhookshot_vk.dlldiffers between A and B. The runs alternated: before, PR, before, PR.RX_VK_COMPACT=1, capped at 60 fps (HOS_FPS_MAX=60), daytime Hyrule Field save. 50 seconds of the camera turning in place after the load, with frames fromRX_VK_FRAME_TRACE. Headless (HOS_HEADLESS=1), with the save loaded and the camera turned throughHOS_INPUT_LIVE.RX_VK_TEST_ALLOWED_MB=7000RX_VK_TEST_ALLOWED_MB=7000RX_VK_TEST_ALLOWED_MB=7000RX_VK_TEST_ALLOWED_MB=7000(1% low here is the average frame rate of the slowest 1% of frames. Uploaded is the sum of the trace's
upload_mb, and retired is the sum of the log's "N textures retired" lines, both over the 50 seconds.)Results:
🤖 Generated with Claude Code
Tested this on an 8 GB AMD card, where the hold matters most, and it removes most of the camera-turn hitches with
RX_VK_COMPACT=1.Setup
633eb435) with this PR'svk_texture.cppandvram_compact.hchanges. The docs hunk conflicts with the release tag and was left out. Built with the builder's llvm-mingw (clang 23.1.5),RX_VK_ONLY=1 RX_RELEASE=1.tests/test_vram_compact.py --cxx clang++passes.RX_VK_FORCE_MESH_FALLBACK=1,RX_VK_CHECKPOINTS=1,RX_VK_RECORDER=thread. OnlyRX_VK_COMPACTdiffers.Method
HOS_INPUT_LIVEscript after the load: camera sweeps left and right plus two short walks. Both arms ended in the same spot.RX_VK_FRAME_LOG.RX_VK_SHOT_EVERY=60on, and each shot costs a 55–65 ms frame. Those presents (and the one after) are left out of the numbers below.RX_VK_COMPACToffRX_VK_COMPACT=1with this PRVRAM:lines)With the PR, textures made settled at 0.4–0.8 a frame, so the churn described above did not show up. Half of the remaining frames over 50 ms came in one burst on the first camera turn, as new views streamed in. No device loss, validation or "not made" lines in either run. A normal play session afterwards showed no visual problems.
Caveats
Hi! I've been troubleshooting the persistent FPS degradation on an RTX 3070 8 GB and wanted to share my results, as I think they might be relevant to this PR.
System:
The issue:
The game can run smoothly at approximately 50–60 FPS while exploring Hyrule Field or Kakariko Village. However, transitioning between these areas can cause a significant, persistent performance drop.
For example:
I've also compared two recordings in the same location using essentially the same route. The main difference was whether I'd travelled through Kakariko beforehand. The session that had gone through Kakariko performed significantly worse.
This makes me suspect something is remaining allocated or otherwise affecting performance after an area transition.
What I've tested:
Initially, I experimented with the following settings:
RX_VK_VRAM_MB=2560RX_VK_FAST_RECLAIM=1RX_VK_FAST_PRESSURE=80RX_VK_FAST_UNUSED=8These helped somewhat, but the persistent degradation still occurred.
I also experimented with
RX_VK_MIRROR_MB=1024, which didn't resolve the problem, so I removed it.Enabling
RX_VK_COMPACT=1made the biggest difference.With compaction enabled alongside the earlier settings, the game became noticeably smoother. Areas that previously struggled could run at approximately 50–60 FPS again, and performance after transitions initially seemed substantially better.
I then tested
RX_VK_COMPACT=1by itself, without the other VRAM overrides.Initially, this also performed very well. However, during a longer session, I experienced another prolonged FPS collapse. Performance eventually recovered, but there were periods around 12–21 FPS.
The longer log (
hookshot_log(8).txt) contains 17 instances of:vk: over the driver's video-memory allowanceThe reported allocation sometimes reached approximately 7.6 GB against an allowance of about 7.1 GB.
Compaction was actively running and returning memory to the driver, but it wasn't consistently preventing these episodes.
Why I'm posting here:
I've read the description of this PR and noticed the problem involving textures being evicted after only 60 presents, then repeatedly reloaded when needed again.
I'm wondering whether that could explain why performance sometimes remains degraded after travelling between Kakariko and Hyrule Field.
I understand that the current fix has been tested with camera movement within an area, but I thought the loading-screen transition behaviour might also be worth investigating.
I've attached logs from the different configurations, including one where compaction substantially helped and a longer run where performance degraded again.
I'd be happy to test the proposed fix on my RTX 3070, particularly with repeated Kakariko ↔ Hyrule Field transitions, if a suitable build becomes available.
Thanks for the work on this project — the improvements over the last few days have already been crazyyy
03_Additional_long_test.txt
02_Compaction_only_long_test.txt
01_Compaction_plus_tweaks.txt
!
Thanks, this is a clear diagnosis and the fix for the hold's pass reads right. Counting the dead render targets first and then not stopping at the first texture past the excess fixes the starvation you describe. The prepass uses the same conditions as the loop (
need_dead,last_value > completed, shifted/drawn_over), sohold_deadis exactly what the loop then lets go: no dead target is counted and then kept. The fence check (tex->last_value > completed) still comes before any texture is chosen, so nothing the GPU may still read gets evicted. The header is plain integer math and builds the same with llvm-mingw and gcc/clang. Joey-Magnus (Linux, 4070 Ti), nullifyshot (Windows, 5080) and Tempus3214 (RX 6600 8 GB) all confirm it on hardware you didn't have. CI is green.One thing I'd like settled before merge. The description says passes from the other rules (the budget, 90%, 97%) are unchanged, and the test says the driver's limit clears both lines. In practice that clearing almost never happens. The 90% rule and the reserve share the same
pressure, so the reserve's excess is always at least the 90% rule's plus 10% of the allowance. Whenever the 90% rule fires underRX_VK_COMPACT,reserve < budgetholds and the pass becomes a hold pass. The 97% rule only wins (budget < evict_to) whenreclaimis over ~17% of the allowance. So past 90% and even past 97%, a live texture now has to be unused for 600 presents before it can go, where main let it go aftermin_unused(60). That is exactly the situation of the 8 GB reports: your own Limits note (93-96% stretches, nothing left to let go) and buasfe's log with the count over the allowance 17 times on a 3070. Paging past the allowance costs more than re-uploads. I'd keep the 600/1800 thresholds for the part of the excess that only the 85/80 lines ask for, and let textures pastneed_unusedgo for the part the 90%/97% rules ask for. For example, record the budget those rules set before the hold applies (rule_to) and givehold_targeta third tier for it, so that rule's excess is served as before. A test case for "count over 90%: a texture unused 100 presents still goes" would pin it.Nits:
if (budget < evict_to) evict_to = hold_to = 0;comment ("the driver's limit asks for more") reads as if this is the usual path near the limit. A word that it rarely fires once the reserve is set would help the next reader.This only fires when the 97% rule's excess (by
used) beats the reserve's (byused - reclaim, 17 points lower), which takesreclaim> ~17% of the allowance. So over 97% the hold pass, with its 600/1800-present thresholds, usually still decides. Unless that's intended, the driver-limit case needs its own target (review body).The reserve's excess is always at least the 90% rule's (same
pressure, a line 10 points lower), so whenever the 90% rule setbudgetabove, this replaces it, and the pass that follows protects live textures for 600 presents instead ofmin_unused. See the review body.Suggestion: a third tier here, where a texture unused for
need_unusedmay go whiletotal_bytes - goingis above what the 90%/97%/size rules alone asked for. Then the hold only changes what its own lines take.Thanks for the follow-up. Since my last review the branch picked up a merge of main, the switch to on-by-default (
vram_compact::enabled, with tests for unset /1/0/ empty) and the new tested-configurations rows. The default flip itself is clean:enabled()matches the docs and the test,RX_VK_COMPACT=0is a clear way back, and the on-vs-off walk numbers (633/408 frames over 25 ms down to 66/36, shadows never given back) are a good case for it.Still open, and it matters more now:
hold_targetare unchanged, so what I described last time still holds: whenever the 90% rule fires, the 80% reserve sets a lower budget from the samepressureand the pass becomes a hold pass, where a live texture has to be unused 600 presents (notmin_unused, 60) before it can go for the hold's line, and 1,800 for the reserve's. The 97% rule only takes over whenreclaimis above ~17% of the allowance. Before this push that only affected people who setRX_VK_COMPACT=1. With the default flipped it affects every 8 GB card, which is where the count really sits past 90% (your own Limits note on 93-96% stretches, buasfe's 3070 log with 17 "over the driver's allowance" lines). I'd still like the third tier (record what the 90%/97%/size rules asked for before the hold applies, and let textures pastneed_unusedgo for that part of the excess) plus a test like "count over 90%: a texture unused 100 presents still goes" before this ships on by default. If you think 600 presents is right even past 90%, say why and I'm happy to look at numbers, but I'd want them from a real 8 GB card, not the simulated allowance.Nits, both still open:
if (budget < evict_to) evict_to = hold_to = 0;(3843) is unchanged; it still reads as the usual path near the limit, when it rarely fires once the reserve is set.CI: everything passes except
appimage, which died on a Launchpad connection reset (RemoteDisconnected) while installing packages, not on the build. A re-run should clear it. Mergeable, no conflicts.@ -281,3 +281,2 @@bool pool_pressed; // (VRAM near what the driver allows: texture_frame's budget)bool compact_on() { // (hookshot, RX_VK_COMPACT=1: the textures in a pool of their own, compacted: compact_step)static const bool v = getenv("RX_VK_COMPACT") && atoi(getenv("RX_VK_COMPACT")) != 0;bool compact_on() { // (hookshot: the textures in a pool of their own, compacted: compact_step. On unless RX_VK_COMPACT=0)With this on by default, the 90%/97% point from my last review (the hold's 600/1,800-present thresholds deciding passes the 90%/97% rules asked for) now applies to every small card without the switch set. I'd land the third tier before the default flips, or flip the default in a follow-up once that's in.
Heads-up: this conflicts with main now. Could you rebase onto main and resolve the conflicts? The point in my last review (the 90%/97% rules' passes still using the hold's 600/1800-present thresholds, now on by default for every 8 GB card) is still open as well. Thanks!
Thanks for the review. You were right, and it mattered more once the default flipped. Pushed:
2ed98c46), with the conflict indocs/tested-configurations.mdresolved. It's a merge and not a rebase because I can't force-push from here.2a407e10). The budget that the size, the 90% rule and the 97% rule ask for on their own is now kept apart from the hold's (rule_budget).hold_targetlets a texture unused forneed_unusedgo until the textures are down to 7/8 of that budget, which is what a pass over the budget cuts to with the compaction off. The 600 and 1,800 presents now only decide what the 85% and 80% lines add on top.tests/test_vram_compact.pyhas "the count over 90%: a texture unused 100 presents still goes", a worked case at 94% of a 7,000 MB allowance, and source checks thatrule_budgetis taken before the hold's lines and joined by the 97% rule's.if (budget < evict_to) evict_to = hold_to = 0;now says it rarely fires once the reserve is set, and why.Not run in the game: this push is the rules' arithmetic and the source checks only, and I still have no real 8 GB card. The AppImage failure was a Launchpad connection reset; this push should re-run it.
@buasfe sorry for the slow answer. Your log (the count over the allowance 17 times on the 3070) is the case this push is for: past 90% the cache can again let go of textures unused for a second. I can't attach a build myself, but if you're able to build from this branch, a few Kakariko ↔ Hyrule Field transitions on your card would cover the two things nobody has tested here, a scene change and a real 8 GB card.
🤖 Generated with Claude Code
Done in
2a407e10:rule_target/ theneed_unusedtier inhold_target, with the worked 94% case in the test. Resolved from my side.@ -281,3 +281,2 @@bool pool_pressed; // (VRAM near what the driver allows: texture_frame's budget)bool compact_on() { // (hookshot, RX_VK_COMPACT=1: the textures in a pool of their own, compacted: compact_step)static const bool v = getenv("RX_VK_COMPACT") && atoi(getenv("RX_VK_COMPACT")) != 0;bool compact_on() { // (hookshot: the textures in a pool of their own, compacted: compact_step. On unless RX_VK_COMPACT=0)With the third tier in, the 90%/97% passes again let textures unused 60 presents go for their part, so the default flip is fine by me.
Thanks, this is what I was after. Since my last review: I merged main into the branch (
40d8e17) to clear the conflict; the only conflict was thedocs/development.mdtable, where I kept yourRX_VK_COMPACTrow and main's newRX_VK_REACTIVErow. Nothing else in the merge differs from what main brought in.Earlier points:
2a407e10).rule_budgetis taken after the size and 90% rules and before the hold's lines, the 97% rule joins it, andhold_targetnow gives a texture unused forneed_unusedthe rules' part (rule_target, 7/8 of their budget, the same cut a pass makes with the compaction off), then the hold's line at 600 and the reserve's at 1,800, taking the lowest that applies. The dead render targets still count first throughhold_met, and the fence andkeep_rendered/ finalized checks still come after, so nothing in use by the GPU is newly exposed. The 94%-of-7,000 MB case in the test works the numbers through, including "unused 100 presents still goes" and stopping once the rules' part is covered. With this, the description's "passes the other rules ask for are as they were" is true again.if (budget < evict_to) evict_to = hold_to = 0;: fixed, it now says it's rare once the reserve is set and why.Overlaps: #1879 landed on main right after the merge; a trial 3-way merge of
vk_texture.cppis clean (it only addsoverwrittentostruct Evict/evict.push_backand the retire tally), and GitHub shows no conflict. #2368 inserts its keep check just above the hold check in the same loop, so whichever lands second resolves that spot and should make the hold's pass count the kept bytes, as agreed there.CI: all green except
Python source checks (windows-2022, part 1/2), which failstests/test_native_astc_texture_count.py(its fixture'sscanfunder-Werroron Windows). Main fails the same test since #2288, so it isn't this PR.Approving.
github-sync referenced this pull request2026-11-12 06:22:54 +00:00