WIP: fix(vk): pause VRAM compaction after a slow allocator pass #2032
No reviewers
Labels
No labels
area/accessibility
area/audio
area/input
area/install
area/menu
area/renderer
area/saves
area/tooling
area/upscaling
beanstalk: approved
beanstalk: failed
beanstalk: needs approval
beanstalk: running
beanstalk: tested
ci: approved
good first issue
gpu/amd
gpu/igpu
gpu/intel
gpu/nvidia
help wanted
kind
bug
kind
chore
kind
docs
kind
feature
kind
performance
kind
question
needs measurements
platform/linux
platform/macos
platform/steam-deck
platform/windows
preview
priority
critical
priority
high
priority
low
regression
risk
high
risk
low
risk
medium
status
blocked
status
confirmed
status
duplicate
status
invalid
status
needs-info
status
wontfix
ci: approved
No milestone
No project
No assignees
2 participants
Notifications
Due date
No due date set.
Dependencies
No dependencies set
Reference
MagicBeanDev/Longshot!2032
Loading…
Reference in a new issue
No description provided.
Delete branch "khuongngo0310/Longshot:codex/vram-compaction-stall"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Summary
A slow VMA defragmentation pass can leave the texture compaction sweep running. Finish the sweep if beginning or ending a pass takes at least 50 ms, and wait five seconds before starting another one.
A slow begin returns the unused temporary allocations. A slow end keeps the completed GPU copies and the freed-byte totals. Keep the image reaper excluded while VMA opens or closes the context because those operations change the pool's sorting mode.
Testing
Tested: Windows 11, RTX 3060 Ti, NVIDIA 621.01, 2560×1440. Loaded the Market and Kakariko Village, then completed a scripted 600-frame camera pan in Hyrule Field. The texture pool opened and closed 29 compaction sweeps with the reduced threshold. The game exited normally. The game-free regression reproduced the unclosed sweep on main and passed with this change. The threaded reaper check and independent code review also passed.
Still to test:
Measurements
Verdict: The game-free tests reproduced the slow-pass failure and verified the recovery. Gameplay exercised normal compaction. Neither build hit a slow allocator pass or a frame of 50 ms or more in the timed route. The patch had higher average FPS and higher p99 frame time in both pairs. These runs leave the effect on the reported long hitch unmeasured.
Two interleaved runs per build on Windows 11, RTX 3060 Ti (NVIDIA 621.01), with the same copied save and 2560x1440 output. Loaded the Market and Kakariko Village, then recorded the same 600-frame camera sweep in Hyrule Field. Both builds used a lowered compaction threshold, disabled image preallocation, uncapped FPS, and frame generation and VSync off.
85a59e6fbcb7cf16Risks
After a slow sweep, the five-second pause may retain VRAM longer under sustained memory pressure. The first slow allocator call can still hitch. The existing RX_VK_COMPACT=0 switch disables texture compaction.
Checklist
Ready for review
main; the title is a Conventional Commit.mise run testpasses (any failure that also fails onmainis named).main, or Measurements says why there is nothing to measure.third_party/untouched; nothing GPL-incompatible.When it applies
mainand passes here.tests/that reads changed source is updated, not weakened.Assisted-by:/Co-Authored-By:trailers.Ready to merge
main, measured again or a comment says why the numbers still hold.Tested in game: Windows 11, RTX 3060 Ti (NVIDIA 621.01), 2560x1440. Loaded Market and Kakariko Village, then completed a 600-frame camera pan in Hyrule Field. All 29 compaction sweeps in the run closed, and the game exited normally.
Possibly related, for context:
Recommendation: Request changes. A slow pass now closes, but repeatedly slow begins can discard every useful move while the machine is short of memory. The tests also miss whether the renderer starts the cooldown and whether the reaper stays excluded until the context closes.
The change stops a texture-compaction sweep after a VMA pass call takes 50 ms or more. It returns unused temporary allocations, preserves completed copies and their freed-byte totals, and consolidates sweep completion in one helper. The cost is delayed memory reclamation and a new reaper mutex acquisition on idle presents.
@fixwill do (8)gpu/vk/vk_vma.cpp:254: Allow useful moves from a slow begin to finish when memory pressure is critical, then close the sweep and back off after committing the copies.gpu/vk/vk_texture.cpp:4815: The moved queue check takes reap_mu on every idle present with compaction enabled, even when the next check is hundreds of presents away. (more on the line)tests/test_vram_compact_stall.py:148: The cooldown case calls compact_finish directly and runs a slice of compact_step. (more on the line)tests/test_image_reaper_threads.py:362: The slow-end case can pass if compact_over moves outside reap_mu while notify_one remains after it. (more on the line)gpu/vk/vk_vma.cpp:252: Log which VMA call triggered the cutoff, its elapsed milliseconds, and the bytes reclaimed so a stall can be matched to frame and memory samples.docs/dev/reference/architecture.md:207: The arbiter docs say texture-block compaction runs at the next present. A pressure increase during the cooldown can advance the frame check but still leaves reclamation deferred. Describe the cooldown and its interaction with the pressure wake-up once the policy is settled.tests/test_vram_compact.py:15: This updated check still pins g++ in the suite, so it ignores CXX=clang++ and can skip on a Windows setup with only llvm-mingw. Use the suite's {cxx} placeholder as the new stall and threaded fixtures do.For the author (2) (a run, a measurement or a decision)
gpu/vk/vk_vma.cpp:254: A fragmented pool whose first begin stays above 50 ms can lose all compaction progress. (more on the line)gpu/vk/vk_vma.cpp:252: The smoke run shows that ordinary sweeps close. (more on the line)Checks: Still running:
CI / Conventional Commit title (pull_request),CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request),CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request),CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request),CI / Workflow lint (actionlint) (pull_request),CI / Website build and checks (pull_request),Guide screenshots / Guide images and local UI source review (pull_request),Third-party parts / As their notes say (pull_request),Third-party parts / The same as upstream (pull_request),PR description / Description follows the template (pull_request_target).More: does it help, other systems, switches, what was read
Does it help: Yes for sweep cleanup in the reported game-free comparison of
85a59e6f83andbcb7cf16be. The author reports 29 completed sweeps in a Windows 11 RTX 3060 Ti smoke run with a reduced threshold. The effect on gameplay hitches and memory pressure is unproven. No timed gameplay comparison or hardware reproduction of the slow pass was supplied.Other systems: The policy applies to NVIDIA, AMD and Intel on Windows and Linux, including Steam Deck, where texture blocks occupy shared RAM. Reaper synchronization also affects dead render targets and their associated resources. The copy path still waits for GPU completion before destroying old images. The diff leaves the public renderer ABI, shaders, saves, audio and input untouched. Only the Windows NVIDIA smoke run is reported.
Switches: None added, removed or given a new default. RX_VK_COMPACT and RX_VK_REAPER remain on by default, so the 50 ms threshold and five-second cooldown apply in the default configuration. RX_VK_COMPACT_MB retains its allowance-based default. The reduced smoke-test setting was unspecified, and the new cooldown is unvalidated under pressure.
Reviewed:
bcb7cf16beagainst85a59e6f83: Read all four changed Python tests in full and all changed C++ sections with their surrounding code. Traced the present caller, GPU completion, allocator accounting, reaper writes and readers, and pressure handling. Read AGENTS.md, CONTRIBUTING.md, the PR template and developer standards, plus the relevant architecture and test-writing rules. No separate review checklist or adjacent Tooling checkout was present. Skimmed the screenshot manifest, workflow and other source-reading tests. The rest of the large renderer files and the external VMA implementation were outside the read. No run logs, tester comments or prior reviews were supplied.; nothing run (a code review).Checked and fine:
Testing: Read-only review. Git diff --check passed. No builds, tests or project scripts were run. The author reports the game-free regression and threaded fixture passed on head, plus the Windows gameplay smoke run. CI was still running, and the full Windows suite and Linux gameplay were outstanding.
Overlaps and merge order: None found in the supplied PR context or local history.
Automated code review by BeanBot (gpt-6.2-sol), on the pull request's opening. Nothing was run; a maintainer decides.
@ -4803,1 +4812,4 @@}// (RX_VK_REAPER: the dead images handed to the reaper are freed before a pass opens, as they were here before: a pass VMA// plans with them still in the blocks moves around them and empties fewer. A present or so later; never a wait here)if (images_to_reap()) return;L2: The moved queue check takes reap_mu on every idle present with compaction enabled, even when the next check is hundreds of presents away. On Linux the reaper can hold that mutex while writing its periodic log line to stderr, so an idle compaction check can stall replay behind logging. Return first when no sweep is active and the frame or cooldown deadline is still pending, then check the queue before opening VMA.
@ -239,3 +249,4 @@reap_done_cv.wait(rk, [] { return !reaping; }); // (the frees in flight, done first)pass_waiting = false;const auto start = compact_now();const VkResult r = vmaBeginDefragmentationPass(vma, compact_ctx, &compact_pass);L5 (minor): The smoke run shows that ordinary sweeps close. It leaves open whether players see fewer hitches at the shipped settings. On a setup that reproduces the slow pass, compare the named base and revised head in at least two interleaved runs each with the same save, route, settings and build. Report pass durations, stalls, average FPS, 1% lows, median and p99 frame times, and peak RAM and VRAM. Cover the affected Tier 1 configurations, including AMD and Steam Deck.
@ -242,2 +253,2 @@if (r != VK_INCOMPLETE) { // (VK_SUCCESS: nothing left to move; or an error. The reaper, held for it, goes on)compact_over(freed_bytes, freed_blocks);const bool stalled = compact_now() - start >= kCompactStall;if (stalled || r != VK_INCOMPLETE) {L1: A fragmented pool whose first begin stays above 50 ms can lose all compaction progress. Even if VMA proposes moves that would empty a block, every retry discards them and starts from scratch. The cooldown also overrides the pressure wake-up. Give critical VRAM or shared-RAM pressure a bounded way to finish useful moves before backing off, and compare reclaimed bytes and frame times with base under sustained pressure.
@ -316,0 +359,4 @@hand_over(m);passes_left = 1;slow_end_us = 51'000;const auto result = image_compact_pass_end(nullptr, nullptr);L4: The slow-end case can pass if compact_over moves outside reap_mu while notify_one remains after it. A reaper already asleep after the hand-over has no wake-up during the five-millisecond spin and therefore records no overlap. Gate the context-close stub and explicitly make a competing free attempt while it is open. Verify the test fails when context close moves outside the lock.
@ -0,0 +145,4 @@void check_cooldown() {using namespace std::chrono_literals;compact.pressed = true;compact_finish(ImageCompactResult::stalled, 240, 3);L3: The cooldown case calls compact_finish directly and runs a slice of compact_step. Removing the renderer's compact_finish call after a stalled begin would leave both this fixture and the source checks green while the game skipped the cooldown. Drive the production scheduling path with a fake clock and allocator, and assert that neither a stalled begin nor a stalled end can reopen a sweep before the deadline.
Added the paired Hyrule Field timings. Both builds exercised compaction and neither reproduced a slow pass. The game-free regressions cover the recovery path.
View command line instructions
Checkout
From your project repository, check out a new branch and test the changes.