perf(vk): the image reaper's queue is bounded and gives its memory back near the VRAM allowance #1914
No reviewers
Labels
No labels
area/accessibility
area/audio
area/input
area/install
area/menu
area/renderer
area/saves
area/tooling
area/upscaling
beanstalk: approved
beanstalk: failed
beanstalk: needs approval
beanstalk: running
beanstalk: tested
ci: approved
good first issue
gpu/amd
gpu/igpu
gpu/intel
gpu/nvidia
help wanted
kind
bug
kind
chore
kind
docs
kind
feature
kind
performance
kind
question
needs measurements
platform/linux
platform/macos
platform/steam-deck
platform/windows
preview
priority
critical
priority
high
priority
low
regression
risk
high
risk
low
risk
medium
status
blocked
status
confirmed
status
duplicate
status
invalid
status
needs-info
status
wontfix
ci: approved
No milestone
No project
No assignees
3 participants
Notifications
Due date
No due date set.
Dependencies
No dependencies set
Reference
MagicBeanDev/Longshot!1914
Loadingβ¦
Reference in a new issue
No description provided.
Delete branch "dyst/Longshot:perf/reaper-bounded"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Summary
A follow-up to #792 (merged as
afc4d1ad), for the two points of BeanBot's review that were still open there: the reaper's queue had no bound and no answer to VRAM pressure (point 2), and its checks read only the source's text (point 4). This commit was pushed to #792's branch after it merged, so it is here.The cap: past an 8th of the driver's VRAM allowance (128 MB to 1 GB; 1.17 GB on a 10 GB card), an image handed over is destroyed where it was, as before #792.
VRAM pressure: the queue is the budget line's pool
reaper(mem_account.h).reap_nowfrees it on the asking thread.reap_nowreturns only what goes back to the driver: its dedicated allocations' bytes (allocation_dedicated, VMA'svmaGetAllocationInfo2). A share of a block freed leaves the block with VMA, so it counts 0 and the arbiter goes on asking the other pools. With only shares queued, it frees every queued entry of that ledger.A pass that waits holds the reaper off its next free. With a queue, the reaper began the next free before the waiting pass woke, so a pass could wait for the whole queue. When no pass begins, the reaper is woken again.
compact_stepopens a pass only once the queue is empty, and nothing is handed over between that look and the pass. The lock is now safe without relying on that.A log line:
vk: reaper:every 10 s says what went, how much was destroyed where it was handed over, and the queue's most.A test that runs the code:
tests/test_image_reaper_threads.pycuts the compaction's and the reaper's code out ofvk_vma.cppand runs it on threads against stub VMA and Vulkan calls. It checks:reap_now;RX_VK_REAPER=0;reap_nowwith dedicated images, with shares of blocks, with mixed ledgers and with a buffer alone.Copies of
vk_vma.cppthat break one rule each fail it:reap_nowthat counts allocation sizes;reap_nowthat chooses by the image's ledger alone.Writing it found the pass's wait and a count the cap's frees took off without having added.
scripts/memtour.pyhas the new pool's columns.Read first:
gpu/vk/vk_vma.cpp, fromdestroy_image_latertoreap_now, andimage_compact_pass;gpu/vk/vk_texture.cpp'saccount_sampleand the poolreaper.See #792.
Testing
Tested:
In the game, on the first cap (
afa2a345, a 32nd of the allowance): Windows 11, Ryzen 9 3900X, RTX 3080, NVIDIA 621.14, packed over base3f2c9b3c. 6 runs interleaved againstRX_VK_REAPER=0on the same runtime.RX_VK_TEST_VRAM_MB=3072).reaperonce (one run);tests/test_image_reaper_threads.py, not by these runs.RX_VK_REAPER=0. Hence the cap of92e963c0, which these runs are not of.afa2a345) againstRX_VK_REAPER=0, the range of the runsRX_VK_REAPER=0tex_frame_msmeantex_frame_msp99tex_frame_msmeantex_frame_msp99tex_frame_msmaxtex_frame_msp99tex_frame_msp99The checks:
tests/run_all.py --changedon this branch over mainafc4d1ad: 85 passed, 10 failed. Each of the 10 also fails on a cleanmain:test_fsr_uniform_ownership,test_linux_pressure,test_low_vram_texture_mips,test_pass_barrier,test_readback_timeout_source,test_replay_write_order,test_sparse_mirror,test_staging_burst,test_texel_buffer_ownership,test_ub_update_batch.tests/test_image_reaper_threads.pypassed three times in a row on92e963c0, and failed against each broken copy.92e963c0the reaper's, accounting and pool checks pass:test_image_reaper,test_image_reaper_threads,test_mem_account,test_target_image_pool,test_target_image_reuse,test_switch_table,test_async_safe_frees.Still to test:
mainafc4d1adinterleaved, 2 runs a side on the same route with nothing else running, expecting the arrival's burst to stay on the reaper (nothing destroyed where it was handed over),tex_frame_msno higher thanmain's, and the peak VRAM and private memory of both sides (by hand)vk: VRAM compactedlines and no device loss (by hand)Measurements
Verdict: not settled. Run again alone in the queue, this PR is ahead of
mainon every row, but its frame-time rows are not a gain.mainwas as slow in a third pair run after this PR's (30.5-33.9 fps turning). The gap is in the game's CPU threads (game_ms24-29 against 20), which this PR doesn't touch, and the two runtimes'longshot.exehave identical link maps. Something about the runtimes or the PC, not the change, makes it. The render thread's own time freeing images is less exposed to that, and it agrees with the cap holding:The 1 GB cap was still reached in one of the two runs (29 images destroyed back on the render thread). An interleaved series alone in the queue is next.
Setup:
3f2c9b3c.main's first, then this PR's.afc4d1ad92e963c0tex_frame_msmeantex_frame_msp99tex_frame_msmeantex_frame_msp99tex_frame_msmaxmain, A B B A. Another job on the PC disturbedmain's two (14 and 29 fps turning), so those are left out. This PR's two had:tex_frame_msp99 of 5.0-11.1 ms and frame time p99 78-105 ms.RX_VK_REAPER=0are in Testing; they chose the cap of92e963c0.Risks
What could break:
92e963c0is 4.6 times larger here and is still to be measured.reap_nowunder pressure: frees on the render thread, as main did at every present. It's only asked near the VRAM allowance.Turning it off:
RX_VK_REAPER=0, as for #792, or revert this commit.After updating: nothing changes for a player beyond where the frees happen.
Checklist
Ready for review
main; the title is a Conventional Commit.92e963c0, has not run in the game yet; its first cap's 6 runs are in Testing.mise run testpasses (any failure that also fails onmainis named). Not ticked: only the checks that read this diff ran, not the whole suite.main, or Measurements says why there is nothing to measure. Not ticked:92e963c0againstmainis building.third_party/untouched; nothing GPL-incompatible.When it applies
tests/that reads changed source is updated, not weakened.Assisted-by:/Co-Authored-By:trailers.Ready to merge
main, measured again or a comment says why the numbers still hold.π€ Generated with Claude Code
Tested in game: Windows 11, Ryzen 9 3900X, RTX 3080, NVIDIA 621.14: the Market with the camera turning and the arrival in Hyrule Field, one run a side against RX_VK_REAPER=0 so far. No device loss; the reaper's line logged; the queue reached its cap (255 MB) on arriving in Hyrule Field and 199 images went back to the render thread, so the cap is still being chosen. The rest of the series (2 runs a side, and a 3 GB VRAM run a side) is running.
Recommendation: Request changes. The cap puts a substantial burst of image destruction back on the render thread in the reported arrival run, and the pressure path can count pooled allocations as memory returned to the driver when their blocks remain allocated. The cap and pressure behavior need correction and measurement against main before this can merge.
The change bounds most of the reaper queue, exposes its allocations in the memory budget, and lets the arbiter destroy queued objects under pressure. That addresses queue growth, but the proposed cap can restore the stutter the reaper was introduced to avoid, and allocation accounting does not establish how much VRAM the driver gets back.
Does it help: Unproven. The PR reports one Windows RTX 3080 run per side against RX_VK_REAPER=0, not repeated runs against base
afc4d1ad. At the arrival, 199 images fell back to the render thread and its freeing-time p99 was 13.4 ms versus 10.7 ms with the reaper off; the PR cites 3.4 ms for #792 without the cap. Repeated matched runs against main, including frame times, 1% lows, peak VRAM and RAM, are needed.Other systems: The changed path also runs on Linux, AMD, Intel and Steam Deck, and interacts with VMA compaction and the memory arbiter. The reported run covers Windows and NVIDIA only; Linux builds and runs, Tier 1 coverage, pressure behavior and sustained compaction remain unverified.
Switches: No switches added, changed or removed. Existing RX_VK_REAPER defaults on and provides a fallback, but the cap applied while on is not yet supported by measurements across affected systems.
Reviewed:
afa2a34502againstafc4d1ad08: Read AGENTS.md, CONTRIBUTING.md, docs/dev/standards.md, the PR template, commit and complete diff; read the changed reaper and compaction sections, accounting and arbiter paths, and the new threaded fixture. Skimmed the rest of the large changed source files and memtour; did not read those files in full. docs/review-checklist.md and ../Tooling/AGENTS.md were absent. No earlier reviews or comments were supplied; overlapping PRs beyond the referenced merged #792 could not be checked offline.; nothing run (a code review).Checked and fine:
Blocking (to change before merge)
Testing: No code or tests were run for this review. The author reports three passes of the threaded fixture and 85 passes plus 10 failures also seen on main for changed checks; the full suite, Linux build, repeated measurements and listed hardware runs remain outstanding.
Overlaps and merge order: Only the merged predecessor #792 was supplied; other overlapping PRs could not be checked with the network closed.
Automated code review by BeanBot (gpt-6.1-terra), on the pull request's opening. Nothing was run; a maintainer decides.
@ -380,1 +427,4 @@}// (past the cap: destroyed here, as before, unless a pass is open, when nothing of the pools may be freed: then it waits// for the pass's end, as it waited in texture_frame's dead list)if (!reap_queue.empty() && reap_held() + r.image_bytes + r.buffer_bytes > cap && !compact_in_pass) {The reported arrival already sends 199 frees back to the render thread, with 13.4 ms freeing-time p99 versus the cited 3.4 ms uncapped. Please choose and measure a bound or pressure strategy that retains the burst benefit, including its VRAM and RAM cost against main.
@ -388,0 +457,4 @@std::lock_guard lk(reap_mu);if (compact_in_pass) return 0; // (nothing of the pools freed inside a pass)for (auto it = reap_queue.begin(); it != reap_queue.end() && got < want;) {if ((it->ledger == 0) != vram) {This filters by the image ledger alone. A RAM image with a VRAM buffer, or a buffer-only entry in VRAM, is skipped for a VRAM request. Please select by either allocation's ledger and test mixed-ledger and buffer-only entries.
@ -388,0 +461,4 @@++it;continue;}got += it->image_bytes + (it->buffer_ledger == it->ledger ? it->buffer_bytes : 0);These are allocation sizes, not necessarily bytes returned to the driver: a freed texture suballocation can leave its VMA block allocated. The arbiter may therefore stop asking other pools while driver usage stays high. Please report actual returned memory or distinguish reusable block space so the arbiter can continue.
The review's three points, in
92e963c0:main(afc4d1ad, the reaper with no cap) is building now: 2 runs a side, with peak VRAM and RAM.reap_nowreturns. Right: a share of a block leaves the block with VMA.allocation_dedicated, VMA'svmaGetAllocationInfo2).tests/test_image_reaper_threads.pyhas the cases: dedicated images, shares, mixed ledgers, a buffer alone. A copy that counts allocation sizes, or chooses by the image's ledger alone, fails it.Measured on the old cap, one run a side with a simulated 3 GB card (
RX_VK_TEST_VRAM_MB=3072):RX_VK_REAPER=0tex_frame_ms) p99, camera turning in the Marketπ€ Generated with Claude Code
Measured
92e963c0againstmainafc4d1ad(#792's reaper, no cap) on Windows 11, RTX 3080, NVIDIA 621.14: 4 runs interleaved A B B A, the camera turning in the Market, then the arrival in Hyrule Field. The table is in Measurements.The 1 GB cap is still reached on arriving in Hyrule Field.
main's 2.4-5.4. Its worst frame: 43-54 ms, againstmain's 75-93.So a fixed cap is the wrong bound. Leaving the Market lets go of more than 1 GB at once.
maindid before #792.main's two runs were disturbed. Another job on the PC ran at the series' start: 14 and 29 fps turning, against 39-49 in every earlier run. Their frame times can't be compared. They're being run again with nothing else in the queue.Peak VRAM, the driver's count: 6,854-7,947 MB here against 7,057-7,093 on
main. It comes from the budget line about every 12 s, so a burst's peak can fall between two lines.Still WIP.
π€ Generated with Claude Code
On Linux this does not compile (clang 23.1.5, the AppImage toolchain), so nightly-627 + this PR fails in
vk_vma.cpp:The lambda in
reap_cap()returns256ull << 20(unsigned long long) on one path andstd::clamp<uint64_t>(...)(unsigned long, sinceuint64_tisunsigned longon Linux) on the other; on Windows both areunsigned long long, which is why it builds there.return static_cast<uint64_t>(256ull << 20);on the fallback path makes both returnsuint64_tand the Linux build proceeds.mainafc4d1adrun again, 2 runs, alone in the queue. Measurements now has these as itsmaincolumn.main, run again92e963c0tex_frame_msp99tex_frame_msp99tex_frame_msmaxmain's, as the 25-65 frees its cap sends back to the render thread there would make it.main. It will change to a bound by the free room under the arbiter's line.mainruns came right after this PR's, not interleaved with them.π€ Generated with Claude Code
This PR @
92e963c0run again, 2 runs, alone in the queue, right aftermain's rerun. Measurements now compares the two reruns.main, run againtex_frame_msp99tex_frame_msmaxNot settled. The sides ran one after the other, not interleaved. This PR is also 25 % faster turning in the Market, where the reaper should change the frame little. So the PC's state probably moved between the two pairs, and the frame times can't be credited to the change.
The render thread's own time freeing images is less exposed to that, and it agrees with the cap holding on these arrivals. One of the two runs still reached the 1 GB cap.
Next: an interleaved series, A B B A, with nothing else in the queue.
π€ Generated with Claude Code
mainrun a third pair, alone in the queue, right after this PR's rerun: 30.5-33.9 fps turning in the Market, against this PR's 44.7-45.5.So
mainwas slower both before and after this PR's pair, and the PC's drift over time doesn't explain the gap.The difference is on the game's CPU threads, not the renderer's. Medians a run:
main(4 runs alone)game_ms, turning in the Marketrender_ms, turning in the Marketgpu_wait_ms, turning in the MarketThis PR changes only the renderer (
longshot_vk.dll). The two runtimes'longshot.exehave identical link maps (1,306,036 lines, all but the link timestamp the same), and their logs' setup lines match: switches, driver, cores, policy.So the frame-time gap between the two runtimes is not this change, and the frame-time rows in Measurements are not a gain. The render thread's own
tex_frame_msrows are what this PR moves.A comparison that leaves the runtime out: one runtime, with only
longshot_vk.dllswapped betweenmain's and this PR's before each run, interleaved.π€ Generated with Claude Code
WIP: perf(vk): the image reaper's queue is bounded and gives its memory back near the VRAM allowanceto perf(vk): the image reaper's queue is bounded and gives its memory back near the VRAM allowance