fix(vk): a submit's capture no longer misses the pages the replay's collection is marking again #1083
No reviewers
Labels
No labels
area/accessibility
area/audio
area/input
area/install
area/menu
area/renderer
area/saves
area/tooling
area/upscaling
beanstalk: approved
beanstalk: failed
beanstalk: needs approval
beanstalk: running
beanstalk: tested
ci: approved
good first issue
gpu/amd
gpu/igpu
gpu/intel
gpu/nvidia
help wanted
kind
bug
kind
chore
kind
docs
kind
feature
kind
performance
kind
question
needs measurements
platform/linux
platform/macos
platform/steam-deck
platform/windows
preview
priority
critical
priority
high
priority
low
regression
risk
high
risk
low
risk
medium
status
blocked
status
confirmed
status
duplicate
status
invalid
status
needs-info
status
wontfix
ci: approved
No milestone
No project
No assignees
3 participants
Notifications
Due date
No due date set.
Dependencies
No dependencies set
Reference
MagicBeanDev/Longshot!1083
Loading…
Reference in a new issue
No description provided.
Delete branch "nyannoying/Longshot:fix/vk-capture-collect-window"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Type: fix · Risk: low · Tested on: Windows 11, Arc B580 · Needs: Linux (any GPU); Windows on AMD or NVIDIA
Why
For one frame at a time, part of the picture is drawn from data the game had already replaced: in Kokiri Forest the
ground mist is gone for a frame. Before each batch of work the renderer copies the memory the game wrote since the last
copy (a submit's capture). The render thread, meanwhile, sorts the game's writes for its own records and puts back the
marks of the pages a capture should take. A capture made on the game's thread at that moment found those pages
unmarked and left them out, so the work read an older copy.
With the defaults this happens rarely, mostly while an area loads (26 and 36 of ~650 captures after a teleport to
Kokiri Forest, 0 or 1 of ~700 standing there). With the render thread replaying jobs as they come
(
RX_VK_FRAME_GATE=0) a third of the captures land in that window, and the mist went missing in 16 to 89 of 3,600frames. The whole-frames gate (#588) was made for a dark frame like this one in the Market and could not name the read
behind it; this may be that read. With the race fixed, turning the gate off is worth about 8% here (Kokiri Forest and the Market), which a separate pull
request will propose with its own measurements.
Whether the one-frame black or stale surfaces reported on other GPUs (#1023, #1045, #740) come from this race was not
checked; none is linked as fixed.
How to reproduce
Seen on: Windows 11 Pro 10.0.29704, Intel Arc B580 12 GB, driver 32.0.101.9154
Settings and switches:
RX_VK_FRAME_GATE=0andRX_VK_PRESENT_AFTER_SIGNAL=1(the race every few frames); withthe defaults it shows mostly while an area loads
Steps: start with
RX_VK_FRAMELOG=<file>, teleport to Kokiri Forest (child) and stand at the arrival spot;create
<file>.go(the log writes while it exists) and remove it a minute later; thenpython -I scripts/framelog.py <file>Expected: no frame where part of the picture is darker than in the frames before and after it
Actual:
framelog.pyfinds 57 events in 3,715 frames (40 s), each 4 or 5 blocks darker than in both neighbouringframes by 24 to 28 levels: the ground mist gone for a frame
What changed
memory_collect_written()(the render thread) andcapture_scan()(a submit's capture, the game's thread) now share amutex,
collect_mu: the collection holds it from taking the write marks to marking the captured pages again, and thecapture's scan takes it, so a capture made in that window waits for the marks instead of missing them.
capture_scan()split out ofcapture_make()ingpu/vk/vk_memory.cpp, unchanged, in a commit of its own, sothe check can run it.
collect_muinmemory_collect_written()andcapture_scan(); the comment above it gives the lock order:collect_mu, thencap_ranges_muand the host's map lock under it;capture_scan()takes it under its capture's order lock (submit_order_mu,sync_order_muwithRX_VK_ASYNC=0, orreplay_mufromresume_queue), none of which is taken undercollect_mu, and takes no lock under it.tests/test_vk_collect_window.py(new) runs the two functions on two threads with stand-ins for the host'smarks and lets the capture in at each end of the collection's window;
tests/test_met_wait_capture.pyfinds the scanin
capture_scan().docs/tested/(2026-11-16-collect-window-nyannoying-run01torun14).capture_scan()is taken under, and the 14 runs say XeSS was on (95c0a7a5).Measurements
Verdict: Fixed: with the render thread replaying jobs as they come, the frames with part of the picture dark went from 17, 16 and 27 of 3,600 to none (
framelog.py: 57 events to 0); with the defaults there were none before or after, and the frame rate is the same (86.5 and 86.5 fps before, 86.4 and 86.6 after).Hardware: Windows 11 Pro 10.0.29704, Intel Arc B580 12 GB (driver 32.0.101.9154), Core i5-13600K (20 threads), 64 GB
Scene: Kokiri Forest (child) at the debug teleport's arrival spot, standing; 1280x720 window, rendered at 960x540 and
upscaled with Intel XeSS 2.0.2 (the Upscaler row's Intel XeSS, Ultra Quality: #459), no frame generation,
HOS_FPS_MAX=240(corrected: this said "no upscaler" before; every run here had XeSS on, both sides alike)Method: each run a new start of the game, the teleport, 40 s to settle, 40 s measured: the frame rate from the log's
5 s
vi:windows (the median of 8); dark frames from a region test in the renderer (a local patch, not in this PR:16x9 regions of every presented frame, a frame counted when 2+ regions are more than 10 levels darker than in both
neighbouring frames while those agree within 4), and in two runs from
scripts/framelog.pyon the frame log. The framelog on in every run, the same on both sides. Ten runs, interleaved: A1 B1 A0 B0 B1 A1 B0 A0 AK BK.
Baseline: main's renderer (
MKWVK_API_VERSION24) does not load in this game folder, whose host is 690d8446's (22).Before is the renderer the folder runs: 690d8446 with the render-thread changes since merged to main (#761, #838,
64ed6a2f), and the region test: 5d3d6ed0, a local commit; After is the same with this PR's code (its first threecommits), the same diff as here.
How to reproduce: the steps above, once with each renderer;
RX_VK_FRAMELOG_DROP=1drops the pictures when onlythe region test's count is wanted (the region test is not in this PR).
fd5238f7on the sameRX_VK_FRAME_GATE=0,RX_VK_PRESENT_AFTER_SIGNAL=1(three runs)scripts/framelog.pyevents, the same (the third run)collect_mu, four area loads a session (runs 11-14: After with a local counter)RX_VK_FRAME_GATE=0: 35,165 and 33,267 waits, 2.65 and 2.51 s, the longest 20.8 and 20.7 ms (at area loads)The check, before the fix (
696531d8) and with it (fd5238f7):No screenshots: what changes is a one-frame dropout, which a still picture before and after doesn't show; the counts
above stand for it.
Flags and settings
None.
Testing
Tested: Windows 11 Pro 10.0.29704, Arc B580 (driver 32.0.101.9154), the renderer built alone with llvm-mingw (clang 23.1.5): 14 runs in
docs/tested/(2026-11-16-collect-window-nyannoying-run01torun14): Kokiri Forest standing before and after, with the defaults and withRX_VK_FRAME_GATE=0, then four area loads a session with a local counter of the waits; no device loss in any.tests/test_vk_collect_window.py(clang 23.1.5): fails at696531d8for the bug's reason, passes atfd5238f7(11 runs of 11), and fails with each of three half fixes: the lock over taking the marks alone, over marking the pages again alone, and the capture's scan without it.python -I -B tests/run_all.py: see the checklist.Not tested: Linux, and any GPU but the B580; g++ (no g++ on this PC, the check compiled with clang 23.1.5 from
llvm-mingw only).
Must be tested:
@Beanstalk linux): build the renderer and runpython -I -B tests/run_all.py test_vk_collect_window test_met_wait_capture, expecting both to pass with g++; then Kokiri Forest and the Market,expecting no new hitch and no frame with part of the picture dark (by hand: the FPS counter's Dev level, a look).
@Beanstalk windows amd/nvidia): Kokiri Forest, then the Market, with thedefaults, expecting the frame rate within a run's noise of main's and no new hitch.
Risks
What could break: a new mutex between the render thread and the game's thread. A submit's capture made while the
collection holds it waits for it (on the B580 with the defaults, about 900 times in a session with four area loads, 0.25 to 0.32 s in all, the longest 1 and 12 ms in two sessions (the 12 ms at an area load); with
RX_VK_FRAME_GATE=0about 34,000 times, 2.5 to 2.7 s, the longest about 21 ms at each load); a collection that comes during a capture's scan waits for the scan. Adeadlock would need a thread holding
cap_ranges_muor the host's map lock to takecollect_mu; the only other taker,capture_scan(), takes it under its capture's order lock (submit_order_mu,sync_order_muorreplay_mu), none of which is taken undercollect_mu, and takes none under it.Turning it off: revert the commit.
Compatibility: Nothing changes.
Issues
None fits for certain; see Why for #588 and the reports that may be related.
Checklist
Everyone
type(scope): description) of the status line's type.main.python -I -B tests/run_all.pypasses (any failure that also fails onmainis named). On Windows with clang 23.1.5 (CXX=clang++): 553 passed, 19 failed, 93 skipped (no Vulkan SDK headers or game folder here). All 19 fail onmain(0691f81c) on this PC too:test_auto_handheld,test_bug_report_defaults(since #773, diagnosed there),test_coverage_run(itstasklistoutput, in this PC's code page, doesn't decode as cp1252),test_crash_dialog,test_crt_options,test_failed_compile_keeps_objects,test_fastfp_fma,test_fpx_sticky,test_guest_importedandtest_pending_guest_import(import_reachundeclared),test_linux_preflight,test_make_game_cache,test_missing_nup,test_pads_after_menu,test_run_all_changed,test_spawn_emit,test_tested_configurations(a CRLF check of the command line's pipe),test_vk_occlusion_queries(its cut ofvk_replay.cppno longer finds the recording's end barrier),test_walk_key.--strict --changed origin/mainat the move and at the fix: the same two import checks, and the four that need the Vulkan headers.third_party/untouched (a new folder with aPROVENANCE.mdinstead).When it applies
Assisted-by:/Co-Authored-By:trailers.docs/tested/for each new measured run (format indocs/tested-configurations.md).mainand passing here.tests/that reads changed source updated in this PR, not weakened.not built here, the Test Farm's Linux run under Must be tested builds it.
🤖 Generated with Claude Code
Tested in game: Windows 11 Pro 10.0.29704, Intel Arc B580 12 GB, driver 32.0.101.9154: Kokiri Forest (child) standing, 10 interleaved runs before and after. With RX_VK_FRAME_GATE=0 the one-frame ground-mist dropouts went from 17, 16 and 27 frames of 3,600 to 0, 0 and 0 (scripts/framelog.py: 57 events to 0); with the defaults 0 before and after, 86.5 and 86.5 fps against 86.4 and 86.6. Then four sessions teleporting Kokiri Forest, the Market, Kokiri Forest, the Market, with a local counter of the lock's waits: no dark frame, no glitch, no device loss.
@Beanstalk two runs for this PR, please:
longshot_vk.so) from this branch and runpython -I -B tests/run_all.py test_vk_collect_window test_met_wait_capturewith g++. Expected: both pass. Then child Link at the debug teleport's arrival in Kokiri Forest, standing, 60 s after 40 s to settle, main against this PR, each withRX_VK_FRAME_GATE=0,RX_VK_PRESENT_AFTER_SIGNAL=1andRX_VK_FRAMELOG=<file>(the log writes while<file>.goexists: create it for the 60 s, remove it after), thenpython -I scripts/framelog.py <file>. Expected: events (one-frame darker patches, the ground mist gone) on main, none with this PR; no device lost. Then Castle Town's market by day with the defaults, 60 s, main against this PR. Expected: the average fps and p99 frame time within a run's noise of main's, no new hitch.The frame log's pictures stay on the tester's PC; only
framelog.py's counts are needed. What it changes: one mutex between the render thread's collection of the game's writes and a submit's capture on the game's thread (gpu/vk/vk_memory.cpp). On an Arc B580 it took the one-frame dropouts withRX_VK_FRAME_GATE=0from 16-27 of 3,600 frames to 0, at no cost in fps with the defaults.@nyannoying only members of
BigBeanscan start the farm.One of them can approve this request by reacting 👍 to it.This request was for7936a6d, and the PR has changed since (95c0a7a), so a 👍 no longer approves it. Comment@Beanstalkagain to ask for the new commit.Correction to the description and the
docs/tested/rows: every run here had the Upscaler row on Intel XeSS (Ultra Quality: the scene rendered at 960x540 and upscaled to the 1280x720 window, #459), on both sides alike, not "no upscaler" as they said. Each run's log hasIntel XeSS: 960x540 -> 1280x720 (Ultra Quality). No number changes. The description's Scene line is fixed now; the 14 rows get the same fix with this branch's next push, not now, so the Test Farm request above stays on this head.Code review only: I read the diff, the host side (
host/rx_backend.ccollect_in,host/nvdrv.c) and every caller, and ran nothing. It's WIP, so this is a comment, not an approval. I found nothing that has to change before merge.What it does for a player: no more one-frame dropouts where part of the picture comes from an older capture (Kokiri Forest's ground mist). These are rare with the defaults, mostly while an area loads, and common with
RX_VK_FRAME_GATE=0.Correctness: the lock covers the window exactly.
collect_muis held fromhost->collect_written(taking every mark) to the lastmark_writtenof a captured page, and the capture's scan takes it.shadow_written_collected,cap_view_bumpandpages_collectedstay outside the lock, which is right. I went through the possible orderings: a capture before the collection, during it (it waits) and after it. Each one ends with page P either in this capture or marked again for the next one.Lock order: I found no cycle. Under
collect_muthe collection takes onlycap_ranges_mu(capture_ranges_merged→refresh_pool_ranges_locked→host->maps_generation/gpu_pools) and nvdrv'slock(hos_gpu_guest_ranges,hos_gpu_ptr). Both are leaves: neither calls back into the renderer. The scan is reached from three places:memory_capture_submiton the game's thread, undersubmit_order_mu(orsync_order_muwithRX_VK_ASYNC=0).memory_capture_nowon the render thread, fromresume_queueunderreplay_mu.capture_makelets go ofcap_ranges_mubeforecapture_scan. Nothing undercollect_mutakessubmit_order_mu,sync_order_muorreplay_mu, so none of these can deadlock. The comment's "capture_scan takes collect_mu holding no other lock" isn't accurate, though (inline). The deadlock argument in Risks has the same wording.Cost (a suggestion, your call). The game's thread now waits for the whole
host->collect_writtenscan, over every GPU range.capture_make's own comment puts that at about 20 ms, and your longest waits (12 ms with the defaults at a load, about 21 ms with the gate off) match it. The wait also happens undersubmit_order_mu, so any other thread that submits waits behind it. design.md §4 rule 7 (the CPU saturates first) makes this worth watching on a slower CPU than the 13600K, so please make sure the Test Farm's AMD and NVIDIA runs include an area load. A follow-up could remove the window instead of serialising it: the collection would not take the marks inside the captured ranges at all. It would collect over the GPU ranges minuscapture_ranges_merged(), which needs no re-mark and no lock. That takes a host entry (and anMKWVK_API_VERSIONbump), and those pages would then reachshadow_written_collectedonly through the capture's own report. It's a design change, not something for this PR. #936 (fastercollect_in) also shortens the window and so the waits.The check.
tests/test_vk_collect_window.pyis good. It cuts withsrccut.block, takes--cxxthe way the other 147 checks do, and lets the capture in at each end of the window with a handshake instead of a timer, so it doesn't depend on timing. It asserts the right things: page 9 is in the capture with its mark cleared, and page 20 is stamped and not captured.test_met_wait_capture.pyis updated, not weakened. I did not run either one.Testing. The in-game run on the Arc B580 is posted (10 interleaved runs plus 4 sessions with the counter). Still missing: Linux (g++ and the game) and Windows on AMD or NVIDIA. Your Test Farm request is waiting for a BigBean's 👍. Noted that the 14
docs/tested/rows still say "no upscaler" and that you'll fix them on your next push. I didn't touch them.Mergeable. The forge says it conflicts, but
main(a5b112b6) hasn't touched any of the PR's files since its base (0691f81c). A trial merge withmainis textually clean, so the flag looks stale and your next push should recompute it. Nothing needed resolving, and I pushed nothing, which also keeps the head your Test Farm request is on.Siblings and merge order:
mark_audit_*calls on the lines around the scan call that this PR replaces. A trial merge with this PR is clean and keeps both: its audit calls end up aroundcapture_scan(mirrored, &pages). Either order works. Landing this first is the smaller step, and its race fix gives 1034's oracle a cleaner baseline.host/rx_backend.c. It merges cleanly with this PR and the two are independent, in either order.mainitself, in the late-fix code.Suggested order: this PR, then 1034. 936 and 759 are independent of it.
@ -710,0 +717,4 @@// waits for the collection. On an Arc B580, in sessions with four area loads (~110,000 collections each): with whole// frames the captures waited ~900 times, 0.25 to 0.32 s in all, the longest 1 and 12 ms (at a load); with// RX_VK_FRAME_GATE=0 ~34,000 times, ~2.6 s, the longest ~21 ms at each load. Lock order: collect_mu, then cap_ranges_mu// (capture_ranges_merged) and the host's map lock (collect_written's ranges, gpu_ptr); capture_scan takes collect_muNit: "capture_scan takes collect_mu holding no other lock" isn't quite right. It's taken under
submit_order_mu(game thread,memory_capture_submit),sync_order_mu(RX_VK_ASYNC=0) andreplay_mu(resume_queue→memory_capture_now, render thread). It's still safe, because nothing undercollect_mutakes any of them. Maybe: "capture_scan takes it holding neither cap_ranges_mu nor the map lock (its callers' submit_order_mu, sync_order_mu or replay_mu, none of which is taken under collect_mu)". The deadlock line under Risks has the same wording.@ -760,1 +762,3 @@continue;int64_t n = 0;{ // (from taking the marks to marking the captured pages again: no capture scans in between, collect_mu)std::lock_guard lk(collect_mu);This holds
collect_muover the wholehost->collect_writtenscan of every GPU range (~20 ms per capture_make's own comment), and a capture that arrives then waits for the rest of it on the game's thread, undersubmit_order_mu. That matches your 12 ms and 21 ms longest waits. It's acceptable for the fix, but please make sure an area load is part of the Test Farm runs. A follow-up that collects only outsidecapture_ranges_merged()would have no window to lock (see the summary).WIP: fix(vk): a submit's capture no longer misses the pages the replay's collection is marking againto fix(vk): a submit's capture no longer misses the pages the replay's collection is marking againApproved at
95c0a7a5. The code review (above) found nothing blocking. On top of the author's commits I pushed one fix,95c0a7a5:capture_scanis taken under (submit_order_mu,sync_order_muorreplay_mu);docs/tested/rows now say XeSS was on, as nyannoying's correction asked;Merged at a maintainer's request, without the Linux build or the AMD/NVIDIA and Linux game runs that Must be tested names. Those runs are still wanted against main.