WIP: perf(vk): a capture's late pages wait for the part that writes them back (RX_VK_CAPTURE_WAIT_PART=0 to disable) #2052
No reviewers
Labels
No labels
area/accessibility
area/audio
area/input
area/install
area/menu
area/renderer
area/saves
area/tooling
area/upscaling
beanstalk: approved
beanstalk: failed
beanstalk: needs approval
beanstalk: running
beanstalk: tested
ci: approved
good first issue
gpu/amd
gpu/igpu
gpu/intel
gpu/nvidia
help wanted
kind
bug
kind
chore
kind
docs
kind
feature
kind
performance
kind
question
needs measurements
platform/linux
platform/macos
platform/steam-deck
platform/windows
preview
priority
critical
priority
high
priority
low
regression
risk
high
risk
low
risk
medium
status
blocked
status
confirmed
status
duplicate
status
invalid
status
needs-info
status
wontfix
ci: approved
No milestone
No project
No assignees
3 participants
Notifications
Due date
No due date set.
Dependencies
No dependencies set
Reference
MagicBeanDev/Longshot!2052
Loading…
Reference in a new issue
No description provided.
Delete branch "siantm/Longshot:perf/vk-capture-wait-part"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Summary
When the GPU cannot fix a capture's late pages (pages a submission still on the GPU writes back),
take_capturewaited for that whole submission, its last part.
With this,
capture_makekeeps, beside the late value, the newest part that wrote a late page back from a mirror andthe newest value written in place, and the take waits for that part (
vk_timeline_wait_partingpu/vk/vk_replay.cpp, throughwait_recorded_partas a CPU read of GPU results does) under the readbacks' guards:the whole-value wait stays when a page was written in place by a submission not done, when a later part reads or
writes guest memory in place, when the part is not known, without a timeline or per-queue fences, with unordered
replay and with the render thread. A part wait that times out says the part's recording and the recordings' timeline;
a lost device is not said as a timeout. The 600-take line says how many late-page waits were for their part. The
result expected: shorter waits at the take when the late pages' part ends before the submission does.
The switch:
RX_VK_CAPTURE_WAIT_PART(new, a kill switch, bool, its row ingpu/vk/switches.def), on by default as thebranch shipped it, read once;
=0waits for the submission's last part, as main does.From d1e33769 on
exp/x86-asm-sync, with db8f6a69's fix of the timeout line folded in.Read first:
capture_make's late pages andtake_captureingpu/vk/vk_memory.cpp,vk_timeline_wait_partingpu/vk/vk_replay.cpp, thentests/test_readback_part.py.Testing
Tested: Windows 11, i7-9700, RTX 4060, the builder's llvm-mingw clang 23; head
db1d385con main75317e1f. Build jobs of this PC's job queue, the checks in a hidden console:tests/run_all.py --strict --exactontests/test_readback_part.py,tests/test_capture_wait_log.pyand every check--changedpicks for the files this changes (batchCapCut3-b3, logt-CapCut3-vk-capture-wait-part-r2.log): 197 checks, 165 passed, 32 failed, 0 skipped; its own checks pass, the switch's read included (fresh processes); the failing ones fail on their base too, run from a base worktree (t-CapCut3-base-main.log(batchCapCut3-b1)):tests/test_capture_read_once.py(fails on main @75317e1ftoo, logt-CapCut3-base-main.log),tests/test_capture_readback_visibility.py(fails on main @75317e1ftoo, logt-CapCut3-base-main.log),tests/test_crash_dialog.py(fails on main @75317e1ftoo, logt-CapCut3-base-main.log),tests/test_crash_record.py(fails on main @75317e1ftoo, logt-CapCut3-base-main.log),tests/test_dev_ab.py(fails on main @75317e1ftoo, logt-CapCut3-base-main.log),tests/test_dev_packs.py(fails on main @75317e1ftoo, logt-CapCut3-base-main.log),tests/test_device_lost_exit.py(fails on main @75317e1ftoo, logt-CapCut3-base-main.log),tests/test_draw_stream_visibility.py(fails on main @75317e1ftoo, logt-CapCut3-base-main.log),tests/test_flush_nothing_held.py(fails on main @75317e1ftoo, logt-CapCut3-base-main.log),tests/test_frame_overlap.py(fails on main @75317e1ftoo, logt-CapCut3-base-main.log),tests/test_fsr_image_retirement.py(fails on main @75317e1ftoo, logt-CapCut3-base-main.log),tests/test_fsr_uniform_ownership.py(fails on main @75317e1ftoo, logt-CapCut3-base-main.log),tests/test_gpu_wait_stats.py(fails on main @75317e1ftoo, logt-CapCut3-base-main.log),tests/test_guest_imported.py(fails on main @75317e1ftoo, logt-CapCut3-base-main.log),tests/test_linux_pressure.py(fails on main @75317e1ftoo, logt-CapCut3-base-main.log),tests/test_mesh_conv.py(fails on main @75317e1ftoo, logt-CapCut3-base-main.log),tests/test_pending_guest_import.py(fails on main @75317e1ftoo, logt-CapCut3-base-main.log),tests/test_perf_report.py(fails on main @75317e1ftoo, logt-CapCut3-base-main.log),tests/test_present_queue_default.py(fails on main @75317e1ftoo, logt-CapCut3-base-main.log),tests/test_present_source_crop.py(fails on main @75317e1ftoo, logt-CapCut3-base-main.log),tests/test_profiling_mode.py(fails on main @75317e1ftoo, logt-CapCut3-base-main.log),tests/test_recorder_own_idle.py(fails on main @75317e1ftoo, logt-CapCut3-base-main.log),tests/test_renderer_new_handler.py(fails on main @75317e1ftoo, logt-CapCut3-base-main.log),tests/test_replay_write_order.py(fails on main @75317e1ftoo, logt-CapCut3-base-main.log),tests/test_slow_frames.py(fails on main @75317e1ftoo, logt-CapCut3-base-main.log),tests/test_sparse_mirror.py(fails on main @75317e1ftoo, logt-CapCut3-base-main.log),tests/test_target_reuse_memory.py(fails on main @75317e1ftoo, logt-CapCut3-base-main.log),tests/test_texel_buffer_ownership.py(fails on main @75317e1ftoo, logt-CapCut3-base-main.log),tests/test_ub_update_batch.py(fails on main @75317e1ftoo, logt-CapCut3-base-main.log),tests/test_upscaler_rows.py(fails on main @75317e1ftoo, logt-CapCut3-base-main.log),tests/test_vk_occlusion_queries.py(fails on main @75317e1ftoo, logt-CapCut3-base-main.log);tests/test_present_semaphores.pyfailed with no output at all (it passes on main; it failed the same way, with no output, in runs of other branches that do not change what it reads, e.g.t-AsmCut3-asm-crc32.log) in that run and passed when run again alone: batchCapCut4-b1, logt-CapCut4-vk-capture-wait-part-r3.log, batchCapCut4-b1, logt-CapCut4-vk-capture-wait-part-r3b.log.It has NOT run in the game on this PC yet.
Still to test:
75317e1f, this branch withRX_VK_CAPTURE_WAIT_PART=0and this branch with it unset (on), each warmed up a minute in the scene first, interleaved (ABBA), at least two runs each, expecting the same picture, no FATAL and no part-wait timeout line, the 600-take line counting waits for their part, and a median frame time unset lower than withRX_VK_CAPTURE_WAIT_PART=0and than main, andRX_VK_CAPTURE_WAIT_PART=0the same as main (by hand)75317e1f, this branch withRX_VK_CAPTURE_WAIT_PART=0and this branch with it unset (on), warmed up, ABBA, expecting the same picture, no FATAL and no part-wait timeout line, and a median frame time unset lower than withRX_VK_CAPTURE_WAIT_PART=0and than main, andRX_VK_CAPTURE_WAIT_PART=0the same as main (@Beanstalk linux+nvidia, linux+amd, windows+amd, windows+intel, steamdeck: Kokiri Forest three minutes, Threaded rendering off: main @75317e1f, this branch with RX_VK_CAPTURE_WAIT_PART=0 and this branch with it unset (on), warmed up, ABBA, the log, frame times and a screenshot)python3 -I -B tests/test_readback_part.py, expecting PASS (by hand)Measurements
Verdict: not measured: no micro-benchmark of this port and no run in the game on this PC yet; the in-game A/B is
in Still to test.
RX_VK_CAPTURE_WAIT_PARTships on, as the branch shipped it; no measurement chose that default yet, the in-game A/B in Still to test does.Risks
What could break: a take that waited for a part before the one that writes its late pages back would copy those
pages before the GPU's values reached them: a stale texture or a wrong value read back for a frame. Every guard falls
back to the whole submission's wait, and the test checks each guard, the part and in-place value kept, the switch off,
the timeout line and the lost-device case. Turn it off with
RX_VK_CAPTURE_WAIT_PART=0. Renderer code: the runtime isrebuilt; nothing changes for players' saves, options or caches.
Checklist
Ready for review
main; the title is a Conventional Commit.mise run testpasses (any failure that also fails onmainis named).main, or Measurements says why there is nothing to measure.third_party/untouched; nothing GPL-incompatible.When it applies
gpu/vk/switches.def.tests/that reads changed source is updated, not weakened.Assisted-by:/Co-Authored-By:trailers.Ready to merge
main, measured again or a comment says why the numbers still hold.- When the late pages' GPU fix cannot apply, take_capture waited for the whole submission that writes a late page back (its last part). Now capture_make keeps, beside the late value, the newest part that wrote a late page back from a mirror and the newest value written in place (wb_parts, wb_direct; capture_late_at_replay adds the pages it finds), and the take waits for that part (vk_replay.cpp vk_timeline_wait_part, through wait_recorded_part as a readback does) under the readbacks' guards: the whole-value wait stays when a page was written in place by a submission not done, when a later part reads or writes guest memory in place (memory_in_place_part), when the part is not known, without a timeline or per-queue fences, with unordered replay and, as on the branch, with the render thread (RX_VK_ASYNC). The 600-take line says how many late-page waits were for their part. - A part wait that times out (250 ms) says the part's recording, the recordings' timeline and the newest recording submitted, apart from the submissions' timeline; a lost device is not said as a timeout (the branch's fix db8f6a69, folded in: main has no part wait to fix). - Switch RX_VK_CAPTURE_WAIT_PART (kill, on); =0 waits for the submission's last part as before. - Branch (d1e33769, collect-once integration): not measured there ("the saving is not measured yet"); not measured in play on main. - Test: tests/test_readback_part.py cuts capture_make's late-page bookkeeping, take_capture's choice and vk_timeline_wait_part, and checks the part and in-place value kept, every guard falling back to the value, the switch off, the timed-out line's numbers and the lost-device case. Switch reads: the test also runs RX_VK_CAPTURE_WAIT_PART as the source reads it, in fresh processes (unset, 0, 1, 2, off, each twice, the variable flipped after the first read: read once). The timeout line says 0 for the recordings' timeline when the driver cannot read it. Assisted-by: Claude CodePossibly related, for context:
@Beanstalk This adds RX_VK_CAPTURE_WAIT_PART, a kill switch on by default: with Threaded rendering off, a capture's late pages wait for their own write-back's recording part (gpu/vk); RX_VK_CAPTURE_WAIT_PART=0 is main's way. It has not run in the game on these PCs yet. Please play Kokiri Forest for three minutes with Threaded rendering off in the Insert menu, the PR against main on the same PC, on the PR's side with the switch unset and with RX_VK_CAPTURE_WAIT_PART=0, with a picture compared with main's, on Linux with NVIDIA, Linux with AMD (RADV), Windows with Intel and the Steam Deck. The same picture, no crash, FATAL line or part-wait timeout line; unset, frame time p99 equal to main's or better and to the switch off.
@siantm only members of
BigBeanscan start the farm. One of them can approve this request by reacting 👍 to it or to this comment.View command line instructions
Checkout
From your project repository, check out a new branch and test the changes.