WIP: feat(vk): count what the submit captures' copies are used for (RX_VK_CAPTURE_USE=1), and a RX_VK_CAPTURE_SKIP experiment #2051
No reviewers
Labels
No labels
area/accessibility
area/audio
area/input
area/install
area/menu
area/renderer
area/saves
area/tooling
area/upscaling
beanstalk: approved
beanstalk: failed
beanstalk: needs approval
beanstalk: running
beanstalk: tested
ci: approved
good first issue
gpu/amd
gpu/igpu
gpu/intel
gpu/nvidia
help wanted
kind
bug
kind
chore
kind
docs
kind
feature
kind
performance
kind
question
needs measurements
platform/linux
platform/macos
platform/steam-deck
platform/windows
preview
priority
critical
priority
high
priority
low
regression
risk
high
risk
low
risk
medium
status
blocked
status
confirmed
status
duplicate
status
invalid
status
needs-info
status
wontfix
ci: approved
No milestone
No project
No assignees
3 participants
Notifications
Due date
No due date set.
Dependencies
No dependencies set
Reference
MagicBeanDev/Longshot!2051
Loadingβ¦
Reference in a new issue
No description provided.
Delete branch "siantm/Longshot:perf/vk-capture-use"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Summary
A submit's capture copies the pages written since the last one (
capture_make), for whatever reads them beforethey are captured again: a mirror's sync or a new mirror copying them in, the replay's own reads of guest memory as
submitted, the texture shadow. Nothing said how many of those copies any reader takes, which is what any attempt to copy
less has to know first.
RX_VK_CAPTURE_USE=1(new, a diagnostic, bool, off) counts it: every 600 submits the copied pages by their pool's kind(a mirrored pool, the texture storage, a small pool with no mirror, other) and how many the capture before (or one of
the five before it) copied too; every 600 takes, which reader read each copy first, and the copies replaced or let go
unread (
cap_slot_read,cap_slot_unread,cap_buf_retire). Off, no copy changes and nothing is counted.RX_VK_CAPTURE_SKIP=n(new, an attempt, int, off) is an experiment and is NOT the same GPU-visible memory: a page whoselast n copies nothing read is left out of the next captures (never a late page or the texture storage's, and every
32nd capture of a page copies it again); a reader that then wants it takes guest memory as it is, counted in the log
(
cap_skip_check). It turns the counts on too. Both have their rows ingpu/vk/switches.def; unset, the code is main's.Adapted to main: the replay's reads as submitted are counted in
submitted_read_locked; main'smemory_as_submittedreads the texture shadow, not the captures, so it has no hook.
From a38a894b on
exp/x86-asm-sync.Read first: the block above
cap_buf_retire(cap_slot_read,cap_slot_unread,cap_skip_check,capture_leaves_out),capture_makeandtake_captureingpu/vk/vk_memory.cpp, thentests/test_capture_use.py.Testing
Tested: Windows 11, i7-9700, RTX 4060, the builder's llvm-mingw clang 23; head
3a28d0b5on main75317e1f. Build jobs of this PC's job queue, the checks in a hidden console:tests/run_all.py --strict --exactontests/test_capture_use.pyand every check--changedpicks for the files this changes (batchCapCut3-b2, logt-CapCut3-vk-capture-use.log): 96 checks, 82 passed, 14 failed, 0 skipped; its own check passes, the switch's read included (fresh processes); the failing ones fail on their base too, run from a base worktree (t-CapCut3-base-main.log(batchCapCut3-b1)):tests/test_capture_read_once.py(fails on main @75317e1ftoo, logt-CapCut3-base-main.log),tests/test_crash_dialog.py(fails on main @75317e1ftoo, logt-CapCut3-base-main.log),tests/test_dev_ab.py(fails on main @75317e1ftoo, logt-CapCut3-base-main.log),tests/test_frame_overlap.py(fails on main @75317e1ftoo, logt-CapCut3-base-main.log),tests/test_guest_imported.py(fails on main @75317e1ftoo, logt-CapCut3-base-main.log),tests/test_linux_pressure.py(fails on main @75317e1ftoo, logt-CapCut3-base-main.log),tests/test_pending_guest_import.py(fails on main @75317e1ftoo, logt-CapCut3-base-main.log),tests/test_present_queue_default.py(fails on main @75317e1ftoo, logt-CapCut3-base-main.log),tests/test_replay_write_order.py(fails on main @75317e1ftoo, logt-CapCut3-base-main.log),tests/test_sparse_mirror.py(fails on main @75317e1ftoo, logt-CapCut3-base-main.log),tests/test_target_reuse_memory.py(fails on main @75317e1ftoo, logt-CapCut3-base-main.log),tests/test_texel_buffer_ownership.py(fails on main @75317e1ftoo, logt-CapCut3-base-main.log),tests/test_vk_occlusion_queries.py(fails on main @75317e1ftoo, logt-CapCut3-base-main.log);tests/test_direct_buffer_writes.pytimed out at 60 s (it passes on main) in that run and passed when run again alone: batchCapCut4-b1, logt-CapCut4-vk-capture-use-r3.log.It has NOT run in the game on this PC yet.
Still to test:
RX_VK_CAPTURE_USE=1against unset, and once withRX_VK_CAPTURE_SKIP=4, expecting thecapture copiesandcapture uselines every 600 submits and takes with it on, no FATAL, the same picture and frame times within the runs' spread withRX_VK_CAPTURE_USE=1; withRX_VK_CAPTURE_SKIP=4the pages left out in the log and any difference in the picture noted (by hand)RX_VK_CAPTURE_USE=1against unset, expecting no FATAL, the same picture and the two log lines (@Beanstalk linux+nvidia, linux+amd, windows+amd, windows+intel, steamdeck: Kokiri Forest three minutes, RX_VK_CAPTURE_USE=1 against unset, the log, frame times and a screenshot)python3 -I -B tests/test_capture_use.py, expecting PASS (by hand)Measurements
Nothing to measure in play yet: a diagnostic off by default and an experiment off by default; no micro-benchmark of this port and no run in the game on this PC yet.
Risks
What could break: with
RX_VK_CAPTURE_USE=1, a little CPU time a submit for the counts; nothing else changes. WithRX_VK_CAPTURE_SKIP=n, by design, a reader of a page left out gets guest memory as it is then, not as it was at itssubmit: a flicker, a stale or a wrong value is possible, which is what the experiment is for; it is never on unless
set. The test drives the counts, the tables, the choice of pages and
copy_partthrough 4,000 random captures against amodel. Turn it off: leave both unset. Renderer code: the runtime is rebuilt; nothing changes for players' saves, options
or caches.
Checklist
Ready for review
main; the title is a Conventional Commit.mise run testpasses (any failure that also fails onmainis named).main, or Measurements says why there is nothing to measure.third_party/untouched; nothing GPL-incompatible.When it applies
gpu/vk/switches.def.tests/that reads changed source is updated, not weakened.Assisted-by:/Co-Authored-By:trailers.Ready to merge
main, measured again or a comment says why the numbers still hold.Possibly related, for context:
@Beanstalk This adds RX_VK_CAPTURE_USE, a diagnostic off by default: with RX_VK_CAPTURE_USE=1 the renderer counts how submit captures' copies are used and logs capture-use lines every 600 submits (gpu/vk); nothing else changes. It has not run in the game on these PCs yet. Please play Kokiri Forest for two minutes, walking into Link's house and out, the PR against main on the same PC, on the PR's side with RX_VK_CAPTURE_USE=1 and with it unset, with a picture compared with main's, on Windows with Intel and on Linux with AMD (RADV). The same picture, no crash or FATAL line; set, the capture-use lines every 600 submits; frame times alike either way. Please report those lines.
@siantm only members of
BigBeanscan start the farm. One of them can approve this request by reacting π to it or to this comment.View command line instructions
Checkout
From your project repository, check out a new branch and test the changes.