WIP: perf(vk): the present lets the replay's order lock go once the frame's copy is on the queue (RX_VK_PRESENT_UNLOCKED=0 to disable) #2053
No reviewers
Labels
No labels
area/accessibility
area/audio
area/input
area/install
area/menu
area/renderer
area/saves
area/tooling
area/upscaling
beanstalk: approved
beanstalk: failed
beanstalk: needs approval
beanstalk: running
beanstalk: tested
ci: approved
good first issue
gpu/amd
gpu/igpu
gpu/intel
gpu/nvidia
help wanted
kind
bug
kind
chore
kind
docs
kind
feature
kind
performance
kind
question
needs measurements
platform/linux
platform/macos
platform/steam-deck
platform/windows
preview
priority
critical
priority
high
priority
low
regression
risk
high
risk
low
risk
medium
status
blocked
status
confirmed
status
duplicate
status
invalid
status
needs-info
status
wontfix
ci: approved
No milestone
No project
No assignees
4 participants
Notifications
Due date
No due date set.
Dependencies
No dependencies set
Reference
MagicBeanDev/Longshot!2053
Loading…
Reference in a new issue
No description provided.
Delete branch "siantm/Longshot:perf/vk-present-unlocked"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Summary
With the replay on the submitting thread (Threaded rendering off),
present()held the replay's order lock(
sync_order_mu) and the replay's core (ReplayPin) until it returned: over the hand-over to the presenter thread(which waits there for the frame before's present) and over the frame's pacing wait after it (NVIDIA Reflex's sleep,
AMD Anti-Lag's input wait). Meanwhile no other game thread could submit, take or wait for a queue's fence, or let
stopped work go on.
What the lock protects is fixed once the frame's copy into the window is submitted: the copy is on the queue behind the
frame's work and ahead of anything replayed later.
present_imagenow lets the lock and the pin go right after thatsubmit (
present_order_release), before the hand-over; a present that ends earlier (no window, a deferred copy, framegeneration) keeps both until
present()returns, as before. Reflex's sleep leaves the frame unmarked until it wakes, sothe other threads' submits during it go untagged instead of as the frame just shown (
gpu/vk/vk_reflex.cpp). The renderthread's path holds no order lock at its present and is unchanged. The result expected: the game's other threads no
longer wait for the present's hand-over and pacing.
The switch:
RX_VK_PRESENT_UNLOCKED(new, a kill switch, bool, its row ingpu/vk/switches.def), on by default as thebranch shipped it, read once;
=0holds both until the present returns, as main does.From bd3ea307 on
exp/x86-asm-sync(its replay-parts work, with 4eed88f7's Reflex line).Read first:
present_order_release,present_imageandpresentingpu/vk/vk_replay.cpp,reflex_sleepingpu/vk/vk_reflex.cpp, thentests/test_present_unlocked.py.Testing
Tested: Windows 11, i7-9700, RTX 4060, the builder's llvm-mingw clang 23; head
3edef0f6on main8aa74983(rebased from4c0c0b7aon75317e1f). Build jobrb-2053-2054of this PC's job queue, the checks in a hidden console:tests/run_all.py --strict --exactontests/test_present_unlocked.py,tests/test_present_acquire_ahead.pyandtests/test_switch_table.py(logrb2053-own-head.log): 3 checks, 3 passed, 0 failed, 0 skipped; its own check passes, the switch's read included (fresh processes). The syntax compile ofgpu/vk/vk_replay.cppandgpu/vk/vk_reflex.cppwith the game build's own commands passes (logrb2053-syntax-head.log). The checks--changedpicks were not run again at this head.It has NOT run in the game on this PC yet.
Still to test:
8aa74983, this branch withRX_VK_PRESENT_UNLOCKED=0and this branch with it unset (on), each warmed up a minute in the scene first, interleaved (ABBA), at least two runs each, expecting the same picture, no FATAL, no stutter or hang at the present, and a median frame time unset lower than withRX_VK_PRESENT_UNLOCKED=0and than main, andRX_VK_PRESENT_UNLOCKED=0the same as main (by hand)8aa74983, this branch withRX_VK_PRESENT_UNLOCKED=0and this branch with it unset (on), warmed up, ABBA, expecting the same picture, no FATAL and no hang, and a median frame time unset lower than withRX_VK_PRESENT_UNLOCKED=0and than main, andRX_VK_PRESENT_UNLOCKED=0the same as main (@Beanstalk linux+nvidia, linux+amd, windows+amd, windows+intel, steamdeck: Kokiri Forest three minutes, Threaded rendering off: main @8aa74983, this branch with RX_VK_PRESENT_UNLOCKED=0 and this branch with it unset (on), warmed up, ABBA, the log, frame times and a screenshot)python3 -I -B tests/test_present_unlocked.py, expecting PASS (by hand)Measurements
Verdict: not measured: no micro-benchmark of this port and no run in the game on this PC yet; the in-game A/B is
in Still to test.
RX_VK_PRESENT_UNLOCKEDships on, as the branch shipped it; no measurement chose that default yet, the in-game A/B in Still to test does.zeldaootr's run on a Steam Deck OLED (#issuecomment-71560), a third party's run, not ours, as posted. It ran the earlier head
4c0c0b7a, merged into main9dd41326. Besides moving onto main8aa74983, the rebase to3edef0f6changed tests only: outsidetests/its added and removed lines are 4c0c0b7a's, with only their context moved by main (present_image_with_trial,present_tagged,present_out's new arguments), so the run stands for this head's renderer change, measured over main9dd41326rather than8aa74983, which this head sits on.9dd413264c0c0b7a(merged into main9dd41326)RX_VK_PRESENT_UNLOCKED=0against unset on the PR's build, three pairs: 44.1 fps against 44.0 on the median (+0.3%), median frame time 22.57 against 22.53 ms.dev.py ab's verdict on both: no difference shown, within the runs' spread. The unset runs logged the switch's line endingon, the=0runs the same line endingoff; noFATAL, device-lost,VK_ERROR, validation or hang line in his 12 logs.Hardware and settings: Steam Deck OLED, SteamOS 3.9.5, RADV VANGOGH, Mesa 26.2.0-devel, GPU clock set to 1600 MHz, headless; Minimum, render 50 % (640x400), output 1280x800, uncapped, V-Sync and frame generation off; Threaded rendering at Auto, off on the Deck. Method:
dev.py ab(with a local change to compare two prebuilt runtimes), Kokiri Forest, three interleaved pairs of 180 s after the warm-up, the median. Its own stated limits: main9dd41326rather than75317e1fas the base; Link standing where the scene puts him, not three minutes of play, in ABAB order rather than ABBA; no picture judged and no screenshot (headless); the Deck has no Reflex; in this scene the Deck's runs agree within about 1 %, so a change of 1-2 % would show only as a hint; the switch is not read throughHOS_AB, so no in-run comparison.Risks
What could break: another thread's submit replayed between the frame's work and its copy into the window (a frame
shown with the next one's work in it), or the lock released twice (an abort) or never (a hang). The lock is let go only
after the copy's submit, and the test runs
present()'s hold through the realReplayPinandpresent_order_release(let go at the copy, held without one, a nested pin,=0). AMD Anti-Lag's wait is covered byreading the order only. Turn it off with
RX_VK_PRESENT_UNLOCKED=0. Renderer code: the runtime is rebuilt; nothingchanges for players' saves, options or caches.
Checklist
Ready for review
main; the title is a Conventional Commit.mise run testpasses (any failure that also fails onmainis named).main, or Measurements says why there is nothing to measure.third_party/untouched; nothing GPL-incompatible.When it applies
gpu/vk/switches.def.tests/that reads changed source is updated, not weakened.Assisted-by:/Co-Authored-By:trailers.Ready to merge
main, measured again or a comment says why the numbers still hold.Possibly related, for context:
@Beanstalk This adds RX_VK_PRESENT_UNLOCKED, a kill switch on by default: with Threaded rendering off, the present no longer holds the replay's order lock (gpu/vk); RX_VK_PRESENT_UNLOCKED=0 is main's way. It has not run in the game on these PCs yet. Please play Hyrule Field for three minutes with Threaded rendering off in the Insert menu, the PR against main on the same PC, on the PR's side with the switch unset and with RX_VK_PRESENT_UNLOCKED=0, with a picture compared with main's, on Windows with NVIDIA with Reflex on and with it off, on Linux with AMD (RADV), and on the Steam Deck. The same picture, no stutter, hang, crash or FATAL line at the present; unset, frame time p99 and hitches equal to main's or better and to the switch off.
@siantm only members of
BigBeanscan start the farm.One of them can approve this request by reacting 👍 to it or to this comment.This request was for4c0c0b7, and the PR has changed since (3edef0f), so a 👍 no longer approves it. Comment@Beanstalkagain to ask for the new commit.4c0c0b7a653edef0f621Rebased onto main @
8aa74983(unreviewed WIP): it conflicted with main's7c23d2be(#1859, Output resolution preview) intests/test_present_acquire_ahead.py, resolved by keeping both changes, andtests/test_present_unlocked.pynow reads main's renamedpresent_tagged,present_image_with_trial,submit_or_retryandpresent_outsignature; the range-diff is otherwise=. Jobrb-2053-2054:tests/test_present_unlocked.py,tests/test_present_acquire_ahead.py,tests/test_switch_table.pyand the syntax compile ofgpu/vk/vk_replay.cppandgpu/vk/vk_reflex.cpppass, with no behaviour change; the earlier head4c0c0b7ais gone.@Beanstalk This adds RX_VK_PRESENT_UNLOCKED, a kill switch on by default: with Threaded rendering off, the present lets the replay's order lock go once the frame's copy is on the queue (gpu/vk). This head is a rebase onto main
8aa74983with no change in behaviour; it replaces my request 65623, made for the earlier head. Please start from the kokiri-start save and take the picture right after it loads and settles, before any input, with a main-against-itself picture on the same PC at that point; then, for frame times only, play Kokiri Forest for three minutes; at least two starts a side, interleaved, the PR against main on the same PC, on the PR's side unset (the default, on) and with RX_VK_PRESENT_UNLOCKED=0 (off), on Windows with an RTX 4070, Windows with an Arc B580, Linux with an RX 6600 (RADV) and the Steam Deck. The farm cannot set Threaded rendering, so with the default settings this run checks that nothing changes; the Threaded-rendering-off path (with Reflex on and off on NVIDIA) is checked by hand, unless a tester has a CPU with fewer than 4 cores, where the game runs with Threaded rendering off; there the median frame time unset should be lower than with RX_VK_PRESENT_UNLOCKED=0 and than main's. The same picture as main within main's own difference from itself; with it unset, frame time p99 equal to main's or better, within the run-to-run spread, and equal to RX_VK_PRESENT_UNLOCKED=0 or better. The log must have no FATAL, device lost or validation error line. No stutter or hang at the present.@siantm only members of
BigBeanscan start the farm. One of them can approve this request by reacting 👍 to it or to this comment.@Beanstalk One correction to my request 71008 above, for its picture point: please take each picture 5 s (300 frames at 60 fps) after the first frame whose picture is not a single uniform colour, with no input in between, and take the main-against-itself picture from the same point. A picture taken during the load's fade-in from black is not comparable: run 70269 on #2156 caught the fade in 3 of 4 starts.
@siantm only members of
BigBeanscan start the farm. One of them can approve this request by reacting 👍 to it or to this comment.Steam Deck result: I ran this PR against main, and its switch
RX_VK_PRESENT_UNLOCKED=0against unset, on a Steam Deck OLED standing in Kokiri Forest with Threaded rendering off, three pairs of three minutes each way. Neither showed a difference the Deck can resolve (-0.3 % and +0.3 % on the median, the runs overlapping), and there were no errors or hangs in any log.Main against this PR:
9dd413264c0c0b7adev.py ab's verdict: no difference shown: -0.3% fps (median 43.8 against 43.9), within the runs' spread (A 43.6 to 44.1, B 43.7 to 44.0).RX_VK_PRESENT_UNLOCKED=0against unset, on this PR's build:dev.py ab's verdict: no difference shown: +0.3% fps (median 44.1 against 44.0), within the runs' spread (A 43.6 to 44.3, B 43.8 to 44.2). (The tool's column header names main's commit because it takes the label from the scripts' folder; both sides ran this PR's build.) So the median frame time with the switch unset was not lower than with=0or than main's here, and=0was the same as main, within about 1 %.From the logs: every one of the 12 reads
vk: Threaded rendering: Auto: off (3 cores of 6 logical processors; the render thread from 4 cores). The PR's runs with the switch unset logvk: the present made with the replay's order lock let go, once the frame's copy is on the queue: on (RX_VK_PRESENT_UNLOCKED=0: held until the present returns), and with=0the same line ends: off. NoFATAL, device-lost,VK_ERROR, validation or hang line in any of the 12 logs; every run exited normally.Note: this is the head I built,
4c0c0b7a, merged onto main9dd41326. The PR has since been rebased (3edef0f6); that head was not run.Hardware: Steam Deck OLED, SteamOS 3.9.5, RADV VANGOGH, Mesa 26.2.0-devel, GPU clock manual 1600 MHz, headless.
Settings: Minimum, Render 50 % (640x400), output 1280x800, uncapped, V-Sync and frame generation off; Threaded rendering at Auto (off on the Deck).
Game data: complete (the dump, with the three damaged textures replaced).
Built: main @
9dd41326and main9dd41326with this PR's head4c0c0b7amerged in (its 1 commit on75317e1f;tests/test_present_acquire_ahead.pyconflicted and was taken from the PR, the code merged cleanly), x86-64-v3, shaders prepared on the Deck; only the renderer differs.Measured:
dev.py ab, Kokiri Forest, 3 interleaved pairs of 180 s after the warm-up (until no frame waits for a pipeline, at least 10 s), the median (the project's tool, with a local change to compare two prebuilt runtimes, since the Deck does not build).Limits: main @
9dd41326rather than75317e1fas the base;dev.py abstands Link still where the scene puts him, so this is not three minutes of play, and the order is ABAB rather than ABBA; the picture can't be judged headless, and there's no screenshot. The Deck has no Reflex. In this standing scene the Deck's runs agree within about 1 %, so a change of 1-2 % would show only as a hint. The switch isn't read throughHOS_AB, so an in-run comparison wasn't possible.@zeldaootr thank you for the Deck runs, and for saying which head you built. Besides moving onto main
8aa74983, the rebase to3edef0f6changed tests only (tests/test_present_unlocked.pyreads main's renamedpresent_tagged,present_image_with_trial,submit_or_retryandpresent_out's new arguments;tests/test_present_acquire_ahead.pykeeps both sides): outsidetests/the added and removed lines are the same as 4c0c0b7a's, only their context moved with main. So your run stands for this head's renderer change, measured over main9dd41326rather than8aa74983. It is now in Measurements as your run, with its limits (Link standing, no Reflex, runs agreeing within about 1 %): no difference and no error or hang on the Deck.@Beanstalk One more correction to my request 71008 above: the farm can turn Threaded rendering off after all, through its switch RX_VK_ASYNC=0 (the setting the Threaded rendering row stands for, on main too). This change acts only with Threaded rendering off, so please set RX_VK_ASYNC=0 on both sides (main and the PR, the PR's side unset and with RX_VK_PRESENT_UNLOCKED=0); the earlier sentence saying the path is checked by hand no longer holds. There the median frame time unset should be lower than with RX_VK_PRESENT_UNLOCKED=0 and than main's.
@siantm only members of
BigBeanscan start the farm. One of them can approve this request by reacting 👍 to it or to this comment.View command line instructions
Checkout
From your project repository, check out a new branch and test the changes.