perf(vk): RX_VK_ACQUIRE_AHEAD=1 asks for the window's next image right after a present, so the render thread no longer waits for it (off by default) #1574
No reviewers
Labels
No labels
area/accessibility
area/audio
area/input
area/install
area/menu
area/renderer
area/saves
area/tooling
area/upscaling
beanstalk: approved
beanstalk: failed
beanstalk: needs approval
beanstalk: running
beanstalk: tested
ci: approved
good first issue
gpu/amd
gpu/igpu
gpu/intel
gpu/nvidia
help wanted
kind
bug
kind
chore
kind
docs
kind
feature
kind
performance
kind
question
needs measurements
platform/linux
platform/macos
platform/steam-deck
platform/windows
preview
priority
critical
priority
high
priority
low
regression
risk
high
risk
low
risk
medium
status
blocked
status
confirmed
status
duplicate
status
invalid
status
needs-info
status
wontfix
ci: approved
No milestone
No project
No assignees
3 participants
Notifications
Due date
No due date set.
Dependencies
No dependencies set
Reference
MagicBeanDev/Longshot!1574
Loading…
Reference in a new issue
No description provided.
Delete branch "klimovich008/Longshot:perf/acquire-ahead"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Type: perf · Risk: low · Tested on: Windows 11, RX 9070 XT (AMD 26.11.1); Test Farm: Windows RTX 3090, Linux RTX 4070; Steam Deck OLED (dukenheimer) · Needs: Intel on Windows; Linux RADV (X11 and Wayland); NVIDIA with the switch's log lines
TL;DR
Since #1458 the renderer waits on the CPU for the window's next image right before it draws into it. On an RX 9070 XT in a window that wait is about 3 ms of every frame on the thread the frame waits for. This PR adds an opt-in switch,
RX_VK_ACQUIRE_AHEAD=1(off by default), that asks for the next image right after each present, so it comes back while the next frame is made. In the standard test in a window it went from 64.6 to 70.0 fps (+8.4 %, 2 pairs), with the game clock at 1.000 and no new stutter. The code is the same on every GPU; the Test Farm is asked for the others.Breakdown
0keeps #1458's order); #1458's fix for the window behind another one (no GPU-side wait anywhere); frame generation, the presenter thread and V-Sync On (FIFO) keep #1458's order even with the switch on; no shader cache or settings change.Why
#1458 (HumanBean) moved the wait for the window's image from the GPU's queue to the CPU, so that a window behind another one no longer stops the GPU's work (#1376). Its text left one question open: "Frame pacing on AMD with the CPU-side wait is the open question for the Test Farm." On this PC (RX 9070 XT, AMD's Windows driver, a 1280x720 window on a 144 Hz desktop) that wait costs 2.8-3.2 ms of each frame on the render thread: its presents take 3.6-4.1 ms a frame since #1458 against 0.9-1.0 ms before it, the CPU sampler finds 19.8 % of the thread blocked in that fence wait, and the render thread is what the frame waits for. Full screen has no such wait (0.88-0.90 ms presents).
Why the gain should carry beyond this PC: the change is not AMD-specific. Every driver makes the program wait for the window's image somewhere, in the acquire's fence (AMD's on Windows, which hands the image back at once and leaves the wait to the fence) or inside the acquire call itself (NVIDIA's, Mesa's, as #1458 says). Asking for the image a present earlier, without waiting, gives it the next frame's work to come back in, whichever of the two holds the wait. Where the image is already back early (full screen here), nothing changes but where the acquire is called. This PC can only show AMD's Windows driver, hence the Test Farm request below and the switch off by default.
See #1458, #1376.
What changed
One switch, off by default. With
RX_VK_ACQUIRE_AHEAD=1, right after each present of a game frame the renderer asks for the next present's image with a timeout of 0 and keeps it as an image already out, the state #1458 already has for an image it acquired and found busy. Its present then looks at it without a new acquire.gpu/vk/vk_replay.cppacquire_ahead()(new), called at the end ofpresent_image()afterpresent_out()returned the present as made, with the queue's lock let go: nothing while an image is held, no swapchain or no unsignaled fence (acquire_fence_ready(), #1458's rule);vkAcquireNextImageKHRwith a timeout of 0 intoacquire_fence.VK_NOT_READY,VK_TIMEOUT, out of date or a lost surface: nothing is out and the next present acquires as without the switch. A lost device is reported as at every other acquire.HeldImagegetsahead. At its present an image asked for ahead is looked at without waiting; not free yet, it is waited for up tokImageFreeNs(250 ms), as an image acquired at that present would be, and only then held as #1458 holds one. So a slow compositor drops no frame that is shown today.held_image_drop()(a swapchain made again while an image is out) also lets go an image asked for ahead.acquire_ahead_wanted(): the switch read once by value (0,off,no,falseor unset: off), logged at the first present; not while frame generation runs, with the presenter thread (RX_VK_PRESENT_THREAD=1: the swapchain is one thread's at a time), or under FIFO (V-Sync On, where the acquire is what paces). Each change of that answer is logged once:vk: acquire ahead: not now, <why>.vk: the window's image (600 presents): waited for X ms a present (its acquire and its fence), and what became of the images asked for ahead; with the switch only. Unset or0, nothing is counted or logged beyond the oneacquire ahead: offline at the first present: the switch costs nothing while off.gpu/vk/switches.def: the row,attempt, off on every PC.docs/development.md: the row besideRX_VK_PRESENT_THREAD.tests/test_present_acquire_ahead.py(new, run bytests/run_all.py) cutspresent_image's acquire, its copy's submit, its end and the new helpers out ofvk_replay.cpp, compiles them against a model presentation engine (AMD-like: an image at once, its fence later, never while the window is hidden; NVIDIA-like: the wait in the acquire, nothing at a timeout of 0) and runs both_WIN32sides with the switch unset,0and1.tests/test_present_device_loss.pycounts the new acquire among the WSI sites that must route a lost device;tests/test_present_thread.py's needle readspresent_image's end with the present's result kept (its later check fails on main too);tests/test_present_image_free.py's fixture gets the switch readerenv_on, answering off (#1458's order, which it checks).docs/tested/: the four admitted standard-test runs, one file each.Measurements
Verdict: helps on AMD's Windows driver in a window: +8.4 % fps (64.59 to 69.99, median of 2 pairs), CPU time a frame -6.7 %, the render thread's presents 3.9-4.2 to 0.73-0.76 ms a frame, every switch-on run faster than every control run; game clock 1.000-1.001 in both arms, p99 not worse, no frame over 100 ms in either. n = 2 pairs (the harness rejected the third pair's control run and skipped the pair, by its rules), so the harness labels it "diagnostic". Not measured on any other GPU or system.
Baseline: the runs were made before this branch was rebased onto main
430eef4f: "Before" is main as it was then,ef6038ab, and "After" is this PR's renderer at 0ceaa6ab over the same game build. The rebase changed none of this PR's commits (git range-diff: identical), and of main's 13 newer commits none touchespresent_imageor the acquire (vk_replay.cppchanged in 5 lines elsewhere); later commits here change a comment, the docs row, the test anddocs/tested/, and make the image-wait counter run with the switch only (a052b5eb: with the switch set, the same calls and counts as the measured renderer; unset, no counting).Hardware: Windows 11 Home (26300), AMD Radeon RX 9070 XT (driver 26.11.1), Ryzen 7 5700X3D (8 cores, 16 threads), 64 GB; one 5120x1440 144 Hz display
Scene: Hyrule Field, the camera turning at full speed; a 1280x720 window (swapchain 1280x720, 4 images, immediate), 2048x1152 output (16:9), a fixed options profile (frame generation off, V-Sync off), no frame rate cap (
HOS_FPS_MAX=240), threaded rendering on (RX_VK_ASYNC=1), a fixed background GPU load beside the game (500 blended quads a frame at 100 fps; it keeps the GPU's clock steady between runs)Method: our standard test: a 20 s window from the area's load, alternating pairs control / switch on, no warm-up runs, a rejected run skips its pair; one game build (exe 0a53b387) in both arms, only
longshot_vk.dlldiffers (control ed1771bb, this PR 59884440); fps, CPU time a frame, p99 and worst frame from the presents in the window, the game clock against real time fromHOS_FPS_LOG=1, the render thread's time from its 600-present log lineHow to reproduce: a window in Hyrule Field with V-Sync Off (immediate) and frame generation off; run once without and once with
RX_VK_ACQUIRE_AHEAD=1inlongshot_env.txt, and compare the log'svk: the render thread (600 presents ...)lines (presentsin them) and the FPS counteref6038abThe frame over 50 ms (90.4, 95.7 and 80.7 ms, 47.7 in the fourth run) is one and the same event in every run, about 1 s into the window: the turn's first stream-in. The third pair's control run was rejected for the game clock's coverage (a probe at 1.013 after a 26 s pause in the load); its pair was skipped, not replaced.
Flags and settings
One new switch, off by default on every PC; nothing else changes.
RX_VK_ACQUIRE_AHEAD0: as before;=1: the next image asked for right after each present, never waited for thereRX_VK_PRESENT_THREAD=1or FIFO (V-Sync On), each loggedTesting
Tested:
In the game, Windows 11, RX 9070 XT, AMD 26.11.1, this PR's renderer at 0ceaa6ab with the switch on, besides the A/B above:
RX_VK_VALIDATION=1, the Khronos layer,VK_LAYER_VALIDATE_SYNC=1): no message about the swapchain, the acquire, the present or fences (the messages there were about shader outputs, image creation and query pools, code this PR does not touch).acquire ahead: not now, frame generation runs, interpolation ran, no error line. V-Sync On:acquire ahead: not now, the swapchain is FIFO, presents 4.0 ms a frame as before.Test Farm (results, commit
2f24a499against main430eef4f). The plan:vsync=2(V-Sync Off: the immediate mode, which the switch does not hold off) and frame generation off.field-default) and withRX_VK_ACQUIRE_AHEAD=1on the PR side (field-acquire-ahead).The reports came from two PCs:
vk: the window's image (600 presents)line, which the switch prints every 600 presents whenever it is set, so they cannot show the switch's state either way.Steam Deck OLED (dukenheimer; SteamOS, RADV, 1280x800 through gamescope's mailbox swapchain, Threaded rendering at Auto (off)): the same build on main
430eef4f, the switch unset against=1, three interleaved rounds.tests/test_present_acquire_ahead.py: the order (the next image asked for after the present is queued, never with the queue's lock held, one asked for ahead at most, none while one is held, never with a signaled or pending fence); an image given back after 20 ms (none dropped, waited for no longer than needed); #1458's hidden window (no GPU wait in any copy, no copy into an image not given back, the first hidden present waits exactly 250 ms, shown again once given back); the swapchain made again with an image out, by a resize and by the present's own SUBOPTIMAL; an ahead acquire answered out of date; the gates (frame generation, the presenter thread, FIFO, the start-up screen's presents); NVIDIA-like (the ahead acquire never waits); a lost device at the ahead acquire; with the switch unset or0, no counter line. Eleven broken variants each fail for their own reason (among them: the ahead acquire before the present, the wait at the present removed, the drop at a new swapchain removed, each gate removed, the switch's value ignored, the counter run with the switch off). It fails on main and passes here; it passes on Windows (llvm-mingw clang++) and on Linux (WSL Ubuntu, g++ and clang++). #1458's owntests/test_present_image_free.pypasses (its fixture only givenenv_on).python -I -B tests/run_all.py --jobs 4(Windows, llvm-mingw clang 20261104) on this branch @2f24a499and on main @430eef4f: this branch 684 passed, 46 failed, 51 skipped (the Linux-only checks); main 683 passed, 46 failed, 51 skipped. The 46 failures are the same list on both (they fail on this Windows checkout on main too, among themtest_present_thread.py, whose later check expects an opt-in-only presenter queue, andtest_tested_configurations.py's line-ending check of its CLI on Windows); the extra pass istests/test_present_acquire_ahead.py.test_present_image_free.pyandtest_present_device_loss.pypass.At
2f24a499,gpu/vk/vk_replay.cppcompiled to an object for Linux (WSL clang 21 with the Linux build's flags) and for Windows (llvm-mingw clang++ 20261104,-Wall: one warning, main's own, aVkClearColorValueinitializer it does not touch); the whole renderer was built with llvm-mingw at 0ceaa6ab for the A/B (210 files, no warnings). The game-free tests pass on WSL Ubuntu with g++ and clang++ at2f24a499too.Not testable by hand: NVIDIA, Intel, Linux and the Steam Deck: not on this PC. The game-free test's NVIDIA-like model stands in for the "wait inside the acquire" drivers (the ahead acquire never waits there). The Test Farm and a Deck owner ran what is above; the rest is below.
Not tested:
SYNC-message at all), and the same session without the switch.Must be tested:
RX_VK_ACQUIRE_AHEAD=1inlongshot_env.txt.acquire ahead: on, and the log'svk: the window's imageline near 0 ms a present.Risks
What could break: only with the switch set. The swapchain's image is asked for earlier: on a driver that answers a timeout of 0 with "not ready", the switch does nothing (the counter's "none given at once" says so). Swapchain remakes (resize, full screen, HDR, V-Sync) let an image asked for ahead go; its fence is then replaced by a new one when it is not signaled yet, as #1458 does after a 250 ms hold, so one
VkFencecan be left over per remake made in the ~3 ms before the image came back (bounded, a small object; reusing such fences once signaled is a follow-up that would touch #1458's path for everyone). Right after frame generation is turned off its last presents ahead may still hold images, and the image asked for ahead is then one more out, as the next present's own acquire would be without the switch (it cannot block: a timeout of 0). Also left as found: a failedvkResetFences(out of memory) after a wait leaves the fence signaled, on #1458's path as on this one.Turning it off: leave
RX_VK_ACQUIRE_AHEADunset or set0.Compatibility: Nothing changes for a player: no save, options file, shader or pipeline cache, or vk ABI change (no
vk_api.hor shader translator file touched).Checklist
Everyone
type(scope): description) of the status line's type.can't be run by hand is in Testing's Not testable by hand, with why.
in Risks.
main.python -I -B tests/run_all.pypasses (any failure that also fails onmainis named).third_party/untouched (a new folder with aPROVENANCE.mdinstead).When it applies
Assisted-by:/Co-Authored-By:trailers.docs/tested/for each new measured run (format indocs/tested-configurations.md).mainand passing here.tests/that reads changed source updated in this PR, not weakened.🤖 Generated with Claude Code
Tested in game: Windows 11, RX 9070 XT (AMD 26.11.1), Ryzen 7 5700X3D: this PR's renderer (at 0ceaa6ab, before the rebase; same code with the switch set) in Hyrule Field's standard test in a 1280x720 window, 2 pairs against main
ef6038ab: 64.6 -> 70.0 fps, game clock 1.000-1.001 both arms, no frame over 100 ms; window resizes, F11 full screen and back, Insert, a Vulkan validation session (no swapchain/acquire/present/fence message), frame generation 2x and V-Sync On (the switch holds off and logs why), and a full-screen window covered for 40 s twice: no error line, no hang.WIP: perf(vk): RX_VK_ACQUIRE_AHEAD=1 asks for the window's next image right after a present, so the render thread no longer waits for it (off by default)to perf(vk): RX_VK_ACQUIRE_AHEAD=1 asks for the window's next image right after a present, so the render thread no longer waits for it (off by default)@Beanstalk nvidia, intel, linux, steamdeck: in Hyrule Field, in a window, with V-Sync Off and frame generation off, play 3 minutes without the switch, then 3 minutes with
RX_VK_ACQUIRE_AHEAD=1inlongshot_env.txt, and post both runs' fps, your Threaded rendering setting, the switch run'svk: the window's image (600 presents)lines, and with Threaded rendering on both runs'vk: the render thread (600 presents ...)lines. Expect with the switch:acquire ahead: onin the log, the window's image waited for near 0 ms a present, fps not lower. On Linux, also cover the window with another one for 40 s, then uncover it: frames must come back. Fail on a hang, a crash, a lost device, flashing or repeated frames, lower fps, or error lines in the log.@klimovich008 only members of
BigBeanscan start the farm. One of them can approve this request by reacting 👍 to it or to this comment.@klimovich008 Yes, we'll run it on our Steam Deck OLED (SteamOS, RADV, handheld 1280×800, Threaded rendering at Auto, so off). It'll be the same build with
RX_VK_ACQUIRE_AHEADunset vs=1, interleaved, in Kokiri standing and Hyrule Field turning. We'll check engagement fromacquire ahead: onand report fps, p99, frames over 50 ms and thevk: the window's image (600 presents)lines.It's queued after the block that's running on the Deck now, so expect results later today. The Deck presents through gamescope, not a desktop window, so we'll also note the swapchain and present mode from the log, since the acquire wait may look different there.
Beanstalk test farm
Commit
2f24a49· tier reviewed · approved by futondestroyer (👍 on klimovich008's request) · dashboardWhat will be tested · AI plan (
gpt-6.1-terra)On NVIDIA, Intel, Linux and Steam Deck PCs, the plan compares main and the PR in Hyrule Field for three minutes per run, with acquire-ahead off and on. It measures FPS, frame times and hitches, compares still images, and fails on lower FPS, crashes, device-loss or fatal GPU-hang logs. Intel and Steam Deck targets will wait for matching testers. The headless farm cannot cover and uncover a desktop window, verify repeated frames throughout a run, or enforce the requested per-present wait and threaded-rendering log details. No tester online now matches intel, steamdeck: those targets wait for one.
1(PR only).Started at 13:08 UTC.
Estimate: done in about 1 h 15 min, from the queue and the testers online now; some jobs can't start yet.
NVIDIA GeForce RTX 3090, 621.24.0; 2: linux+nvidia+nvidia-proprietary+dgpuNVIDIA GeForce RTX 4070, 615.71.9NVIDIA GeForce RTX 4070, 615.71.9; 2: linux+otherunknown, unknownMeasurements · nvidia
Verdict: fail on nvidia (2/1 report):
field-acquire-ahead field.fps_mean.min_delta_pct (+0.0%)failed.Hardware: 1: windows+nvidia+nvidia-proprietary+dgpu
NVIDIA GeForce RTX 3090, 621.24.0AMD Ryzen 9 5900X 12-Core Processor, 128 GB RAM; 2: linux+nvidia+nvidia-proprietary+dgpuNVIDIA GeForce RTX 4070, 615.71.9AMD Ryzen 7 5800X3D 8-Core Processor, 64 GB RAMScene: field-default, from a new save (vsync=2, fullscreen=0, frame_generation=0); field-acquire-ahead, from a new save (vsync=2, fullscreen=0, frame_generation=0); each cell's settings and switches as the table names them
Method: the Beanstalk test farm: main and this PR built and run on the same PC in one session, game starts alternating (ABBA); field-default: 1 game start a side, field 180 s after 10 s warm-up; field-acquire-ahead: 1 game start a side, field 180 s after 10 s warm-up; each number the median of the runs, then of 2 reports
How to reproduce: the farm's plan above (
@Beanstalkon this PR runs it again)430eef42f24a49RX_VK_ACQUIRE_AHEAD=1 (PR only)· field · frame rateRX_VK_ACQUIRE_AHEAD=1 (PR only)· forbidden log linesRX_VK_ACQUIRE_AHEAD=1 (PR only)· field · frame time (p99) (ms)Flags and settings
RX_VK_ACQUIRE_AHEAD1Markdown for the PR description
Measurements · linux
Verdict: pass on linux (2/1 report): no rule failed and no PR run crashed.
Hardware: 1: linux+nvidia+nvidia-proprietary+dgpu
NVIDIA GeForce RTX 4070, 615.71.9AMD Ryzen 7 5800X3D 8-Core Processor, 64 GB RAM; 2: linux+otherunknown, unknownAMD Ryzen 7 5700X3D 8-Core Processor, 128 GB RAMScene: field-default, from a new save (vsync=2, fullscreen=0, frame_generation=0); field-acquire-ahead, from a new save (vsync=2, fullscreen=0, frame_generation=0); each cell's settings and switches as the table names them
Method: the Beanstalk test farm: main and this PR built and run on the same PC in one session, game starts alternating (ABBA); field-default: 1 game start a side, field 180 s after 10 s warm-up; field-acquire-ahead: 1 game start a side, field 180 s after 10 s warm-up; each number the median of the runs, then of 2 reports
How to reproduce: the farm's plan above (
@Beanstalkon this PR runs it again)430eef42f24a49RX_VK_ACQUIRE_AHEAD=1 (PR only)· field · frame rateRX_VK_ACQUIRE_AHEAD=1 (PR only)· forbidden log linesRX_VK_ACQUIRE_AHEAD=1 (PR only)· field · frame time (p99) (ms)Flags and settings
RX_VK_ACQUIRE_AHEAD1Markdown for the PR description
@klimovich008 the Steam Deck run you asked for.
Setup: Steam Deck OLED (SteamOS, RADV), built-in 1280×800 display through gamescope's mailbox swapchain, Minimum preset with a fixed 50% render, Threaded rendering at Auto (off). The same build of this PR on main
430eef4f, withRX_VK_ACQUIRE_AHEADunset (U) vs=1(A), three interleaved rounds.Engagement: ahead acquisition actually succeeded, and the images were reported ready at their next present. Every A counter line ending inside a measured window read 600 free, 0 waited, 0 dropped, 0 none given.
Kokiri standing, three pairs:
That's a small positive directional signal, not an established practical or general win.
Hyrule Field: no admitted measured pair.
What this can't show:
No GPU fault or reset in any run.
No default recommendation from this.
@dukenheimer, thank you: that is the case I could not run here (gamescope's mailbox swapchain, threaded rendering off). How I read it:
I'll add it to Testing. The farm's NVIDIA cell shows p99 19.6 -> 22.1 ms with the switch set (its fps rule fails at +0.0 % because both sides sit at the 60 cap). I'm checking whether the switch was engaged in those runs at all, and I'll report here.
Following up on my comment above about the farm's NVIDIA cell. I went through the four reports.
Did the switch engage? The settings allowed it, but the reports don't show it.
vsync=2, which is V-Sync Off (the immediate mode), with frame generation off. Threaded rendering was on. The switch holds off only for FIFO (V-Sync On), frame generation and the presenter thread, so from the source it should have engaged whereverRX_VK_ACQUIRE_AHEAD=1reached the game.vk: the window's image (600 presents)is printed every 600 presents whether or not it engages, and none of the published log ends has that line. So the reports can't show whether the switch was on in the measured runs, either way.Is the p99 rise real? It comes from one run on one PC. That run neither proves nor rules out an effect.
Testing now has the farm's three reports with their p99, stutters and fps, and the Arch Linux report that timed out before play. It also has the Steam Deck run (thank you @dukenheimer): engaged, Kokiri about +0.4 % fps, no GPU faults, Hyrule with no admitted pair. Must be tested lists the NVIDIA repeat as the open question. No code changed (head
2f24a499).View command line instructions
Checkout
From your project repository, check out a new branch and test the changes.