WIP: perf(vk): read the GPU times of a run of finished recordings in one query call (RX_VK_TS_RUN=1, opt-in) #150
No reviewers
Labels
No labels
area/accessibility
area/audio
area/input
area/install
area/menu
area/renderer
area/saves
area/upscaling
good first issue
gpu/amd
gpu/igpu
gpu/intel
gpu/nvidia
help wanted
kind
bug
kind
chore
kind
docs
kind
feature
kind
performance
kind
question
needs measurements
platform/linux
platform/steam-deck
platform/windows
preview
priority
critical
priority
high
priority
low
status
blocked
status
confirmed
status
duplicate
status
invalid
status
needs-info
status
wontfix
testing wanted
No milestone
No project
No assignees
1 participant
Notifications
Due date
No due date set.
Dependencies
No dependencies set
Reference
MagicBeanDev/Longshot!150
Loading…
Reference in a new issue
No description provided.
Delete branch "klimovich008/Longshot:perf/gpu-time-run"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Why
The replay reads each recording's GPU timestamps with its own
vkGetQueryPoolResultswhen the slot comes round: about 31calls a frame (about 31.5 recordings a frame in a counter run of the Hyrule standard test). In sampled profiles of the same test,
the driver time under these calls was 0.16 ms of the presentation thread a frame (RX 9070 XT, AMD's Windows driver). This reads a
run of finished slots in one call instead. It is opt-in,
RX_VK_TS_RUN=1, and the default is unchanged. A performance change,not a bug fix.
Ported from NotRexed/Hookshot, where it was prepared and reviewed but not opened; the measurements below were made there.
What changed
gpu_time_collectandgpu_time_read_runingpu/vk/vk_replay.cpp. WithRX_VK_TS_RUN=1the slot's own times andthose of up to 15 unread slots after it are read in one call with
VK_QUERY_RESULT_WITH_AVAILABILITY_BIT. The times read aheadare kept (
ts_read_ahead,ts_ahead) and counted when their own slot comes round, as the default counts them, so each120-present window the consumers read sees the same GPU time. A slot the GPU has not finished stays pending for its own turn.
Each recording is counted once.
so each slot's two queries are also reset on the host (
vkResetQueryPool) when the slot comes round, and the pool once whenit is made (
gpu_time_begin). An available result is then always the current use's.hostQueryReset. The device enables every Vulkan 1.2 feature it reports (vk_main.cpp); withoutit,
RX_VK_TS_RUN=1falls back to one slot a call and logs that once (ts_run).vk: GPU timelog line, the movie network check and the Auto (this PC)preset's tuning in play (Hookshot #2850). With the switch unset none of
them change.
tests/test_gpu_time_run.pyandtests/gpu_time_run_test.cpp(game-free, inrun_all).docs/development.md(Time and profiles); two rows indocs/tested-configurations.md.Flags and settings
RX_VK_TS_RUN(set)Measurements
main:
839e85ff(2026-11-13)this PR: perf/gpu-time-run, its commits on
839e85ff@ 05e79bfa (the same diff as this branch's)Hardware: RX 9070 XT, AMD 26.10.2 (Vulkan driver 0x8001a0), Ryzen 7 5700X3D, 64 GB, Windows 11
Scene: Hyrule Field, the standard test: the save loaded, the camera turning at full speed, frame rate unlocked
(
HOS_FPS_MAX=240), 1280x720 window, 2048x1152 output (16:9), no frame generationMethod: a 20 s window after the load, warm caches, 3 alternating pairs, medians; fps and CPU ms a frame from the present log and
the process times.
How to reproduce: the build above, the switch unset against
RX_VK_TS_RUN=1, alternated, with our standard-test script.Baseline: Longshot
839e85ff+ this PR @ 05e79bfa, RX_VK_TS_RUN unsetMain was not the comparison: the switch unset is main's collector, on the same build.
Measured on main
839e85ff; main is at514591a3since. Its 36 commits in between add the occlusion query reports (#197, #260) and the Damage taken row, among others, and change none of the code this PR changes (gpu_time_collect,gpu_time_beginand the timestamp pool invk_replay.cpp): the fps on today's main will differ, the difference between the arms is this PR's own. A measurement on the next base will replace these.839e85ff+ this PR @ 05e79bfa, RX_VK_TS_RUN unset839e85ff+ this PR @ 05e79bfa, RX_VK_TS_RUN=1No gain: the difference is under 5 % with 3 pairs, diagnostic, and goes the other way. The 543 ms frame and three more over
100 ms were in one run with the switch on (61.79 fps); without that run the two arms are level (66.79 and 65.59 against 66.29
and 65.59 fps).
Earlier, on Hookshot and with the earlier collector (same query calls, accounting changed since; not this head):
Hardware: RX 9070 XT, AMD 26.10.2 (Vulkan driver 0x8001a0), Ryzen 7 5700X3D, 64 GB, Windows 11
Scene: Hyrule Field, the standard test as it was then, at 32:9 (5120x1440; it is 16:9 since): the camera turning at full speed,
frame rate unlocked (
HOS_FPS_MAX=240), no upscaler change, no frame generationMethod: 20 s window after the load, warm caches, 2 warm-ups, then alternating pairs. A run counts only if the game clock reads
1.000 ± 0.01 in the window and no frame there is over 250 ms. fps and CPU ms a frame from the present log and the process times.
How to reproduce: our
std_test.py(claude-perf) with--arm each=RX_VK_TS_EACH=1on a build of the earlier collector.Then the run read was the branch's default and
RX_VK_TS_EACH=1gave the one-slot read. Since then the switch was flipped (whichpath is the default), the fallback above was added, and the times read ahead are counted at their own slot's turn. An
independent probe through the real collector (360 frames, 312 reuses) shows the query calls unchanged: 20 with the switch vs 312
without, with the same sum and maximum as the one-slot path after every reuse.
Baseline: Hookshot
46780503+ earlier collector @ b4e4b082, RX_VK_TS_EACH=146780503+ earlier collector @ b4e4b082, RX_VK_TS_EACH=146780503+ earlier collector @ b4e4b082, run readThe fps difference is small; one more A/B could move it. Not measured on other GPUs, drivers, Linux or the Steam Deck.
Observation, not a claim: runs with a frame over 250 ms inside the window were 4 of 12 with the one-slot read and 1 of 12 with
the run read. In each, the window's clock reading over that stall dipped to 0.88-0.96. Too few to say the run read causes fewer.
Testing
Tested: game-free:
tests/test_gpu_time_run.pycompiles the realgpu_time_collectblock fromvk_replay.cppagainst afake query pool. Cases: a run of 20 finished slots counted once in at most 2 calls, slots the GPU is behind on, a slot whose own
queries were never written, a half-written slot, the ring's end, a slot never timed, stale availability after reuse, the sum and
the longest time after every reuse (cleared every 5 reuses, as a consumer's window) matching the default's in one call, the
default's one call a slot, and the switch on a device without
hostQueryReset(one slot a call, no host reset, said once).Controls:
--brokendrops the host reset of a slot that comes round and fails the stale case alone; main'svk_replay.cppfails run, ring_end and cadence (one call a slot, not one for the run) and no_host_reset (main has no switch to log about).
On this branch (
3e653ca8, on main39647e1a; Windows, llvm-mingw clang): the 10 cases pass, the renderer DLL compiles andlinks, and
tests/run_all.pygives 499 passed, 4 failed, 36 skipped. The 4 fail on main too:nx2/test_nup_paths.py,tests/test_check_third_party.py,tests/test_setup_runtime.py,tests/test_release.py. The last commit adds only theswitch's line in
docs/development.md.Content (a second agent, on the same collector built on Hookshot
46780503, its collector andgpu_time_beginbyte-identical tothis branch's):
network check showed that moving GPU time between 120-present windows: the movie decision came at 240 presents without the
switch and at 360 with it, though the totals matched. With the times counted at their own slot's turn, the same probe gives
the one-slot path's sum and maximum after every reuse and in all three windows.
on the RX 9070 XT: 360 fenced submissions over the 48 slots (96 queries), with each slot's own queries read before every reuse
as a reference. The one-slot path matched it at all 312 reuses; the first batched version read a wrong time at the first
reuse (frame 49: 26,000 ns where the reference had 1,680 ns). This version, switch off and on, matches the reference at every
reuse and the 120-present windows' totals and maxima, with 312 query calls off and 20 on. The Khronos validation layer was on
and reported no errors in the four cases; that synchronization validation was active was not confirmed. Evidence for the
component on real hardware, not a validation pass of the whole game.
foliage, Link, the HUD and the camera moving as expected, with nothing blank. No bit-exact frame comparison, and no speed
claim from these runs.
Tested in the game: the speed above (RX 9070 XT, Windows 11, Hyrule Field).
Not tested: a game run with Vulkan validation (Hyrule, switch on) was stopped by our test
machine's memory guard before the first present: the layer loaded and reported no VUID before the stop, but that is not a
validation or synchronization-validation pass. A movie (the movie network check) and the Auto (this PC) preset's tuning in play
were not run in the game. The Longshot rename on this branch was not run in the game (the collector is unchanged by it). Not built on Linux (the change is portable Vulkan code, but no Linux build was made).
Needs other testers:
vk: GPU timeline and the Breakdown's GPU time the same withRX_VK_TS_RUN=1as without, inthe same scene (request to open)
vk: RX_VK_TS_RUN=1 needs hostQueryResetline (request to open)Screenshots
Nothing on screen.
Risk and rollback
Off unless
RX_VK_TS_RUN=1is set; unset, the collector is main's. With it set, a wrong or late GPU time would show in the FPScounter's Breakdown and the
vk: GPU timeline, and could move the movie network check or Auto (this PC)'s tuning; unsettingthe switch turns it off, and reverting the commits is enough.
Reviewer notes
Duplicate check (2026-11-13): GitHub NotRexed/Hookshot open and merged PRs and issues for
vkGetQueryPoolResults, timestampquery, query pool, GPU time and
gpu_time_collect, and Longshot's open and closed PRs and issues. Nothing batches the timestampreads. Hookshot #2897 (#60 here) wraps
vkGetQueryPoolResultsto time callsthat wait (
VK_QUERY_RESULT_WAIT_BIT); this change's call does not wait, and the two do not touch the same code.Not a shader-cache change: it touches none of the files
nx2/build.pyhashes into the stamp.Issues
None.
Checklist
For everyone
type(scope): description).main.python -I -B tests/run_all.pyrun and passing (any failure that also fails onmainis named above).third_party/untouched (a new folder with aPROVENANCE.mdinstead).If applicable
Assisted-by:/Co-Authored-By:trailers.docs/tested-configurations.mdfor each measured run.mainand passing here.🤖 Generated with Claude Code
Measured on Longshot (base
839e85ff+ this PR as a pack of the same patch, Hyrule standard test at 16:9, 3 alternating pairs on an RX 9070 XT): RX_VK_TS_RUN=1 65.59 fps against 66.14 off (-0.8 %, diagnostic), CPU 63.13 against 60.87 ms a frame, and 4 frames over 100 ms (one 543 ms) against none. No gain on this PC; the switch stays off. Description updated; merging current main next.Measured this head on Longshot
839e85ff(+ this PR's commits, 05e79bfa, same diff), Hyrule standard test at 16:9, 3 pairs: no gain. RX_VK_TS_RUN=1 65.59 fps / 63.13 ms CPU against 66.14 / 60.87 unset (diagnostic); one run with the switch had a 543 ms frame and three more over 100 ms, without it the arms are level. The earlier 32:9 numbers stay below, labelled as the earlier collector's. Rows added to docs/tested-configurations.md.514591a3into perf/gpu-time-run 3f007d5655Says why the
839e85ffnumbers hold on today's main: main's later commits do not touch the timestamp collector.839e85ffA/B (the per-arm ones)Closing: on Longshot this switch gives no gain. The standard test at 16:9 on an RX 9070 XT / Ryzen 7 5700X3D (base
839e85ffwith this PR as a pack of the same patch, 3 alternating pairs): RX_VK_TS_RUN=1 65.59 fps against 66.14 off (-0.8 %, diagnostic), CPU 63.13 against 60.87 ms a frame, and 4 frames over 100 ms (one 543 ms) against none. The earlier Hookshot difference (+0.9 to +1.2 % at 32:9) does not hold here. The branch stays on the fork with its game-free test and the GPU probe results above, should a driver where per-slot query reads cost more ever make it worth another look.Pull request closed