WIP: perf(vk): the render thread replays jobs as they come; whole frames are a test #1121
Labels
No labels
area/accessibility
area/audio
area/input
area/install
area/menu
area/renderer
area/saves
area/tooling
area/upscaling
beanstalk: approved
beanstalk: failed
beanstalk: needs approval
beanstalk: running
beanstalk: tested
ci: approved
good first issue
gpu/amd
gpu/igpu
gpu/intel
gpu/nvidia
help wanted
kind
bug
kind
chore
kind
docs
kind
feature
kind
performance
kind
question
needs measurements
platform/linux
platform/macos
platform/steam-deck
platform/windows
preview
priority
critical
priority
high
priority
low
regression
risk
high
risk
low
risk
medium
status
blocked
status
confirmed
status
duplicate
status
invalid
status
needs-info
status
wontfix
ci: approved
No milestone
No project
No assignees
6 participants
Notifications
Due date
No due date set.
Dependencies
No dependencies set
Reference
MagicBeanDev/Longshot!1121
Loading…
Reference in a new issue
No description provided.
Delete branch "nyannoying/Longshot:perf/vk-frames-as-they-come"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Type: perf · Risk: medium · Tested on: Windows 11, Arc B580 · Needs: Linux, AMD and NVIDIA (Kokiri Forest and the Market standing, the frame log's flash count)
Why
With Threaded rendering on (Auto turns it on with 12 or more logical processors), the render thread waits for each frame's present before it replays any of that frame's work ("whole frames", PR 588). The wait was added because replaying a frame's first submits while the game was still writing the rest drew the lit scene black for one frame. Those moments were also where a submit's capture missed the pages the replay's collection was marking again, which #1083 fixed (merged). With that fix, replaying jobs as they come drew no such frame here, and the wait costs about 6% of the frame rate in Kokiri Forest and 8% in the Market standing, and 10% and 4% walking in Kakariko Village and the Market on an Arc B580.
What changed
The render thread replays jobs as they come again; whole frames become a test switch (
RX_VK_FRAME_GATE=1). The present is still handed over after the main thread's signal for its frame, as it was with whole frames, which is what was measured.gate_on()ingpu/vk/vk_replay.cppis off unlessRX_VK_FRAME_GATEis on;present()'s wait for the CPU's signal no longer follows the gate (on unlessRX_VK_PRESENT_AFTER_SIGNAL=0). The comments say why.docs/development.md'sRX_VK_ASYNC=1row;vk_memory.cpp'scollect_mucomment names the new default.Measurements
Verdict: Faster, and no dark frame standing either way. With the defaults, 4 and 4 interleaved sessions standing: Kokiri Forest 86.95 -> 92.4 fps (+6.3%), the Market 53.6 -> 58.0 fps (+8.2%), the medians; the region test counted 0 dark frames in 21,600 before and 0 in 23,400 with this PR. Walking, where the walks take the same way for their first ~9 s (four 120-present windows): Kakariko Village 50.15 -> 55.2 fps (+10.1%), the Market 53.23 -> 55.6 (+4.5%), 4 and 4 sessions; and on one build with only the switch differing (4 sessions with
RX_VK_FRAME_GATE=1, 4 with the default), 50.83 -> 54.0 (+6.2%) and 53.62 -> 55.78 (+4.0%). Walking, the region test flags frames with whole frames on as with this PR (the camera passing people, doorways and shadows): over those first 600 frames of each area, 5 darker and 7 brighter in 8 walks with whole frames, 5 and 9 in 8 with this PR; later, once the walks have parted, the Market's are alike (62 and 85 against 61 and 78) and Kakariko Village's are not (7 and 2 against 30 and 16, most of them one walk's burst on its own way). Whole pictures dark while walking: 0 in 8 walks with whole frames, 4 in 8 with this PR and 3 in 2 earlier ones; the 2 the brightness window logged were all still as dark 10 frames later, with the final pass's exposure unchanged: the picture changed and stayed so, not the one-frame black of PR 588. A later walk with whole frames on had 4 too (3 in Kakariko Village, 1 in the Market; a #446 session whose guard bound nothing in place, so the build before's picture): they come with where a walk goes, not with the switch. On the build before,RX_VK_FRAME_GATE=0(this PR's default) took the Market from 56.8 to 59.85 fps standing (5 and 6 interleaved sessions), 0 and 0 dark frames in 9,000 and 13,800.Hardware: Windows 11 Pro 10.0.29704, Intel Arc B580 12 GB (driver 32.0.101.9154), Core i5-13600K (20 threads), 64 GB
Scene: Kokiri Forest and Castle Town Market standing, Kakariko Village and the Market walking (child); 1280x720 window, the scene rendered at 960x540 and upscaled with Intel XeSS 2.0.2 (Ultra Quality, #459), no frame generation,
HOS_FPS_MAX=240Method: each session a new start of the game: Kokiri Forest, then the Market by the debug teleport, 40 s to settle and 40 s measured in each: the frame rate from the log's 5 s
vi:windows (the median of 8 or 9); dark frames from the region test in the renderer (a local patch, not in this PR, as in #1083: 16x9 regions of every presented frame, a frame counted when 2+ regions are more than 10 levels darker than in both neighbouring frames), over both measured windows. The frame log on over every measured window (RX_VK_FRAMELOG_DROP=1), both sides alike. 8 sessions interleaved (B P P B B P P B). Walking: Kakariko Village, then the Market, the stick held forward 150 frames, right 20, forward 90, left 45, over and over. The walks take the same way for about 10 s and then part (the game's characters move; the draws a frame show it), so their frame rate is compared over each walk's first four windows of 120 presents (the renderer'svk: N presentslines; the median of a session's four, then of the sessions), and their region flashes over the first 600 frames of each area and after. 8 walking sessions interleaved (B P P B B P P B), 2 earlier with this PR alone, and 8 on this PR's build with a second local diagnostic (the picture's brightness 6 frames before and 10 after each whole picture dark, and the final pass's exposure by present), whole frames switched on (RX_VK_FRAME_GATE=1, G) and off (P), interleaved (G P P G G P P G). Before those, the Market alone on the build before, 11 sessions interleaved: whole frames on againstRX_VK_FRAME_GATE=0withRX_VK_PRESENT_AFTER_SIGNAL=1. The Market's frame rate follows its crowd (its draws a frame went from 2,410 to 2,724 between sessions), so it is the noisier of the twoHow to reproduce:
longshot_ctl.py teleport KokiriForest_Child --arrive 60thenMarket_Child; the FPS counter'svi: ... frames presentedlines over the window; the frame logRX_VK_FRAMELOG=<file>withscripts/framelog.py <file>.Baseline: the game folder here runs host 690d8446 (renderer API 22); main's renderer (API 24) does not load in it, so both columns are 690d8446 + #761, #838 + #1083's commits, built from the same source, with and without this PR's commit.
fd5238f7RX_VK_FRAME_GATE=1, left) and off (4 and 4 sessions), Kakariko VillageRX_VK_FRAME_GATE=1, left) and off (4 and 4 sessions), the MarketRX_VK_FRAME_GATE=0+RX_VK_PRESENT_AFTER_SIGNAL=1(right), 5 and 6 sessionsFlags and settings
One switch changes its default, one becomes independent of it.
RX_VK_FRAME_GATE=0turned it off)=1turns it on)RX_VK_PRESENT_AFTER_SIGNAL=0turns it off)Testing
Tested: Windows 11 Pro 10.0.29704, Arc B580 (driver 32.0.101.9154), the renderer built alone with llvm-mingw: 37 runs in
docs/tested/(2026-11-16-frames-as-they-come-nyannoying-run01torun37): the Market standing on the build before with whole frames on and withRX_VK_FRAME_GATE=0(11 sessions), Kokiri Forest and the Market standing before and after (8 sessions, no dark frame in any), Kakariko Village and the Market walking before and after (8 sessions), with this PR alone (2) and with this PR's build and a brightness diagnostic, whole frames switched on and off (8); no device loss in any.python -I -B tests/run_all.pyat892c106e(llvm-mingw, Windows, the toolchain's Vulkan headers): 621 passed, 25 failed, 44 skipped;main60bf15bdon this PC: the same 621, 25 and 44, and the same 25 fail on bothtest_coverage_runpassed in both suites but fails on both whenever no game is running (scripts/coverage/cover.pyreadstasklist's German "no tasks" line as cp1252): run alone afterwards, it failed on both.Not tested: AMD, NVIDIA, Linux, an Intel iGPU, the Steam Deck. Areas other than Kokiri Forest, the Market and their loads.
Must be tested:
RX_VK_FRAMELOG=<file>(pictures kept on your PC), thenpython -I scripts/framelog.py <file>, expecting no one-frame dark patch more than on main, and fps at least main's (Test Farm: @Beanstalk)RX_VK_FRAMELOG=<file>(pictures kept on your PC), thenpython -I scripts/framelog.py <file>, expecting no one-frame dark patch more than on main, and fps at least main's (Test Farm: @Beanstalk)Risks
What could break: a one-frame black lit scene or ground patch where the render thread catches up with the game, on hardware or in areas not run here.
Turning it off:
RX_VK_FRAME_GATE=1, or revert the commit.Compatibility: Nothing changes: no option, cache or API.
Checklist
Everyone
type(scope): description) of the status line's type.can't be run by hand is in Testing's Not testable by hand, with why.
in Risks. Not ticked yet: Measurements shows whole pictures dark in 4 of 8 walks with this PR against 0 of 8 without it.
main. (main60bf15bd: the perf commit'sRX_VK_ASYNCrow indocs/development.mdmet main's change to the row above it, and each side's row is kept;git range-diffshows the code unchanged)python -I -B tests/run_all.pypasses (any failure that also fails onmainis named). At892c106e, with the toolchain's llvm-mingw (binand its runtime folder first onPATH) and Vulkan headers (RX_CPM): 621 passed, 25 failed, 44 skipped;main60bf15bdthe same (621, 25, 44), the same 25 failing:test_audio_output_devices,test_auto_handheld,test_bug_report_defaults,test_crash_dialog,test_crt_options,test_dlss_feature_size,test_failed_compile_keeps_objects,test_fastfp_fma,test_fpx_sticky,test_fsr_uniform_ownership,test_guest_imported,test_import_reach,test_job_queue,test_linux_preflight,test_make_game_cache,test_missing_nup,test_pads_after_menu,test_parallel_init_order,test_pending_guest_import,test_replay_write_order,test_run_all_changed,test_spawn_emit,test_tested_configurations,test_vk_occlusion_queries,test_walk_keythird_party/untouched (a new folder with aPROVENANCE.mdinstead).When it applies
Assisted-by:/Co-Authored-By:trailers.docs/tested/for each new measured run (format indocs/tested-configurations.md).🤖 Generated with Claude Code
Tested in game: Windows 11 Pro 10.0.29704, Intel Arc B580 12 GB (driver 32.0.101.9154), Core i5-13600K. Kokiri Forest and Castle Town Market standing, 4 and 4 interleaved sessions: 86.95 -> 92.4 fps and 53.6 -> 58.0 fps (medians); the region test counted 0 dark frames in 23,400 with this PR (0 in 21,600 before). Kakariko Village and the Market walking, 4 and 4 sessions: 50.15 -> 55.2 and 53.23 -> 55.6 fps over the walks' first ~9 s. Whole pictures dark while walking come with where a walk goes, with whole frames on as well as off (4 in a later walk with them on). No device loss in 37 runs.
@Beanstalk please run this PR against main on Linux (any GPU), and on Windows with an AMD and an NVIDIA card: Kokiri Forest and Castle Town Market standing, 60 s each, with the frame log on (RX_VK_FRAMELOG=; the pictures stay on your PC), then python -I scripts/framelog.py . Expected: no more one-frame dark patches than on main, and fps at least main's. With this PR the render thread replays a frame's jobs as they come instead of waiting for its present (whole frames, now RX_VK_FRAME_GATE=1).
Possibly related, for context:
@nyannoying only members of
BigBeanscan start the farm.One of them can approve this request by reacting 👍 to it.This request was for9d5e089, and the PR has changed since (892c106), so a 👍 no longer approves it. Comment@Beanstalkagain to ask for the new commit.Tested on Linux NVIDIA, which "Must be tested" asks for. Both commits are rebased onto
main@43f40b87with no conflicts, as one build. The default (jobs as they come) is run againstRX_VK_FRAME_GATE=1(whole frames, main's behaviour), interleaved, two sessions each.Hardware: RTX 3090, NVIDIA 620.70.06, Ryzen 9 7900X (24 threads:
render thread and the recorder thread), CachyOS.Settings: DLAA 2560x1440, uncapped.
Route: Hyrule Field, then the Market (teleports), walking and then turning.
The switch does what it says. With
=1the log has thewhole frames (600 presents)lines, as main does. With the default there are none, andpresents after the CPU's signalshows the present still waits for the signal.RX_VK_FRAME_GATE=1(whole frames)On this PC there is no measurable difference: everything is within 1 to 2%, and in no consistent direction. The whole-frames wait doesn't cost anything here, unlike the 6 to 8% on your Arc B580. Frame-time tails are the same either way, with p99 17 to 24 ms in both arms.
Not counted: dark frames. My bench walks and turns, headless and with no frame log, so neither the region test you describe nor
scripts/framelog.pywas run here.🤖 Generated with Claude Code
theguyser referenced this pull request2026-11-16 13:51:42 +00:00
Description only, no code change:
After: this PR @ 862f73d7, on the same(the template's form).What is left before ready: the Linux build, which the Test Farm's Linux run under Must be tested makes (its request waits for a BigBean's approval).
🤖 Generated with Claude Code
9d5e0897f0892c106ef0Rebased onto
main60bf15bd: head892c106e.docs/development.md:maindropped ", GCN" from the Threaded rendering row (#1187), and this PR rewrites theRX_VK_ASYNCrow under it. Each side's row is kept.gpu/vk/vk_replay.cppandgpu/vk/vk_memory.cppmerged without a conflict;git range-diffshows this PR's code unchanged.Checked at
892c106e:RX_VK_ONLY, llvm-mingw), 0 warnings.python -I -B tests/run_all.py(llvm-mingw and the toolchain's Vulkan headers): 621 passed, 25 failed, 44 skipped, the same asmain60bf15bd, with the same 25 failing. More checks run than at the last count (559 passed), because the Vulkan headers were on hand this time.Why the numbers still hold, not measured again: the code diff is the same, and the A/B was this diff on and off on one build of the game folder's source (the Baseline line). Of
main's changes to the same files, #977's early reads never run with the render thread (readback_early.h), and #1187's mirrors are for GCN cards.🤖 Generated with Claude Code
Linux / AMD result for this PR's configuration, since the status line asks for it.
Verdict: faster on every segment in every run, and no dark frame in either arm.
What was run: not this branch. A local build of nightly-338 (main @
e9e7cf0b, which has #1083) plus local patches, where the two arms differ only by the environment:RX_VK_FRAME_GATE=0 RX_VK_PRESENT_AFTER_SIGNAL=1(this PR's behaviour, as its description states it) against both unset (whole frames).MESA_VK_ENABLE_SUBMIT_THREAD=1was set in both arms.Hardware: Linux (CachyOS, kernel 7.4.0-rc1), Ryzen 7 5800X, 32 GB, Radeon RX 9070 XT, RADV (Mesa 26.2.6), gamescope
Scene: a save in Hyrule Field, the character running a fixed route (two running legs, then a teleport to Kakariko and back as
load); 1440p output, render scale 1, frame generation off, uncappedMethod: 3 and 3 runs, interleaved (PR configuration first), median frame time per segment from the per-present frame times
Leg 1 per run, fps (median) / p99 ms / 1 % low fps: whole frames 65.9 / 21.2 / 41, 62.3 / 22.3 / 39, 67.3 / 23.5 / 29; jobs as they come 69.4 / 20.7 / 41, 71.5 / 19.8 / 45, 71.6 / 19.2 / 43.
The number the change targets, from run 1 of each arm:
So the game's frame-end GPU wait falls from about 3.0 ms to about 1.6 ms a frame here; the render thread's own work is unchanged (7.2 ms).
Dark frames: one run per arm on a second route (Castle Town market, Hyrule Castle, Hyrule Field, twice; capped 60) with
RX_VK_FRAMELOGrecording every present at 320x180. Counted with a script of ours, notscripts/framelog.py: a frame whose mean luma is below 0.6 of both neighbours'. 0 of 10,633 frames with jobs as they come, 0 of 10,693 with whole frames. The captures were deleted after counting (3.5 GB each), so I cannot runframelog.pyon them; I can record again if that count is the one you need.Not covered: this branch built from its own commit, NVIDIA, Windows, a standing camera in Kokiri Forest or the Market (the scenes in the description).
Brought up to the ready-for-review standard (#1454): the checklist's Everyone list has two new boxes, added here from what this PR's description and comments show (please check they match what you know).
Verdict: Request changes. Replaying the render thread's jobs as they come is a real gain where it shows (+6-8% standing and +4-10% walking on an Arc B580, 8-17% on an RX 9070 XT), and the code change is small and right. But this PR's own walking A/B shows a cost nobody has explained: whole pictures dark in 4 of 8 walks with it, 0 of 8 without. The default changes on every PC where Auto turns Threaded rendering on (12 or more logical processors). Also, the switch table on main still describes the old defaults.
What it does for players: with Threaded rendering on, the render thread no longer waits for each frame's present before replaying that frame's work, so the game runs faster where the replay was the wait. Whole frames stay behind
RX_VK_FRAME_GATE=1.Reviewed:
892c106eagainst its merge base60bf15bd. Read in full:3cbcb533(gate_on,present()'safter, the comments invk_replay.cppandvk_memory.cpp, theRX_VK_ASYNCrow). Skimmed: the 37docs/tested/rows. Also main'sgpu/vk/switches.defrows for both switches, and a trial merge with maine8e92f97. Read too: every comment with a run (the author's, Molier's RX 9070 XT, our RTX 3090). Nothing run (a code review).Fixed in this review: nothing.
Checked and fine:
gate_on()is now on only withRX_VK_FRAME_GATEset on;env_onalso takestrue,yesand so on.present()keeps the hand-over after the CPU's signal unlessRX_VK_PRESENT_AFTER_SIGNAL=0, which is the configuration the runs measured: the log's "presents after the CPU's signal" line has the same counts with the gate off.RX_VK_FRAME_GATE=1still gives main's behaviour exactly: theafterdefault follows no gate now, and with the gate on,afteris on as before.collect_mucomment names the new default.docs/development.md'sRX_VK_ASYNCrow conflicts, as text.Blocking
The dark whole pictures while walking. In this PR's own same-build A/B (whole frames switched on and off, 4 and 4 walks, interleaved), the pictures that went dark and stayed dark were 0 with the gate and 2 without. Against the build before (WB/WP) it was 0 and 2; that makes 4 of 8 walks against 0 of 8. The one walk with whole frames on that also had them (4) was a different build (#446's guard build, "the build before's picture"), so it isn't the same comparison. A picture that changes and stays dark for 10 frames, with the final pass's exposure unchanged, is not the one-frame black of PR 588, but it is still a picture that differs from main's. Please do one of these before ready:
If it is this PR's, it is a regression and blocks. If it is the route, the numbers will say so.
The switch table on main describes the old defaults. This branch predates main's #1309, so it doesn't touch
gpu/vk/switches.def. There,RX_VK_FRAME_GATEsays "on whenever the render thread is on (async_on)" with "=0 replays the render thread's jobs as they come".RX_VK_PRESENT_AFTER_SIGNALsays "follows the frame gate: on with the render thread unless RX_VK_FRAME_GATE=0". After this PR both are wrong: the gate is off unless=1, and the present waits for the signal unless=0. The table's own note says the code is right and the row wants correcting. Correct both rows in this PR: the gate's kind is now a test (diag/attempt), not a kill switch, and=1is the old way. Then rebase onto main: theRX_VK_ASYNCrow indocs/development.mdconflicts with main's text, and both sides' sentences need keeping.Suggestions
43f40b87, and ran it. Say so beside the box rather than waiting on the farm for the build alone.Testing:
9d5e089and needs asking again for the new head).Overlaps and merge order: none open in
gate_on/present(). Main's #1309 owns the switch rows above.View command line instructions
Manual merge helper
Use this merge commit message when completing the merge manually.
Checkout
From your project repository, check out a new branch and test the changes.