perf(vk): early reads: a read of GPU results waits for its writer's part, not its whole submission (RX_VK_READBACK_EARLY, on without VRAM mirrors) #977
No reviewers
Labels
No labels
area/accessibility
area/audio
area/input
area/install
area/menu
area/renderer
area/saves
area/tooling
area/upscaling
beanstalk: approved
beanstalk: failed
beanstalk: needs approval
beanstalk: running
beanstalk: tested
ci: approved
good first issue
gpu/amd
gpu/igpu
gpu/intel
gpu/nvidia
help wanted
kind
bug
kind
chore
kind
docs
kind
feature
kind
performance
kind
question
needs measurements
platform/linux
platform/macos
platform/steam-deck
platform/windows
preview
priority
critical
priority
high
priority
low
regression
risk
high
risk
low
risk
medium
status
blocked
status
confirmed
status
duplicate
status
invalid
status
needs-info
status
wontfix
ci: approved
No milestone
No project
No assignees
4 participants
Notifications
Due date
No due date set.
Dependencies
No dependencies set
Reference
MagicBeanDev/Longshot!977
Loading…
Reference in a new issue
No description provided.
Delete branch "zeldaootr/Longshot:early-reads-port"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Type: perf · Risk: low · Tested on: Steam Deck OLED (SteamOS 3.9.5, RADV): this branch on v3.0.0-rc.5 (dukenheimer) and its
gpu/code on an older main (ours); RX 580 (Linux, RADV; GCN, so no VRAM mirrors): this head on mainc167ec67(HumanBean); Linux, game-free checks; the renderer built for Windows (llvm-mingw) · Needs: Windows, NVIDIA, Intel (switch unset and on); a same-frame exposure check on the Deck before any defaultWhy
On the Steam Deck the game keeps waiting for GPU results the GPU has already computed. Its jobs read the culling camera block every frame, and the renderer makes that read wait for the whole 22-26 ms batch of GPU work it came in, not for the part that wrote it, so the GPU sits idle while the game waits. Letting the read go once its writer's part has run took Kokiri Forest at handheld settings from 11.40 to 13.89 fps (+21.8%) and Hyrule Field at 1280x800 from 14.98 to 18.47 fps (+23.3%) on a Deck OLED, with a byte-for-byte probe finding none of 4,200 such reads different from what main's wait returns. dukenheimer's run of this branch itself on a Deck OLED gave 15.70 to 20.73 fps (+32.1%) with the same recipe, and 0 of 1,100 probed reads missed. This PR brings it to Longshot on by default where the buffer pools have no VRAM mirror (integrated GPUs: the Steam Deck, AMD APUs, Intel), with
RX_VK_READBACK_EARLY=0as the way back. It was first ported off by default; HumanBean turned it on at review (468f360d; the decision: #977 (comment)), with the exposure check still to come (Risks).The design and nearly all of the code are sheep's and dukenheimer's. sheep built it in sheep/BunnyHood#2, on dukenheimer's #303, which started the early reads and wrote the byte probe used here. This PR only exists to bring their work to Longshot with their authorship kept: their 18 commits are cherry-picked one by one (
cherry-pick -x, authors unchanged, a[port: ...]line where a commit needed more than the fork's names mapped back to Longshot's). If either of them would rather open it themselves, this PR will be closed in favour of theirs.The tracker was searched ("readback early", "early read", "late defer",
READBACK_EARLY): no issue, and no open PR besides #303, which dukenheimer has since closed in favour of this one.What changed
By default on GPUs whose pools have no VRAM mirror (the Steam Deck, integrated GPUs), a CPU read of GPU results written in place waits for the part of the submission its last writer was recorded in, once that submission is recorded to its end, instead of for the whole submission. The capture's late pages are then left to the CPU reads that need them (
RX_VK_LATE_DEFER, which follows the switch) instead of holding up the replay. Discrete cards (mirrors) stay as on main unless=1is set, and=0is main everywhere.gpu/vk/vk_memory.cppnotes each in-place writer's part (early_in_place_write,memory_in_place_writer_part) for ranges whose reads waited;gpu/vk/vk_replay.cppinvalidatewaits for that part, afterearly_recorded_wait(the writer's submission recorded to its end, at most 250 ms), andreplay()ends a part after such a writer (RX_VK_READBACK_EARLY_GAPdraws into it).gpu/vk/vk_draw.cpp: a program that writes storage buffers rebuilds its descriptors once the tracking is in use, so each write is noted.gpu/vk/vk_memory.cpp,late_defer_splitandlate_defer_cpu_readfrommemory_read_as_submitted; a late page in a texture's own storage still waits at the take.RX_VK_LOG_RB_WRITERSand the byte probeRX_VK_LOG_EARLY_CHECK, log only.f2e73429(#443's rule): main'sRX_VK_SPLIT_BEFORE_DRAW(on by default on Intel) lets a part end only right before a plain draw. The fork ended a writer's part right after the writer, which can start the next part with a dispatch or a mesh draw.early_splitnow askspart_ends_before_drawasmaybe_splitdoes; where it holds, the writer's part runs on to the next plain draw. Elsewhere, the Deck included, nothing changes.tests/test_early_split_before_draw.pychecks it.468f360d(HumanBean, at review): on by default where the pools have no mirror.readback_early.h's rule takes the pools' mirror (Facts::mirrors, frommirror_on()): unset isON_DEFAULTwithout mirrors andOFF_MIRRORSwith them;=0is off and=1on everywhere; the render thread andRX_VK_PARALLEL>= 2 still leave it off, now asked or not.RX_VK_LATE_DEFERfollows it as before.tests/test_late_defer.py's rule checks andtests/test_readback_early.py's runs (unset with and without mirrors,=0,=1with mirrors) follow, and the doc rows say so.918372f3(off by default, before468f360d):gpu/vk/readback_early.hkeeps it off on every GPU unless asked for (the fork has it on for the Deck), and off even when asked for withRX_VK_PARALLEL>= 2 or the render thread.2ddd90fa(docs): the switches' rows indocs/development.md.13e90a5emet #1083'scollect_muinmemory_collect_written: in6cf7a3d1(the byte probe's commit) the probe's collection number is taken inside main's lock, with the marks, and its page report runs just after the lock is released, so the probe adds no lock undercollect_mu; the capture's report follows main's newcapture_scan. Both do nothing unlessRX_VK_LOG_EARLY_CHECKis set.tests/test_readback_early.py,tests/test_late_defer.pyandtests/test_early_split_before_draw.pyare new and cut withtests/srccut.py(3eb400b7);tests/test_readback_part.py,tests/test_late_fix_build.py,tests/test_capture_wait_log.py,tests/test_frame_trace_waits.py,tests/parallel_rb_refresh_test.cppandtests/direct_buffer_writes_test.cppfollow the code, andtests/fsr_image_retirement_test.cppgets a stand-in forrec_close, whichfinish_recordingnow calls (1d95536f: the check needs the Vulkan headers and had been skipped here without them);tests/test_vk_collect_window.py, which cutsmemory_collect_writtenout of the source, gets stand-ins for the probe's two hooks (36b0ccbb).docs/tested/(e161d7e6).Measurements
Verdict: it helps on the Steam Deck OLED where the game waits on its reads, and shows no measurable change where it doesn't. This branch itself (dukenheimer, on v3.0.0-rc.5): Kokiri Forest at handheld settings 15.70 -> 20.73 fps (+32.1%), the host's GPU waits 32 -> 6 ms a frame, the same GPU time, 0 of 1,100 probed reads missed. Our pack of its
gpu/code on an older main: +21.8% in Kokiri Forest at handheld settings, +23.3% in Hyrule Field at 1280x800, 0 of 4,200 probed reads missed. At a play build's settings (frame generation 3x,RX_VK_READBACK_LATE=newest, the GPU at 1600 MHz, dukenheimer): no measurable fps change, the two unset runs bracketing both on runs; captures stopped waiting for late pages, and 0 of 2,200 probed reads missed. On a desktop RX 580 (GCN, whose pools have no VRAM mirror by default), this head merged with mainc167ec67: fps up 10 to 24% in every phase of a four-place route, in both=1runs, GPU time unchanged, the byte probe 0 changed, and unset at main's speed (HumanBean). With the render thread on, the early reads leave themselves off, and the gain is not there (below). The byte probe checks only what a read returns: whether the game, let go early, writes memory the rest of the submission still reads is not measured (Risks).Baseline: dukenheimer's run (the first tables) is this branch on v3.0.0-rc.5 (main
e4d035e9), their rebase of148f14a0onto it, which matches1d1263e6in every built file; one build, "Before" withRX_VK_READBACK_EARLYunset (main's path), "After" with=1. The head now,36b0ccbb, is1d1263e6rebased onto13e90a5e:git range-diffshows every commit unchanged but6cf7a3d1, whose byte-probe hook inmemory_collect_writtennow takes its collection number inside #1083'scollect_mu, and one new test-only commit. Both act only withRX_VK_LOG_EARLY_CHECK, so the fps numbers hold for this head; the probe's numbers were taken with the hook's earlier form.dukenheimer's run of this branch (#977 (comment); their numbers):
Hardware: Steam Deck OLED, AMD Custom GPU 0932 (RADV VANGOGH), Mesa 26.2.0-devel, SteamOS 3.9.5, 16 GB; docked, in Game Mode through Steam (not headless). Built with the release toolchain (LLVM 23.1.5, glibc 2.35),
x86-64-v3, the play profileScene: Kokiri Forest beside Saria, output 1280x720, render resolution 50%, the game's own upscaler, quality preset Custom, frame generation off, threaded rendering Auto, stock clocks, Profile Docked, HDR off
Method: warm shader caches copied fresh for each start,
RX_VK_PRECOMPILE=0, nolongshot_env.txt; 20 s to settle, then 45 s standing; a warm-up, then unset,=1,=1, unset; fps and frame times from the presents, the other rows the medians of the log's periodic lines over each windowRX_VK_READBACK_EARLYunset: this PR @1d1263e6on rc.5RX_VK_READBACK_EARLY=1: this PR @1d1263e6on rc.5Byte probe (
=1,RX_VK_LOG_EARLY_CHECK=4,RX_VK_LOG_READBACK_WAITS=1; 30 s standing, then a moving Kokiri route for 88 s): 1,100 reads checked, 0 missed, 0 ambiguous, 0 lost. A plain start first (withoutRX_VK_PRECOMPILE=0, after--prepare-shaders) reached play at 14.35 fps with RSS at most 6.6 GB and MemAvailable at least 1.4 GB.In a Deck play build (rc.5 with #225, #227, #222 and this PR): Profile Docked, output 1280x800, the game's upscaler at 100%, frame generation 3x, HDR10,
RX_VK_READBACK_LATE=newestinlongshot_env.txt, the GPU fixed at 1600 MHz; a moving Kokiri route, 80 s; a warm-up, then unset,=1,=1, unset:=1(B1 / B2)=1runs)Byte probe on that build: Kokiri to the Deku Tree and back twice,
RX_VK_LOG_EARLY_CHECK=4, 800 checked; the moving route,=1, 1,400 checked: 0 missed, ambiguous or lost in either. No device loss, device fault, stalled GPU or memory kill in any run.HumanBean's run on a desktop RX 580 (#977 (comment); their numbers): RX 580 8 GB (Polaris10), RADV, 8 threads, KDE. Main keeps the VRAM mirrors off on GCN (
mirror_on()), so the early reads act there as on the Deck. Build: this PR's head merged withmainc167ec67, as adev_packs.pypack on a base of that main. Route: Kokiri Forest, Hyrule Field, the Market (debug teleport), Hyrule Field again; standing, then turning. One session per arm: main's own build, this PR unset (E0), and=1with the byte probe (RX_VK_LOG_EARLY_CHECK=4) twice (E1). Not interleaved, and the probe's checked reads wait as main's would, so E1 carries the probe's cost.=1, run 1=1, run 2fps 10 to 24% higher in every phase in both
=1runs, GPU time unchanged: the wait was on the CPU's side. Readbacks: with it off, 364 to 421 of 3,000 waited 10 to 16 ms each; with it on, 637 to 652 waited for their writer's part, 4 to 6 ms each. Byte probe, both runs:early -> main's release: 100 unchanged, 0 changed on a page the CPU did not ... 0 lost. Unset is main's speed.Our pack of its
gpu/code (the earlier measurements):Baseline for the pack: the Deck's development base is main @
9c9d69ed, and a renderer-only pack can't take this branch (it sits on a newer main). The pack measured,early2(tree d552c919, local), is9c9d69edplus #443'svk_replay.cppchange (82cee56e, off on AMD) plus this branch'sgpu/diff. Both columns are that one pack: "Before" withRX_VK_READBACK_EARLYunset (main's path), "After" with=1. What makes the numbers carry over:gpu/vk/readback_early.his byte-identical to this branch's, and the lines this PR adds and removes invk_replay.cpp,vk_memory.cpp,vk_draw.cppandvk_internal.hare the same, line for line, as the pack's (compared with the line numbers stripped; the pack's only extra lines are #443's). The code around them is not the same: main has about 30 renderer commits since9c9d69ed, some in the same paths (#526 the late pages' fix, #590 unread bookkeeping without mirrors, #769, #588 the render thread), which is why dukenheimer's run of the branch itself (above) is the one on today's code. The rebase onto maine4d035e9left this comparison as it was: the branch's owngpu/lines are the same, line for line, before and after it, and none of the main changes it met touches what an early read waits for: #700 adds a count to the readback line (it counts only reads that waited for their whole submission, so an early read is not in it), #670 says a wait that comes back device-lost as a lost device, and #431's import-table walk, which the early reads' note inmemory_note_gpu_writesits beside, is main's with the switch unset or set.Hardware: Steam Deck OLED, AMD Custom GPU 0932 (RADV VANGOGH), Mesa 26.2.0-devel, SteamOS 3.9.5, kernel 7.2.7-valve2, Zen 2 4C/8T, 16 GB shared, stock GPU clock
Scene: Kokiri Forest outside Link's house beside Saria (about 2,200 draws a frame), output 1280x720, render resolution 50%, the game's upscaler; Hyrule Field at the Lost Woods bridge, output 1280x800, render resolution 100%. Both: quality preset Custom, frame generation off, render thread off (Auto), headless
Method: fps and frame times from the present trace over 45 s standing after settling; GPU and wait figures from the log's periodic lines. Interleaved off, on, on, off after a warm-up, the median of each arm. Warm shader caches copied fresh,
RX_VK_PRECOMPILE=0, nolongshot_env.txt. Probe:RX_VK_READBACK_EARLY=1 RX_VK_LOG_EARLY_CHECK=4 RX_VK_LOG_READBACK_WAITS=1, theearly-read checksummaries summedHow to reproduce: a child Kokiri save;
RX_VK_READBACK_EARLY=1against unset. Probe routes: 30 s standing then 12 walk-and-pan cycles outdoors in Kokiri Forest; teleports Kokiri Forest, Deku Tree, Kokiri Forest, Deku Tree with camera pans at eachRX_VK_READBACK_EARLYunset: packearly2@ d552c919RX_VK_READBACK_EARLY=1: packearly2@ d552c919RX_VK_READBACK_EARLYunset: packearly2@ d552c919RX_VK_READBACK_EARLY=1: packearly2@ d552c919RX_VK_LOG_EARLY_CHECK=4#303's own version missed 29 of 800 and 21 of 1,100 on these two routes. An earlier port of the fork's head (14e83e9c onto
9c9d69ed, on by default on the Deck), in Hyrule Field native on the same Deck: main 15.20 fps, the port 18.47 (+21.5%), the port withRX_VK_LATE_DEFER=017.58, the port withRX_VK_READBACK_EARLY=015.03; its probe found 0 of 4,000 reads missed across the Field, the Lost Woods and the Market. GPU time is the same in every arm: the gain is the CPU no longer waiting, not less GPU work.With the render thread on, none of it.
RX_VK_READBACK_EARLYleaves itself off when the render thread is on (RX_VK_ASYNC=1, Threaded rendering;gpu/vk/readback_early.h: it was not measured with it). On our Deck OLED on 2026-11-16, Kokiri Forest at handheld settings withRX_VK_READBACK_EARLY=1, #906'sRX_VK_MESH_FB_LAYOUT=soaand #1053'sRX_VK_GPU_CLOCK=1all set: 17.8-19.5 fps without the render thread, 14.7-15.1 with it (one run each, not interleaved: an indication, not a measurement of this PR). The Deck's Auto leaves the render thread off.With the other two Deck switches (our pack of this PR, #906 and #1053 on main
9c9d69ed; same Deck OLED, empty shader caches in every run, each arm twice in mirrored order, the median of each; the Field series and Kokiri's two ends also at #1053 (comment)):The clock hold still pays with the early reads on (14.05 -> 19.05): together they take the host's wait on the GPU from 51.8 to under 1 ms a frame. Nothing broke with all three on (exit 0 in all 18 runs).
The exposure check: Hyrule Field and the Lost Woods in turn, the camera turning, a shot at each of 7 stops, main against the pack with the switch on. No frame too bright or too dark, no flash and nothing popping at the screen's edges, but the two runs' camera did not stop on the same frame (a turn timed in game frames lands elsewhere when the frame rate differs), so the shots are not pixel-comparable.
RX_VK_FLASH=1is not in these builds.Flags and settings
Three switches and two diagnostics are added.
RX_VK_READBACK_EARLY, andRX_VK_LATE_DEFERwith it, are on by default where the pools have no mirror; the rest are off.RX_VK_READBACK_EARLY=1on,=0off; never withRX_VK_PARALLEL>= 2 or the render threadRX_VK_READBACK_LATE=newest, GPU at 1600 MHz), where captures stopped waiting for late pages; left off with the render thread, so no gain there; idle with VRAM mirrors (discrete cards but GCN); a desktop RX 580 (GCN: no mirrors by default) +10 to +24% fps in every phase, GPU time the same, the probe clean (HumanBean); untested on other integrated GPUs and on Intel with #443's rule=0is the first thing to try for a one-frame flash or a HUD or object out of place there; the exposure check's result can still turn it back off (Risks)RX_VK_READBACK_EARLY_GAPRX_VK_READBACK_EARLY_RECORDEDRX_VK_READBACK_EARLY)=0for an A/B onlyRX_VK_LATE_DEFERRX_VK_READBACK_EARLYwhere pools have no mirror; off with mirrors orRX_VK_PARALLEL> 1RX_VK_READBACK_LATE=newestin a play build, the captures that waited for late pages went from 3 and 39 of 600 to 0 (dukenheimer); gains nothing aloneRX_VK_LOG_RB_WRITERS=1); release counts only (=2)=1waits and copies, changing timingRX_VK_LOG_EARLY_CHECKTesting
Tested:
1d1263e6's tree, built with the release toolchain), Kokiri Forest at handheld settings unset against=1, interleaved, and the byte probe on a moving Kokiri route; and the branch in a Deck play build at native with frame generation 3x, two interleaved runs each and two probe runs. No device loss, fault or memory kill.mainc167ec67in adev_packs.pypack, main's build, unset and=1twice with the byte probe, over Kokiri Forest, Hyrule Field and the Market, standing and turning (Measurements; #977 (comment)). Unset at main's speed,=1+10 to +24%, the probe 0 changed and 0 lost.gpu/code on main9c9d69ed): the byte probe on two Kokiri routes, the timed pairs in Kokiri Forest (handheld) and Hyrule Field (native), the exposure pair. The engaged lines in every on arm, none in the off arms; exit 0, no device loss in any run.36b0ccbb, main13e90a5e): the fullpython3 -I -B tests/run_all.py(clang++ 23.1.5, the Vulkan and SPIR-V headers found) after the rebase: 627 passed, 23 failed, 26 skipped. To name them, the checks that failed in that run or in other recent full runs here (28) were run again on this head and on main13e90a5e, the same way. 19 fail on both:test_audio_output_devices,test_bug_report_defaults,test_crt_options,test_failed_compile_keeps_objects,test_fsr_uniform_ownership,test_guest_imported,test_import_reach,test_linux_preflight,test_linux_update,test_make_game_cache,test_missing_nup,test_mods_size_table,test_nonuniform_image_arrays,test_parallel_init_order,test_pending_guest_import,test_replay_write_order,test_run_all_changed,test_vk_occlusion_queriesandtest_walk_key. The rest depend on the PC's load, on this branch and on main alike:test_fpx_stickyandtest_recorder_synctimed out in the full run and pass on both;test_bug_reportandtest_hardware_reportpass on both;test_pipeline_cache_save(a timed check) failed once in the re-run here and passes twice alone, andtest_seed_readerfailed once on main and passes here36b0ccbb;RX_VK_ONLY=1 RX_TARGET=windows:longshot_vk.dll, PE32+ x86-64, no warning, no error)1d1263e6(maine4d035e9), with the Vulkan headers found (LONGSHOT_VULKAN_HEADERS; earlier runs here skipped the checks that need them):MAGICBEAN_TOOLING=<Tooling> python3 -I -B tests/run_all.py(clang++ 23.1.5): 610 passed, 16 failed, 40 skipped. All 16 fail on maine4d035e9too, run there with the same headers and clang++:test_bug_report_defaults,test_crt_options,test_fsr_uniform_ownership,test_guest_imported,test_import_reach,test_linux_preflight,test_linux_update,test_make_game_cache,test_missing_nup,test_mods_size_table,test_parallel_init_order,test_pending_guest_import,test_replay_write_order,test_run_all_changed,test_vk_occlusion_queries,test_walk_key.run_all.py --changed forge/main: withCXX=g++242 passed, 8 failed, 5 skipped; withCXX=clang++241 passed, 9 failed, 5 skipped. The 8 are among the 16 above and fail on main with both compilers. The 9th,test_pipeline_cache_save(a timed check, at a load average over 300), failed on main too when run alone there at the same load, passed in the full run and with g++, and passes run directly on this branch; the function it cuts is byte-identical to main's.test_readback_early,test_late_defer,test_early_split_before_draw,test_readback_part,test_late_fix_build,test_capture_wait_log,test_frame_trace_waits,test_parallel_rb_refresh,test_direct_buffer_writes,test_split_before_draw,test_parallel_small_cb,test_split_ub_flush,test_srccut,test_lost_wait_said,test_fsr_image_retirement): all pass with g++ 13.3 and clang++ 23.1.5,--strict.test_import_reach(#431's) fails to compile its fixture on maine4d035e9as here, with both compilers (bg_inflight's type in its stand-ins). The three new checks also passed withSRCCUT_CHECK_NAIVE=1before the rebase (every cut the same as their old cutter's).test_split_before_draw,test_parallel_small_cbandtest_split_ub_flushare main's, unchanged;test_late_deferchecks the rule unset on every GPU;test_early_split_before_drawgets main's part ends with the switch off. Every new path is behindreadback_early_on(),late_defer_on(),rb_writers_on()orearly_check_on().-fsyntax-onlyofvk_replay.cpp,vk_memory.cppandvk_draw.cppfor Linux and Windows (llvm-mingw 20261104), before the rebases onto27a669baande4d035e9(not run again since; the branch's own lines are unchanged): no error on Linux; on Windows the same errors as main's files, none new.Not tested:
36b0ccbb) on the Deck: the Deck runs were of1d1263e6's tree and of the pack; the head differs from1d1263e6only in the probe's hook and a test stand-in. It has run in the game on an RX 580 (HumanBean).RX_VK_READBACK_EARLY=1where parts end only before a plain draw: Intel by default).RX_VK_LATE_DEFER=0on this pack (it was measured on the earlier port only).Must be tested:
RX_VK_READBACK_EARLY=1and with it unset, shots at the doorway, expecting no flash or dark frame and nothing popping at the edges against unset (by hand: compare the shots)RX_VK_READBACK_EARLYunset and once withRX_VK_READBACK_EARLY=1, expecting the same frame rate and picture as main in both (with VRAM mirrors the switch stays idle) and no error in the log (Test Farm: @Beanstalk windows nvidia, Hyrule Field and Kokiri Forest with RX_VK_READBACK_EARLY unset and with it set to 1, no regression against main)RX_VK_READBACK_EARLYunset and with=1, expecting no white shards in either, the same frame rate as main unset, and with it on the log'sreadbacks waited for their writer's partline (Test Farm: @Beanstalk windows intel, Kokiri Forest moving with RX_VK_READBACK_EARLY unset and with it set to 1, look for white shards and compare fps with main)Risks
What could break: only with
RX_VK_READBACK_EARLY=1:RX_VK_LATE_DEFER, which follows the switch, a deferred late page is copied from guest memory at its first CPU read rather than at the take, so a game write made after the submit and before that read lands in the replay's "as submitted" copy: a longer window than the old wait's (design rule 2, "nothing the replay reads comes from live guest memory"). The two mechanisms also come as one switch: on the earlier port the early reads alone gave 17.58 fps and both 18.47 (from the same reading and the earlier port's runs). A Deck OLED tester of the fork found the deferral cost about 2 fps where the replay thread is the limit (internal 768x432 with their ownRX_VK_READBACK_LATErule); this PR's handheld runs (640x360 internal) gained with both.invalidate's comment: they "keep the wait for all of its submission, whose later parts read and write guest memory itself"), and why main's own part wait, for a mirror's page, is refused when a later part reads guest memory in place (in_place_after); the early wait has no such test. The job may then write memory a later part still reads: the next frame's uniforms or vertices, or the exposure meter (the pattern of issue 997). The byte probe cannot see this:early_check_readcompares only the bytes of the range the CPU reads against what main's wait returns (and a checked read itself waits as main's would), so its 0 missed reads clear the risk above, not this one. The evidence that can clear it is a same-frame exposure check: the same frames with the switch unset and on, compared pixel for pixel. The one here was not pixel-comparable (the camera did not stop on the same frames), so dyst's review (#977 (comment)) asked for no default before such a check. It is on by default anyway (468f360d, #977 (comment)): the gain is large where it acts, nothing has been seen by eye or probe, and a correct game guards that memory with its own fences. TheRX_VK_LOG_INPLACE_EXPOSUREruns continue; a higher rate after an early release turns the default back off, or holds what they point at, before the next release.With mirrors (discrete cards) and the switch unset, or with
=0anywhere, nothing changes: the tracking, the splits, the descriptor rebuilds and the recording tags are behind the switches.Turning it off:
RX_VK_READBACK_EARLY=0;RX_VK_LATE_DEFER=0alone for the late pages.Compatibility: Nothing changes: no shader translator file (the shader caches stay valid),
MKWVK_API_VERSIONunchanged (vk_api.huntouched), no option, ini key or menu row.Works with / conflicts with
memory_read_as_submittedintosubmitted_read_unlockedandsubmitted_read_lockedand addsmemory_read_as_submitted_many, the batched reader of a draw's uniform copies (on by default; on Intel iGPUs without mirrors, where the uniform snapshotub_snapshot_onis on, every uniform range is read through it). This PR'slate_defer_cpu_readcall sits in the per-page loop that moves intosubmitted_read_locked, so the two conflict invk_memory.cpp. Whichever lands second moves the call intosubmitted_read_locked, which both readers go through, and makestests/test_late_defer.pylook for it there; otherwise a batched read would take a deferred late page without copying it again. If #952 lands first, this PR does it.Checklist
Everyone
(The early reads with the two things they need:
RX_VK_LATE_DEFERgains nothing on its own and exists because the early reads make the late pages the replay would otherwise wait ~19 ms a frame for; it follows the switch and can be turned off alone. The diagnostics are the probe that measured it. Our review of the fork's PR asked for the deferral as its own PR; the authors' commits for it are mainly the first three and could be split out if the maintainers prefer)type(scope): description) of the status line's type.main: at review onto26ce01b5, commit by commit with no conflict, head468f360d(the default on top). Before that: on13e90a5e, head36b0ccbb; it merges with currentmain71b1edd8without a conflict (git merge-tree). The rebase frome4d035e9had one conflict, invk_memory.cpp'smemory_collect_writtenwith #1083'scollect_mu, resolved in6cf7a3d1(the probe's hook, What changed), and added36b0ccbb(a test stand-in). The earlier rebase ontoe4d035e9kept #700's whole-submission count ininvalidate(52160308,6d10a356) and #431's walk comment inmemory_note_gpu_write(fda1d573,6cf7a3d1), each named in that commit's[port: ...]line. Old head1d1263e6→ new36b0ccbb; the hashes in this description are the new ones.python -I -B tests/run_all.pypasses (any failure that also fails onmainis named). Run at36b0ccbb;468f360dwas not run (a code-only review):tests/test_late_defer.pyandtests/test_readback_early.pyare the checks it changes. (Testing: the fullpython3 -I -B tests/run_all.py(clang++ 23.1.5, the Vulkan and SPIR-V headers found) after the rebase: 627 passed, 23 failed, 26 skipped. To name them, the checks that failed in that run or in other recent full runs here (28) were run again on this head and on main13e90a5e, the same way. 19 fail on both:test_audio_output_devices,test_bug_report_defaults,test_crt_options,test_failed_compile_keeps_objects,test_fsr_uniform_ownership,test_guest_imported,test_import_reach,test_linux_preflight,test_linux_update,test_make_game_cache,test_missing_nup,test_mods_size_table,test_nonuniform_image_arrays,test_parallel_init_order,test_pending_guest_import,test_replay_write_order,test_run_all_changed,test_vk_occlusion_queriesandtest_walk_key. The rest depend on the PC's load, on this branch and on main alike:test_fpx_stickyandtest_recorder_synctimed out in the full run and pass on both;test_bug_reportandtest_hardware_reportpass on both;test_pipeline_cache_save(a timed check) failed once in the re-run here and passes twice alone, andtest_seed_readerfailed once on main and passes here)third_party/untouched (a new folder with aPROVENANCE.mdinstead).When it applies
Assisted-by:/Co-Authored-By:trailers.=1,=1, unset after a warm-up, the median of each arm, at handheld settings and in a play build; the pack: ours, the same way. The render-thread pair is one run each and is labelled as an indication; HumanBean's RX 580 run is one session per arm, not interleaved, with=1twice, and is labelled so)docs/tested/for each new measured run (format indocs/tested-configurations.md). (Ours:e161d7e6. dukenheimer's and HumanBean's runs are cited from their comments, as theirs)mainand passing here. (tests/test_early_split_before_draw.py: it fails with the fork's end right after the writer on Intel)tests/that reads changed source updated in this PR, not weakened.36b0ccbb;RX_VK_ONLY=1 RX_TARGET=windows:longshot_vk.dll, PE32+ x86-64, no warning, no error); Linux: dukenheimer's release-toolchain build; no path code changed)Tested in game: Steam Deck OLED (SteamOS 3.9.5, RADV, Mesa 26.2.0-devel), this branch's gpu/ code as a renderer pack on an older main: Kokiri Forest at handheld settings and Hyrule Field at 1280x800, switch off and on, interleaved; a byte probe on a Kokiri walking route and Kokiri to the Deku Tree and back: 4,200 early reads checked, none different from main's; +21.8% and +23.3% fps, no flash or dark frame seen.
Hi @sheep and @dukenheimer,
This PR is your early reads, brought over to Longshot: sheep's work from sheep/BunnyHood#2, built on dukenheimer's #303. Every commit of yours is here with you as its author. We only added three small ones on top: it starts switched off (
RX_VK_READBACK_EARLY=1turns it on), it stays off with the render thread, and it keeps to the rule Longshot added for Intel graphics in #443.How it did on our Steam Deck OLED:
It's yours to take over, whichever way you prefer: we can close this one and you open your own (we're happy to hand over the numbers and how we measured them), you send commits for this branch, or we keep carrying it with your names on it. No rush either way.
Thank you both for the careful work. It's the biggest Steam Deck gain we know of.
zeldaootr
Thanks @zeldaootr and @sheep, please carry this forward; I won't take it over. Your #443 part-end rule closes exactly the gap our last probe on #303 found (writes later in the same submission), and 0 differences in 4,200 checked reads with +22-23% on the Deck is the result #303 was missing. I'm closing #303 in favour of this one. Happy to run a Deck OLED A/B or the byte probe on a newer head if that helps the review.
Thanks @dukenheimer, will do. And yes please to the Deck OLED run: the one thing this PR still lacks is a run of the branch itself, since our numbers come from its
gpu/change on an older main. An A/B of the current head (148f14a0) withRX_VK_READBACK_EARLY=1against unset, plus your byte probe (RX_VK_LOG_EARLY_CHECK=4), would let it leave WIP. One more data point from tonight, same Deck, Kokiri at handheld settings: early reads together with the record layout from #906 gave 11.47 to 14.31 fps (+24.8%), against +24.0% for early reads alone; nothing broke with both on.zeldaootr referenced this pull request2026-11-16 05:04:11 +00:00
148f14a09f1d1263e6beRebased onto main
e4d035e9(head1d1263e6, was148f14a0): four conflicts, all where main's #700 added the whole-submission count to the readback line (invalidate) and #431 reworded the import walk's comment inmemory_note_gpu_write; each was resolved in the commit it arose in, keeping both sides, and named in that commit's[port: ...]line, and the branch's owngpu/lines are unchanged line for line, so the Deck numbers still describe this code. One commit of ours on top givestests/fsr_image_retirement_test.cppa stand-in forrec_close, whichfinish_recordingcalls since the early reads (the check needs the Vulkan headers and had been skipped here). Re-run with the Vulkan headers: the port's and the pinning checks with g++ and clang++ (all pass),run_all.py --changed forge/mainwith both, and the full suite (610 passed, 16 failed, 40 skipped), every failure also failing on main.@dukenheimer the head is now
1d1263e6(rebased onto current main; the early reads' owngpu/change is line for line what it was). If you do run it on your Deck, that is the commit to use.Deck OLED: this branch on rc.5 with your recipe, and in a Deck play build
In short: with your recipe, fps went from 15.70 to 20.73 (+32%), and the byte probe found 0 missed of 1,100 reads. In the owner's play build at his settings (frame generation 3x, render 100%, GPU at 1600 MHz) there was no measurable fps change on a moving route, and 0 missed of 2,200 probed reads.
Hardware: Steam Deck OLED, AMD Custom GPU 0932 (RADV VANGOGH), Mesa 26.2.0-devel, SteamOS 3.9.5, 16 GB. Docked, in Game Mode through Steam (not headless).
Builds: Linux, the release toolchain (LLVM 23.1.5, glibc 2.35),
x86-64-v3with the play profile.Code: v3.0.0-rc.5
e4d035e9+ this PR. I rebased148f14a0onto rc.5 before your rebase and hit the same two conflicts (#700's count ininvalidate, #431's comment inmemory_note_gpu_write); I kept both sides. The tree built matches1d1263e6in every built file:git diffagainst it shows only yourrec_closestand-in intests/fsr_image_retirement_test.cpp.1. This branch alone, your recipe
Settings, as your run02: output 1280x720, render resolution 50%, the game's own upscaler, quality preset Custom, frame generation off, threaded rendering Auto, stock clocks. Also Profile Docked, so the output and render rows apply as set, and HDR off, since Game Mode would give this display HDR10. The remaining rows are the owner's (anisotropic 2x, the detail rows at 100%, contact shadows off).
Method: warm shader caches copied fresh for each start (taken after the warm-up),
RX_VK_PRECOMPILE=0, nolongshot_env.txt. Kokiri Forest beside Saria, 20 s to settle, then 45 s standing. A warm-up first, then unset,=1,=1, unset.=1=1Median fps: unset 15.70,
=120.73 (+32.1%). GPU time is the same in both arms; the host's GPU waits drop from 32 to 6 ms a frame. fps and frame times come from the presents; the other rows are medians of the log's periodic lines over each 45 s window.Byte probe (
=1,RX_VK_LOG_EARLY_CHECK=4,RX_VK_LOG_READBACK_WAITS=1; 30 s standing, then our harness's moving Kokiri route for 88 s): 1,100 reads checked (1,090 skipped), 0 missed, 0 ambiguous, 0 lost. After main's release, all 1,100 were unchanged.A plain start came first (K0: without
RX_VK_PRECOMPILE=0, after--prepare-shaders, the same settings, the moving Kokiri route). It reached play at 14.35 fps (p95 119 ms), with RSS at most 6.6 GB and MemAvailable at least 1.4 GB: no memory kill.2. In a Deck play build: rc.5 + #225 + #227 + #222 + this PR
Settings (the owner's): Profile Docked, output 1280x800, the game's own upscaler at 100%, frame generation 3x, HDR native (HDR10 output in Game Mode).
longshot_env.txtsetsRX_VK_READBACK_LATE=newest. GPU fixed at 1600 MHz.Route: the moving Kokiri route, an 80 s window. Warm-up first, then unset,
=1,=1, unset.=1=1a capture's wait for write-back ... timed out(its 250 ms cap), lines in the whole runMedian game fps: unset 15.9,
=116.25. The two unset runs (14.7 and 17.1) bracket both=1runs, so there is no measurable change at these settings: the moving route varies more than the arms differ. The waits did change: captures stopped waiting for late pages, and the readbacks' waits were shorter.Byte probe on this build (
=1, the same settings):RX_VK_LOG_EARLY_CHECK=4)RX_VK_LOG_EARLY_CHECK=1)At
RX_VK_LOG_EARLY_CHECK=4the moving route completed fewer than 100 checks, so it printed no summary; I ran it again atRX_VK_LOG_EARLY_CHECK=1.No device loss, device fault, stalled GPU or memory kill in any run. The 250 ms capture-wait cap was also reached once in the warm-up (unset) and once in each probe run.
Suite
Linux container (clang 23.1.5): rc.5 608 passed / 22 failed. rc.5 + this PR (
148f14a0rebased, before the stand-in) 610 / 23; the one more istest_fsr_image_retirement. With arec_closestand-in, as in1d1263e6, it passes (28 of 28).1d1263e6be36b0ccbb35Rebased onto main
13e90a5e(head36b0ccbb); it also merges cleanly with main8a4aa282. The one conflict was invk_memory.cppwith thecollect_muof #1083. The early check's collection number is now taken inside main's lock, together with the marks, and its page report runs just after the lock is released, so it adds no lock undercollect_mu; the capture's report follows main's newcapture_scan. Both calls do nothing unlessRX_VK_LOG_EARLY_CHECKis set. One new test-only commit stubs the early check's hooks intests/test_vk_collect_window.py, which cuts that function out of the source. Every other commit is unchanged bygit range-diff, with its author as it was.tests/run_all.pyon Linux: 627 passed, 23 failed, 26 skipped; 21 of the failures also fail on main13e90a5e, andtest_fpx_stickyandtest_recorder_synctimed out only in the loaded full run and pass alone.Careful port, with the authorship trail kept clear. Everything new sits behind
readback_early_on(),late_defer_on()or the diagnostics. Reading the unset path (take_capture's restructured wait, the replay loop,same_descindraw()), it does what main does.RX_VK_READBACK_LATE=newest, GPU at 1600 MHz, no measurable fps change, fewer capture waits. It bounds where the gain shows. The head and base hashes (1d1263e6one4d035e9) are also two rebases old.vk_memory.cpp. #952 splitsmemory_read_as_submittedintosubmitted_read_unlockedandsubmitted_read_lockedand adds a batched reader. Whichever lands second should movelate_defer_cpu_readintosubmitted_read_lockedand updatetest_late_defer.py. Otherwise the batched uniform reads on Intel iGPUs would read a deferred late page without copying it again.This is a code read only: nothing was built or run.
🤖 Generated with Claude Code
@ -1396,6 +1403,7 @@ bool memory_read_as_submitted(uint64_t gpu, void* dst, uint64_t n) {auto b = cap_bufs.find(e.gen);if (b == cap_bufs.end() || !b->second.map) continue;late_fix_cpu_read((uint32_t)page, b->second.buffer, (uint8_t*)b->second.map + (uint64_t)e.slot * 4096);if (late_defer_any()) late_defer_cpu_read((uint32_t)page, e.gen, e.slot, b->second.alloc, (uint8_t*)b->second.map);Heads-up: #952 moves this per-page loop into
submitted_read_locked(), shared bymemory_read_as_submittedand a newmemory_read_as_submitted_many, and the two branches conflict here. Whichever lands second should put this hook there, andtests/test_late_defer.pyshould look for it there, so the batched uniform reads also copy a deferred late page again.@ -3497,0 +4049,4 @@// (only a read with a page written in place pending: with VRAM mirrors (a dGPU) a page is written back by part already,// and nothing is noted, split or rebuilt for it: on an RTX 5090 the switch cost ~2 fps doing that and waited early 0 times.// A sparse mirror's block with no memory yet is written in place for a submission: those pages take the early wait)if (pending && !late && readback_early_on() && unforced && by_part && whole > completed()) {Here the game's job is let go while later parts of
wholestill read guest memory in place: the case main's comment onby_partgives as the reason pages written in place wait for the whole submission.RX_VK_LOG_EARLY_CHECKchecks only the read range against main's wait, so it can't catch the job writing memory a later part reads. A same-frame exposure check is the evidence this branch still needs before any default.Description brought up to the thread (dyst's review). dukenheimer's run of this branch now leads Measurements: +32.1% on the Kokiri handheld recipe and 0 of 1,100 probed reads missed. Their play-build run showed no measurable fps change and fewer capture waits. The hashes are this head's (
36b0ccbbon13e90a5e; it merges cleanly with71b1edd8). Risks now says plainly that the byte probe can't see the game, let go early, writing memory a later part still reads, and that a same-frame exposure check comes before any Deck default. Also added: the conflict with #952 and who moves the late-page hook. The early reads leave themselves off with the render thread: Kokiri handheld ran 17.8-19.5 fps without it and 14.7-15.1 with it, one run each. The three-switch series is in too. The Numbers box is ticked. No code changed.Thanks @dyst, all three hold, and the description now says so.
Behind the thread: fixed. dukenheimer's run of this branch now leads Measurements: Kokiri at handheld settings 15.70 to 20.73 fps (+32.1%), 0 of 1,100 probed reads missed. Their play-build run is there too (frame generation 3x,
RX_VK_READBACK_LATE=newest, GPU at 1600 MHz: no measurable fps change, the captures' waits for late pages 3 and 39 of 600 down to 0). It is also in the flag table's row, as the bound on where the gain shows. The Verdict, Tested / Not tested and the Numbers box follow from it. The hashes are this head's:36b0ccbbon13e90a5e, which merges cleanly with71b1edd8.git range-diffagainst their1d1263e6shows only the probe's hook and one test stand-in changed, so their fps numbers hold for this head.What the probe can't see: agreed, and checked in the code.
invalidate's comment gives the later parts reading and writing guest memory as the reason pages written in place wait for the whole submission. Main's own part wait for a mirror's page is refused when a later part reads in place (in_place_after). The early branch waits formax(writer, part)with no such test.early_check_readcopies only[gpu, gpu + size)before and after main's wait, and the checked read itself then waits as main's would. So its 0 missed reads clear risk 1 and say nothing about the game's job, let go early, writing memory a later part still reads. Risks now says that plainly. It also says that a same-frame exposure check is the evidence needed before any Deck default; our exposure pair wasn't pixel-comparable. The flag table's recommendation says the same.#952: agreed. In its split the per-page loop lives in
submitted_read_locked, which bothmemory_read_as_submittedandmemory_read_as_submitted_manygo through. The latter isub_reads_flush's batched read of a draw's uniform copies. It's on by default, and on Intel iGPUs without mirrors (the uniform snapshot,ub_snapshot_on) every uniform range goes through it, whileRX_VK_LATE_DEFERfollows the early reads there. So whichever lands second must move the hook. If #952 lands first, we'll movelate_defer_cpu_readintosubmitted_read_lockedhere. We'll also maketests/test_late_defer.py's source check look for it there. That check now cutsmemory_read_as_submittedand asks that it takes the late copy again before copying from it. The conflict and who does what are under Risks, "Works with / conflicts with".One more thing in the update: the early reads leave themselves off with the render thread (
RX_VK_ASYNC=1). With the clock hold and the record layout on, Kokiri at handheld settings ran 17.8-19.5 fps without the render thread and 14.7-15.1 with it (one run each), so Threaded rendering forgoes the gain. That's now in Measurements and Not tested.WIP: perf(vk): early reads: a read of GPU results waits for its writer's part, not its whole submission (RX_VK_READBACK_EARLY, off by default)to perf(vk): early reads: a read of GPU results waits for its writer's part, not its whole submission (RX_VK_READBACK_EARLY, off by default)theguyser referenced this pull request2026-11-16 13:51:42 +00:00
Tested on a desktop GCN card. The description has a Steam Deck only.
Hardware: RX 580 8 GB (Polaris10), RADV, 8 threads, KDE.
Build: this PR's head merged with
mainc167ec67, as a dev_packs pack on a base of that main.Route: Kokiri Forest, Hyrule Field, the Market (debug teleport), Hyrule Field again; standing, then turning. One session per arm.
Arms: E0 is this PR with
RX_VK_READBACK_EARLYunset. E1 is=1, with the byte probeRX_VK_LOG_EARLY_CHECK=4, run twice. M is main's own build.early -> main's release: 100 unchanged, 0 changed on a page the CPU did not ... 0 lostin both runs.A related result from the same lab, for whoever looks at GCN next. On plain main,
RX_VK_MIRROR=1gave +40 to +55% fps on this card: Kokiri 20.9 to 29.6, Hyrule Field 27.5 to 42.3, the Market standing 24.2 to 31.6.mirror_on()keeps the mirrors off on GCN because of an older RX 580 measurement. That's a separate change, and I haven't run it with this PR's switch together.🤖 Generated with Claude Code
Description updated, no code change (head still
36b0ccbb): HumanBean's RX 580 run (#977 (comment)) is in the Verdict, Measurements, Testing and the switch's row. On a desktop GCN card, whose pools have no VRAM mirror by default, this head gave fps 10-24% higher in every phase with=1, GPU time the same, the byte probe 0 changed, and unset at main's speed. It is the first run of this head in the game, and the first off the Deck. Must be tested is unchanged: the Deck's same-frame exposure check, Windows with NVIDIA and Windows with Intel all remain.Thanks @HumanBean. This is the first run of this head in the game and the first off the Deck: +10-24% in every phase on the RX 580 with GPU time unchanged, the byte probe clean in both runs, and unset at main's speed. It's in the description now (Verdict, Measurements, Testing and the switch's row), labelled as one session per arm. It fits the rule: main keeps the VRAM mirrors off on GCN, so the early reads act there as they do on the Deck. It doesn't clear anything under Must be tested, though. The Deck's same-frame exposure check, Windows with NVIDIA and Windows with Intel are all still open.
Thanks for the
RX_VK_MIRROR=1result too. +40-55% on plain main is a big number, and worth a look of its own. The rule that keeps mirrors off on GCN came from an older RX 580 measurement. It predates the sparse mirrors (#250) that your main already had, so that measurement may no longer hold. It's a separate change, and we haven't measured it. Note that with mirrors on, this PR's early reads stand aside (they act only where the pools have no mirror), so the two gains won't add up on that card.Suggest turning this on by default where it measured a gain, rather than leaving it opt-in:
RX_VK_READBACK_EARLYon for AMD/RADV, at least wherever the VRAM mirrors are off (integrated GPUs, the Deck, GCN).The evidence:
100 unchanged, 0 changed ... 0 lostin both sessions.Off by default, the gain reaches only players who know the switch. Keep
RX_VK_READBACK_EARLY=0as the way back.🤖 Generated with Claude Code
Thanks @HumanBean. We want this on by default on the Deck too, and we are not there yet, for the reason in @dyst's review: the byte probe checks the range the CPU reads, and cannot see the game, let go early, writing memory that a later part of the same submission still reads in place. On GPUs without mirrors that read happens when the GPU runs the part, so the window is real by construction; main has had a bug of this kind before (the stretched HUD and white flashes of issue 997). Whether this game writes in that window is what we do not know.
That check is running on a Steam Deck today: a log-only diagnostic (
RX_VK_LOG_INPLACE_EXPOSURE, on a branch on top of this one) samples parts, copies what they bind in place before the submit, and compares after the GPU has finished, with the early reads off and on. If the rate after an early release is no higher than without one, we will send the default for AMD without mirrors as a follow-up with that evidence; if it is higher, the addresses say what to snapshot or hold. Results will be posted here either way.One more input for the default: the early reads leave themselves off with the render thread, so the default would only reach players whose Threaded rendering is off (Auto leaves it off on the Deck's 8 threads).
A quick check on an RTX 3090 (NVIDIA 620.70.06, Linux), and a correction to my default-on suggestion above.
Setup: this PR rebased onto
main@43f40b87(no conflicts). One session each ofRX_VK_READBACK_EARLYunset and=1withRX_VK_LOG_EARLY_CHECK=4, both withRX_VK_ASYNC=0, since early reads never run with the render thread and Auto turns it on with 24 threads. DLAA 2560x1440, Hyrule Field then the Market.On a discrete card with VRAM mirrors it doesn't act:
=1, every line readsreadbacks waited for their writer's part (RX_VK_READBACK_EARLY): 0, 0.00 ms each.Correction to "on for AMD/RADV": its gain lives where the pools have no mirrors. #1187 turns the VRAM mirrors on for discrete GCN cards: on the same RX 580 they gave +28 to +54%, against this PR's +10 to +24%. With #1187 in, GCN no longer takes this path either. So the right default is on for integrated GPUs, where mirrors stay off by design: the Steam Deck (your +22 to 32%) and AMD APUs. Discrete cards don't need it.
🤖 Generated with Claude Code
We're turning
RX_VK_READBACK_EARLYon by default in this PR rather than in a follow-up. I'll push it on top of a rebase, and the review follows.Why now:
RX_VK_PARALLEL>= 2.RX_VK_READBACK_EARLY=0is the way back.What we're accepting: the exposure check, the risk dyst raised, isn't done. If the game writes memory that a later part of the submission still reads in place, it would show as an occasional one-frame glitch: a flash, or the HUD or an object out of place. A correct game guards that memory with its own fences, and the byte probe and the runs so far saw nothing. So we'd rather ship the gain and name the switch as the first thing to try for that kind of report.
Please keep the
RX_VK_LOG_INPLACE_EXPOSUREruns going and post them here. If they show a higher rate after an early release, we turn the default back off, or hold what they point at, before the next release.🤖 Generated with Claude Code
36b0ccbb35468f360d63perf(vk): early reads: a read of GPU results waits for its writer's part, not its whole submission (RX_VK_READBACK_EARLY, off by default)to perf(vk): early reads: a read of GPU results waits for its writer's part, not its whole submission (RX_VK_READBACK_EARLY, on without VRAM mirrors)I read it as code only and ran nothing. I've pushed a rebase and a commit that turns the early reads on by default where the pools have no VRAM mirror, as decided in #977 (comment). dyst's three points still stand, and I haven't repeated them.
Pushed (head
468f360d):main26ce01b5, with no conflict, and their authors kept.468f360d, the default:readback_early::Factstakesmirrors, frommirror_on(). Unset now givesON_DEFAULTwithout mirrors andOFF_MIRRORSwith them.=0is off everywhere, and=1is on with mirrors too.RX_VK_PARALLEL>= 2 still leave it off. They are now asked whether the switch is set or not:async_on()is decided at start, andRX_VK_PARALLELdefaults to 0.RX_VK_LATE_DEFERfollows it unchanged.mirror_on()is declared in the anonymous namespace beforereadback_early_on, where it is defined.tests/test_late_defer.py's rule checks andtests/test_readback_early.py's runs cover unset with and without mirrors,=0, and=1with mirrors.invalidate's "(opt-in)" comment, the title and the description.What I checked in the code:
invalidate's early branch only runs with a page written in place still pending (whole > completed()), so on a mirrored card it does nothing, as the RTX 3090 run showed. Where no writer is known, or the recording wait passes 250 ms, it falls back to main's whole-submission wait.early_recorded_wait: if the writer's submission hasn't started recording, it returns at once. That is safe, because no writer is known yet, so the read waits for the whole submission.readback_early_on(),late_defer_on()and the diagnostics. With=0, or with mirrors and the switch unset, the path is main's, as dyst read it.What the default now ships with, from your own Risks section, accepted in the decision comment:
in_place_afterguard, which main's own per-part wait has.RX_VK_LATE_DEFERgives it a longer window than the old wait did.If any of these shows up, it would be a one-frame glitch on an integrated GPU, and the doc row names
RX_VK_READBACK_EARLY=0as the first thing to try.Who gets it, as far as I can tell from the code: the Steam Deck (Auto leaves the render thread off on its 8 threads), AMD APUs and Intel iGPUs with the render thread off. An APU or Intel laptop with enough threads for Auto to turn the render thread on gets nothing, because the early reads stand aside there.
Still missing:
f2e73429(a writer's part running on to the next plain draw, #443's rule) has never run in the game. A Windows Intel run, unset against=0, is the most useful next test. The Test Farm could do it.RX_VK_LOG_INPLACE_EXPOSURE): please post it here when it's done. That is why this review is a comment, not an approval.The checks were not run on
468f360d. The two changed checks are listed above.🤖 Generated with Claude Code
@ -0,0 +36,4 @@inline Why choose(const Facts& f) {if (f.env && env_value_off(f.env)) return OFF_SWITCH; // (an empty value is off too: env_on's reading)if (parallel(f.parallel)) return OFF_PARALLEL;Default added here (
468f360d): unset is on where the pools have no mirror and off where they have one, and the render thread and the parallel replay are now asked whether the switch is set or not.The exposure check has run on a Steam Deck, as asked. It does not clear the risk: parts recorded after an early release saw CPU-side changes to memory they bind in place about 3.6 times as often as with main's wait. It does not yet show a fault either, because the check cannot say whose writes those are. Details, and what we are doing next.
Setup: Steam Deck OLED (RADV), headless, Kokiri Forest at output 1280x720 with 50% render, GPU at a manual 1600 MHz. The build is an older main (
9c9d69ed) with this PR'sgpu/code, thesoalayout and the log-only diagnosticRX_VK_LOG_INPLACE_EXPOSURE=8: for one part in eight it notes the pages bound in place while the part is recorded, copies them before the part's submit, waits for the part on the GPU and compares, leaving out pages the GPU itself wrote. Two runs an arm, order 1 2 2 1, then the third arm twice.RX_VK_UB_SNAPSHOT=1, parts after an early releaseRX_VK_UB_SNAPSHOT=1, the other partsThe two runs of each arm agree (18.2 and 20.4%; 71.7 and 67.3%; 45.8 and 46.9%), and nothing was lost or unreadable.
What it shows:
What it does not show:
Next, today: the diagnostic gets its address lines kept apart for parts after an early release and matched against the readback ranges, so each change is classed as the replay's own write-back, a uniform page, or something else; then the same three arms again. If "something else" is there, the addresses say what to snapshot or hold. We will also take frame pairs with the switch on and off in a moving scene and look for one-frame faults.
Our reading for the default: nothing here says to turn it off today, and nothing here says the risk is cleared.
RX_VK_READBACK_EARLY=0remains the first thing to try for a report of a one-frame flash or an object out of place on a GPU without mirrors.Tested by an AI agent (Claude Code) for zeldaootr.
A correction to my comment above: "reported no glitches" says more than we know. The Deck's owner played about five minutes in Kokiri Forest with early reads on and did not mention any glitch; he was not asked to look for one-frame faults. Treat it as no evidence either way.