perf(vk): early reads: a read of GPU results waits for its writer's part, not its whole submission (RX_VK_READBACK_EARLY, on without VRAM mirrors) #977

Merged
HumanBean merged 26 commits from zeldaootr/Longshot:early-reads-port into main 2026-11-16 17:36:46 +00:00
Contributor

Type: perf · Risk: low · Tested on: Steam Deck OLED (SteamOS 3.9.5, RADV): this branch on v3.0.0-rc.5 (dukenheimer) and its gpu/ code on an older main (ours); RX 580 (Linux, RADV; GCN, so no VRAM mirrors): this head on main c167ec67 (HumanBean); Linux, game-free checks; the renderer built for Windows (llvm-mingw) · Needs: Windows, NVIDIA, Intel (switch unset and on); a same-frame exposure check on the Deck before any default

Why

On the Steam Deck the game keeps waiting for GPU results the GPU has already computed. Its jobs read the culling camera block every frame, and the renderer makes that read wait for the whole 22-26 ms batch of GPU work it came in, not for the part that wrote it, so the GPU sits idle while the game waits. Letting the read go once its writer's part has run took Kokiri Forest at handheld settings from 11.40 to 13.89 fps (+21.8%) and Hyrule Field at 1280x800 from 14.98 to 18.47 fps (+23.3%) on a Deck OLED, with a byte-for-byte probe finding none of 4,200 such reads different from what main's wait returns. dukenheimer's run of this branch itself on a Deck OLED gave 15.70 to 20.73 fps (+32.1%) with the same recipe, and 0 of 1,100 probed reads missed. This PR brings it to Longshot on by default where the buffer pools have no VRAM mirror (integrated GPUs: the Steam Deck, AMD APUs, Intel), with RX_VK_READBACK_EARLY=0 as the way back. It was first ported off by default; HumanBean turned it on at review (468f360d; the decision: #977 (comment)), with the exposure check still to come (Risks).

The design and nearly all of the code are sheep's and dukenheimer's. sheep built it in sheep/BunnyHood#2, on dukenheimer's #303, which started the early reads and wrote the byte probe used here. This PR only exists to bring their work to Longshot with their authorship kept: their 18 commits are cherry-picked one by one (cherry-pick -x, authors unchanged, a [port: ...] line where a commit needed more than the fork's names mapped back to Longshot's). If either of them would rather open it themselves, this PR will be closed in favour of theirs.

The tracker was searched ("readback early", "early read", "late defer", READBACK_EARLY): no issue, and no open PR besides #303, which dukenheimer has since closed in favour of this one.

What changed

By default on GPUs whose pools have no VRAM mirror (the Steam Deck, integrated GPUs), a CPU read of GPU results written in place waits for the part of the submission its last writer was recorded in, once that submission is recorded to its end, instead of for the whole submission. The capture's late pages are then left to the CPU reads that need them (RX_VK_LATE_DEFER, which follows the switch) instead of holding up the replay. Discrete cards (mirrors) stay as on main unless =1 is set, and =0 is main everywhere.

  • vk, the early reads (dukenheimer, sheep): gpu/vk/vk_memory.cpp notes each in-place writer's part (early_in_place_write, memory_in_place_writer_part) for ranges whose reads waited; gpu/vk/vk_replay.cpp invalidate waits for that part, after early_recorded_wait (the writer's submission recorded to its end, at most 250 ms), and replay() ends a part after such a writer (RX_VK_READBACK_EARLY_GAP draws into it). gpu/vk/vk_draw.cpp: a program that writes storage buffers rebuilds its descriptors once the tracking is in use, so each write is noted.
  • vk, the late pages left to their reads (sheep): gpu/vk/vk_memory.cpp, late_defer_split and late_defer_cpu_read from memory_read_as_submitted; a late page in a texture's own storage still waits at the take.
  • vk, diagnostics (dukenheimer, sheep): RX_VK_LOG_RB_WRITERS and the byte probe RX_VK_LOG_EARLY_CHECK, log only.
  • Our commits:
    • f2e73429 (#443's rule): main's RX_VK_SPLIT_BEFORE_DRAW (on by default on Intel) lets a part end only right before a plain draw. The fork ended a writer's part right after the writer, which can start the next part with a dispatch or a mesh draw. early_split now asks part_ends_before_draw as maybe_split does; where it holds, the writer's part runs on to the next plain draw. Elsewhere, the Deck included, nothing changes. tests/test_early_split_before_draw.py checks it.
    • 468f360d (HumanBean, at review): on by default where the pools have no mirror. readback_early.h's rule takes the pools' mirror (Facts::mirrors, from mirror_on()): unset is ON_DEFAULT without mirrors and OFF_MIRRORS with them; =0 is off and =1 on everywhere; the render thread and RX_VK_PARALLEL >= 2 still leave it off, now asked or not. RX_VK_LATE_DEFER follows it as before. tests/test_late_defer.py's rule checks and tests/test_readback_early.py's runs (unset with and without mirrors, =0, =1 with mirrors) follow, and the doc rows say so.
    • 918372f3 (off by default, before 468f360d): gpu/vk/readback_early.h keeps it off on every GPU unless asked for (the fork has it on for the Deck), and off even when asked for with RX_VK_PARALLEL >= 2 or the render thread.
    • 2ddd90fa (docs): the switches' rows in docs/development.md.
    • The rebase onto 13e90a5e met #1083's collect_mu in memory_collect_written: in 6cf7a3d1 (the byte probe's commit) the probe's collection number is taken inside main's lock, with the marks, and its page report runs just after the lock is released, so the probe adds no lock under collect_mu; the capture's report follows main's new capture_scan. Both do nothing unless RX_VK_LOG_EARLY_CHECK is set.
  • tests: tests/test_readback_early.py, tests/test_late_defer.py and tests/test_early_split_before_draw.py are new and cut with tests/srccut.py (3eb400b7); tests/test_readback_part.py, tests/test_late_fix_build.py, tests/test_capture_wait_log.py, tests/test_frame_trace_waits.py, tests/parallel_rb_refresh_test.cpp and tests/direct_buffer_writes_test.cpp follow the code, and tests/fsr_image_retirement_test.cpp gets a stand-in for rec_close, which finish_recording now calls (1d95536f: the check needs the Vulkan headers and had been skipped here without them); tests/test_vk_collect_window.py, which cuts memory_collect_written out of the source, gets stand-ins for the probe's two hooks (36b0ccbb).
  • docs: three runs in docs/tested/ (e161d7e6).

Measurements

Verdict: it helps on the Steam Deck OLED where the game waits on its reads, and shows no measurable change where it doesn't. This branch itself (dukenheimer, on v3.0.0-rc.5): Kokiri Forest at handheld settings 15.70 -> 20.73 fps (+32.1%), the host's GPU waits 32 -> 6 ms a frame, the same GPU time, 0 of 1,100 probed reads missed. Our pack of its gpu/ code on an older main: +21.8% in Kokiri Forest at handheld settings, +23.3% in Hyrule Field at 1280x800, 0 of 4,200 probed reads missed. At a play build's settings (frame generation 3x, RX_VK_READBACK_LATE=newest, the GPU at 1600 MHz, dukenheimer): no measurable fps change, the two unset runs bracketing both on runs; captures stopped waiting for late pages, and 0 of 2,200 probed reads missed. On a desktop RX 580 (GCN, whose pools have no VRAM mirror by default), this head merged with main c167ec67: fps up 10 to 24% in every phase of a four-place route, in both =1 runs, GPU time unchanged, the byte probe 0 changed, and unset at main's speed (HumanBean). With the render thread on, the early reads leave themselves off, and the gain is not there (below). The byte probe checks only what a read returns: whether the game, let go early, writes memory the rest of the submission still reads is not measured (Risks).

Baseline: dukenheimer's run (the first tables) is this branch on v3.0.0-rc.5 (main e4d035e9), their rebase of 148f14a0 onto it, which matches 1d1263e6 in every built file; one build, "Before" with RX_VK_READBACK_EARLY unset (main's path), "After" with =1. The head now, 36b0ccbb, is 1d1263e6 rebased onto 13e90a5e: git range-diff shows every commit unchanged but 6cf7a3d1, whose byte-probe hook in memory_collect_written now takes its collection number inside #1083's collect_mu, and one new test-only commit. Both act only with RX_VK_LOG_EARLY_CHECK, so the fps numbers hold for this head; the probe's numbers were taken with the hook's earlier form.

dukenheimer's run of this branch (#977 (comment); their numbers):

Hardware: Steam Deck OLED, AMD Custom GPU 0932 (RADV VANGOGH), Mesa 26.2.0-devel, SteamOS 3.9.5, 16 GB; docked, in Game Mode through Steam (not headless). Built with the release toolchain (LLVM 23.1.5, glibc 2.35), x86-64-v3, the play profile
Scene: Kokiri Forest beside Saria, output 1280x720, render resolution 50%, the game's own upscaler, quality preset Custom, frame generation off, threaded rendering Auto, stock clocks, Profile Docked, HDR off
Method: warm shader caches copied fresh for each start, RX_VK_PRECOMPILE=0, no longshot_env.txt; 20 s to settle, then 45 s standing; a warm-up, then unset, =1, =1, unset; fps and frame times from the presents, the other rows the medians of the log's periodic lines over each window

Kokiri Forest, handheld, 45 s standing Before, RX_VK_READBACK_EARLY unset: this PR @ 1d1263e6 on rc.5 After, RX_VK_READBACK_EARLY=1: this PR @ 1d1263e6 on rc.5
fps (run 1 / run 2) 15.71 / 15.69 20.69 / 20.78
median fps 15.70 20.73 (+32.1%)
frame time p50 / p95 / p99 (ms) 62.5, 62.4 / 70.1, 69.9 / 94.1, 86.0 46.8, 46.5 / 54.8, 54.4 / 77.0, 76.1
frames over 100 ms 8, 8 8, 8
GPU time (ms a frame) 39.1, 39.0 38.1, 38.3
host time in GPU resource waits (ms a frame) 32.4, 32.4 5.6, 5.9
GPU busy, mean clock 69%, 925 MHz; 71%, 914 MHz 81%, 1001 MHz; 83%, 1002 MHz
replay memory sync (ms a frame) 1.55, 1.60 0.71, 0.72

Byte probe (=1, RX_VK_LOG_EARLY_CHECK=4, RX_VK_LOG_READBACK_WAITS=1; 30 s standing, then a moving Kokiri route for 88 s): 1,100 reads checked, 0 missed, 0 ambiguous, 0 lost. A plain start first (without RX_VK_PRECOMPILE=0, after --prepare-shaders) reached play at 14.35 fps with RSS at most 6.6 GB and MemAvailable at least 1.4 GB.

In a Deck play build (rc.5 with #225, #227, #222 and this PR): Profile Docked, output 1280x800, the game's upscaler at 100%, frame generation 3x, HDR10, RX_VK_READBACK_LATE=newest in longshot_env.txt, the GPU fixed at 1600 MHz; a moving Kokiri route, 80 s; a warm-up, then unset, =1, =1, unset:

Kokiri, moving route, play build unset (A1 / A2) =1 (B1 / B2)
game fps (shown with 3x) 17.1 (49.3) / 14.7 (41.9) 16.2 (46.6) / 16.3 (47.5)
median game fps 15.9 16.25: no measurable change (the unset runs bracket both =1 runs)
shown frame time p99 (ms) 89.0 / 104.9 103.8 / 104.6
captures that waited for late pages (of the last 600) 3 / 39 0 / 0
capture waits that hit their 250 ms cap, in the run 2 / 3 0 / 0
readbacks that waited for their write-back (of 3,000), ms each 630, 2.66 / 775, 2.07 1,275, 0.50 / 1,689, 0.49
readbacks that waited for their writer's part (of 3,000), ms each - 690, 0.67 / 918, 0.23
MemAvailable at its lowest (MB) 1,049 / 1,303 1,282 / 1,248

Byte probe on that build: Kokiri to the Deku Tree and back twice, RX_VK_LOG_EARLY_CHECK=4, 800 checked; the moving route, =1, 1,400 checked: 0 missed, ambiguous or lost in either. No device loss, device fault, stalled GPU or memory kill in any run.

HumanBean's run on a desktop RX 580 (#977 (comment); their numbers): RX 580 8 GB (Polaris10), RADV, 8 threads, KDE. Main keeps the VRAM mirrors off on GCN (mirror_on()), so the early reads act there as on the Deck. Build: this PR's head merged with main c167ec67, as a dev_packs.py pack on a base of that main. Route: Kokiri Forest, Hyrule Field, the Market (debug teleport), Hyrule Field again; standing, then turning. One session per arm: main's own build, this PR unset (E0), and =1 with the byte probe (RX_VK_LOG_EARLY_CHECK=4) twice (E1). Not interleaved, and the probe's checked reads wait as main's would, so E1 carries the probe's cost.

RX 580, fps (GPU ms) (HumanBean) main E0, unset E1, =1, run 1 E1, =1, run 2
Kokiri, standing 20.9 (35.60) 21.0 (35.70) 24.5 (35.40) 24.2 (35.50)
Kokiri, turning 22.4 (31.70) 22.7 (31.20) 26.3 (31.35) 26.3 (31.40)
Hyrule Field, standing 27.5 (24.30) 27.8 (24.10) 33.7 (23.80) 32.3 (24.10)
Hyrule Field, turning 33.6 (20.25) 34.3 (19.90) 39.6 (19.75) 40.0 (19.60)
the Market, standing 24.2 (28.90) 24.7 (28.35) 26.6 (28.50) 27.1 (27.80)
the Market, turning 36.5 (17.50) 35.8 (17.70) 44.6 (17.55) 44.5 (16.90)
Hyrule Field again, turning 34.6 (20.05) 32.9 (20.05) 38.2 (19.95) 39.5 (19.60)

fps 10 to 24% higher in every phase in both =1 runs, GPU time unchanged: the wait was on the CPU's side. Readbacks: with it off, 364 to 421 of 3,000 waited 10 to 16 ms each; with it on, 637 to 652 waited for their writer's part, 4 to 6 ms each. Byte probe, both runs: early -> main's release: 100 unchanged, 0 changed on a page the CPU did not ... 0 lost. Unset is main's speed.

Our pack of its gpu/ code (the earlier measurements):

Baseline for the pack: the Deck's development base is main @ 9c9d69ed, and a renderer-only pack can't take this branch (it sits on a newer main). The pack measured, early2 (tree d552c919, local), is 9c9d69ed plus #443's vk_replay.cpp change (82cee56e, off on AMD) plus this branch's gpu/ diff. Both columns are that one pack: "Before" with RX_VK_READBACK_EARLY unset (main's path), "After" with =1. What makes the numbers carry over: gpu/vk/readback_early.h is byte-identical to this branch's, and the lines this PR adds and removes in vk_replay.cpp, vk_memory.cpp, vk_draw.cpp and vk_internal.h are the same, line for line, as the pack's (compared with the line numbers stripped; the pack's only extra lines are #443's). The code around them is not the same: main has about 30 renderer commits since 9c9d69ed, some in the same paths (#526 the late pages' fix, #590 unread bookkeeping without mirrors, #769, #588 the render thread), which is why dukenheimer's run of the branch itself (above) is the one on today's code. The rebase onto main e4d035e9 left this comparison as it was: the branch's own gpu/ lines are the same, line for line, before and after it, and none of the main changes it met touches what an early read waits for: #700 adds a count to the readback line (it counts only reads that waited for their whole submission, so an early read is not in it), #670 says a wait that comes back device-lost as a lost device, and #431's import-table walk, which the early reads' note in memory_note_gpu_write sits beside, is main's with the switch unset or set.
Hardware: Steam Deck OLED, AMD Custom GPU 0932 (RADV VANGOGH), Mesa 26.2.0-devel, SteamOS 3.9.5, kernel 7.2.7-valve2, Zen 2 4C/8T, 16 GB shared, stock GPU clock
Scene: Kokiri Forest outside Link's house beside Saria (about 2,200 draws a frame), output 1280x720, render resolution 50%, the game's upscaler; Hyrule Field at the Lost Woods bridge, output 1280x800, render resolution 100%. Both: quality preset Custom, frame generation off, render thread off (Auto), headless
Method: fps and frame times from the present trace over 45 s standing after settling; GPU and wait figures from the log's periodic lines. Interleaved off, on, on, off after a warm-up, the median of each arm. Warm shader caches copied fresh, RX_VK_PRECOMPILE=0, no longshot_env.txt. Probe: RX_VK_READBACK_EARLY=1 RX_VK_LOG_EARLY_CHECK=4 RX_VK_LOG_READBACK_WAITS=1, the early-read check summaries summed
How to reproduce: a child Kokiri save; RX_VK_READBACK_EARLY=1 against unset. Probe routes: 30 s standing then 12 walk-and-pan cycles outdoors in Kokiri Forest; teleports Kokiri Forest, Deku Tree, Kokiri Forest, Deku Tree with camera pans at each

Kokiri Forest, handheld (1280x720 output, 50%) Before, RX_VK_READBACK_EARLY unset: pack early2 @ d552c919 After, RX_VK_READBACK_EARLY=1: pack early2 @ d552c919
fps (run 1 / run 2) 11.82 / 10.98 14.11 / 13.67
median fps 11.40 13.89 (+21.8%)
frame time p50 / p99 (ms) 79.6, 83.5 / 127.3, 157.9 67.3, 70.2 / 119.0, 126.5
frames over 100 ms 99, 146 36, 44
GPU time (ms a frame) 52.4, 55.1 51.7, 53.0
host GPU waits (ms a frame) 48.5, 51.0 22.6, 24.1
GPU busy 75%, 77% 87%, 86%
replay memory sync (ms a frame) 2.0-2.7 1.8-1.9
Hyrule Field, native (1280x800, 100%) Before, RX_VK_READBACK_EARLY unset: pack early2 @ d552c919 After, RX_VK_READBACK_EARLY=1: pack early2 @ d552c919
fps (run 1 / run 2) 14.97 / 14.99 18.45 / 18.48
median fps 14.98 18.47 (+23.3%)
frame time p50 / p99 (ms) 66.6, 66.5 / 75.9, 73.8 51.7, 52.0 / 69.4, 69.4
GPU time (ms a frame) 44.9, 44.8 44.1, 44.3
host GPU waits (ms a frame) 39.0, 39.0 22.6, 22.2
GPU busy 76%, 74% 89%, 88%
replay memory sync (ms a frame) 0.92-1.17 0.78-0.80
Byte probe, RX_VK_LOG_EARLY_CHECK=4 Kokiri, walking Kokiri to the Deku Tree and back, twice
reads checked 2,300 1,900
changed by main's release on pages the CPU did not write (missed) 0 0
ambiguous / lost 0 / 0 0 / 0

#303's own version missed 29 of 800 and 21 of 1,100 on these two routes. An earlier port of the fork's head (14e83e9c onto 9c9d69ed, on by default on the Deck), in Hyrule Field native on the same Deck: main 15.20 fps, the port 18.47 (+21.5%), the port with RX_VK_LATE_DEFER=0 17.58, the port with RX_VK_READBACK_EARLY=0 15.03; its probe found 0 of 4,000 reads missed across the Field, the Lost Woods and the Market. GPU time is the same in every arm: the gain is the CPU no longer waiting, not less GPU work.

With the render thread on, none of it. RX_VK_READBACK_EARLY leaves itself off when the render thread is on (RX_VK_ASYNC=1, Threaded rendering; gpu/vk/readback_early.h: it was not measured with it). On our Deck OLED on 2026-11-16, Kokiri Forest at handheld settings with RX_VK_READBACK_EARLY=1, #906's RX_VK_MESH_FB_LAYOUT=soa and #1053's RX_VK_GPU_CLOCK=1 all set: 17.8-19.5 fps without the render thread, 14.7-15.1 with it (one run each, not interleaved: an indication, not a measurement of this PR). The Deck's Auto leaves the render thread off.

With the other two Deck switches (our pack of this PR, #906 and #1053 on main 9c9d69ed; same Deck OLED, empty shader caches in every run, each arm twice in mirrored order, the median of each; the Field series and Kokiri's two ends also at #1053 (comment)):

median fps nothing on early reads early reads + clock hold all three
Kokiri Forest, handheld 11.42 (12.97, 9.87) 14.05 19.05 19.61
Hyrule Field, native 15.07 20.99 (with the layout) 24.64

The clock hold still pays with the early reads on (14.05 -> 19.05): together they take the host's wait on the GPU from 51.8 to under 1 ms a frame. Nothing broke with all three on (exit 0 in all 18 runs).

vk: reads of GPU results written in place wait for their writer's part (RX_VK_READBACK_EARLY)
vk: late pages left to their reads on, with the early reads (RX_VK_LATE_DEFER=0: off)
vk: late pages left to their reads ... 212 pages in 40 takes (212 kept with their capture), 78 read on the CPU, 0 of them waited (0.00 ms in all); 9 still late    (Kokiri, on, run 1)
vk: late pages left to their reads ... 384 pages in 69 takes (384 kept with their capture), 138 read on the CPU, 0 of them waited (0.00 ms in all); 8 still late   (Field, on)

The exposure check: Hyrule Field and the Lost Woods in turn, the camera turning, a shot at each of 7 stops, main against the pack with the switch on. No frame too bright or too dark, no flash and nothing popping at the screen's edges, but the two runs' camera did not stop on the same frame (a turn timed in game frames lands elsewhere when the frame rate differs), so the shots are not pixel-comparable. RX_VK_FLASH=1 is not in these builds.

Flags and settings

Three switches and two diagnostics are added. RX_VK_READBACK_EARLY, and RX_VK_LATE_DEFER with it, are on by default where the pools have no mirror; the rest are off.

Flag What it is Before this PR After this PR Effect Helps / doesn't help / untested Recommendation
RX_VK_READBACK_EARLY A read of GPU results written in place waits for its writer's part absent on where the pools have no mirror, off with mirrors; =1 on, =0 off; never with RX_VK_PARALLEL >= 2 or the render thread Frees the game's jobs from the reads they poll Helps the Deck OLED: this branch +32.1% in Kokiri handheld (dukenheimer), the pack +21.8% handheld, +23.3% native; no measurable fps change at a play build's settings (frame generation 3x, RX_VK_READBACK_LATE=newest, GPU at 1600 MHz), where captures stopped waiting for late pages; left off with the render thread, so no gain there; idle with VRAM mirrors (discrete cards but GCN); a desktop RX 580 (GCN: no mirrors by default) +10 to +24% fps in every phase, GPU time the same, the probe clean (HumanBean); untested on other integrated GPUs and on Intel with #443's rule On where the pools have no mirror (shipped so); =0 is the first thing to try for a one-frame flash or a HUD or object out of place there; the exposure check's result can still turn it back off (Risks)
RX_VK_READBACK_EARLY_GAP Draws into a part before a writer may end it absent 16 Writers in a row end one part 16 measured on the Deck; other values untested As shipped
RX_VK_READBACK_EARLY_RECORDED The early read first waits for its writer's submission to be recorded to its end absent on (with RX_VK_READBACK_EARLY) The part chosen is the range's last writer's Measured free in the fork (19.00 to 19.06 fps, Deck LCD) As shipped; =0 for an A/B only
RX_VK_LATE_DEFER A capture's late pages wait for the CPU read that needs them, not at the take absent follows RX_VK_READBACK_EARLY where pools have no mirror; off with mirrors or RX_VK_PARALLEL > 1 Keeps the replay's memory sync at main's level with the early reads Helps with the early reads (earlier port: 17.58 to 18.47 fps); with RX_VK_READBACK_LATE=newest in a play build, the captures that waited for late pages went from 3 and 39 of 600 to 0 (dukenheimer); gains nothing alone As shipped
RX_VK_LOG_RB_WRITERS Diagnostic: what wrote the pages a read waits for (=1); release counts only (=2) absent off =1 waits and copies, changing timing For investigation Off
RX_VK_LOG_EARLY_CHECK Diagnostic: one early read in n compared byte by byte with what main's wait returns absent off The checked reads wait as main's would For probe runs Off

Testing

Tested:

  • Steam Deck OLED, SteamOS 3.9.5, RADV, by dukenheimer: this branch on v3.0.0-rc.5 (1d1263e6's tree, built with the release toolchain), Kokiri Forest at handheld settings unset against =1, interleaved, and the byte probe on a moving Kokiri route; and the branch in a Deck play build at native with frame generation 3x, two interleaved runs each and two probe runs. No device loss, fault or memory kill.
  • RX 580 (Polaris10, GCN), Linux, RADV, by HumanBean: this head merged with main c167ec67 in a dev_packs.py pack, main's build, unset and =1 twice with the byte probe, over Kokiri Forest, Hyrule Field and the Market, standing and turning (Measurements; #977 (comment)). Unset at main's speed, =1 +10 to +24%, the probe 0 changed and 0 lost.
  • Steam Deck OLED, SteamOS 3.9.5, RADV, ours: the pack above (this PR's gpu/ code on main 9c9d69ed): the byte probe on two Kokiri routes, the timed pairs in Kokiri Forest (handheld) and Hyrule Field (native), the exposure pair. The engaged lines in every on arm, none in the off arms; exit 0, no device loss in any run.
  • Game-free, Linux x86-64, on this head (36b0ccbb, main 13e90a5e): the full python3 -I -B tests/run_all.py (clang++ 23.1.5, the Vulkan and SPIR-V headers found) after the rebase: 627 passed, 23 failed, 26 skipped. To name them, the checks that failed in that run or in other recent full runs here (28) were run again on this head and on main 13e90a5e, the same way. 19 fail on both: test_audio_output_devices, test_bug_report_defaults, test_crt_options, test_failed_compile_keeps_objects, test_fsr_uniform_ownership, test_guest_imported, test_import_reach, test_linux_preflight, test_linux_update, test_make_game_cache, test_missing_nup, test_mods_size_table, test_nonuniform_image_arrays, test_parallel_init_order, test_pending_guest_import, test_replay_write_order, test_run_all_changed, test_vk_occlusion_queries and test_walk_key. The rest depend on the PC's load, on this branch and on main alike: test_fpx_sticky and test_recorder_sync timed out in the full run and pass on both; test_bug_report and test_hardware_report pass on both; test_pipeline_cache_save (a timed check) failed once in the re-run here and passes twice alone, and test_seed_reader failed once on main and passes here
  • The renderer built and linked for Windows with llvm-mingw 20261104 on this head (36b0ccbb; RX_VK_ONLY=1 RX_TARGET=windows: longshot_vk.dll, PE32+ x86-64, no warning, no error)
  • Before the last rebase, on 1d1263e6 (main e4d035e9), with the Vulkan headers found (LONGSHOT_VULKAN_HEADERS; earlier runs here skipped the checks that need them): MAGICBEAN_TOOLING=<Tooling> python3 -I -B tests/run_all.py (clang++ 23.1.5): 610 passed, 16 failed, 40 skipped. All 16 fail on main e4d035e9 too, run there with the same headers and clang++: test_bug_report_defaults, test_crt_options, test_fsr_uniform_ownership, test_guest_imported, test_import_reach, test_linux_preflight, test_linux_update, test_make_game_cache, test_missing_nup, test_mods_size_table, test_parallel_init_order, test_pending_guest_import, test_replay_write_order, test_run_all_changed, test_vk_occlusion_queries, test_walk_key.
  • run_all.py --changed forge/main: with CXX=g++ 242 passed, 8 failed, 5 skipped; with CXX=clang++ 241 passed, 9 failed, 5 skipped. The 8 are among the 16 above and fail on main with both compilers. The 9th, test_pipeline_cache_save (a timed check, at a load average over 300), failed on main too when run alone there at the same load, passed in the full run and with g++, and passes run directly on this branch; the function it cuts is byte-identical to main's.
  • The port's checks and the ones pinning main's part ends and the paths the rebase met (test_readback_early, test_late_defer, test_early_split_before_draw, test_readback_part, test_late_fix_build, test_capture_wait_log, test_frame_trace_waits, test_parallel_rb_refresh, test_direct_buffer_writes, test_split_before_draw, test_parallel_small_cb, test_split_ub_flush, test_srccut, test_lost_wait_said, test_fsr_image_retirement): all pass with g++ 13.3 and clang++ 23.1.5, --strict. test_import_reach (#431's) fails to compile its fixture on main e4d035e9 as here, with both compilers (bg_inflight's type in its stand-ins). The three new checks also passed with SRCCUT_CHECK_NAIVE=1 before the rebase (every cut the same as their old cutter's).
  • Unset is main: test_split_before_draw, test_parallel_small_cb and test_split_ub_flush are main's, unchanged; test_late_defer checks the rule unset on every GPU; test_early_split_before_draw gets main's part ends with the switch off. Every new path is behind readback_early_on(), late_defer_on(), rb_writers_on() or early_check_on().
  • Whole-file -fsyntax-only of vk_replay.cpp, vk_memory.cpp and vk_draw.cpp for Linux and Windows (llvm-mingw 20261104), before the rebases onto 27a669ba and e4d035e9 (not run again since; the branch's own lines are unchanged): no error on Linux; on Windows the same errors as main's files, none new.

Not tested:

  • This head (36b0ccbb) on the Deck: the Deck runs were of 1d1263e6's tree and of the pack; the head differs from 1d1263e6 only in the probe's hook and a test stand-in. It has run in the game on an RX 580 (HumanBean).
  • Intel and NVIDIA, with the switch unset or on; Windows in the game; other integrated GPUs (AMD Phoenix and Strix); other GCN cards (Vega), and the RX 580 in interleaved runs.
  • #443's path in the game (RX_VK_READBACK_EARLY=1 where parts end only before a plain draw: Intel by default).
  • The early reads with the render thread: they leave themselves off with it even when asked for. With it on, Kokiri at handheld settings ran 14.7-15.1 fps against 17.8-19.5 without it (one run each, with the clock hold and the layout on; Measurements): a player with Threaded rendering on gets none of the gain.
  • Walking out of a house and back (the harness could not script it), and an exposure check at the same frames: what can show whether the game, let go early, writes memory a later part still reads (Risks): needed before any default.
  • RX_VK_LATE_DEFER=0 on this pack (it was measured on the earlier port only).

Must be tested:

  • steamdeck, SteamOS: walk out of Link's house into Kokiri Forest and back several times with RX_VK_READBACK_EARLY=1 and with it unset, shots at the doorway, expecting no flash or dark frame and nothing popping at the edges against unset (by hand: compare the shots)
  • windows, nvidia: Hyrule Field and Kokiri Forest, once with RX_VK_READBACK_EARLY unset and once with RX_VK_READBACK_EARLY=1, expecting the same frame rate and picture as main in both (with VRAM mirrors the switch stays idle) and no error in the log (Test Farm: @Beanstalk windows nvidia, Hyrule Field and Kokiri Forest with RX_VK_READBACK_EARLY unset and with it set to 1, no regression against main)
  • windows, intel: Kokiri Forest while moving, with RX_VK_READBACK_EARLY unset and with =1, expecting no white shards in either, the same frame rate as main unset, and with it on the log's readbacks waited for their writer's part line (Test Farm: @Beanstalk windows intel, Kokiri Forest moving with RX_VK_READBACK_EARLY unset and with it set to 1, look for white shards and compare fps with main)

Risks

What could break: only with RX_VK_READBACK_EARLY=1:

  • A CPU read let go before a later GPU write to the same bytes (a stale culling camera: objects popping at the screen's edges). #303's version did it in 2-4% of camera-block reads; the recording wait closed that on the routes probed here (0 of 4,200 on the pack, 0 of 4,000 on the earlier port, 0 of 3,300 on this branch: dukenheimer's 1,100 and 2,200).
  • A submission recorded in two recordings (it stops at an event wait and resumes under the same value) is let go after its first recording, before a writer the second adds. Main's own wait has the same gap, so it is not a regression, but "every writer of the submission is noted before the part is chosen" does not hold there and the probe can't see it (found reading the code; still open).
  • With RX_VK_LATE_DEFER, which follows the switch, a deferred late page is copied from guest memory at its first CPU read rather than at the take, so a game write made after the submit and before that read lands in the replay's "as submitted" copy: a longer window than the old wait's (design rule 2, "nothing the replay reads comes from live guest memory"). The two mechanisms also come as one switch: on the earlier port the early reads alone gave 17.58 fps and both 18.47 (from the same reading and the earlier port's runs). A Deck OLED tester of the fork found the deferral cost about 2 fps where the replay thread is the limit (internal 768x432 with their own RX_VK_READBACK_LATE rule); this PR's handheld runs (640x360 internal) gained with both.
  • Not covered by the probe: with it on, the game's job is let go once its writer's part has run, while the later parts of the same submission may still read guest memory in place. That is why main waits for the whole submission for pages written in place (invalidate's comment: they "keep the wait for all of its submission, whose later parts read and write guest memory itself"), and why main's own part wait, for a mirror's page, is refused when a later part reads guest memory in place (in_place_after); the early wait has no such test. The job may then write memory a later part still reads: the next frame's uniforms or vertices, or the exposure meter (the pattern of issue 997). The byte probe cannot see this: early_check_read compares only the bytes of the range the CPU reads against what main's wait returns (and a checked read itself waits as main's would), so its 0 missed reads clear the risk above, not this one. The evidence that can clear it is a same-frame exposure check: the same frames with the switch unset and on, compared pixel for pixel. The one here was not pixel-comparable (the camera did not stop on the same frames), so dyst's review (#977 (comment)) asked for no default before such a check. It is on by default anyway (468f360d, #977 (comment)): the gain is large where it acts, nothing has been seen by eye or probe, and a correct game guards that memory with its own fences. The RX_VK_LOG_INPLACE_EXPOSURE runs continue; a higher rate after an early release turns the default back off, or holds what they point at, before the next release.

With mirrors (discrete cards) and the switch unset, or with =0 anywhere, nothing changes: the tracking, the splits, the descriptor rebuilds and the recording tags are behind the switches.

Turning it off: RX_VK_READBACK_EARLY=0; RX_VK_LATE_DEFER=0 alone for the late pages.

Compatibility: Nothing changes: no shader translator file (the shader caches stay valid), MKWVK_API_VERSION unchanged (vk_api.h untouched), no option, ini key or menu row.

Works with / conflicts with

  • #952 (a recording's uniform copies shared, their reads made once a draw) splits memory_read_as_submitted into submitted_read_unlocked and submitted_read_locked and adds memory_read_as_submitted_many, the batched reader of a draw's uniform copies (on by default; on Intel iGPUs without mirrors, where the uniform snapshot ub_snapshot_on is on, every uniform range is read through it). This PR's late_defer_cpu_read call sits in the per-page loop that moves into submitted_read_locked, so the two conflict in vk_memory.cpp. Whichever lands second moves the call into submitted_read_locked, which both readers go through, and makes tests/test_late_defer.py look for it there; otherwise a batched read would take a deferred late page without copying it again. If #952 lands first, this PR does it.
  • #906 and #1053: measured together with this one (Measurements); no shared lines.

Checklist

Everyone

  • One topic.
    (The early reads with the two things they need: RX_VK_LATE_DEFER gains nothing on its own and exists because the early reads make the late pages the replay would otherwise wait ~19 ms a frame for; it follows the switch and can be turned off alone. The diagnostics are the probe that measured it. Our review of the fork's PR asked for the deferral as its own PR; the authors' commits for it are mainly the first three and could be split out if the maintainers prefer)
  • The title is a Conventional Commit (type(scope): description) of the status line's type.
  • Why gives the reason plainly and links the issues that fit (the tracker searched).
  • Measurements has a verdict, or one line saying why there is nothing to measure.
  • Testing names what was tested, what wasn't, and what must still be tested, where and how.
  • Rebased on current main: at review onto 26ce01b5, commit by commit with no conflict, head 468f360d (the default on top). Before that: on 13e90a5e, head 36b0ccbb; it merges with current main 71b1edd8 without a conflict (git merge-tree). The rebase from e4d035e9 had one conflict, in vk_memory.cpp's memory_collect_written with #1083's collect_mu, resolved in 6cf7a3d1 (the probe's hook, What changed), and added 36b0ccbb (a test stand-in). The earlier rebase onto e4d035e9 kept #700's whole-submission count in invalidate (52160308, 6d10a356) and #431's walk comment in memory_note_gpu_write (fda1d573, 6cf7a3d1), each named in that commit's [port: ...] line. Old head 1d1263e6 → new 36b0ccbb; the hashes in this description are the new ones.
  • python -I -B tests/run_all.py passes (any failure that also fails on main is named). Run at 36b0ccbb; 468f360d was not run (a code-only review): tests/test_late_defer.py and tests/test_readback_early.py are the checks it changes. (Testing: the full python3 -I -B tests/run_all.py (clang++ 23.1.5, the Vulkan and SPIR-V headers found) after the rebase: 627 passed, 23 failed, 26 skipped. To name them, the checks that failed in that run or in other recent full runs here (28) were run again on this head and on main 13e90a5e, the same way. 19 fail on both: test_audio_output_devices, test_bug_report_defaults, test_crt_options, test_failed_compile_keeps_objects, test_fsr_uniform_ownership, test_guest_imported, test_import_reach, test_linux_preflight, test_linux_update, test_make_game_cache, test_missing_nup, test_mods_size_table, test_nonuniform_image_arrays, test_parallel_init_order, test_pending_guest_import, test_replay_write_order, test_run_all_changed, test_vk_occlusion_queries and test_walk_key. The rest depend on the PC's load, on this branch and on main alike: test_fpx_sticky and test_recorder_sync timed out in the full run and pass on both; test_bug_report and test_hardware_report pass on both; test_pipeline_cache_save (a timed check) failed once in the re-run here and passes twice alone, and test_seed_reader failed once on main and passes here)
  • No game files, dumps, build output or generated files.
  • third_party/ untouched (a new folder with a PROVENANCE.md instead).
  • The license is OK: GPL-3.0-or-later, nothing incompatible and nothing decompiled.

When it applies

  • AI assistance recorded with Assisted-by: / Co-Authored-By: trailers.
  • Numbers from the same PC, scene and settings, at least 2 interleaved runs each, the median reported. (This branch: dukenheimer's unset, =1, =1, unset after a warm-up, the median of each arm, at handheld settings and in a play build; the pack: ours, the same way. The render-thread pair is one run each and is labelled as an indication; HumanBean's RX 580 run is one session per arm, not interleaved, with =1 twice, and is labelled so)
  • Switches documented (docs/development.md; the README for a menu row) and shipped at their recommended value.
  • A file in docs/tested/ for each new measured run (format in docs/tested-configurations.md). (Ours: e161d7e6. dukenheimer's and HumanBean's runs are cited from their comments, as theirs)
  • A game-free test added for the fix, failing on main and passing here. (tests/test_early_split_before_draw.py: it fails with the fork's end right after the writer on Intel)
  • A check under tests/ that reads changed source updated in this PR, not weakened.
  • Host code builds on both Windows (llvm-mingw) and Linux, with UTF-8 paths. (The renderer built and linked for Windows with llvm-mingw 20261104 on this head (36b0ccbb; RX_VK_ONLY=1 RX_TARGET=windows: longshot_vk.dll, PE32+ x86-64, no warning, no error); Linux: dukenheimer's release-toolchain build; no path code changed)
  • Docs updated.
**Type:** perf · **Risk:** low · **Tested on:** Steam Deck OLED (SteamOS 3.9.5, RADV): this branch on v3.0.0-rc.5 (dukenheimer) and its `gpu/` code on an older main (ours); RX 580 (Linux, RADV; GCN, so no VRAM mirrors): this head on main c167ec67 (HumanBean); Linux, game-free checks; the renderer built for Windows (llvm-mingw) · **Needs:** Windows, NVIDIA, Intel (switch unset and on); a same-frame exposure check on the Deck before any default ## Why On the Steam Deck the game keeps waiting for GPU results the GPU has already computed. Its jobs read the culling camera block every frame, and the renderer makes that read wait for the whole 22-26 ms batch of GPU work it came in, not for the part that wrote it, so the GPU sits idle while the game waits. Letting the read go once its writer's part has run took Kokiri Forest at handheld settings from 11.40 to 13.89 fps (+21.8%) and Hyrule Field at 1280x800 from 14.98 to 18.47 fps (+23.3%) on a Deck OLED, with a byte-for-byte probe finding none of 4,200 such reads different from what main's wait returns. dukenheimer's run of this branch itself on a Deck OLED gave 15.70 to 20.73 fps (+32.1%) with the same recipe, and 0 of 1,100 probed reads missed. This PR brings it to Longshot **on by default where the buffer pools have no VRAM mirror** (integrated GPUs: the Steam Deck, AMD APUs, Intel), with `RX_VK_READBACK_EARLY=0` as the way back. It was first ported off by default; HumanBean turned it on at review (`468f360d`; the decision: https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/977#issuecomment-37118), with the exposure check still to come (Risks). **The design and nearly all of the code are sheep's and dukenheimer's.** sheep built it in https://git.magicbean.dev/sheep/BunnyHood/pulls/2, on dukenheimer's https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/303, which started the early reads and wrote the byte probe used here. This PR only exists to bring their work to Longshot with their authorship kept: their 18 commits are cherry-picked one by one (`cherry-pick -x`, authors unchanged, a `[port: ...]` line where a commit needed more than the fork's names mapped back to Longshot's). If either of them would rather open it themselves, this PR will be closed in favour of theirs. The tracker was searched ("readback early", "early read", "late defer", `READBACK_EARLY`): no issue, and no open PR besides https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/303, which dukenheimer has since closed in favour of this one. ## What changed By default on GPUs whose pools have no VRAM mirror (the Steam Deck, integrated GPUs), a CPU read of GPU results written in place waits for the part of the submission its last writer was recorded in, once that submission is recorded to its end, instead of for the whole submission. The capture's late pages are then left to the CPU reads that need them (`RX_VK_LATE_DEFER`, which follows the switch) instead of holding up the replay. Discrete cards (mirrors) stay as on main unless `=1` is set, and `=0` is main everywhere. - vk, the early reads (dukenheimer, sheep): `gpu/vk/vk_memory.cpp` notes each in-place writer's part (`early_in_place_write`, `memory_in_place_writer_part`) for ranges whose reads waited; `gpu/vk/vk_replay.cpp` `invalidate` waits for that part, after `early_recorded_wait` (the writer's submission recorded to its end, at most 250 ms), and `replay()` ends a part after such a writer (`RX_VK_READBACK_EARLY_GAP` draws into it). `gpu/vk/vk_draw.cpp`: a program that writes storage buffers rebuilds its descriptors once the tracking is in use, so each write is noted. - vk, the late pages left to their reads (sheep): `gpu/vk/vk_memory.cpp`, `late_defer_split` and `late_defer_cpu_read` from `memory_read_as_submitted`; a late page in a texture's own storage still waits at the take. - vk, diagnostics (dukenheimer, sheep): `RX_VK_LOG_RB_WRITERS` and the byte probe `RX_VK_LOG_EARLY_CHECK`, log only. - Our commits: - `f2e73429` (https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/443's rule): main's `RX_VK_SPLIT_BEFORE_DRAW` (on by default on Intel) lets a part end only right before a plain draw. The fork ended a writer's part right after the writer, which can start the next part with a dispatch or a mesh draw. `early_split` now asks `part_ends_before_draw` as `maybe_split` does; where it holds, the writer's part runs on to the next plain draw. Elsewhere, the Deck included, nothing changes. `tests/test_early_split_before_draw.py` checks it. - `468f360d` (HumanBean, at review): on by default where the pools have no mirror. `readback_early.h`'s rule takes the pools' mirror (`Facts::mirrors`, from `mirror_on()`): unset is `ON_DEFAULT` without mirrors and `OFF_MIRRORS` with them; `=0` is off and `=1` on everywhere; the render thread and `RX_VK_PARALLEL` >= 2 still leave it off, now asked or not. `RX_VK_LATE_DEFER` follows it as before. `tests/test_late_defer.py`'s rule checks and `tests/test_readback_early.py`'s runs (unset with and without mirrors, `=0`, `=1` with mirrors) follow, and the doc rows say so. - `918372f3` (off by default, before `468f360d`): `gpu/vk/readback_early.h` keeps it off on every GPU unless asked for (the fork has it on for the Deck), and off even when asked for with `RX_VK_PARALLEL` >= 2 or the render thread. - `2ddd90fa` (docs): the switches' rows in `docs/development.md`. - The rebase onto 13e90a5e met https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/1083's `collect_mu` in `memory_collect_written`: in `6cf7a3d1` (the byte probe's commit) the probe's collection number is taken inside main's lock, with the marks, and its page report runs just after the lock is released, so the probe adds no lock under `collect_mu`; the capture's report follows main's new `capture_scan`. Both do nothing unless `RX_VK_LOG_EARLY_CHECK` is set. - tests: `tests/test_readback_early.py`, `tests/test_late_defer.py` and `tests/test_early_split_before_draw.py` are new and cut with `tests/srccut.py` (`3eb400b7`); `tests/test_readback_part.py`, `tests/test_late_fix_build.py`, `tests/test_capture_wait_log.py`, `tests/test_frame_trace_waits.py`, `tests/parallel_rb_refresh_test.cpp` and `tests/direct_buffer_writes_test.cpp` follow the code, and `tests/fsr_image_retirement_test.cpp` gets a stand-in for `rec_close`, which `finish_recording` now calls (`1d95536f`: the check needs the Vulkan headers and had been skipped here without them); `tests/test_vk_collect_window.py`, which cuts `memory_collect_written` out of the source, gets stand-ins for the probe's two hooks (`36b0ccbb`). - docs: three runs in `docs/tested/` (`e161d7e6`). ## Measurements **Verdict:** it helps on the Steam Deck OLED where the game waits on its reads, and shows no measurable change where it doesn't. **This branch itself** (dukenheimer, on v3.0.0-rc.5): Kokiri Forest at handheld settings **15.70 -> 20.73 fps (+32.1%)**, the host's GPU waits 32 -> 6 ms a frame, the same GPU time, 0 of 1,100 probed reads missed. Our pack of its `gpu/` code on an older main: +21.8% in Kokiri Forest at handheld settings, +23.3% in Hyrule Field at 1280x800, 0 of 4,200 probed reads missed. At a play build's settings (frame generation 3x, `RX_VK_READBACK_LATE=newest`, the GPU at 1600 MHz, dukenheimer): no measurable fps change, the two unset runs bracketing both on runs; captures stopped waiting for late pages, and 0 of 2,200 probed reads missed. **On a desktop RX 580** (GCN, whose pools have no VRAM mirror by default), this head merged with main c167ec67: fps up 10 to 24% in every phase of a four-place route, in both `=1` runs, GPU time unchanged, the byte probe 0 changed, and unset at main's speed (HumanBean). With the render thread on, the early reads leave themselves off, and the gain is not there (below). The byte probe checks only what a read returns: whether the game, let go early, writes memory the rest of the submission still reads is not measured (Risks). Baseline: dukenheimer's run (the first tables) is this branch on v3.0.0-rc.5 (main e4d035e9), their rebase of `148f14a0` onto it, which matches `1d1263e6` in every built file; one build, "Before" with `RX_VK_READBACK_EARLY` unset (main's path), "After" with `=1`. The head now, `36b0ccbb`, is `1d1263e6` rebased onto 13e90a5e: `git range-diff` shows every commit unchanged but `6cf7a3d1`, whose byte-probe hook in `memory_collect_written` now takes its collection number inside https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/1083's `collect_mu`, and one new test-only commit. Both act only with `RX_VK_LOG_EARLY_CHECK`, so the fps numbers hold for this head; the probe's numbers were taken with the hook's earlier form. **dukenheimer's run of this branch** (https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/977#issuecomment-34349; their numbers): Hardware: Steam Deck OLED, AMD Custom GPU 0932 (RADV VANGOGH), Mesa 26.2.0-devel, SteamOS 3.9.5, 16 GB; docked, in Game Mode through Steam (not headless). Built with the release toolchain (LLVM 23.1.5, glibc 2.35), `x86-64-v3`, the play profile Scene: Kokiri Forest beside Saria, output 1280x720, render resolution 50%, the game's own upscaler, quality preset Custom, frame generation off, threaded rendering Auto, stock clocks, Profile Docked, HDR off Method: warm shader caches copied fresh for each start, `RX_VK_PRECOMPILE=0`, no `longshot_env.txt`; 20 s to settle, then 45 s standing; a warm-up, then unset, `=1`, `=1`, unset; fps and frame times from the presents, the other rows the medians of the log's periodic lines over each window | Kokiri Forest, handheld, 45 s standing | Before, `RX_VK_READBACK_EARLY` unset: this PR @ 1d1263e6 on rc.5 | After, `RX_VK_READBACK_EARLY=1`: this PR @ 1d1263e6 on rc.5 | |---|---|---| | fps (run 1 / run 2) | 15.71 / 15.69 | 20.69 / 20.78 | | median fps | 15.70 | 20.73 (+32.1%) | | frame time p50 / p95 / p99 (ms) | 62.5, 62.4 / 70.1, 69.9 / 94.1, 86.0 | 46.8, 46.5 / 54.8, 54.4 / 77.0, 76.1 | | frames over 100 ms | 8, 8 | 8, 8 | | GPU time (ms a frame) | 39.1, 39.0 | 38.1, 38.3 | | host time in GPU resource waits (ms a frame) | 32.4, 32.4 | 5.6, 5.9 | | GPU busy, mean clock | 69%, 925 MHz; 71%, 914 MHz | 81%, 1001 MHz; 83%, 1002 MHz | | replay memory sync (ms a frame) | 1.55, 1.60 | 0.71, 0.72 | Byte probe (`=1`, `RX_VK_LOG_EARLY_CHECK=4`, `RX_VK_LOG_READBACK_WAITS=1`; 30 s standing, then a moving Kokiri route for 88 s): 1,100 reads checked, 0 missed, 0 ambiguous, 0 lost. A plain start first (without `RX_VK_PRECOMPILE=0`, after `--prepare-shaders`) reached play at 14.35 fps with RSS at most 6.6 GB and MemAvailable at least 1.4 GB. In a Deck play build (rc.5 with https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/225, https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/227, https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/222 and this PR): Profile Docked, output 1280x800, the game's upscaler at 100%, frame generation 3x, HDR10, `RX_VK_READBACK_LATE=newest` in `longshot_env.txt`, the GPU fixed at 1600 MHz; a moving Kokiri route, 80 s; a warm-up, then unset, `=1`, `=1`, unset: | Kokiri, moving route, play build | unset (A1 / A2) | `=1` (B1 / B2) | |---|---|---| | game fps (shown with 3x) | 17.1 (49.3) / 14.7 (41.9) | 16.2 (46.6) / 16.3 (47.5) | | median game fps | 15.9 | 16.25: no measurable change (the unset runs bracket both `=1` runs) | | shown frame time p99 (ms) | 89.0 / 104.9 | 103.8 / 104.6 | | captures that waited for late pages (of the last 600) | 3 / 39 | 0 / 0 | | capture waits that hit their 250 ms cap, in the run | 2 / 3 | 0 / 0 | | readbacks that waited for their write-back (of 3,000), ms each | 630, 2.66 / 775, 2.07 | 1,275, 0.50 / 1,689, 0.49 | | readbacks that waited for their writer's part (of 3,000), ms each | - | 690, 0.67 / 918, 0.23 | | MemAvailable at its lowest (MB) | 1,049 / 1,303 | 1,282 / 1,248 | Byte probe on that build: Kokiri to the Deku Tree and back twice, `RX_VK_LOG_EARLY_CHECK=4`, 800 checked; the moving route, `=1`, 1,400 checked: 0 missed, ambiguous or lost in either. No device loss, device fault, stalled GPU or memory kill in any run. **HumanBean's run on a desktop RX 580** (https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/977#issuecomment-35943; their numbers): RX 580 8 GB (Polaris10), RADV, 8 threads, KDE. Main keeps the VRAM mirrors off on GCN (`mirror_on()`), so the early reads act there as on the Deck. Build: this PR's head merged with `main` c167ec67, as a `dev_packs.py` pack on a base of that main. Route: Kokiri Forest, Hyrule Field, the Market (debug teleport), Hyrule Field again; standing, then turning. One session per arm: main's own build, this PR unset (E0), and `=1` with the byte probe (`RX_VK_LOG_EARLY_CHECK=4`) twice (E1). Not interleaved, and the probe's checked reads wait as main's would, so E1 carries the probe's cost. | RX 580, fps (GPU ms) (HumanBean) | main | E0, unset | E1, `=1`, run 1 | E1, `=1`, run 2 | |---|---|---|---|---| | Kokiri, standing | 20.9 (35.60) | 21.0 (35.70) | 24.5 (35.40) | 24.2 (35.50) | | Kokiri, turning | 22.4 (31.70) | 22.7 (31.20) | 26.3 (31.35) | 26.3 (31.40) | | Hyrule Field, standing | 27.5 (24.30) | 27.8 (24.10) | 33.7 (23.80) | 32.3 (24.10) | | Hyrule Field, turning | 33.6 (20.25) | 34.3 (19.90) | 39.6 (19.75) | 40.0 (19.60) | | the Market, standing | 24.2 (28.90) | 24.7 (28.35) | 26.6 (28.50) | 27.1 (27.80) | | the Market, turning | 36.5 (17.50) | 35.8 (17.70) | 44.6 (17.55) | 44.5 (16.90) | | Hyrule Field again, turning | 34.6 (20.05) | 32.9 (20.05) | 38.2 (19.95) | 39.5 (19.60) | fps 10 to 24% higher in every phase in both `=1` runs, GPU time unchanged: the wait was on the CPU's side. Readbacks: with it off, 364 to 421 of 3,000 waited 10 to 16 ms each; with it on, 637 to 652 waited for their writer's part, 4 to 6 ms each. Byte probe, both runs: `early -> main's release: 100 unchanged, 0 changed on a page the CPU did not ... 0 lost`. Unset is main's speed. **Our pack of its `gpu/` code** (the earlier measurements): Baseline for the pack: the Deck's development base is main @ 9c9d69ed, and a renderer-only pack can't take this branch (it sits on a newer main). The pack measured, `early2` (tree d552c919, local), is 9c9d69ed plus https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/443's `vk_replay.cpp` change (82cee56e, off on AMD) plus this branch's `gpu/` diff. Both columns are that one pack: "Before" with `RX_VK_READBACK_EARLY` unset (main's path), "After" with `=1`. What makes the numbers carry over: `gpu/vk/readback_early.h` is byte-identical to this branch's, and the lines this PR adds and removes in `vk_replay.cpp`, `vk_memory.cpp`, `vk_draw.cpp` and `vk_internal.h` are the same, line for line, as the pack's (compared with the line numbers stripped; the pack's only extra lines are https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/443's). The code around them is not the same: main has about 30 renderer commits since 9c9d69ed, some in the same paths (https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/526 the late pages' fix, https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/590 unread bookkeeping without mirrors, https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/769, https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/588 the render thread), which is why dukenheimer's run of the branch itself (above) is the one on today's code. The rebase onto main e4d035e9 left this comparison as it was: the branch's own `gpu/` lines are the same, line for line, before and after it, and none of the main changes it met touches what an early read waits for: https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/700 adds a count to the readback line (it counts only reads that waited for their whole submission, so an early read is not in it), https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/670 says a wait that comes back device-lost as a lost device, and https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/431's import-table walk, which the early reads' note in `memory_note_gpu_write` sits beside, is main's with the switch unset or set. Hardware: Steam Deck OLED, AMD Custom GPU 0932 (RADV VANGOGH), Mesa 26.2.0-devel, SteamOS 3.9.5, kernel 7.2.7-valve2, Zen 2 4C/8T, 16 GB shared, stock GPU clock Scene: Kokiri Forest outside Link's house beside Saria (about 2,200 draws a frame), output 1280x720, render resolution 50%, the game's upscaler; Hyrule Field at the Lost Woods bridge, output 1280x800, render resolution 100%. Both: quality preset Custom, frame generation off, render thread off (Auto), headless Method: fps and frame times from the present trace over 45 s standing after settling; GPU and wait figures from the log's periodic lines. Interleaved off, on, on, off after a warm-up, the median of each arm. Warm shader caches copied fresh, `RX_VK_PRECOMPILE=0`, no `longshot_env.txt`. Probe: `RX_VK_READBACK_EARLY=1 RX_VK_LOG_EARLY_CHECK=4 RX_VK_LOG_READBACK_WAITS=1`, the `early-read check` summaries summed How to reproduce: a child Kokiri save; `RX_VK_READBACK_EARLY=1` against unset. Probe routes: 30 s standing then 12 walk-and-pan cycles outdoors in Kokiri Forest; teleports Kokiri Forest, Deku Tree, Kokiri Forest, Deku Tree with camera pans at each | Kokiri Forest, handheld (1280x720 output, 50%) | Before, `RX_VK_READBACK_EARLY` unset: pack `early2` @ d552c919 | After, `RX_VK_READBACK_EARLY=1`: pack `early2` @ d552c919 | |---|---|---| | fps (run 1 / run 2) | 11.82 / 10.98 | 14.11 / 13.67 | | median fps | 11.40 | 13.89 (+21.8%) | | frame time p50 / p99 (ms) | 79.6, 83.5 / 127.3, 157.9 | 67.3, 70.2 / 119.0, 126.5 | | frames over 100 ms | 99, 146 | 36, 44 | | GPU time (ms a frame) | 52.4, 55.1 | 51.7, 53.0 | | host GPU waits (ms a frame) | 48.5, 51.0 | 22.6, 24.1 | | GPU busy | 75%, 77% | 87%, 86% | | replay memory sync (ms a frame) | 2.0-2.7 | 1.8-1.9 | | Hyrule Field, native (1280x800, 100%) | Before, `RX_VK_READBACK_EARLY` unset: pack `early2` @ d552c919 | After, `RX_VK_READBACK_EARLY=1`: pack `early2` @ d552c919 | |---|---|---| | fps (run 1 / run 2) | 14.97 / 14.99 | 18.45 / 18.48 | | median fps | 14.98 | 18.47 (+23.3%) | | frame time p50 / p99 (ms) | 66.6, 66.5 / 75.9, 73.8 | 51.7, 52.0 / 69.4, 69.4 | | GPU time (ms a frame) | 44.9, 44.8 | 44.1, 44.3 | | host GPU waits (ms a frame) | 39.0, 39.0 | 22.6, 22.2 | | GPU busy | 76%, 74% | 89%, 88% | | replay memory sync (ms a frame) | 0.92-1.17 | 0.78-0.80 | | Byte probe, `RX_VK_LOG_EARLY_CHECK=4` | Kokiri, walking | Kokiri to the Deku Tree and back, twice | |---|---|---| | reads checked | 2,300 | 1,900 | | changed by main's release on pages the CPU did not write (missed) | 0 | 0 | | ambiguous / lost | 0 / 0 | 0 / 0 | https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/303's own version missed 29 of 800 and 21 of 1,100 on these two routes. An earlier port of the fork's head (14e83e9c onto 9c9d69ed, on by default on the Deck), in Hyrule Field native on the same Deck: main 15.20 fps, the port 18.47 (+21.5%), the port with `RX_VK_LATE_DEFER=0` 17.58, the port with `RX_VK_READBACK_EARLY=0` 15.03; its probe found 0 of 4,000 reads missed across the Field, the Lost Woods and the Market. GPU time is the same in every arm: the gain is the CPU no longer waiting, not less GPU work. **With the render thread on, none of it.** `RX_VK_READBACK_EARLY` leaves itself off when the render thread is on (`RX_VK_ASYNC=1`, Threaded rendering; `gpu/vk/readback_early.h`: it was not measured with it). On our Deck OLED on 2026-11-16, Kokiri Forest at handheld settings with `RX_VK_READBACK_EARLY=1`, https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/906's `RX_VK_MESH_FB_LAYOUT=soa` and https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/1053's `RX_VK_GPU_CLOCK=1` all set: 17.8-19.5 fps without the render thread, 14.7-15.1 with it (one run each, not interleaved: an indication, not a measurement of this PR). The Deck's Auto leaves the render thread off. **With the other two Deck switches** (our pack of this PR, https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/906 and https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/1053 on main 9c9d69ed; same Deck OLED, empty shader caches in every run, each arm twice in mirrored order, the median of each; the Field series and Kokiri's two ends also at https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/1053#issuecomment-34267): | median fps | nothing on | early reads | early reads + clock hold | all three | |---|---|---|---|---| | Kokiri Forest, handheld | 11.42 (12.97, 9.87) | 14.05 | 19.05 | 19.61 | | Hyrule Field, native | 15.07 | 20.99 (with the layout) | | 24.64 | The clock hold still pays with the early reads on (14.05 -> 19.05): together they take the host's wait on the GPU from 51.8 to under 1 ms a frame. Nothing broke with all three on (exit 0 in all 18 runs). ``` vk: reads of GPU results written in place wait for their writer's part (RX_VK_READBACK_EARLY) vk: late pages left to their reads on, with the early reads (RX_VK_LATE_DEFER=0: off) vk: late pages left to their reads ... 212 pages in 40 takes (212 kept with their capture), 78 read on the CPU, 0 of them waited (0.00 ms in all); 9 still late (Kokiri, on, run 1) vk: late pages left to their reads ... 384 pages in 69 takes (384 kept with their capture), 138 read on the CPU, 0 of them waited (0.00 ms in all); 8 still late (Field, on) ``` The exposure check: Hyrule Field and the Lost Woods in turn, the camera turning, a shot at each of 7 stops, main against the pack with the switch on. No frame too bright or too dark, no flash and nothing popping at the screen's edges, but the two runs' camera did not stop on the same frame (a turn timed in game frames lands elsewhere when the frame rate differs), so the shots are **not pixel-comparable**. `RX_VK_FLASH=1` is not in these builds. ### Flags and settings Three switches and two diagnostics are added. `RX_VK_READBACK_EARLY`, and `RX_VK_LATE_DEFER` with it, are on by default where the pools have no mirror; the rest are off. | Flag | What it is | Before this PR | After this PR | Effect | Helps / doesn't help / untested | Recommendation | |---|---|---|---|---|---|---| | `RX_VK_READBACK_EARLY` | A read of GPU results written in place waits for its writer's part | absent | on where the pools have no mirror, off with mirrors; `=1` on, `=0` off; never with `RX_VK_PARALLEL` >= 2 or the render thread | Frees the game's jobs from the reads they poll | Helps the Deck OLED: this branch +32.1% in Kokiri handheld (dukenheimer), the pack +21.8% handheld, +23.3% native; no measurable fps change at a play build's settings (frame generation 3x, `RX_VK_READBACK_LATE=newest`, GPU at 1600 MHz), where captures stopped waiting for late pages; left off with the render thread, so no gain there; idle with VRAM mirrors (discrete cards but GCN); a desktop RX 580 (GCN: no mirrors by default) +10 to +24% fps in every phase, GPU time the same, the probe clean (HumanBean); untested on other integrated GPUs and on Intel with https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/443's rule | On where the pools have no mirror (shipped so); `=0` is the first thing to try for a one-frame flash or a HUD or object out of place there; the exposure check's result can still turn it back off (Risks) | | `RX_VK_READBACK_EARLY_GAP` | Draws into a part before a writer may end it | absent | 16 | Writers in a row end one part | 16 measured on the Deck; other values untested | As shipped | | `RX_VK_READBACK_EARLY_RECORDED` | The early read first waits for its writer's submission to be recorded to its end | absent | on (with `RX_VK_READBACK_EARLY`) | The part chosen is the range's last writer's | Measured free in the fork (19.00 to 19.06 fps, Deck LCD) | As shipped; `=0` for an A/B only | | `RX_VK_LATE_DEFER` | A capture's late pages wait for the CPU read that needs them, not at the take | absent | follows `RX_VK_READBACK_EARLY` where pools have no mirror; off with mirrors or `RX_VK_PARALLEL` > 1 | Keeps the replay's memory sync at main's level with the early reads | Helps with the early reads (earlier port: 17.58 to 18.47 fps); with `RX_VK_READBACK_LATE=newest` in a play build, the captures that waited for late pages went from 3 and 39 of 600 to 0 (dukenheimer); gains nothing alone | As shipped | | `RX_VK_LOG_RB_WRITERS` | Diagnostic: what wrote the pages a read waits for (`=1`); release counts only (`=2`) | absent | off | `=1` waits and copies, changing timing | For investigation | Off | | `RX_VK_LOG_EARLY_CHECK` | Diagnostic: one early read in n compared byte by byte with what main's wait returns | absent | off | The checked reads wait as main's would | For probe runs | Off | ## Testing **Tested:** - Steam Deck OLED, SteamOS 3.9.5, RADV, by dukenheimer: this branch on v3.0.0-rc.5 (`1d1263e6`'s tree, built with the release toolchain), Kokiri Forest at handheld settings unset against `=1`, interleaved, and the byte probe on a moving Kokiri route; and the branch in a Deck play build at native with frame generation 3x, two interleaved runs each and two probe runs. No device loss, fault or memory kill. - RX 580 (Polaris10, GCN), Linux, RADV, by HumanBean: this head merged with `main` c167ec67 in a `dev_packs.py` pack, main's build, unset and `=1` twice with the byte probe, over Kokiri Forest, Hyrule Field and the Market, standing and turning (Measurements; https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/977#issuecomment-35943). Unset at main's speed, `=1` +10 to +24%, the probe 0 changed and 0 lost. - Steam Deck OLED, SteamOS 3.9.5, RADV, ours: the pack above (this PR's `gpu/` code on main 9c9d69ed): the byte probe on two Kokiri routes, the timed pairs in Kokiri Forest (handheld) and Hyrule Field (native), the exposure pair. The engaged lines in every on arm, none in the off arms; exit 0, no device loss in any run. - Game-free, Linux x86-64, on this head (`36b0ccbb`, main 13e90a5e): the full `python3 -I -B tests/run_all.py` (clang++ 23.1.5, the Vulkan and SPIR-V headers found) after the rebase: 627 passed, 23 failed, 26 skipped. To name them, the checks that failed in that run or in other recent full runs here (28) were run again on this head and on main 13e90a5e, the same way. 19 fail on both: `test_audio_output_devices`, `test_bug_report_defaults`, `test_crt_options`, `test_failed_compile_keeps_objects`, `test_fsr_uniform_ownership`, `test_guest_imported`, `test_import_reach`, `test_linux_preflight`, `test_linux_update`, `test_make_game_cache`, `test_missing_nup`, `test_mods_size_table`, `test_nonuniform_image_arrays`, `test_parallel_init_order`, `test_pending_guest_import`, `test_replay_write_order`, `test_run_all_changed`, `test_vk_occlusion_queries` and `test_walk_key`. The rest depend on the PC's load, on this branch and on main alike: `test_fpx_sticky` and `test_recorder_sync` timed out in the full run and pass on both; `test_bug_report` and `test_hardware_report` pass on both; `test_pipeline_cache_save` (a timed check) failed once in the re-run here and passes twice alone, and `test_seed_reader` failed once on main and passes here - The renderer built and linked for Windows with llvm-mingw 20261104 on this head (`36b0ccbb`; `RX_VK_ONLY=1 RX_TARGET=windows`: `longshot_vk.dll`, PE32+ x86-64, no warning, no error) - Before the last rebase, on `1d1263e6` (main e4d035e9), with the Vulkan headers found (`LONGSHOT_VULKAN_HEADERS`; earlier runs here skipped the checks that need them): `MAGICBEAN_TOOLING=<Tooling> python3 -I -B tests/run_all.py` (clang++ 23.1.5): 610 passed, 16 failed, 40 skipped. All 16 fail on main e4d035e9 too, run there with the same headers and clang++: `test_bug_report_defaults`, `test_crt_options`, `test_fsr_uniform_ownership`, `test_guest_imported`, `test_import_reach`, `test_linux_preflight`, `test_linux_update`, `test_make_game_cache`, `test_missing_nup`, `test_mods_size_table`, `test_parallel_init_order`, `test_pending_guest_import`, `test_replay_write_order`, `test_run_all_changed`, `test_vk_occlusion_queries`, `test_walk_key`. - `run_all.py --changed forge/main`: with `CXX=g++` 242 passed, 8 failed, 5 skipped; with `CXX=clang++` 241 passed, 9 failed, 5 skipped. The 8 are among the 16 above and fail on main with both compilers. The 9th, `test_pipeline_cache_save` (a timed check, at a load average over 300), failed on main too when run alone there at the same load, passed in the full run and with g++, and passes run directly on this branch; the function it cuts is byte-identical to main's. - The port's checks and the ones pinning main's part ends and the paths the rebase met (`test_readback_early`, `test_late_defer`, `test_early_split_before_draw`, `test_readback_part`, `test_late_fix_build`, `test_capture_wait_log`, `test_frame_trace_waits`, `test_parallel_rb_refresh`, `test_direct_buffer_writes`, `test_split_before_draw`, `test_parallel_small_cb`, `test_split_ub_flush`, `test_srccut`, `test_lost_wait_said`, `test_fsr_image_retirement`): all pass with g++ 13.3 and clang++ 23.1.5, `--strict`. `test_import_reach` (https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/431's) fails to compile its fixture on main e4d035e9 as here, with both compilers (`bg_inflight`'s type in its stand-ins). The three new checks also passed with `SRCCUT_CHECK_NAIVE=1` before the rebase (every cut the same as their old cutter's). - Unset is main: `test_split_before_draw`, `test_parallel_small_cb` and `test_split_ub_flush` are main's, unchanged; `test_late_defer` checks the rule unset on every GPU; `test_early_split_before_draw` gets main's part ends with the switch off. Every new path is behind `readback_early_on()`, `late_defer_on()`, `rb_writers_on()` or `early_check_on()`. - Whole-file `-fsyntax-only` of `vk_replay.cpp`, `vk_memory.cpp` and `vk_draw.cpp` for Linux and Windows (llvm-mingw 20261104), before the rebases onto 27a669ba and e4d035e9 (not run again since; the branch's own lines are unchanged): no error on Linux; on Windows the same errors as main's files, none new. **Not tested:** - This head (`36b0ccbb`) on the Deck: the Deck runs were of `1d1263e6`'s tree and of the pack; the head differs from `1d1263e6` only in the probe's hook and a test stand-in. It has run in the game on an RX 580 (HumanBean). - Intel and NVIDIA, with the switch unset or on; Windows in the game; other integrated GPUs (AMD Phoenix and Strix); other GCN cards (Vega), and the RX 580 in interleaved runs. - https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/443's path in the game (`RX_VK_READBACK_EARLY=1` where parts end only before a plain draw: Intel by default). - The early reads with the render thread: they leave themselves off with it even when asked for. With it on, Kokiri at handheld settings ran 14.7-15.1 fps against 17.8-19.5 without it (one run each, with the clock hold and the layout on; Measurements): a player with Threaded rendering on gets none of the gain. - Walking out of a house and back (the harness could not script it), and an exposure check at the same frames: what can show whether the game, let go early, writes memory a later part still reads (Risks): needed before any default. - `RX_VK_LATE_DEFER=0` on this pack (it was measured on the earlier port only). **Must be tested:** - steamdeck, SteamOS: walk out of Link's house into Kokiri Forest and back several times with `RX_VK_READBACK_EARLY=1` and with it unset, shots at the doorway, expecting no flash or dark frame and nothing popping at the edges against unset (by hand: compare the shots) - windows, nvidia: Hyrule Field and Kokiri Forest, once with `RX_VK_READBACK_EARLY` unset and once with `RX_VK_READBACK_EARLY=1`, expecting the same frame rate and picture as main in both (with VRAM mirrors the switch stays idle) and no error in the log (Test Farm: @Beanstalk windows nvidia, Hyrule Field and Kokiri Forest with RX_VK_READBACK_EARLY unset and with it set to 1, no regression against main) - windows, intel: Kokiri Forest while moving, with `RX_VK_READBACK_EARLY` unset and with `=1`, expecting no white shards in either, the same frame rate as main unset, and with it on the log's `readbacks waited for their writer's part` line (Test Farm: @Beanstalk windows intel, Kokiri Forest moving with RX_VK_READBACK_EARLY unset and with it set to 1, look for white shards and compare fps with main) ## Risks **What could break:** only with `RX_VK_READBACK_EARLY=1`: - A CPU read let go before a later GPU write to the same bytes (a stale culling camera: objects popping at the screen's edges). https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/303's version did it in 2-4% of camera-block reads; the recording wait closed that on the routes probed here (0 of 4,200 on the pack, 0 of 4,000 on the earlier port, 0 of 3,300 on this branch: dukenheimer's 1,100 and 2,200). - A submission recorded in two recordings (it stops at an event wait and resumes under the same value) is let go after its first recording, before a writer the second adds. Main's own wait has the same gap, so it is not a regression, but "every writer of the submission is noted before the part is chosen" does not hold there and the probe can't see it (found reading the code; still open). - With `RX_VK_LATE_DEFER`, which follows the switch, a deferred late page is copied from guest memory at its first CPU read rather than at the take, so a game write made after the submit and before that read lands in the replay's "as submitted" copy: a longer window than the old wait's (design rule 2, "nothing the replay reads comes from live guest memory"). The two mechanisms also come as one switch: on the earlier port the early reads alone gave 17.58 fps and both 18.47 (from the same reading and the earlier port's runs). A Deck OLED tester of the fork found the deferral cost about 2 fps where the replay thread is the limit (internal 768x432 with their own `RX_VK_READBACK_LATE` rule); this PR's handheld runs (640x360 internal) gained with both. - **Not covered by the probe:** with it on, the game's job is let go once its writer's part has run, while the later parts of the same submission may still read guest memory in place. That is why main waits for the whole submission for pages written in place (`invalidate`'s comment: they "keep the wait for all of its submission, whose later parts read and write guest memory itself"), and why main's own part wait, for a mirror's page, is refused when a later part reads guest memory in place (`in_place_after`); the early wait has no such test. The job may then write memory a later part still reads: the next frame's uniforms or vertices, or the exposure meter (the pattern of issue 997). The byte probe cannot see this: `early_check_read` compares only the bytes of the range the CPU reads against what main's wait returns (and a checked read itself waits as main's would), so its 0 missed reads clear the risk above, not this one. The evidence that can clear it is a same-frame exposure check: the same frames with the switch unset and on, compared pixel for pixel. The one here was not pixel-comparable (the camera did not stop on the same frames), so dyst's review (https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/977#issuecomment-35427) asked for no default before such a check. **It is on by default anyway** (`468f360d`, https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/977#issuecomment-37118): the gain is large where it acts, nothing has been seen by eye or probe, and a correct game guards that memory with its own fences. The `RX_VK_LOG_INPLACE_EXPOSURE` runs continue; a higher rate after an early release turns the default back off, or holds what they point at, before the next release. With mirrors (discrete cards) and the switch unset, or with `=0` anywhere, nothing changes: the tracking, the splits, the descriptor rebuilds and the recording tags are behind the switches. **Turning it off:** `RX_VK_READBACK_EARLY=0`; `RX_VK_LATE_DEFER=0` alone for the late pages. **Compatibility:** Nothing changes: no shader translator file (the shader caches stay valid), `MKWVK_API_VERSION` unchanged (`vk_api.h` untouched), no option, ini key or menu row. ### Works with / conflicts with - https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/952 (a recording's uniform copies shared, their reads made once a draw) splits `memory_read_as_submitted` into `submitted_read_unlocked` and `submitted_read_locked` and adds `memory_read_as_submitted_many`, the batched reader of a draw's uniform copies (on by default; on Intel iGPUs without mirrors, where the uniform snapshot `ub_snapshot_on` is on, every uniform range is read through it). This PR's `late_defer_cpu_read` call sits in the per-page loop that moves into `submitted_read_locked`, so the two conflict in `vk_memory.cpp`. Whichever lands second moves the call into `submitted_read_locked`, which both readers go through, and makes `tests/test_late_defer.py` look for it there; otherwise a batched read would take a deferred late page without copying it again. If https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/952 lands first, this PR does it. - https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/906 and https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/1053: measured together with this one (Measurements); no shared lines. ## Checklist ### Everyone - [x] One topic. (The early reads with the two things they need: `RX_VK_LATE_DEFER` gains nothing on its own and exists because the early reads make the late pages the replay would otherwise wait ~19 ms a frame for; it follows the switch and can be turned off alone. The diagnostics are the probe that measured it. Our review of the fork's PR asked for the deferral as its own PR; the authors' commits for it are mainly the first three and could be split out if the maintainers prefer) - [x] The title is a Conventional Commit (`type(scope): description`) of the status line's type. - [x] Why gives the reason plainly and links the issues that fit (the tracker searched). - [x] Measurements has a verdict, or one line saying why there is nothing to measure. - [x] Testing names what was tested, what wasn't, and what must still be tested, where and how. - [x] Rebased on current `main`: at review onto 26ce01b5, commit by commit with no conflict, head `468f360d` (the default on top). Before that: on 13e90a5e, head `36b0ccbb`; it merges with current `main` 71b1edd8 without a conflict (`git merge-tree`). The rebase from e4d035e9 had one conflict, in `vk_memory.cpp`'s `memory_collect_written` with https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/1083's `collect_mu`, resolved in `6cf7a3d1` (the probe's hook, What changed), and added `36b0ccbb` (a test stand-in). The earlier rebase onto e4d035e9 kept https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/700's whole-submission count in `invalidate` (`52160308`, `6d10a356`) and https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/431's walk comment in `memory_note_gpu_write` (`fda1d573`, `6cf7a3d1`), each named in that commit's `[port: ...]` line. Old head `1d1263e6` → new `36b0ccbb`; the hashes in this description are the new ones. - [x] `python -I -B tests/run_all.py` passes (any failure that also fails on `main` is named). Run at `36b0ccbb`; `468f360d` was not run (a code-only review): `tests/test_late_defer.py` and `tests/test_readback_early.py` are the checks it changes. (Testing: the full `python3 -I -B tests/run_all.py` (clang++ 23.1.5, the Vulkan and SPIR-V headers found) after the rebase: 627 passed, 23 failed, 26 skipped. To name them, the checks that failed in that run or in other recent full runs here (28) were run again on this head and on main 13e90a5e, the same way. 19 fail on both: `test_audio_output_devices`, `test_bug_report_defaults`, `test_crt_options`, `test_failed_compile_keeps_objects`, `test_fsr_uniform_ownership`, `test_guest_imported`, `test_import_reach`, `test_linux_preflight`, `test_linux_update`, `test_make_game_cache`, `test_missing_nup`, `test_mods_size_table`, `test_nonuniform_image_arrays`, `test_parallel_init_order`, `test_pending_guest_import`, `test_replay_write_order`, `test_run_all_changed`, `test_vk_occlusion_queries` and `test_walk_key`. The rest depend on the PC's load, on this branch and on main alike: `test_fpx_sticky` and `test_recorder_sync` timed out in the full run and pass on both; `test_bug_report` and `test_hardware_report` pass on both; `test_pipeline_cache_save` (a timed check) failed once in the re-run here and passes twice alone, and `test_seed_reader` failed once on main and passes here) - [x] No game files, dumps, build output or generated files. - [x] `third_party/` untouched (a new folder with a `PROVENANCE.md` instead). - [x] The license is OK: GPL-3.0-or-later, nothing incompatible and nothing decompiled. ### When it applies - [x] AI assistance recorded with `Assisted-by:` / `Co-Authored-By:` trailers. - [x] Numbers from the same PC, scene and settings, at least 2 interleaved runs each, the median reported. (This branch: dukenheimer's unset, `=1`, `=1`, unset after a warm-up, the median of each arm, at handheld settings and in a play build; the pack: ours, the same way. The render-thread pair is one run each and is labelled as an indication; HumanBean's RX 580 run is one session per arm, not interleaved, with `=1` twice, and is labelled so) - [x] Switches documented (docs/development.md; the README for a menu row) and shipped at their recommended value. - [x] A file in `docs/tested/` for each new measured run (format in `docs/tested-configurations.md`). (Ours: `e161d7e6`. dukenheimer's and HumanBean's runs are cited from their comments, as theirs) - [x] A game-free test added for the fix, failing on `main` and passing here. (`tests/test_early_split_before_draw.py`: it fails with the fork's end right after the writer on Intel) - [x] A check under `tests/` that reads changed source updated in this PR, not weakened. - [x] Host code builds on both Windows (llvm-mingw) and Linux, with UTF-8 paths. (The renderer built and linked for Windows with llvm-mingw 20261104 on this head (`36b0ccbb`; `RX_VK_ONLY=1 RX_TARGET=windows`: `longshot_vk.dll`, PE32+ x86-64, no warning, no error); Linux: dukenheimer's release-toolchain build; no path code changed) - [x] Docs updated.
Where the buffer pools have no mirror in VRAM (an integrated GPU: the Steam
Deck), nothing on the GPU reads a capture's copy of a page: the draws bind
guest memory in place. A late page (one the CPU wrote while a submission
still on the GPU writes it) outside the texture storage is left as the
capture copied it, noted, and copied again after its write-back at the
first CPU read of it (memory_read_as_submitted). The texture storage's late
pages wait as before. RX_VK_LATE_DEFER=0: the wait at the take, as before.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
(cherry picked from commit 462103d9c0ddf03be9ac5a09289bbf67abf6aa62)
[port: the fork's names as Longshot's (the fork's attributions in comments dropped)]
Co-Authored-By: Claude Opus 5.5 <[email protected]>
(cherry picked from commit 21b859bc637da43785ea5828255e557d7405ea9f)
[port: the fork's names as Longshot's (compat/longshot_env.h)]
The shadow's spans are whole pools (up to 256 MB) textures were uploaded
from, and OoT's GPU results share pools with textures: on the Steam Deck
every capture that waited had such a page, and no late page was left to its
reads. texture_storage_any (vk_replay.cpp's tex_pages) says whether a
texture the game made has its storage on the page.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
(cherry picked from commit 1645f66f3233286c98073e9bf396b82a70deecb0)
[port: the fork's names as Longshot's]
Diagnostics, off by default. The replay notes the op it records (and a storage buffer's
program) and counts its draws and dispatches; each page a write-back is noted for keeps the
first and last such writer of its newest submission. A read of GPU results that waits then
logs (with RX_VK_LOG_READBACK_WAITS) its writers, their part and draw in the submission, how
long into the wait the last writer's part was done (it waits for that part first, then for the
rest), and the bytes the wait changed before and after that part; every 3000 reads, a summary
by size. On the Steam Deck it named the dispatch that writes the camera block the game's jobs
poll, halfway through its submission, with the data final 12.5 of 18.9 ms before the wait ended.

Co-authored-by: Copilot App <[email protected]>
(cherry picked from commit 5bb1b6eb98)
(cherry picked from commit 258be232d9f32f95f60ae22b20dfccdc760f777b)
Opt-in. On a GPU whose pools have no VRAM mirror (the Steam Deck: every page) a CPU read of
GPU results waited for the whole submission that wrote it. With RX_VK_READBACK_EARLY=1, once a
range's read has waited, every GPU write in place to it keeps the part it was recorded in (any
size), a later read waits for that part when its newest write is the range's newest (else the
whole submission, as before), and the write ends its part right after it (at least
RX_VK_READBACK_EARLY_GAP draws, 16, into the part). A draw of a program that writes storage
buffers builds its descriptors so each write is noted.

Steam Deck, Kokiri Forest, stock clocks: the camera-block reads waited 18.9 ms each, 1.7 now;
the frame's critical path 72.4 -> 57.6 ms; 14-15 -> 17.1 real fps (preliminary, one run).

tests/test_readback_early.py; test_parallel_small_cb and the direct-write fixture follow
maybe_split through split_part and the note in note_direct_write. docs/development.md.

Co-authored-by: Copilot App <[email protected]>
(cherry picked from commit 863fbbe00b)
(cherry picked from commit 5f6b962517c1513ec6c2a4806c0ff90f58a4ebab)
[port: since #443 on main, maybe_split only counts and split_part(cmd) ends the part: the fork's split_part(cmd, true) is split_part(cmd), counted in st_early_splits when a part did end. tests/test_parallel_small_cb.py kept as main has it; tests/test_readback_early.py follows]
On an RTX 5090 (VRAM mirrors: no page written in place) the switch waited early 0 times and cost
~2 fps at M1152 (1.2 at MP165): every waiting read noted its range and walked the ranges under
wb_pages_mu, which made every storage-writing draw rebuild its descriptors, and every op the
replay recorded did an atomic exchange. Now a range is noted only by a read with a page written
in place pending, the descriptor rebuild waits for one, and the per-op check is a load.

Co-authored-by: Copilot App <[email protected]>
(cherry picked from commit 9e4c3ca961)
(cherry picked from commit db62901cb2e5eb98c2201c1d2240301e969e57f0)
tests/test_split_ub_flush.py (#189) cut maybe_split, now a wrapper of split_part, and tests/test_late_fix_build.py (#248) built wb_pages_note without RX_VK_LOG_RB_WRITERS's hook.

Co-authored-by: Copilot App <[email protected]>
(cherry picked from commit 025d8e98d2)
(cherry picked from commit 7bf9e9372fdf9a42b6ff2c39440c51ae99ffe683)
[port: tests/test_split_ub_flush.py left as main has it: main's split_part(cmd) is already the part's end it cuts]
From mooki's review of #303:
- memory_in_place_writer_part skipped pages it had no write for: a range widened over a page the
  same submission wrote before (in a later part) was let go at an earlier part. Now every page of
  the range whose in-place write-back is the read's submission (as memory_writeback_value found
  it) needs that write seen, else the whole submission as before; and a write to any byte of a
  page a waited range reaches is kept for the page (two ranges in one page), while only a write
  of the waited bytes ends its part.
- invalidate: RX_VK_LOG_RB_WRITERS's wait for the writer's part came before the late rule
  (RX_VK_READBACK_LATE), so a read late waited with the diagnostics on. It comes after it now,
  and only for a read that waits.
- RX_VK_LOG_RB_WRITERS's submission span was closed only outside a burst: a burst's kept no last
  part or total work. rb_span_close before every finish_recording.
- tests: the widened range and two ranges of a page (test_readback_early.py), a write really
  over 1 MiB, the diagnostics with a late rule (test_readback_part.py), a burst's span (cut
  out and run).

Co-authored-by: Copilot App <[email protected]>
(cherry picked from commit 52306c0cc3)
(cherry picked from commit 72712e459ea6da5a5853200ea877191682510307)
[port: tests/test_readback_early.py: its checks of maybe_split and split_part as they are since #443]
memory_in_place_writer_part skipped a page whose wb_pages entry named another
submission, but note_direct_write notes a write over 1 MiB in early_pages
only: a later part's large write was missed and the read came back before
it. The fix and the two cases are from zeldaootr's and mooki's reviews of
#303.

Co-Authored-By: zeldaootr <[email protected]>
Co-Authored-By: Claude Opus 5.5 <[email protected]>
(cherry picked from commit b8d45cd6aef3a49a9bf70d127399de1df16e45e1)
A bare storage pointer's guessed range and a wide binding (vk_draw.cpp) are
written in place where pools have no mirror, and no page record held them:
a read let go at an earlier writer's part could come back before them. On
the Steam Deck, Kokiri Forest to the Market, Kakariko and Hyrule Field
(RX_VK_LOG_RB_WRITERS), 3 of 637 waits for the 0x3980 camera block had 4-21
bytes changed after their writer's part. memory_note_gpu_write now notes
them for the early wait. Finding 4 of zeldaootr's review of
#303.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
(cherry picked from commit 5ec95383bd730e1082363e1b945eddb3da342ac0)
[port: the fork's names as Longshot's]
RX_VK_READBACK_EARLY (#303)
is on by default on the Steam Deck's APU (gpu_clock.h's steam_deck rule) while
the render thread is off, where it was measured; elsewhere opt-in, a value given
winning, never with the parallel replay. RX_VK_LATE_DEFER follows it unless set:
alone with RX_VK_READBACK_LATE=newest the late pages' wait held the game back
from reads that cost more. The rules are in gpu/vk/readback_early.h.

Kokiri Forest, Steam Deck LCD, 1280x720, two interleaved runs each: 15.3 fps
with neither, 17.8 / 18.2 with the early reads, 18.6 / 18.6 with both.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
(cherry picked from commit a94c60de0b3ec43f7bdc25d58edc0a1ae5334c6b)
[port: the fork's names as Longshot's; gpu_clock.h (the fork's GPU clock hold, not in Longshot) gives way to readback_early::steam_deck, vk_main.cpp's hw.handheld rule (SteamDeck=1 counts on an integrated GPU only); docs/development.md keeps main's rows beside the fork's]
With the early reads on, the diagnostics' check waited for the last writer its
own record knew (the writes wb_pages_note sees), not for the part the early
wait uses (which also holds the writes over 1 MiB and the bare and wide
bindings), and checked reads the early wait sends to the whole submission:
it counted bytes an early read never misses. Now it waits for the early
wait's part, and leaves those reads out. Diagnostics only: nothing changes
without RX_VK_LOG_RB_WRITERS.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
(cherry picked from commit 808eb437ebddaf57988fae74fb8d50eb85ceafe5)
[port: the fork's names as Longshot's]
Use the same target as the early read and wait for the submission timeline before taking the diagnostic snapshot. Discard timed-out comparisons and update the parallel refresh fixture for the writer hook.

Assisted-by: OpenAI Codex
(cherry picked from commit 7f163a360e04098174b05007d78fffb7d3066866)
Assisted-by: OpenAI Codex
(cherry picked from commit e6807148e2c81079c2b0666f8da9a5d7de69a5dc)
A corrected diagnostic found camera-buffer changes between the early part and whole-submission completion. Since concurrent CPU writes cannot yet be ruled out, preserve the previous waits by default while retaining the experiment for investigation.

Assisted-by: OpenAI Codex
(cherry picked from commit 9b78ed86e4737987ff64814d046d5dcf50078dc1)
[port: the fork's names as Longshot's; the steam_deck rule that stood for gpu_clock.h goes with the default; docs/development.md keeps main's rows]
RX_VK_READBACK_EARLY let a read go at the part of the newest writer of its range recorded so far. A read that came while the
rest of that submission was still being recorded took a writer that was not the range's last: on the Steam Deck over
scene changes, 269 of 745 reads of the culling camera block were let go at dispatch 7984161db3e053b5's part, and
fd958faab99f20a2, recorded after the read, wrote ~1170 of its bytes later in the same submission. Now the early wait first
waits, at most 250 ms, until the recording that writes back with the read's value is finished (rec_open_tag, set where a
recording opens and cleared in finish_recording; early_recorded_wait), so every writer of that submission, the uniform
updates of its last part included, is noted before the part is chosen. Past 250 ms, the whole submission as before.
RX_VK_READBACK_EARLY_RECORDED=0: not waited for, for an A/B. The writers' diagnostic computes the same target.

RX_VK_LOG_RB_WRITERS=2 (and =1): every 3000 reads, by size, the reads let go before their writer's submission was done,
how many while it was still recorded, and how many before a writer of the range recorded later in the same submission
(and of those, the thread's last read of the range for it). =2 does only this, with no wait and no copy.

On the Steam Deck LCD over scene changes, camera block: main's wait, 455 releases, 56 before a later writer (21 the
thread's last read); the early wait without this, 287 releases, 138 before a later writer (65 the last read). Kokiri
Forest standing, interleaved: main's wait 15.77 fps, early 19.00, early with this 19.06.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
(cherry picked from commit cf419f69347a352b94f3c4bd4083f8f35b4b704e)
[port: the fork's names as Longshot's]
This reverts 2f2251d4, which left RX_VK_READBACK_EARLY opt-in everywhere while a read at the early boundary was found
going before a later writer of its range in the same submission. The early wait now waits for that submission to be
recorded to its end first (141708c3), so the part it chooses is the range's last writer's. With that, on the Steam Deck
LCD in Kokiri Forest at 1280x720, three interleaved runs each on one runtime: 15.77 fps median with the reads waiting for
their whole submission (RX_VK_READBACK_EARLY=0), 19.06 with the early reads and the wait for the recording, 19.00 without
that wait; frame time 63.4 -> 52.5 ms, GPU time unchanged at 42.4 ms.

Unset, the switch is on again on the Steam Deck's APU alone (gpu_clock.h steam_deck), not with the render thread or the
parallel replay; elsewhere it stays opt-in, and a value given wins. RX_VK_LATE_DEFER follows it as before.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
(cherry picked from commit c5fa783ede0e4f275732088bf32311a68144ca51)
[port: the fork's names as Longshot's; gpu_clock.h's steam_deck as readback_early::steam_deck again (vk_main.cpp's hw.handheld rule); docs/development.md keeps main's rows]
RX_VK_READBACK_EARLY ends the part a readback's writer was recorded in right after the writer, so the read waits for the
GPU's work up to it. Main's #443 (RX_VK_SPLIT_BEFORE_DRAW, on by default on Intel) lets a part end only right before a
plain draw: on an Intel Arc B580 a part that began with mesh draws drew the plain draws after it as exploded triangles.
An end right after the writer (a dispatch, the culling camera block's) began the next part with whatever came next, a
dispatch or a mesh draw, which is what #443 rules out.

early_split now asks part_ends_before_draw as maybe_split does: where parts end only before a plain draw, the writer's
part runs on to the next one (replay() ends it there), else it ends at once, as on the Steam Deck where the early reads
were measured (AMD: the rule is off there, so nothing changes for those numbers). The writer's part never ends before
the writer, so a read that waits for it still waits for its writer, only for a few more dispatches and mesh draws; with no
plain draw left the submission's end comes first and the read waits for the whole submission, as without the switch.
A part ended that way counts in "parts ended after a readback's writer" when it ends, not when it is asked for.

tests/test_early_split_before_draw.py runs the replay loop's steps with these functions cut from vk_replay.cpp: Intel and
RX_VK_SPLIT_BEFORE_DRAW=1 end at the plain draw and no part begins with a mesh draw or a dispatch; AMD and
RX_VK_SPLIT_BEFORE_DRAW=0 end right after the writer; with the switch off only maybe_split's ends. It fails with the
end at once on Intel.

Assisted-by: Claude
The fork turns the early reads on by default on the Steam Deck's APU. Here they stay opt-in everywhere until the byte probe
has run on the routes where #303's version missed GPU writes (the Kokiri moving route, Kokiri into the Deku Tree) and the
picture is checked where the exposure changes: a read let go at its writer's part is a release main's wait for the whole
submission never makes. RX_VK_LATE_DEFER follows it unless given, so it is off by default too.

- readback_early.h: unset is off on every GPU, whatever the device; the Steam Deck rule that stood for the fork's
  gpu_clock.h goes. Asked for, it is still left off with the parallel replay (RX_VK_PARALLEL >= 2) and now also with the
  render thread (RX_VK_ASYNC, Threaded rendering), which it was not measured with and which records behind the game; the
  log says so. The render thread is asked about only when the switch is given.
- vk_replay.cpp: rec_open and rec_close (the open recording's tag, for the wait for a writer's recording and the release
  probe) do nothing unless RX_VK_READBACK_EARLY or RX_VK_LOG_RB_WRITERS is on, so unset no lock or notify is added to a
  recording's end.
- tests/test_late_defer.py: the rule's cases, unset off on the Deck too, asked for off with the render thread.

Assisted-by: Claude
sheep's probe (Bunny Hood #2) saw the 0x3980 camera block change after the early boundary in 269 of
745 compares; it holds the reader for the whole submission, more than main waits for, and cannot tell
the game's writes from the GPU's. Here one read in RX_VK_LOG_EARLY_CHECK=<n> let go at its writer's
part is copied as it returns, then makes main's own wait for it (invalidate's usual choice for pages
written in place: its submission's newest submitted part, else the whole submission) and is copied
again: what main would have returned, at the time main would have. Those reads come back as main's
would, so a check run is not a measurement. A thread of its own copies the range again once the whole
submission is done (informative: parts recorded after the read, which main misses too). The soft write
watch, which only CPU stores and marked host writes set, tells the game's own writes apart by page,
until the second full collection begun after a copy (a mark published late); a change on a page the
CPU wrote is counted as ambiguous, and the checks with no CPU page are counted apart. GPU writes the
replay records after the read in a later part of the submission are counted by kind.
tests/test_readback_early.py cuts the comparison and the intervals' bookkeeping out and runs them, and
checks the wiring (the sampled read's wait follows invalidate's usual choice).

A first version (non-blocking, its main-boundary copy taken by the thread after a timeline wait) was
reviewed before use: a late copy could miss a write later parts restore, a timeout of the whole
submission's wait discarded the first interval, and a change on a page the CPU also wrote was counted
as the CPU's.

Co-authored-by: Copilot App <[email protected]>
(cherry picked from commit 92004c13f3)
[port: from #303 onto sheep/BunnyHood#2's early reads, as measured on the Steam Deck (0 of 4,000 checked reads missed); #303's vk_render_thread_on is not carried (replay_render_thread_on here); the conflicts were additive]
The rows said what the fork ships: on by default on the Steam Deck's APU. Here both are off unless asked for, the early
reads are left off with the render thread too, and where a part ends only before a plain draw (RX_VK_SPLIT_BEFORE_DRAW)
the writer's part runs on to it. The numbers are the Deck LCD's and OLED's, both on an earlier build; the release counts
of RX_VK_LOG_RB_WRITERS=2 that the row quoted are left out (by how that probe counts, a read let go after its recording
closed is never counted against the early wait); RX_VK_LOG_EARLY_CHECK is what checks it.

Assisted-by: Claude
`tests/test_readback_early.py`, `tests/test_late_defer.py` and `tests/test_early_split_before_draw.py` came with their own
copy of the brace counter (`block`, `between`, `section`); main's `tests/srccut.py` (#27) is now the one a new check
uses. Each cuts the same text as before: all three pass with `SRCCUT_CHECK_NAIVE=1` (every cut also made the old way and
compared), with g++ and clang++, and no marker is missing or repeated. Main's own checks this branch updates
(`test_readback_part`, `test_late_fix_build`, `test_frame_trace_waits`) keep their cutters, as #27 left them.

Assisted-by: Claude
docs(tested): the early reads on the Steam Deck OLED: the byte probe, Kokiri handheld and Hyrule Field native
Some checks failed
Website / Website on Cloudflare (pull_request_target) Has been skipped
PR description / Description follows the template (pull_request_target) Successful in 4s
CI / Conventional Commit title (pull_request) Has been cancelled
CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Has been cancelled
CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Has been cancelled
CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Has been cancelled
CI / Workflow lint (actionlint) (pull_request) Has been cancelled
Guide screenshots / Guide images and local UI source review (pull_request) Has been cancelled
Third-party parts / As their notes say (pull_request) Has been cancelled
Third-party parts / The same as upstream (pull_request) Has been cancelled
Website / Website on Cloudflare (pull_request) Has been cancelled
148f14a09f
Three runs of `RX_VK_READBACK_EARLY` on the Deck, one file each: the byte probe (4,200 reads checked, 0 missed), Kokiri
Forest at handheld settings (median +21.8% fps) and Hyrule Field at 1280x800 (median +23.3%). The pack measured is this
branch's `gpu/` diff on main 9c9d69ed with #443's replay change, as each row says.

Assisted-by: Claude
Author
Contributor

Tested in game: Steam Deck OLED (SteamOS 3.9.5, RADV, Mesa 26.2.0-devel), this branch's gpu/ code as a renderer pack on an older main: Kokiri Forest at handheld settings and Hyrule Field at 1280x800, switch off and on, interleaved; a byte probe on a Kokiri walking route and Kokiri to the Deku Tree and back: 4,200 early reads checked, none different from main's; +21.8% and +23.3% fps, no flash or dark frame seen.

Tested in game: Steam Deck OLED (SteamOS 3.9.5, RADV, Mesa 26.2.0-devel), this branch's gpu/ code as a renderer pack on an older main: Kokiri Forest at handheld settings and Hyrule Field at 1280x800, switch off and on, interleaved; a byte probe on a Kokiri walking route and Kokiri to the Deku Tree and back: 4,200 early reads checked, none different from main's; +21.8% and +23.3% fps, no flash or dark frame seen.
Author
Contributor

Hi @sheep and @dukenheimer,

This PR is your early reads, brought over to Longshot: sheep's work from sheep/BunnyHood#2, built on dukenheimer's #303. Every commit of yours is here with you as its author. We only added three small ones on top: it starts switched off (RX_VK_READBACK_EARLY=1 turns it on), it stays off with the render thread, and it keeps to the rule Longshot added for Intel graphics in #443.

How it did on our Steam Deck OLED:

  • Kokiri Forest at handheld settings: 11.4 to 13.9 fps (+22%).
  • Hyrule Field at 1280x800: 15.0 to 18.5 fps (+23%).
  • dukenheimer's byte check: 4,200 reads compared with what main would have returned, none different, on the two Kokiri routes where #303's first version missed some.

It's yours to take over, whichever way you prefer: we can close this one and you open your own (we're happy to hand over the numbers and how we measured them), you send commits for this branch, or we keep carrying it with your names on it. No rush either way.

Thank you both for the careful work. It's the biggest Steam Deck gain we know of.

zeldaootr

Hi @sheep and @dukenheimer, This PR is your early reads, brought over to Longshot: sheep's work from https://git.magicbean.dev/sheep/BunnyHood/pulls/2, built on dukenheimer's https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/303. Every commit of yours is here with you as its author. We only added three small ones on top: it starts switched off (`RX_VK_READBACK_EARLY=1` turns it on), it stays off with the render thread, and it keeps to the rule Longshot added for Intel graphics in #443. How it did on our Steam Deck OLED: - Kokiri Forest at handheld settings: 11.4 to 13.9 fps (+22%). - Hyrule Field at 1280x800: 15.0 to 18.5 fps (+23%). - dukenheimer's byte check: 4,200 reads compared with what main would have returned, none different, on the two Kokiri routes where #303's first version missed some. It's yours to take over, whichever way you prefer: we can close this one and you open your own (we're happy to hand over the numbers and how we measured them), you send commits for this branch, or we keep carrying it with your names on it. No rush either way. Thank you both for the careful work. It's the biggest Steam Deck gain we know of. zeldaootr
Contributor

Thanks @zeldaootr and @sheep, please carry this forward; I won't take it over. Your #443 part-end rule closes exactly the gap our last probe on #303 found (writes later in the same submission), and 0 differences in 4,200 checked reads with +22-23% on the Deck is the result #303 was missing. I'm closing #303 in favour of this one. Happy to run a Deck OLED A/B or the byte probe on a newer head if that helps the review.

Thanks @zeldaootr and @sheep, please carry this forward; I won't take it over. Your #443 part-end rule closes exactly the gap our last probe on #303 found (writes later in the same submission), and 0 differences in 4,200 checked reads with +22-23% on the Deck is the result #303 was missing. I'm closing #303 in favour of this one. Happy to run a Deck OLED A/B or the byte probe on a newer head if that helps the review.
Author
Contributor

Thanks @dukenheimer, will do. And yes please to the Deck OLED run: the one thing this PR still lacks is a run of the branch itself, since our numbers come from its gpu/ change on an older main. An A/B of the current head (148f14a0) with RX_VK_READBACK_EARLY=1 against unset, plus your byte probe (RX_VK_LOG_EARLY_CHECK=4), would let it leave WIP. One more data point from tonight, same Deck, Kokiri at handheld settings: early reads together with the record layout from #906 gave 11.47 to 14.31 fps (+24.8%), against +24.0% for early reads alone; nothing broke with both on.

Thanks @dukenheimer, will do. And yes please to the Deck OLED run: the one thing this PR still lacks is a run of the branch itself, since our numbers come from its `gpu/` change on an older main. An A/B of the current head (148f14a0) with `RX_VK_READBACK_EARLY=1` against unset, plus your byte probe (`RX_VK_LOG_EARLY_CHECK=4`), would let it leave WIP. One more data point from tonight, same Deck, Kokiri at handheld settings: early reads together with the record layout from https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/906 gave 11.47 to 14.31 fps (+24.8%), against +24.0% for early reads alone; nothing broke with both on.
zeldaootr force-pushed early-reads-port from 148f14a09f
Some checks failed
Website / Website on Cloudflare (pull_request_target) Has been skipped
PR description / Description follows the template (pull_request_target) Successful in 4s
CI / Conventional Commit title (pull_request) Has been cancelled
CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Has been cancelled
CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Has been cancelled
CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Has been cancelled
CI / Workflow lint (actionlint) (pull_request) Has been cancelled
Guide screenshots / Guide images and local UI source review (pull_request) Has been cancelled
Third-party parts / As their notes say (pull_request) Has been cancelled
Third-party parts / The same as upstream (pull_request) Has been cancelled
Website / Website on Cloudflare (pull_request) Has been cancelled
to 1d1263e6be
Some checks failed
PR description / Description follows the template (pull_request_target) Successful in 6s
CI / Conventional Commit title (pull_request) Has been cancelled
CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Has been cancelled
CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Has been cancelled
CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Has been cancelled
CI / Workflow lint (actionlint) (pull_request) Has been cancelled
Guide screenshots / Guide images and local UI source review (pull_request) Has been cancelled
Third-party parts / As their notes say (pull_request) Has been cancelled
Third-party parts / The same as upstream (pull_request) Has been cancelled
Website / Website on Cloudflare (pull_request) Has been cancelled
2026-11-16 05:05:58 +00:00
Compare
Author
Contributor

Rebased onto main e4d035e9 (head 1d1263e6, was 148f14a0): four conflicts, all where main's #700 added the whole-submission count to the readback line (invalidate) and #431 reworded the import walk's comment in memory_note_gpu_write; each was resolved in the commit it arose in, keeping both sides, and named in that commit's [port: ...] line, and the branch's own gpu/ lines are unchanged line for line, so the Deck numbers still describe this code. One commit of ours on top gives tests/fsr_image_retirement_test.cpp a stand-in for rec_close, which finish_recording calls since the early reads (the check needs the Vulkan headers and had been skipped here). Re-run with the Vulkan headers: the port's and the pinning checks with g++ and clang++ (all pass), run_all.py --changed forge/main with both, and the full suite (610 passed, 16 failed, 40 skipped), every failure also failing on main.

Rebased onto main e4d035e9 (head `1d1263e6`, was `148f14a0`): four conflicts, all where main's #700 added the whole-submission count to the readback line (`invalidate`) and #431 reworded the import walk's comment in `memory_note_gpu_write`; each was resolved in the commit it arose in, keeping both sides, and named in that commit's `[port: ...]` line, and the branch's own `gpu/` lines are unchanged line for line, so the Deck numbers still describe this code. One commit of ours on top gives `tests/fsr_image_retirement_test.cpp` a stand-in for `rec_close`, which `finish_recording` calls since the early reads (the check needs the Vulkan headers and had been skipped here). Re-run with the Vulkan headers: the port's and the pinning checks with g++ and clang++ (all pass), `run_all.py --changed forge/main` with both, and the full suite (610 passed, 16 failed, 40 skipped), every failure also failing on main.
Author
Contributor

@dukenheimer the head is now 1d1263e6 (rebased onto current main; the early reads' own gpu/ change is line for line what it was). If you do run it on your Deck, that is the commit to use.

@dukenheimer the head is now 1d1263e6 (rebased onto current main; the early reads' own `gpu/` change is line for line what it was). If you do run it on your Deck, that is the commit to use.
Contributor

Deck OLED: this branch on rc.5 with your recipe, and in a Deck play build

In short: with your recipe, fps went from 15.70 to 20.73 (+32%), and the byte probe found 0 missed of 1,100 reads. In the owner's play build at his settings (frame generation 3x, render 100%, GPU at 1600 MHz) there was no measurable fps change on a moving route, and 0 missed of 2,200 probed reads.

Hardware: Steam Deck OLED, AMD Custom GPU 0932 (RADV VANGOGH), Mesa 26.2.0-devel, SteamOS 3.9.5, 16 GB. Docked, in Game Mode through Steam (not headless).
Builds: Linux, the release toolchain (LLVM 23.1.5, glibc 2.35), x86-64-v3 with the play profile.
Code: v3.0.0-rc.5 e4d035e9 + this PR. I rebased 148f14a0 onto rc.5 before your rebase and hit the same two conflicts (#700's count in invalidate, #431's comment in memory_note_gpu_write); I kept both sides. The tree built matches 1d1263e6 in every built file: git diff against it shows only your rec_close stand-in in tests/fsr_image_retirement_test.cpp.

1. This branch alone, your recipe

Settings, as your run02: output 1280x720, render resolution 50%, the game's own upscaler, quality preset Custom, frame generation off, threaded rendering Auto, stock clocks. Also Profile Docked, so the output and render rows apply as set, and HDR off, since Game Mode would give this display HDR10. The remaining rows are the owner's (anisotropic 2x, the detail rows at 100%, contact shadows off).
Method: warm shader caches copied fresh for each start (taken after the warm-up), RX_VK_PRECOMPILE=0, no longshot_env.txt. Kokiri Forest beside Saria, 20 s to settle, then 45 s standing. A warm-up first, then unset, =1, =1, unset.

Kokiri Forest, 45 s standing K1 unset K2 =1 K3 =1 K4 unset
fps 15.71 20.69 20.78 15.69
frame time p50 / p95 / p99 (ms) 62.5 / 70.1 / 94.1 46.8 / 54.8 / 77.0 46.5 / 54.4 / 76.1 62.4 / 69.9 / 86.0
frames over 100 ms 8 8 8 8
GPU time (ms a frame) 39.1 38.1 38.3 39.0
host time in GPU resource waits (ms a frame) 32.4 5.6 5.9 32.4
GPU busy, mean clock 69%, 925 MHz 81%, 1001 MHz 83%, 1002 MHz 71%, 914 MHz
replay memory sync (ms a frame) 1.55 0.71 0.72 1.60

Median fps: unset 15.70, =1 20.73 (+32.1%). GPU time is the same in both arms; the host's GPU waits drop from 32 to 6 ms a frame. fps and frame times come from the presents; the other rows are medians of the log's periodic lines over each 45 s window.

Byte probe (=1, RX_VK_LOG_EARLY_CHECK=4, RX_VK_LOG_READBACK_WAITS=1; 30 s standing, then our harness's moving Kokiri route for 88 s): 1,100 reads checked (1,090 skipped), 0 missed, 0 ambiguous, 0 lost. After main's release, all 1,100 were unchanged.

A plain start came first (K0: without RX_VK_PRECOMPILE=0, after --prepare-shaders, the same settings, the moving Kokiri route). It reached play at 14.35 fps (p95 119 ms), with RSS at most 6.6 GB and MemAvailable at least 1.4 GB: no memory kill.

2. In a Deck play build: rc.5 + #225 + #227 + #222 + this PR

Settings (the owner's): Profile Docked, output 1280x800, the game's own upscaler at 100%, frame generation 3x, HDR native (HDR10 output in Game Mode). longshot_env.txt sets RX_VK_READBACK_LATE=newest. GPU fixed at 1600 MHz.
Route: the moving Kokiri route, an 80 s window. Warm-up first, then unset, =1, =1, unset.

Kokiri, moving route A1 unset B1 =1 B2 =1 A2 unset
game fps (shown with 3x) 17.1 (49.3) 16.2 (46.6) 16.3 (47.5) 14.7 (41.9)
shown frame time p50 / p95 / p99 (ms) 16.6 / 49.6 / 89.0 16.6 / 50.9 / 103.8 16.6 / 49.8 / 104.6 17.1 / 53.0 / 104.9
shown frames over 100 ms 34 42 42 41
captures that waited for late pages (of the last 600) 3 0 0 39
a capture's wait for write-back ... timed out (its 250 ms cap), lines in the whole run 2 0 0 3
readbacks that waited for their write-back (of the last 3,000), ms each 630, 2.66 1,275, 0.50 1,689, 0.49 775, 2.07
readbacks that waited for their writer's part (of 3,000), ms each - 690, 0.67 918, 0.23 -
MemAvailable at its lowest (MB) 1,049 1,282 1,248 1,303

Median game fps: unset 15.9, =1 16.25. The two unset runs (14.7 and 17.1) bracket both =1 runs, so there is no measurable change at these settings: the moving route varies more than the arms differ. The waits did change: captures stopped waiting for late pages, and the readbacks' waits were shorter.

Byte probe on this build (=1, the same settings):

Kokiri Forest to the Deku Tree and back, twice, with camera turns (RX_VK_LOG_EARLY_CHECK=4) the moving Kokiri route (RX_VK_LOG_EARLY_CHECK=1)
reads checked (skipped) 800 (2,635) 1,400 (415)
missed / ambiguous / lost 0 / 0 / 0 0 / 0 / 0
after main's release: unchanged 800 1,400

At RX_VK_LOG_EARLY_CHECK=4 the moving route completed fewer than 100 checks, so it printed no summary; I ran it again at RX_VK_LOG_EARLY_CHECK=1.

No device loss, device fault, stalled GPU or memory kill in any run. The 250 ms capture-wait cap was also reached once in the warm-up (unset) and once in each probe run.

Suite

Linux container (clang 23.1.5): rc.5 608 passed / 22 failed. rc.5 + this PR (148f14a0 rebased, before the stand-in) 610 / 23; the one more is test_fsr_image_retirement. With a rec_close stand-in, as in 1d1263e6, it passes (28 of 28).

**Deck OLED: this branch on rc.5 with your recipe, and in a Deck play build** In short: with your recipe, fps went from 15.70 to 20.73 (+32%), and the byte probe found 0 missed of 1,100 reads. In the owner's play build at his settings (frame generation 3x, render 100%, GPU at 1600 MHz) there was no measurable fps change on a moving route, and 0 missed of 2,200 probed reads. Hardware: Steam Deck OLED, AMD Custom GPU 0932 (RADV VANGOGH), Mesa 26.2.0-devel, SteamOS 3.9.5, 16 GB. Docked, in Game Mode through Steam (not headless). Builds: Linux, the release toolchain (LLVM 23.1.5, glibc 2.35), `x86-64-v3` with the play profile. Code: v3.0.0-rc.5 `e4d035e9` + this PR. I rebased `148f14a0` onto rc.5 before your rebase and hit the same two conflicts (#700's count in `invalidate`, #431's comment in `memory_note_gpu_write`); I kept both sides. The tree built matches `1d1263e6` in every built file: `git diff` against it shows only your `rec_close` stand-in in `tests/fsr_image_retirement_test.cpp`. ### 1. This branch alone, your recipe Settings, as your run02: output 1280x720, render resolution 50%, the game's own upscaler, quality preset Custom, frame generation off, threaded rendering Auto, stock clocks. Also Profile Docked, so the output and render rows apply as set, and HDR off, since Game Mode would give this display HDR10. The remaining rows are the owner's (anisotropic 2x, the detail rows at 100%, contact shadows off). Method: warm shader caches copied fresh for each start (taken after the warm-up), `RX_VK_PRECOMPILE=0`, no `longshot_env.txt`. Kokiri Forest beside Saria, 20 s to settle, then 45 s standing. A warm-up first, then unset, `=1`, `=1`, unset. | Kokiri Forest, 45 s standing | K1 unset | K2 `=1` | K3 `=1` | K4 unset | |---|---|---|---|---| | fps | 15.71 | 20.69 | 20.78 | 15.69 | | frame time p50 / p95 / p99 (ms) | 62.5 / 70.1 / 94.1 | 46.8 / 54.8 / 77.0 | 46.5 / 54.4 / 76.1 | 62.4 / 69.9 / 86.0 | | frames over 100 ms | 8 | 8 | 8 | 8 | | GPU time (ms a frame) | 39.1 | 38.1 | 38.3 | 39.0 | | host time in GPU resource waits (ms a frame) | 32.4 | 5.6 | 5.9 | 32.4 | | GPU busy, mean clock | 69%, 925 MHz | 81%, 1001 MHz | 83%, 1002 MHz | 71%, 914 MHz | | replay memory sync (ms a frame) | 1.55 | 0.71 | 0.72 | 1.60 | Median fps: unset 15.70, `=1` 20.73 (**+32.1%**). GPU time is the same in both arms; the host's GPU waits drop from 32 to 6 ms a frame. fps and frame times come from the presents; the other rows are medians of the log's periodic lines over each 45 s window. Byte probe (`=1`, `RX_VK_LOG_EARLY_CHECK=4`, `RX_VK_LOG_READBACK_WAITS=1`; 30 s standing, then our harness's moving Kokiri route for 88 s): 1,100 reads checked (1,090 skipped), 0 missed, 0 ambiguous, 0 lost. After main's release, all 1,100 were unchanged. A plain start came first (K0: without `RX_VK_PRECOMPILE=0`, after `--prepare-shaders`, the same settings, the moving Kokiri route). It reached play at 14.35 fps (p95 119 ms), with RSS at most 6.6 GB and MemAvailable at least 1.4 GB: no memory kill. ### 2. In a Deck play build: rc.5 + #225 + #227 + #222 + this PR Settings (the owner's): Profile Docked, output 1280x800, the game's own upscaler at 100%, frame generation 3x, HDR native (HDR10 output in Game Mode). `longshot_env.txt` sets `RX_VK_READBACK_LATE=newest`. GPU fixed at 1600 MHz. Route: the moving Kokiri route, an 80 s window. Warm-up first, then unset, `=1`, `=1`, unset. | Kokiri, moving route | A1 unset | B1 `=1` | B2 `=1` | A2 unset | |---|---|---|---|---| | game fps (shown with 3x) | 17.1 (49.3) | 16.2 (46.6) | 16.3 (47.5) | 14.7 (41.9) | | shown frame time p50 / p95 / p99 (ms) | 16.6 / 49.6 / 89.0 | 16.6 / 50.9 / 103.8 | 16.6 / 49.8 / 104.6 | 17.1 / 53.0 / 104.9 | | shown frames over 100 ms | 34 | 42 | 42 | 41 | | captures that waited for late pages (of the last 600) | 3 | 0 | 0 | 39 | | `a capture's wait for write-back ... timed out` (its 250 ms cap), lines in the whole run | 2 | 0 | 0 | 3 | | readbacks that waited for their write-back (of the last 3,000), ms each | 630, 2.66 | 1,275, 0.50 | 1,689, 0.49 | 775, 2.07 | | readbacks that waited for their writer's part (of 3,000), ms each | - | 690, 0.67 | 918, 0.23 | - | | MemAvailable at its lowest (MB) | 1,049 | 1,282 | 1,248 | 1,303 | Median game fps: unset 15.9, `=1` 16.25. The two unset runs (14.7 and 17.1) bracket both `=1` runs, so there is no measurable change at these settings: the moving route varies more than the arms differ. The waits did change: captures stopped waiting for late pages, and the readbacks' waits were shorter. Byte probe on this build (`=1`, the same settings): | | Kokiri Forest to the Deku Tree and back, twice, with camera turns (`RX_VK_LOG_EARLY_CHECK=4`) | the moving Kokiri route (`RX_VK_LOG_EARLY_CHECK=1`) | |---|---|---| | reads checked (skipped) | 800 (2,635) | 1,400 (415) | | missed / ambiguous / lost | 0 / 0 / 0 | 0 / 0 / 0 | | after main's release: unchanged | 800 | 1,400 | At `RX_VK_LOG_EARLY_CHECK=4` the moving route completed fewer than 100 checks, so it printed no summary; I ran it again at `RX_VK_LOG_EARLY_CHECK=1`. No device loss, device fault, stalled GPU or memory kill in any run. The 250 ms capture-wait cap was also reached once in the warm-up (unset) and once in each probe run. ### Suite Linux container (clang 23.1.5): rc.5 608 passed / 22 failed. rc.5 + this PR (`148f14a0` rebased, before the stand-in) 610 / 23; the one more is `test_fsr_image_retirement`. With a `rec_close` stand-in, as in `1d1263e6`, it passes (28 of 28).
zeldaootr force-pushed early-reads-port from 1d1263e6be
Some checks failed
PR description / Description follows the template (pull_request_target) Successful in 6s
CI / Conventional Commit title (pull_request) Has been cancelled
CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Has been cancelled
CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Has been cancelled
CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Has been cancelled
CI / Workflow lint (actionlint) (pull_request) Has been cancelled
Guide screenshots / Guide images and local UI source review (pull_request) Has been cancelled
Third-party parts / As their notes say (pull_request) Has been cancelled
Third-party parts / The same as upstream (pull_request) Has been cancelled
Website / Website on Cloudflare (pull_request) Has been cancelled
to 36b0ccbb35
Some checks failed
PR description / Description follows the template (pull_request_target) Successful in 4s
CI / Conventional Commit title (pull_request) Has been cancelled
CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Has been cancelled
CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Has been cancelled
CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Has been cancelled
CI / Workflow lint (actionlint) (pull_request) Has been cancelled
Guide screenshots / Guide images and local UI source review (pull_request) Has been cancelled
Third-party parts / As their notes say (pull_request) Has been cancelled
Third-party parts / The same as upstream (pull_request) Has been cancelled
Website / Website on Cloudflare (pull_request) Has been cancelled
2026-11-16 11:18:06 +00:00
Compare
Author
Contributor

Rebased onto main 13e90a5e (head 36b0ccbb); it also merges cleanly with main 8a4aa282. The one conflict was in vk_memory.cpp with the collect_mu of #1083. The early check's collection number is now taken inside main's lock, together with the marks, and its page report runs just after the lock is released, so it adds no lock under collect_mu; the capture's report follows main's new capture_scan. Both calls do nothing unless RX_VK_LOG_EARLY_CHECK is set. One new test-only commit stubs the early check's hooks in tests/test_vk_collect_window.py, which cuts that function out of the source. Every other commit is unchanged by git range-diff, with its author as it was. tests/run_all.py on Linux: 627 passed, 23 failed, 26 skipped; 21 of the failures also fail on main 13e90a5e, and test_fpx_sticky and test_recorder_sync timed out only in the loaded full run and pass alone.

Rebased onto main 13e90a5e (head 36b0ccbb); it also merges cleanly with main 8a4aa282. The one conflict was in `vk_memory.cpp` with the `collect_mu` of https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/1083. The early check's collection number is now taken inside main's lock, together with the marks, and its page report runs just after the lock is released, so it adds no lock under `collect_mu`; the capture's report follows main's new `capture_scan`. Both calls do nothing unless `RX_VK_LOG_EARLY_CHECK` is set. One new test-only commit stubs the early check's hooks in `tests/test_vk_collect_window.py`, which cuts that function out of the source. Every other commit is unchanged by `git range-diff`, with its author as it was. `tests/run_all.py` on Linux: 627 passed, 23 failed, 26 skipped; 21 of the failures also fail on main 13e90a5e, and `test_fpx_sticky` and `test_recorder_sync` timed out only in the loaded full run and pass alone.
dyst left a comment

Careful port, with the authorship trail kept clear. Everything new sits behind readback_early_on(), late_defer_on() or the diagnostics. Reading the unset path (take_capture's restructured wait, the replay loop, same_desc in draw()), it does what main does.

  1. The description is behind the thread. dukenheimer ran this branch on rc.5 (+32.1% on your Kokiri recipe, 0 of 1,100 probed reads missed). That run covers most of what the Verdict, "Not tested" and the unticked "Numbers" box still say is missing. His play-build run belongs in the flag table too: frame generation 3x, RX_VK_READBACK_LATE=newest, GPU at 1600 MHz, no measurable fps change, fewer capture waits. It bounds where the gain shows. The head and base hashes (1d1263e6 on e4d035e9) are also two rebases old.
  2. Main keeps the whole-submission wait for pages written in place because "its later parts read and write guest memory itself". The early path lifts exactly that (your risk 4). The byte probe compares the range the CPU reads against main's wait, which is risk 1. It can't see the job, let go early, writing memory a later part still reads in place. So a same-frame exposure check is what can clear risk 4 before any Deck default. I'd say so in Risks.
  3. This conflicts with #952 in vk_memory.cpp. #952 splits memory_read_as_submitted into submitted_read_unlocked and submitted_read_locked and adds a batched reader. Whichever lands second should move late_defer_cpu_read into submitted_read_locked and update test_late_defer.py. Otherwise the batched uniform reads on Intel iGPUs would read a deferred late page without copying it again.

This is a code read only: nothing was built or run.

🤖 Generated with Claude Code

Careful port, with the authorship trail kept clear. Everything new sits behind `readback_early_on()`, `late_defer_on()` or the diagnostics. Reading the unset path (take_capture's restructured wait, the replay loop, `same_desc` in `draw()`), it does what main does. 1. The description is behind the thread. dukenheimer ran this branch on rc.5 (+32.1% on your Kokiri recipe, 0 of 1,100 probed reads missed). That run covers most of what the Verdict, "Not tested" and the unticked "Numbers" box still say is missing. His play-build run belongs in the flag table too: frame generation 3x, `RX_VK_READBACK_LATE=newest`, GPU at 1600 MHz, no measurable fps change, fewer capture waits. It bounds where the gain shows. The head and base hashes (1d1263e6 on e4d035e9) are also two rebases old. 2. Main keeps the whole-submission wait for pages written in place because "its later parts read and write guest memory itself". The early path lifts exactly that (your risk 4). The byte probe compares the range the CPU reads against main's wait, which is risk 1. It can't see the job, let go early, writing memory a later part still reads in place. So a same-frame exposure check is what can clear risk 4 before any Deck default. I'd say so in Risks. 3. This conflicts with #952 in `vk_memory.cpp`. #952 splits `memory_read_as_submitted` into `submitted_read_unlocked` and `submitted_read_locked` and adds a batched reader. Whichever lands second should move `late_defer_cpu_read` into `submitted_read_locked` and update `test_late_defer.py`. Otherwise the batched uniform reads on Intel iGPUs would read a deferred late page without copying it again. This is a code read only: nothing was built or run. 🤖 Generated with [Claude Code](https://claude.com/claude-code)
@ -1396,6 +1403,7 @@ bool memory_read_as_submitted(uint64_t gpu, void* dst, uint64_t n) {
auto b = cap_bufs.find(e.gen);
if (b == cap_bufs.end() || !b->second.map) continue;
late_fix_cpu_read((uint32_t)page, b->second.buffer, (uint8_t*)b->second.map + (uint64_t)e.slot * 4096);
if (late_defer_any()) late_defer_cpu_read((uint32_t)page, e.gen, e.slot, b->second.alloc, (uint8_t*)b->second.map);
Member

Heads-up: #952 moves this per-page loop into submitted_read_locked(), shared by memory_read_as_submitted and a new memory_read_as_submitted_many, and the two branches conflict here. Whichever lands second should put this hook there, and tests/test_late_defer.py should look for it there, so the batched uniform reads also copy a deferred late page again.

Heads-up: #952 moves this per-page loop into `submitted_read_locked()`, shared by `memory_read_as_submitted` and a new `memory_read_as_submitted_many`, and the two branches conflict here. Whichever lands second should put this hook there, and `tests/test_late_defer.py` should look for it there, so the batched uniform reads also copy a deferred late page again.
@ -3497,0 +4049,4 @@
// (only a read with a page written in place pending: with VRAM mirrors (a dGPU) a page is written back by part already,
// and nothing is noted, split or rebuilt for it: on an RTX 5090 the switch cost ~2 fps doing that and waited early 0 times.
// A sparse mirror's block with no memory yet is written in place for a submission: those pages take the early wait)
if (pending && !late && readback_early_on() && unforced && by_part && whole > completed()) {
Member

Here the game's job is let go while later parts of whole still read guest memory in place: the case main's comment on by_part gives as the reason pages written in place wait for the whole submission. RX_VK_LOG_EARLY_CHECK checks only the read range against main's wait, so it can't catch the job writing memory a later part reads. A same-frame exposure check is the evidence this branch still needs before any default.

Here the game's job is let go while later parts of `whole` still read guest memory in place: the case main's comment on `by_part` gives as the reason pages written in place wait for the whole submission. `RX_VK_LOG_EARLY_CHECK` checks only the read range against main's wait, so it can't catch the job writing memory a later part reads. A same-frame exposure check is the evidence this branch still needs before any default.
Author
Contributor

Description brought up to the thread (dyst's review). dukenheimer's run of this branch now leads Measurements: +32.1% on the Kokiri handheld recipe and 0 of 1,100 probed reads missed. Their play-build run showed no measurable fps change and fewer capture waits. The hashes are this head's (36b0ccbb on 13e90a5e; it merges cleanly with 71b1edd8). Risks now says plainly that the byte probe can't see the game, let go early, writing memory a later part still reads, and that a same-frame exposure check comes before any Deck default. Also added: the conflict with #952 and who moves the late-page hook. The early reads leave themselves off with the render thread: Kokiri handheld ran 17.8-19.5 fps without it and 14.7-15.1 with it, one run each. The three-switch series is in too. The Numbers box is ticked. No code changed.

Description brought up to the thread (dyst's review). dukenheimer's run of this branch now leads Measurements: +32.1% on the Kokiri handheld recipe and 0 of 1,100 probed reads missed. Their play-build run showed no measurable fps change and fewer capture waits. The hashes are this head's (`36b0ccbb` on 13e90a5e; it merges cleanly with 71b1edd8). Risks now says plainly that the byte probe can't see the game, let go early, writing memory a later part still reads, and that a same-frame exposure check comes before any Deck default. Also added: the conflict with https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/952 and who moves the late-page hook. The early reads leave themselves off with the render thread: Kokiri handheld ran 17.8-19.5 fps without it and 14.7-15.1 with it, one run each. The three-switch series is in too. The Numbers box is ticked. No code changed.
Author
Contributor

Thanks @dyst, all three hold, and the description now says so.

  1. Behind the thread: fixed. dukenheimer's run of this branch now leads Measurements: Kokiri at handheld settings 15.70 to 20.73 fps (+32.1%), 0 of 1,100 probed reads missed. Their play-build run is there too (frame generation 3x, RX_VK_READBACK_LATE=newest, GPU at 1600 MHz: no measurable fps change, the captures' waits for late pages 3 and 39 of 600 down to 0). It is also in the flag table's row, as the bound on where the gain shows. The Verdict, Tested / Not tested and the Numbers box follow from it. The hashes are this head's: 36b0ccbb on 13e90a5e, which merges cleanly with 71b1edd8. git range-diff against their 1d1263e6 shows only the probe's hook and one test stand-in changed, so their fps numbers hold for this head.

  2. What the probe can't see: agreed, and checked in the code. invalidate's comment gives the later parts reading and writing guest memory as the reason pages written in place wait for the whole submission. Main's own part wait for a mirror's page is refused when a later part reads in place (in_place_after). The early branch waits for max(writer, part) with no such test. early_check_read copies only [gpu, gpu + size) before and after main's wait, and the checked read itself then waits as main's would. So its 0 missed reads clear risk 1 and say nothing about the game's job, let go early, writing memory a later part still reads. Risks now says that plainly. It also says that a same-frame exposure check is the evidence needed before any Deck default; our exposure pair wasn't pixel-comparable. The flag table's recommendation says the same.

  3. #952: agreed. In its split the per-page loop lives in submitted_read_locked, which both memory_read_as_submitted and memory_read_as_submitted_many go through. The latter is ub_reads_flush's batched read of a draw's uniform copies. It's on by default, and on Intel iGPUs without mirrors (the uniform snapshot, ub_snapshot_on) every uniform range goes through it, while RX_VK_LATE_DEFER follows the early reads there. So whichever lands second must move the hook. If #952 lands first, we'll move late_defer_cpu_read into submitted_read_locked here. We'll also make tests/test_late_defer.py's source check look for it there. That check now cuts memory_read_as_submitted and asks that it takes the late copy again before copying from it. The conflict and who does what are under Risks, "Works with / conflicts with".

One more thing in the update: the early reads leave themselves off with the render thread (RX_VK_ASYNC=1). With the clock hold and the record layout on, Kokiri at handheld settings ran 17.8-19.5 fps without the render thread and 14.7-15.1 with it (one run each), so Threaded rendering forgoes the gain. That's now in Measurements and Not tested.

Thanks @dyst, all three hold, and the description now says so. 1. **Behind the thread:** fixed. dukenheimer's run of this branch now leads Measurements: Kokiri at handheld settings 15.70 to 20.73 fps (+32.1%), 0 of 1,100 probed reads missed. Their play-build run is there too (frame generation 3x, `RX_VK_READBACK_LATE=newest`, GPU at 1600 MHz: no measurable fps change, the captures' waits for late pages 3 and 39 of 600 down to 0). It is also in the flag table's row, as the bound on where the gain shows. The Verdict, Tested / Not tested and the Numbers box follow from it. The hashes are this head's: `36b0ccbb` on 13e90a5e, which merges cleanly with 71b1edd8. `git range-diff` against their `1d1263e6` shows only the probe's hook and one test stand-in changed, so their fps numbers hold for this head. 2. **What the probe can't see:** agreed, and checked in the code. `invalidate`'s comment gives the later parts reading and writing guest memory as the reason pages written in place wait for the whole submission. Main's own part wait for a mirror's page is refused when a later part reads in place (`in_place_after`). The early branch waits for `max(writer, part)` with no such test. `early_check_read` copies only `[gpu, gpu + size)` before and after main's wait, and the checked read itself then waits as main's would. So its 0 missed reads clear risk 1 and say nothing about the game's job, let go early, writing memory a later part still reads. Risks now says that plainly. It also says that a same-frame exposure check is the evidence needed before any Deck default; our exposure pair wasn't pixel-comparable. The flag table's recommendation says the same. 3. **https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/952:** agreed. In its split the per-page loop lives in `submitted_read_locked`, which both `memory_read_as_submitted` and `memory_read_as_submitted_many` go through. The latter is `ub_reads_flush`'s batched read of a draw's uniform copies. It's on by default, and on Intel iGPUs without mirrors (the uniform snapshot, `ub_snapshot_on`) every uniform range goes through it, while `RX_VK_LATE_DEFER` follows the early reads there. So whichever lands second must move the hook. If https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/952 lands first, we'll move `late_defer_cpu_read` into `submitted_read_locked` here. We'll also make `tests/test_late_defer.py`'s source check look for it there. That check now cuts `memory_read_as_submitted` and asks that it takes the late copy again before copying from it. The conflict and who does what are under Risks, "Works with / conflicts with". One more thing in the update: the early reads leave themselves off with the render thread (`RX_VK_ASYNC=1`). With the clock hold and the record layout on, Kokiri at handheld settings ran 17.8-19.5 fps without the render thread and 14.7-15.1 with it (one run each), so Threaded rendering forgoes the gain. That's now in Measurements and Not tested.
zeldaootr changed title from WIP: perf(vk): early reads: a read of GPU results waits for its writer's part, not its whole submission (RX_VK_READBACK_EARLY, off by default) to perf(vk): early reads: a read of GPU results waits for its writer's part, not its whole submission (RX_VK_READBACK_EARLY, off by default) 2026-11-16 13:34:56 +00:00
Owner

Tested on a desktop GCN card. The description has a Steam Deck only.

Hardware: RX 580 8 GB (Polaris10), RADV, 8 threads, KDE.
Build: this PR's head merged with main c167ec67, as a dev_packs pack on a base of that main.
Route: Kokiri Forest, Hyrule Field, the Market (debug teleport), Hyrule Field again; standing, then turning. One session per arm.
Arms: E0 is this PR with RX_VK_READBACK_EARLY unset. E1 is =1, with the byte probe RX_VK_LOG_EARLY_CHECK=4, run twice. M is main's own build.

fps (GPU ms) main E0 (unset) E1, run 1 E1, run 2
Kokiri, standing 20.9 (35.60) 21.0 (35.70) 24.5 (35.40) 24.2 (35.50)
Kokiri, turning 22.4 (31.70) 22.7 (31.20) 26.3 (31.35) 26.3 (31.40)
Hyrule Field, standing 27.5 (24.30) 27.8 (24.10) 33.7 (23.80) 32.3 (24.10)
Hyrule Field, turning 33.6 (20.25) 34.3 (19.90) 39.6 (19.75) 40.0 (19.60)
the Market, standing 24.2 (28.90) 24.7 (28.35) 26.6 (28.50) 27.1 (27.80)
the Market, turning 36.5 (17.50) 35.8 (17.70) 44.6 (17.55) 44.5 (16.90)
Hyrule Field again, turning 34.6 (20.05) 32.9 (20.05) 38.2 (19.95) 39.5 (19.60)
  • Readbacks: 364 to 421 of 3,000 waited 10 to 16 ms each with it off. With it on, 637 to 652 waited for their writer's part, 4 to 6 ms each.
  • fps: up 10 to 24% in every phase, in both E1 runs. GPU time is unchanged, so the wait was CPU-side.
  • Byte check: early -> main's release: 100 unchanged, 0 changed on a page the CPU did not ... 0 lost in both runs.
  • Off by default: E0 is main's speed, so unset changes nothing.

A related result from the same lab, for whoever looks at GCN next. On plain main, RX_VK_MIRROR=1 gave +40 to +55% fps on this card: Kokiri 20.9 to 29.6, Hyrule Field 27.5 to 42.3, the Market standing 24.2 to 31.6. mirror_on() keeps the mirrors off on GCN because of an older RX 580 measurement. That's a separate change, and I haven't run it with this PR's switch together.

🤖 Generated with Claude Code

Tested on a desktop GCN card. The description has a Steam Deck only. **Hardware:** RX 580 8 GB (Polaris10), RADV, 8 threads, KDE. **Build:** this PR's head merged with `main` c167ec67, as a dev_packs pack on a base of that main. **Route:** Kokiri Forest, Hyrule Field, the Market (debug teleport), Hyrule Field again; standing, then turning. One session per arm. **Arms:** E0 is this PR with `RX_VK_READBACK_EARLY` unset. E1 is `=1`, with the byte probe `RX_VK_LOG_EARLY_CHECK=4`, run twice. M is main's own build. | fps (GPU ms) | main | E0 (unset) | E1, run 1 | E1, run 2 | |---|---|---|---|---| | Kokiri, standing | 20.9 (35.60) | 21.0 (35.70) | **24.5** (35.40) | **24.2** (35.50) | | Kokiri, turning | 22.4 (31.70) | 22.7 (31.20) | **26.3** (31.35) | **26.3** (31.40) | | Hyrule Field, standing | 27.5 (24.30) | 27.8 (24.10) | **33.7** (23.80) | **32.3** (24.10) | | Hyrule Field, turning | 33.6 (20.25) | 34.3 (19.90) | **39.6** (19.75) | **40.0** (19.60) | | the Market, standing | 24.2 (28.90) | 24.7 (28.35) | **26.6** (28.50) | **27.1** (27.80) | | the Market, turning | 36.5 (17.50) | 35.8 (17.70) | **44.6** (17.55) | **44.5** (16.90) | | Hyrule Field again, turning | 34.6 (20.05) | 32.9 (20.05) | **38.2** (19.95) | **39.5** (19.60) | - **Readbacks:** 364 to 421 of 3,000 waited 10 to 16 ms each with it off. With it on, 637 to 652 waited for their writer's part, 4 to 6 ms each. - **fps:** up 10 to 24% in every phase, in both E1 runs. GPU time is unchanged, so the wait was CPU-side. - **Byte check:** `early -> main's release: 100 unchanged, 0 changed on a page the CPU did not ... 0 lost` in both runs. - **Off by default:** E0 is main's speed, so unset changes nothing. A related result from the same lab, for whoever looks at GCN next. On plain main, `RX_VK_MIRROR=1` gave +40 to +55% fps on this card: Kokiri 20.9 to 29.6, Hyrule Field 27.5 to 42.3, the Market standing 24.2 to 31.6. `mirror_on()` keeps the mirrors off on GCN because of an older RX 580 measurement. That's a separate change, and I haven't run it with this PR's switch together. 🤖 Generated with [Claude Code](https://claude.com/claude-code)
Author
Contributor

Description updated, no code change (head still 36b0ccbb): HumanBean's RX 580 run (#977 (comment)) is in the Verdict, Measurements, Testing and the switch's row. On a desktop GCN card, whose pools have no VRAM mirror by default, this head gave fps 10-24% higher in every phase with =1, GPU time the same, the byte probe 0 changed, and unset at main's speed. It is the first run of this head in the game, and the first off the Deck. Must be tested is unchanged: the Deck's same-frame exposure check, Windows with NVIDIA and Windows with Intel all remain.

Description updated, no code change (head still `36b0ccbb`): HumanBean's RX 580 run (https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/977#issuecomment-35943) is in the Verdict, Measurements, Testing and the switch's row. On a desktop GCN card, whose pools have no VRAM mirror by default, this head gave fps 10-24% higher in every phase with `=1`, GPU time the same, the byte probe 0 changed, and unset at main's speed. It is the first run of this head in the game, and the first off the Deck. Must be tested is unchanged: the Deck's same-frame exposure check, Windows with NVIDIA and Windows with Intel all remain.
Author
Contributor

Thanks @HumanBean. This is the first run of this head in the game and the first off the Deck: +10-24% in every phase on the RX 580 with GPU time unchanged, the byte probe clean in both runs, and unset at main's speed. It's in the description now (Verdict, Measurements, Testing and the switch's row), labelled as one session per arm. It fits the rule: main keeps the VRAM mirrors off on GCN, so the early reads act there as they do on the Deck. It doesn't clear anything under Must be tested, though. The Deck's same-frame exposure check, Windows with NVIDIA and Windows with Intel are all still open.

Thanks for the RX_VK_MIRROR=1 result too. +40-55% on plain main is a big number, and worth a look of its own. The rule that keeps mirrors off on GCN came from an older RX 580 measurement. It predates the sparse mirrors (#250) that your main already had, so that measurement may no longer hold. It's a separate change, and we haven't measured it. Note that with mirrors on, this PR's early reads stand aside (they act only where the pools have no mirror), so the two gains won't add up on that card.

Thanks @HumanBean. This is the first run of this head in the game and the first off the Deck: +10-24% in every phase on the RX 580 with GPU time unchanged, the byte probe clean in both runs, and unset at main's speed. It's in the description now (Verdict, Measurements, Testing and the switch's row), labelled as one session per arm. It fits the rule: main keeps the VRAM mirrors off on GCN, so the early reads act there as they do on the Deck. It doesn't clear anything under Must be tested, though. The Deck's same-frame exposure check, Windows with NVIDIA and Windows with Intel are all still open. Thanks for the `RX_VK_MIRROR=1` result too. +40-55% on plain main is a big number, and worth a look of its own. The rule that keeps mirrors off on GCN came from an older RX 580 measurement. It predates the sparse mirrors (https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/250) that your main already had, so that measurement may no longer hold. It's a separate change, and we haven't measured it. Note that with mirrors on, this PR's early reads stand aside (they act only where the pools have no mirror), so the two gains won't add up on that card.
Owner

Suggest turning this on by default where it measured a gain, rather than leaving it opt-in: RX_VK_READBACK_EARLY on for AMD/RADV, at least wherever the VRAM mirrors are off (integrated GPUs, the Deck, GCN).

The evidence:

  • RX 580, my run above: fps +10 to +24% in every phase. The readback waits went from 10 to 16 ms each to 4 to 6 ms, and GPU time is unchanged. The byte probe found 100 unchanged, 0 changed ... 0 lost in both sessions.
  • Steam Deck, yours: +22 to 32%. With #906's layout and #1053's hold together it reached 15.07 → 24.64 fps in Hyrule Field.
  • NVIDIA: not measured here. Leave it as it is there until someone runs it on a card with the mirrors on.

Off by default, the gain reaches only players who know the switch. Keep RX_VK_READBACK_EARLY=0 as the way back.

🤖 Generated with Claude Code

Suggest turning this on by default where it measured a gain, rather than leaving it opt-in: `RX_VK_READBACK_EARLY` **on for AMD/RADV**, at least wherever the VRAM mirrors are off (integrated GPUs, the Deck, GCN). The evidence: - **RX 580, my run above:** fps +10 to +24% in every phase. The readback waits went from 10 to 16 ms each to 4 to 6 ms, and GPU time is unchanged. The byte probe found `100 unchanged, 0 changed ... 0 lost` in both sessions. - **Steam Deck, yours:** +22 to 32%. With #906's layout and #1053's hold together it reached 15.07 → 24.64 fps in Hyrule Field. - **NVIDIA:** not measured here. Leave it as it is there until someone runs it on a card with the mirrors on. Off by default, the gain reaches only players who know the switch. Keep `RX_VK_READBACK_EARLY=0` as the way back. 🤖 Generated with [Claude Code](https://claude.com/claude-code)
Author
Contributor

Thanks @HumanBean. We want this on by default on the Deck too, and we are not there yet, for the reason in @dyst's review: the byte probe checks the range the CPU reads, and cannot see the game, let go early, writing memory that a later part of the same submission still reads in place. On GPUs without mirrors that read happens when the GPU runs the part, so the window is real by construction; main has had a bug of this kind before (the stretched HUD and white flashes of issue 997). Whether this game writes in that window is what we do not know.

That check is running on a Steam Deck today: a log-only diagnostic (RX_VK_LOG_INPLACE_EXPOSURE, on a branch on top of this one) samples parts, copies what they bind in place before the submit, and compares after the GPU has finished, with the early reads off and on. If the rate after an early release is no higher than without one, we will send the default for AMD without mirrors as a follow-up with that evidence; if it is higher, the addresses say what to snapshot or hold. Results will be posted here either way.

One more input for the default: the early reads leave themselves off with the render thread, so the default would only reach players whose Threaded rendering is off (Auto leaves it off on the Deck's 8 threads).

Thanks @HumanBean. We want this on by default on the Deck too, and we are not there yet, for the reason in @dyst's review: the byte probe checks the range the CPU reads, and cannot see the game, let go early, writing memory that a later part of the same submission still reads in place. On GPUs without mirrors that read happens when the GPU runs the part, so the window is real by construction; main has had a bug of this kind before (the stretched HUD and white flashes of issue 997). Whether this game writes in that window is what we do not know. That check is running on a Steam Deck today: a log-only diagnostic (`RX_VK_LOG_INPLACE_EXPOSURE`, on a branch on top of this one) samples parts, copies what they bind in place before the submit, and compares after the GPU has finished, with the early reads off and on. If the rate after an early release is no higher than without one, we will send the default for AMD without mirrors as a follow-up with that evidence; if it is higher, the addresses say what to snapshot or hold. Results will be posted here either way. One more input for the default: the early reads leave themselves off with the render thread, so the default would only reach players whose Threaded rendering is off (Auto leaves it off on the Deck's 8 threads).
Owner

A quick check on an RTX 3090 (NVIDIA 620.70.06, Linux), and a correction to my default-on suggestion above.

Setup: this PR rebased onto main @ 43f40b87 (no conflicts). One session each of RX_VK_READBACK_EARLY unset and =1 with RX_VK_LOG_EARLY_CHECK=4, both with RX_VK_ASYNC=0, since early reads never run with the render thread and Auto turns it on with 24 threads. DLAA 2560x1440, Hyrule Field then the Market.

On a discrete card with VRAM mirrors it doesn't act:

  • With =1, every line reads readbacks waited for their writer's part (RX_VK_READBACK_EARLY): 0, 0.00 ms each.
  • The reads it handles are GPU results written in place, which here go through the mirrors' write-back instead. Those waits are already short: 1,219 to 1,295 of 3,000 readbacks waited 0.16 to 0.65 ms each, either arm, against 10 to 16 ms on the RX 580.
  • fps differs only within this PC's noise (Market turning 60.3 unset against 54.8, Field walking 69.4 against 50.9 with one swap-heavy phase), so there's no gain or loss to report.

Correction to "on for AMD/RADV": its gain lives where the pools have no mirrors. #1187 turns the VRAM mirrors on for discrete GCN cards: on the same RX 580 they gave +28 to +54%, against this PR's +10 to +24%. With #1187 in, GCN no longer takes this path either. So the right default is on for integrated GPUs, where mirrors stay off by design: the Steam Deck (your +22 to 32%) and AMD APUs. Discrete cards don't need it.

🤖 Generated with Claude Code

A quick check on an RTX 3090 (NVIDIA 620.70.06, Linux), and a correction to my default-on suggestion above. **Setup:** this PR rebased onto `main` @ 43f40b87 (no conflicts). One session each of `RX_VK_READBACK_EARLY` unset and `=1` with `RX_VK_LOG_EARLY_CHECK=4`, both with `RX_VK_ASYNC=0`, since early reads never run with the render thread and Auto turns it on with 24 threads. DLAA 2560x1440, Hyrule Field then the Market. **On a discrete card with VRAM mirrors it doesn't act:** - With `=1`, every line reads `readbacks waited for their writer's part (RX_VK_READBACK_EARLY): 0, 0.00 ms each`. - The reads it handles are GPU results written in place, which here go through the mirrors' write-back instead. Those waits are already short: 1,219 to 1,295 of 3,000 readbacks waited 0.16 to 0.65 ms each, either arm, against 10 to 16 ms on the RX 580. - fps differs only within this PC's noise (Market turning 60.3 unset against 54.8, Field walking 69.4 against 50.9 with one swap-heavy phase), so there's no gain or loss to report. **Correction to "on for AMD/RADV":** its gain lives where the pools have no mirrors. #1187 turns the VRAM mirrors on for discrete GCN cards: on the same RX 580 they gave +28 to +54%, against this PR's +10 to +24%. With #1187 in, GCN no longer takes this path either. So the right default is **on for integrated GPUs**, where mirrors stay off by design: the Steam Deck (your +22 to 32%) and AMD APUs. Discrete cards don't need it. 🤖 Generated with [Claude Code](https://claude.com/claude-code)
Owner

We're turning RX_VK_READBACK_EARLY on by default in this PR rather than in a follow-up. I'll push it on top of a rebase, and the review follows.

Why now:

  • Where it acts, it helps: the Steam Deck +22 to 32% (yours and dukenheimer's), the RX 580 +10 to 24%.
  • Where it doesn't help, it doesn't run: on the RTX 3090, with mirrors, 0 reads took the early path.
  • Same scope as the switch's own rule: pools without a VRAM mirror (the Deck, AMD APUs, Intel iGPUs, and GCN until #1187). It still stands aside with the render thread and with RX_VK_PARALLEL >= 2. RX_VK_READBACK_EARLY=0 is the way back.

What we're accepting: the exposure check, the risk dyst raised, isn't done. If the game writes memory that a later part of the submission still reads in place, it would show as an occasional one-frame glitch: a flash, or the HUD or an object out of place. A correct game guards that memory with its own fences, and the byte probe and the runs so far saw nothing. So we'd rather ship the gain and name the switch as the first thing to try for that kind of report.

Please keep the RX_VK_LOG_INPLACE_EXPOSURE runs going and post them here. If they show a higher rate after an early release, we turn the default back off, or hold what they point at, before the next release.

🤖 Generated with Claude Code

We're turning `RX_VK_READBACK_EARLY` on by default in this PR rather than in a follow-up. I'll push it on top of a rebase, and the review follows. **Why now:** - **Where it acts, it helps:** the Steam Deck +22 to 32% (yours and dukenheimer's), the RX 580 +10 to 24%. - **Where it doesn't help, it doesn't run:** on the RTX 3090, with mirrors, 0 reads took the early path. - **Same scope as the switch's own rule:** pools without a VRAM mirror (the Deck, AMD APUs, Intel iGPUs, and GCN until https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/1187). It still stands aside with the render thread and with `RX_VK_PARALLEL` >= 2. `RX_VK_READBACK_EARLY=0` is the way back. **What we're accepting:** the exposure check, the risk dyst raised, isn't done. If the game writes memory that a later part of the submission still reads in place, it would show as an occasional one-frame glitch: a flash, or the HUD or an object out of place. A correct game guards that memory with its own fences, and the byte probe and the runs so far saw nothing. So we'd rather ship the gain and name the switch as the first thing to try for that kind of report. Please keep the `RX_VK_LOG_INPLACE_EXPOSURE` runs going and post them here. If they show a higher rate after an early release, we turn the default back off, or hold what they point at, before the next release. 🤖 Generated with [Claude Code](https://claude.com/claude-code)
HumanBean force-pushed early-reads-port from 36b0ccbb35
Some checks failed
PR description / Description follows the template (pull_request_target) Successful in 4s
CI / Conventional Commit title (pull_request) Has been cancelled
CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Has been cancelled
CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Has been cancelled
CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Has been cancelled
CI / Workflow lint (actionlint) (pull_request) Has been cancelled
Guide screenshots / Guide images and local UI source review (pull_request) Has been cancelled
Third-party parts / As their notes say (pull_request) Has been cancelled
Third-party parts / The same as upstream (pull_request) Has been cancelled
Website / Website on Cloudflare (pull_request) Has been cancelled
to 468f360d63
Some checks failed
CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Has been skipped
CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Has been skipped
CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Has been skipped
Third-party parts / The same as upstream (pull_request) Has been skipped
Website / Website on Cloudflare (pull_request) Has been skipped
CI / Conventional Commit title (pull_request) Successful in 8s
Guide screenshots / Guide images and local UI source review (pull_request) Failing after 10s
CI / Workflow lint (actionlint) (pull_request) Successful in 11s
Third-party parts / As their notes say (pull_request) Successful in 13s
PR description / Description follows the template (pull_request_target) Successful in 5s
2026-11-16 17:32:56 +00:00
Compare
HumanBean changed title from perf(vk): early reads: a read of GPU results waits for its writer's part, not its whole submission (RX_VK_READBACK_EARLY, off by default) to perf(vk): early reads: a read of GPU results waits for its writer's part, not its whole submission (RX_VK_READBACK_EARLY, on without VRAM mirrors) 2026-11-16 17:33:36 +00:00
HumanBean left a comment

I read it as code only and ran nothing. I've pushed a rebase and a commit that turns the early reads on by default where the pools have no VRAM mirror, as decided in #977 (comment). dyst's three points still stand, and I haven't repeated them.

Pushed (head 468f360d):

  • Rebase: the 24 commits are rebased one by one onto main 26ce01b5, with no conflict, and their authors kept.
  • 468f360d, the default:
    • readback_early::Facts takes mirrors, from mirror_on(). Unset now gives ON_DEFAULT without mirrors and OFF_MIRRORS with them. =0 is off everywhere, and =1 is on with mirrors too.
    • The render thread and RX_VK_PARALLEL >= 2 still leave it off. They are now asked whether the switch is set or not: async_on() is decided at start, and RX_VK_PARALLEL defaults to 0.
    • RX_VK_LATE_DEFER follows it unchanged.
    • mirror_on() is declared in the anonymous namespace before readback_early_on, where it is defined.
    • tests/test_late_defer.py's rule checks and tests/test_readback_early.py's runs cover unset with and without mirrors, =0, and =1 with mirrors.
    • Also updated: the doc rows, invalidate's "(opt-in)" comment, the title and the description.

What I checked in the code:

  • invalidate's early branch only runs with a page written in place still pending (whole > completed()), so on a mirrored card it does nothing, as the RTX 3090 run showed. Where no writer is known, or the recording wait passes 250 ms, it falls back to main's whole-submission wait.
  • early_recorded_wait: if the writer's submission hasn't started recording, it returns at once. That is safe, because no writer is known yet, so the read waits for the whole submission.
  • Everything new is behind readback_early_on(), late_defer_on() and the diagnostics. With =0, or with mirrors and the switch unset, the path is main's, as dyst read it.

What the default now ships with, from your own Risks section, accepted in the decision comment:

  • Later parts read guest memory in place: the early wait has no in_place_after guard, which main's own per-part wait has.
  • A submission recorded in two recordings: it can be let go before the second recording's writer.
  • Design rule 2 ("nothing the replay reads comes from live guest memory"): RX_VK_LATE_DEFER gives it a longer window than the old wait did.

If any of these shows up, it would be a one-frame glitch on an integrated GPU, and the doc row names RX_VK_READBACK_EARLY=0 as the first thing to try.

Who gets it, as far as I can tell from the code: the Steam Deck (Auto leaves the render thread off on its 8 threads), AMD APUs and Intel iGPUs with the render thread off. An APU or Intel laptop with enough threads for Auto to turn the render thread on gets nothing, because the early reads stand aside there.

Still missing:

  • Intel: it is now on by default on Intel iGPUs, and the path through f2e73429 (a writer's part running on to the next plain draw, #443's rule) has never run in the game. A Windows Intel run, unset against =0, is the most useful next test. The Test Farm could do it.
  • The exposure check (RX_VK_LOG_INPLACE_EXPOSURE): please post it here when it's done. That is why this review is a comment, not an approval.

The checks were not run on 468f360d. The two changed checks are listed above.

🤖 Generated with Claude Code

I read it as code only and ran nothing. I've pushed a rebase and a commit that turns the early reads on by default where the pools have no VRAM mirror, as decided in https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/977#issuecomment-37118. dyst's three points still stand, and I haven't repeated them. **Pushed (head `468f360d`):** - **Rebase:** the 24 commits are rebased one by one onto `main` 26ce01b5, with no conflict, and their authors kept. - **`468f360d`, the default:** - `readback_early::Facts` takes `mirrors`, from `mirror_on()`. Unset now gives `ON_DEFAULT` without mirrors and `OFF_MIRRORS` with them. `=0` is off everywhere, and `=1` is on with mirrors too. - The render thread and `RX_VK_PARALLEL` >= 2 still leave it off. They are now asked whether the switch is set or not: `async_on()` is decided at start, and `RX_VK_PARALLEL` defaults to 0. - `RX_VK_LATE_DEFER` follows it unchanged. - `mirror_on()` is declared in the anonymous namespace before `readback_early_on`, where it is defined. - `tests/test_late_defer.py`'s rule checks and `tests/test_readback_early.py`'s runs cover unset with and without mirrors, `=0`, and `=1` with mirrors. - Also updated: the doc rows, `invalidate`'s "(opt-in)" comment, the title and the description. **What I checked in the code:** - **`invalidate`'s early branch** only runs with a page written in place still pending (`whole > completed()`), so on a mirrored card it does nothing, as the RTX 3090 run showed. Where no writer is known, or the recording wait passes 250 ms, it falls back to main's whole-submission wait. - **`early_recorded_wait`:** if the writer's submission hasn't started recording, it returns at once. That is safe, because no writer is known yet, so the read waits for the whole submission. - **Everything new is behind `readback_early_on()`, `late_defer_on()` and the diagnostics.** With `=0`, or with mirrors and the switch unset, the path is main's, as dyst read it. **What the default now ships with**, from your own Risks section, accepted in the decision comment: - **Later parts read guest memory in place:** the early wait has no `in_place_after` guard, which main's own per-part wait has. - **A submission recorded in two recordings:** it can be let go before the second recording's writer. - **Design rule 2 ("nothing the replay reads comes from live guest memory"):** `RX_VK_LATE_DEFER` gives it a longer window than the old wait did. If any of these shows up, it would be a one-frame glitch on an integrated GPU, and the doc row names `RX_VK_READBACK_EARLY=0` as the first thing to try. **Who gets it, as far as I can tell from the code:** the Steam Deck (Auto leaves the render thread off on its 8 threads), AMD APUs and Intel iGPUs with the render thread off. An APU or Intel laptop with enough threads for Auto to turn the render thread on gets nothing, because the early reads stand aside there. **Still missing:** - **Intel:** it is now on by default on Intel iGPUs, and the path through `f2e73429` (a writer's part running on to the next plain draw, https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/443's rule) has never run in the game. A Windows Intel run, unset against `=0`, is the most useful next test. The Test Farm could do it. - **The exposure check (`RX_VK_LOG_INPLACE_EXPOSURE`):** please post it here when it's done. That is why this review is a comment, not an approval. The checks were not run on `468f360d`. The two changed checks are listed above. 🤖 Generated with [Claude Code](https://claude.com/claude-code)
@ -0,0 +36,4 @@
inline Why choose(const Facts& f) {
if (f.env && env_value_off(f.env)) return OFF_SWITCH; // (an empty value is off too: env_on's reading)
if (parallel(f.parallel)) return OFF_PARALLEL;
Owner

Default added here (468f360d): unset is on where the pools have no mirror and off where they have one, and the render thread and the parallel replay are now asked whether the switch is set or not.

Default added here (468f360d): unset is on where the pools have no mirror and off where they have one, and the render thread and the parallel replay are now asked whether the switch is set or not.
HumanBean deleted branch early-reads-port 2026-11-16 17:36:47 +00:00
Author
Contributor

The exposure check has run on a Steam Deck, as asked. It does not clear the risk: parts recorded after an early release saw CPU-side changes to memory they bind in place about 3.6 times as often as with main's wait. It does not yet show a fault either, because the check cannot say whose writes those are. Details, and what we are doing next.

Setup: Steam Deck OLED (RADV), headless, Kokiri Forest at output 1280x720 with 50% render, GPU at a manual 1600 MHz. The build is an older main (9c9d69ed) with this PR's gpu/ code, the soa layout and the log-only diagnostic RX_VK_LOG_INPLACE_EXPOSURE=8: for one part in eight it notes the pages bound in place while the part is recorded, copies them before the part's submit, waits for the part on the GPU and compares, leaving out pages the GPU itself wrote. Two runs an arm, order 1 2 2 1, then the third arm twice.

arm parts checked changed before the GPU finished them
1: early reads off (main's wait) 37,900 19.3% (509 words a changed part)
2: early reads on, parts recorded after an early release of their submission 5,354 69.4% (1,480 words)
2: early reads on, the other parts 40,346 8.8% (43 words)
3: as 2 with RX_VK_UB_SNAPSHOT=1, parts after an early release 3,542 46.4% (399 words)
3: as 2 with RX_VK_UB_SNAPSHOT=1, the other parts 41,958 0.1%

The two runs of each arm agree (18.2 and 20.4%; 71.7 and 67.3%; 45.8 and 46.9%), and nothing was lost or unreadable.

What it shows:

  • Main is not at zero: a fifth of its parts see such changes, and the snapshot arm says nearly all of those are uniform-buffer pages (the game writing the next frame's matrices). That is the known reason the uniform snapshot exists on Intel.
  • After an early release the rate is much higher, and two thirds of it is not uniform buffers.

What it does not show:

  • It is an upper bound: bytes that moved before the part finished on the GPU, not proof the GPU read them after they moved.
  • It cannot tell the game's writes from the replay's own. With early reads, a read's write-back of its range lands in guest memory while later parts are still on the GPU; if those parts bind that range in place, the check counts it, and that is the read behaving as designed. The diagnostic logs the address of the first 32 changed parts, and all 32 were used up at the title before any early read, so no address of an after-release change was logged. That was our recipe's mistake.
  • No picture fault was looked for in these runs. The owner of this Deck played about five minutes of Kokiri Forest with early reads on this morning and reported no glitches; the byte probe has found 0 missed reads in every run so far.

Next, today: the diagnostic gets its address lines kept apart for parts after an early release and matched against the readback ranges, so each change is classed as the replay's own write-back, a uniform page, or something else; then the same three arms again. If "something else" is there, the addresses say what to snapshot or hold. We will also take frame pairs with the switch on and off in a moving scene and look for one-frame faults.

Our reading for the default: nothing here says to turn it off today, and nothing here says the risk is cleared. RX_VK_READBACK_EARLY=0 remains the first thing to try for a report of a one-frame flash or an object out of place on a GPU without mirrors.

Tested by an AI agent (Claude Code) for zeldaootr.

The exposure check has run on a Steam Deck, as asked. **It does not clear the risk: parts recorded after an early release saw CPU-side changes to memory they bind in place about 3.6 times as often as with main's wait.** It does not yet show a fault either, because the check cannot say whose writes those are. Details, and what we are doing next. **Setup:** Steam Deck OLED (RADV), headless, Kokiri Forest at output 1280x720 with 50% render, GPU at a manual 1600 MHz. The build is an older main (9c9d69ed) with this PR's `gpu/` code, the `soa` layout and the log-only diagnostic `RX_VK_LOG_INPLACE_EXPOSURE=8`: for one part in eight it notes the pages bound in place while the part is recorded, copies them before the part's submit, waits for the part on the GPU and compares, leaving out pages the GPU itself wrote. Two runs an arm, order 1 2 2 1, then the third arm twice. | arm | parts checked | changed before the GPU finished them | |---|---|---| | 1: early reads off (main's wait) | 37,900 | **19.3%** (509 words a changed part) | | 2: early reads on, parts recorded after an early release of their submission | 5,354 | **69.4%** (1,480 words) | | 2: early reads on, the other parts | 40,346 | 8.8% (43 words) | | 3: as 2 with `RX_VK_UB_SNAPSHOT=1`, parts after an early release | 3,542 | **46.4%** (399 words) | | 3: as 2 with `RX_VK_UB_SNAPSHOT=1`, the other parts | 41,958 | 0.1% | The two runs of each arm agree (18.2 and 20.4%; 71.7 and 67.3%; 45.8 and 46.9%), and nothing was lost or unreadable. **What it shows:** - Main is not at zero: a fifth of its parts see such changes, and the snapshot arm says nearly all of those are uniform-buffer pages (the game writing the next frame's matrices). That is the known reason the uniform snapshot exists on Intel. - After an early release the rate is much higher, and two thirds of it is **not** uniform buffers. **What it does not show:** - It is an upper bound: bytes that moved before the part finished on the GPU, not proof the GPU read them after they moved. - It cannot tell the game's writes from the replay's own. With early reads, a read's write-back of its range lands in guest memory while later parts are still on the GPU; if those parts bind that range in place, the check counts it, and that is the read behaving as designed. The diagnostic logs the address of the first 32 changed parts, and all 32 were used up at the title before any early read, so no address of an after-release change was logged. That was our recipe's mistake. - No picture fault was looked for in these runs. The owner of this Deck played about five minutes of Kokiri Forest with early reads on this morning and reported no glitches; the byte probe has found 0 missed reads in every run so far. **Next, today:** the diagnostic gets its address lines kept apart for parts after an early release and matched against the readback ranges, so each change is classed as the replay's own write-back, a uniform page, or something else; then the same three arms again. If "something else" is there, the addresses say what to snapshot or hold. We will also take frame pairs with the switch on and off in a moving scene and look for one-frame faults. Our reading for the default: nothing here says to turn it off today, and nothing here says the risk is cleared. `RX_VK_READBACK_EARLY=0` remains the first thing to try for a report of a one-frame flash or an object out of place on a GPU without mirrors. *Tested by an AI agent (Claude Code) for zeldaootr.*
Author
Contributor

A correction to my comment above: "reported no glitches" says more than we know. The Deck's owner played about five minutes in Kokiri Forest with early reads on and did not mention any glitch; he was not asked to look for one-frame faults. Treat it as no evidence either way.

A correction to my comment above: "reported no glitches" says more than we know. The Deck's owner played about five minutes in Kokiri Forest with early reads on and did not mention any glitch; he was not asked to look for one-frame faults. Treat it as no evidence either way.
Author
Contributor

A second correction to the exposure result above, and it makes the result more worrying, not less. I offered "the replay's own write-back of the read's range" as a harmless explanation for part of the 69%. Reading the code, that explanation is empty on a GPU without mirrors:

  • Without mirrors every binding is the imported guest pool itself, so there is no write-back copy at all (wait_writebacks_reading has nothing to do: the GPU writes guest memory), and the early path copies nothing: it waits for the writer's part and the CPU then reads guest memory directly.
  • The GPU's own in-place writes were already left out of the comparison (every page written from 256 parts before the checked one onward).

So the changes the check counted after an early release are CPU-side writes: the game's, or the replay's own CPU writes in stream order (counter reports, occlusion sums, texels written back), which happen with or without early reads and should show at the same rate in the control arm. One real candidate is now named: the game writing a range of GPU results it has just read (for example a meter it reads and sets back to 0) while a later part of the same submission still reads that range in place. With main's wait no later part can see that write.

It is still an upper bound (bytes that changed before the part finished on the GPU, not proof the GPU read them afterwards), and still no visible fault has been looked for. The second version of the diagnostic classes every changed word (a range the CPU read back or the replay wrote, a uniform page, or other), tags which of those it was, logs addresses and binding kinds for parts after an early release, and raises the per-part cap from 4 to 16 MiB; with it comes a capture of 24 consecutive frames in a camera turn, early reads on and off. It is built and queued on the Deck; results here when it has run.

If you would rather not ship the default on an open question, turning it back off for the next release and on again once this is settled would cost Deck players the gain for one release; we would understand either call.

A second correction to the exposure result above, and it makes the result more worrying, not less. I offered "the replay's own write-back of the read's range" as a harmless explanation for part of the 69%. Reading the code, that explanation is empty on a GPU without mirrors: - Without mirrors every binding is the imported guest pool itself, so there is no write-back copy at all (`wait_writebacks_reading` has nothing to do: the GPU writes guest memory), and the early path copies nothing: it waits for the writer's part and the CPU then reads guest memory directly. - The GPU's own in-place writes were already left out of the comparison (every page written from 256 parts before the checked one onward). So the changes the check counted after an early release are CPU-side writes: the game's, or the replay's own CPU writes in stream order (counter reports, occlusion sums, texels written back), which happen with or without early reads and should show at the same rate in the control arm. One real candidate is now named: the game writing a range of GPU results it has just read (for example a meter it reads and sets back to 0) while a later part of the same submission still reads that range in place. With main's wait no later part can see that write. It is still an upper bound (bytes that changed before the part finished on the GPU, not proof the GPU read them afterwards), and still no visible fault has been looked for. The second version of the diagnostic classes every changed word (a range the CPU read back or the replay wrote, a uniform page, or other), tags which of those it was, logs addresses and binding kinds for parts after an early release, and raises the per-part cap from 4 to 16 MiB; with it comes a capture of 24 consecutive frames in a camera turn, early reads on and off. It is built and queued on the Deck; results here when it has run. If you would rather not ship the default on an open question, turning it back off for the next release and on again once this is settled would cost Deck players the gain for one release; we would understand either call.
Sign in to join this conversation.
No reviewers
No milestone
No project
No assignees
4 participants
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
MagicBeanDev/Longshot!977
No description provided.