fix(vk): no more storms of texture re-uploads, gigabytes in seconds with slow frames (a GPU buffer write no longer re-uploads a texture from memory its data never reached) #2164

Open
Veix wants to merge 3 commits from Veix/Longshot:fix/upload-only-arrived into main
Contributor

Summary

The renderer uploads textures again and again from the game's memory although nothing new is there: 7 to 14 GB in the seconds after the pause menu closes, one render target every 5 presents in Castle Town, a steady stream in Hyrule Field. This stops it. The easiest place to see it is the pause menu, but it is not a menu bug: it happens wherever the GPU writes a buffer over memory that a texture also uses.

What goes wrong. The game draws into render targets that share guest memory with other things, among them buffers its compute programs write every frame. When the GPU writes such a buffer, the renderer notes "the GPU wrote here" for every texture over that memory and, at the texture's next use, uploads the whole texture again from guest memory. But a written binding over 1024 KB that the CPU does not read is never copied down into guest memory (RX_VK_WB_MAX_KB, wb_learn.h). So the upload brings back the bytes the memory had before, over what the GPU has drawn since. The copy down is skipped, the copy up happens anyway: a re-upload that can have no new data.

Where it sits in main:

  • gpu_buffer_bind (gpu/vk/vk_memory.cpp) tells the textures of the whole binding (texture_gpu_wrote) in its first line, about 120 lines before it decides whether the binding is written back (wb_leave_out).
  • memory_stamp_gpu stamps the write's pages, and a texture's check for CPU writes reads those stamps as a CPU write.
  • texture_touch (gpu/vk/vk_texture.cpp) uploads the whole texture for any such notice and takes it for memory's.

How to see it on main. Load a save in Kokiri Forest, run for a few seconds so the area streams in, open the pause menu for a few seconds and close it. The log then repeats, every frame or two:

vk: texture #3086 0x10003fc0400 4096x4096 NVN 0x32: the GPU wrote its storage (submission 42524): uploaded again
vk: upload over a drawn image: #21298 0x10003fc0400 1920x1080x1 x1 NVN 0x3f at present 21693 ..., finalized, 0 KB dirty in 0 ranges, a GPU buffer wrote it

and the status line reads 27 to 31 MB uploaded a frame at about 47 fps where the same view runs at 75. A fresh load that only stands still does not show it: the 4096x4096 picture it hits exists only once more of the area is in.

What this changes. One rule: a texture is uploaded from guest memory only where memory holds newer data of its own. - newer only where the write reached memory: the textures are told of a written binding when gpu_buffer_bind returns, after its write-back is decided: of the whole binding where it is written back or written in place, or of the kept ranges alone where the rest is left out, including any kept edge range. Those ranges and any CPU-dirty ranges are uploaded for single-level, single-layer, uncompressed, non-sparse textures; other textures are uploaded whole as before. A drawn picture refreshed in part stays marked as drawn.

  • GPU page stamps are not filtered: an earlier CPU write on the same page may still need uploading. The memory generation is captured before the dirty scan so a later write is not acknowledged prematurely.

On by default; RX_VK_UPLOAD_ARRIVED=0 gives main's behaviour. The log's vk: upload arrived line reports bindings not told whole, part and whole refreshes, and part refreshes that kept a drawn picture every 600 presents.

Read first: the comment above upload_arrived_on in gpu/vk/vk_texture.cpp (the rule), then gpu_buffer_bind's Notice in gpu/vk/vk_memory.cpp, then texture_touch.

Related work, none of which this replaces:

  • #332 (Amateurwizrd, WIP): a render target is not uploaded for a buffer write older than its drawing. This covers the writes that come after the drawing, frame after frame; the two do not touch the same lines.
  • #282 (bluetoe, WIP) and #1223 (TrashPanda, closed when #1553 replaced the write map): upload only the part a GPU write covered. The same Castle Town write is named there. This PR does that on GpuWriteHistory, and first asks whether the data arrived at all.
  • #1162 (merged): a texture the GPU really wrote must be uploaded again, not kept stale. Kept: writes whose data reaches memory and CPU writes are taken for upload.
  • See #750 (a GPU-written texture uploaded again per present in Castle Town on a Steam Deck LCD): the same lines in its log. Not tested there: at that size the written buffer is under the 1024 KB limit and is copied down, so this PR may not change that case (Still to test).

This PR was written with AI assistance (Claude Code); the measurements and the play testing are from my own PC.

Testing

Tested: Windows 11, Ryzen 9 9900X, 64 GB, RTX 4070 SUPER 12 GB (NVIDIA 621.24). Everything in the game ran on main @ e7579b0a; the branch was then rebased onto 8f02906b (one conflict in vk_texture.cpp, with #1987's diagnostics and its moved log line, both kept as main has them), and after it the checks and the renderer's build ran again, the game did not.

  • In the game, by hand: about 25 minutes with the rule on at 3840x2160 DLAA through Kokiri Forest, Kakariko Village, Castle Town and Hyrule Field, with many pause menus. No wrong picture seen (nothing stale, black or flickering that main does not show), no device loss. The storm after the menu that main shows at the same spot did not come once.
  • In the game, scripted: the routine under Measurements, three runs each way.
  • tests/test_direct_buffer_writes.py (extended) runs the real gpu_buffer_bind in every case with the rule off and on, including a binding whose write-back is partly left out. The previously reported 24 of 24 passes were before the latest changes; the updated check still needs to run.
  • tests/test_upload_arrived.py (new) checks notification and dirty-scan ordering, then runs a game-free fixture using the renderer's page scan and write history. The updated check still needs to run.
  • python -I -B tests/run_all.py --changed origin/main on Windows with the pinned llvm-mingw, after the rebase: 133 passed, 14 failed, 36 skipped. All 14 fail on main @ 8f02906b on this PC too, run in the same environment: test_capture_read_once, test_crash_dialog, test_dev_ab, test_frame_overlap, test_guest_imported, test_input_movie, test_intel_prepare_side_by_side, test_linux_pressure, test_pending_guest_import, test_prepare_stall, test_readback_timeout_source, test_sparse_mirror, test_submit_oom, test_switch_table (the table is out of order on main since #2093). None is new. The whole suite did not finish on this PC (it stopped making progress after 319 checks): CI runs it.
  • The renderer builds alone (RX_VK_ONLY=1, llvm-mingw clang) at this head, and built in the game at the measured commit.

Still to test:

  • Any GPU, several areas: the same picture as main, with attention to textures the game fills through buffers (terrain, effects, the HUD's labels) and to anything that updates late or not at all, expecting no difference (the Test Farm, no request posted yet: Kokiri Forest, Castle Town, Hyrule Field and Kakariko with a pause menu opened and closed in each, pictures against main, the log's uploaded again and upload over a drawn image lines counted on both sides)
  • AMD and Intel, Linux: the same, expecting fewer of those lines and no picture difference (the Test Farm)
  • A Steam Deck or another small screen (#750's setup): Castle Town with a camera spin, the upload over a drawn image ... a GPU buffer wrote it lines with the rule on and RX_VK_UPLOAD_ARRIVED=0, expecting fewer; this is the case that may be unchanged, since the written buffer is small enough there to be copied down (by hand)
  • A card of 6 GB or less: VRAM over a session with and without the rule, expecting no more held (by hand; on the 12 GB card it is equal, below)
  • The updated checks at this head, including on Linux and GCC with --cxx g++, and the renderer's build on Linux (by hand or CI; only clang on Windows ran here)
  • Windows, the same PC: the routine under Measurements once more at this head, since the game ran before the rebase, expecting the same (by hand)

Measurements

Earlier measurements: the upload storm after the pause menu and its slow frames were gone with the earlier rule: 7.3 GB uploaded in the 15 seconds after three menu closings without the rule (the median of three runs; 7.2 to 13.7), 0.28 GB with it; frames over 16.7 ms in those seconds from 32 % to 1.4 %. VRAM is the same.

Setup: Windows 11, RTX 4070 SUPER 12 GB (NVIDIA 621.24), Ryzen 9 9900X, 64 GB. 3840x2160 output, DLAA, frame rate uncapped, V-Sync on (a 180 Hz screen), Threaded rendering on. A scripted routine through the control bridge on one save: Kokiri Forest, teleport to Castle Town and run forward 10 s, teleport to Hyrule Field and run forward 10 s, teleport back to Kokiri Forest, then three times the pause menu open 4 s and closed 5 s. Measured from the frame trace (RX_VK_FRAME_TRACE=1): the 5 seconds after each of the three closings, pooled per run. Six runs, alternating: off, on, off, on, on, off.

Both sides are this PR's build on main @ e7579b0a, a switch against a switch: "before" is RX_VK_UPLOAD_ARRIVED=0, which is main's path. No separate build of main was run for these numbers, and they are from before the rebase onto 8f02906b. The first four runs were made at an earlier commit of this branch where the rule was off unless set (so "on" was =1); the last two with the rule on by default, and with =0. The rule's code was the same in both, but has since changed; these measurements need to be repeated at this head.

Before: main @ e7579b0a (its path in this PR's build: RX_VK_UPLOAD_ARRIVED=0) After: this PR @ 9321ba15 (the head before the rebase)
MB uploaded in the 15 s after three closings, by run 13,710 / 7,153 / 7,304 278 / 288 / 244
median 7,304 278
Frames over 16.7 ms in those seconds, by run 47 % / 32 % / 21 % 1.5 % / 1.4 % / 1.1 %
Frame time p95, by run (ms) 22.6 / 18.4 / 22.5 14.0 / 14.4 / 14.6
Frame time p99, by run (ms) 25.7 / 19.5 / 23.1 19.2 / 18.3 / 17.2
Frame rate in those seconds, by run 64.5 / 77.2 / 76.1 85.7 / 84.2 / 88.9
The worst frame, by run (ms) 43.0 / 36.6 / 27.2 32.1 / 36.2 / 36.4
upload over a drawn image lines a run 58 / 36 / 123 0 / 1 / 0
VRAM used, the run's peak (MB, the log's vk: VRAM: line) 7,965 / 8,127 / 9,432 8,061 / 8,146 / 8,189
Textures the GPU drew, during the menus (MB) 1,269 / 1,269 1,292 / 1,251

The worst single frame is not better: those are textures arriving. The one upload over a drawn image line with the earlier rule is a 960x540 picture uploaded once for a write that covered all of it.

What the earlier rule's own line counted in one run with it on (10,800 presents): 146,979 written bindings not written back and so not told to the textures whole; 787 checks that found only the GPU's own page stamps and uploaded nothing (1,262 MB named as written), none of which had a page the CPU wrote; 14 textures refreshed in part (28 MB of 157 MB); 4 times a drawn picture left as it is (98 MB).

By hand, in play with the earlier rule without the routine (one session each way, 3840x2160 DLAA): without the rule one 1920x1080 target in Castle Town was uploaded again every 5 presents, 146 times (1.2 GB), and 4.4 and 4.7 GB were uploaded after two menu closings; with it, over 75,600 presents through four areas, no drawn picture was uploaded over, and the line counted 4,391 echo checks (6.1 GB named), 154 drawn pictures left (4.8 GB) and 46 part refreshes (97 MB of 552 MB). In Hyrule Field the echo alone ran at about 0.8 MB a present.

VRAM: the drawn textures' total, which is the only kind the rule could hold longer, is equal within 30 MB across the four runs it was read from, and the log's list of drawn textures unused for 120 presents holds the same three in all of them. Not measured on a card where VRAM is short.

Risks

  • A stale texture. If the game fills a texture through a buffer and the rule misjudges it, the texture keeps what it had. Writes whose data is written back and CPU writes are taken for upload. Not seen in 25 minutes of play with the earlier rule, and not looked for systematically at this head (Still to test). RX_VK_UPLOAD_ARRIVED=0 turns the rule off without a rebuild; a revert also undoes it.
  • Part uploads go only to single-level, single-layer, uncompressed, non-sparse textures; every other texture is uploaded whole as before when a write's data reached it.
  • After updating: nothing changes for saves, longshot_options.ini or the caches. The renderer is compiled again.

Checklist

Ready for review

  • One topic, on current main; the title is a Conventional Commit.
  • Tested on my own PC, in the game for a change the game runs, as Testing says.
  • mise run test passes (any failure that also fails on main is named). (The checks that read the changed files ran, with the 14 that also fail on main named; the whole suite did not finish on this PC: CI.)
  • Measured against main, or Measurements says why there is nothing to measure. (Against this build with the rule off, which is main's path: a switch against a switch, not a build of main.)
  • No game files, build output or hand-edited generated files; third_party/ untouched; nothing GPL-incompatible.

When it applies

  • A fix: a game-free test that fails on main and passes here. (The updated tests still need to run.)
  • A switch added or changed: it ships at the default the measurements chose, with its row in gpu/vk/switches.def.
  • A check under tests/ that reads changed source is updated, not weakened. (The updated tests still need to run.)
  • Host code builds on Windows (llvm-mingw) and Linux, with UTF-8 paths. (Windows llvm-mingw: builds. Linux not built.)
  • AI assistance recorded with Assisted-by: / Co-Authored-By: trailers.

Ready to merge

  • Everything in Still to test has run, its results in Tested.
  • Tier 1 hardware it can affect is tested (docs/dev/standards.md, "Hardware tiers").
  • Approved in review, every comment answered.
  • CI passes on the head; on current main, measured again or a comment says why the numbers still hold.

🤖 Generated with Claude Code

## Summary **The renderer uploads textures again and again from the game's memory although nothing new is there: 7 to 14 GB in the seconds after the pause menu closes, one render target every 5 presents in Castle Town, a steady stream in Hyrule Field. This stops it.** The easiest place to see it is the pause menu, but it is not a menu bug: it happens wherever the GPU writes a buffer over memory that a texture also uses. **What goes wrong.** The game draws into render targets that share guest memory with other things, among them buffers its compute programs write every frame. When the GPU writes such a buffer, the renderer notes "the GPU wrote here" for every texture over that memory and, at the texture's next use, uploads the whole texture again from guest memory. But a written binding over 1024 KB that the CPU does not read is never copied down into guest memory (`RX_VK_WB_MAX_KB`, `wb_learn.h`). So the upload brings back the bytes the memory had before, over what the GPU has drawn since. The copy down is skipped, the copy up happens anyway: a re-upload that can have no new data. Where it sits in `main`: - `gpu_buffer_bind` (`gpu/vk/vk_memory.cpp`) tells the textures of the whole binding (`texture_gpu_wrote`) in its first line, about 120 lines before it decides whether the binding is written back (`wb_leave_out`). - `memory_stamp_gpu` stamps the write's pages, and a texture's check for CPU writes reads those stamps as a CPU write. - `texture_touch` (`gpu/vk/vk_texture.cpp`) uploads the whole texture for any such notice and takes it for memory's. **How to see it on `main`.** Load a save in Kokiri Forest, run for a few seconds so the area streams in, open the pause menu for a few seconds and close it. The log then repeats, every frame or two: ``` vk: texture #3086 0x10003fc0400 4096x4096 NVN 0x32: the GPU wrote its storage (submission 42524): uploaded again vk: upload over a drawn image: #21298 0x10003fc0400 1920x1080x1 x1 NVN 0x3f at present 21693 ..., finalized, 0 KB dirty in 0 ranges, a GPU buffer wrote it ``` and the status line reads 27 to 31 MB uploaded a frame at about 47 fps where the same view runs at 75. A fresh load that only stands still does not show it: the 4096x4096 picture it hits exists only once more of the area is in. **What this changes.** One rule: a texture is uploaded from guest memory only where memory holds newer data of its own. - **newer only where the write reached memory:** the textures are told of a written binding when `gpu_buffer_bind` returns, after its write-back is decided: of the whole binding where it is written back or written in place, or of the kept ranges alone where the rest is left out, including any kept edge range. Those ranges and any CPU-dirty ranges are uploaded for single-level, single-layer, uncompressed, non-sparse textures; other textures are uploaded whole as before. A drawn picture refreshed in part stays marked as drawn. - GPU page stamps are not filtered: an earlier CPU write on the same page may still need uploading. The memory generation is captured before the dirty scan so a later write is not acknowledged prematurely. On by default; `RX_VK_UPLOAD_ARRIVED=0` gives `main`'s behaviour. The log's `vk: upload arrived` line reports bindings not told whole, part and whole refreshes, and part refreshes that kept a drawn picture every 600 presents. Read first: the comment above `upload_arrived_on` in `gpu/vk/vk_texture.cpp` (the rule), then `gpu_buffer_bind`'s `Notice` in `gpu/vk/vk_memory.cpp`, then `texture_touch`. Related work, none of which this replaces: - #332 (Amateurwizrd, WIP): a render target is not uploaded for a buffer write *older* than its drawing. This covers the writes that come after the drawing, frame after frame; the two do not touch the same lines. - #282 (bluetoe, WIP) and #1223 (TrashPanda, closed when #1553 replaced the write map): upload only the part a GPU write covered. The same Castle Town write is named there. This PR does that on `GpuWriteHistory`, and first asks whether the data arrived at all. - #1162 (merged): a texture the GPU really wrote must be uploaded again, not kept stale. Kept: writes whose data reaches memory and CPU writes are taken for upload. - See #750 (a GPU-written texture uploaded again per present in Castle Town on a Steam Deck LCD): the same lines in its log. Not tested there: at that size the written buffer is under the 1024 KB limit and is copied down, so this PR may not change that case (Still to test). This PR was written with AI assistance (Claude Code); the measurements and the play testing are from my own PC. ## Testing **Tested:** Windows 11, Ryzen 9 9900X, 64 GB, RTX 4070 SUPER 12 GB (NVIDIA 621.24). Everything in the game ran on `main` @ e7579b0a; the branch was then rebased onto 8f02906b (one conflict in `vk_texture.cpp`, with #1987's diagnostics and its moved log line, both kept as `main` has them), and after it the checks and the renderer's build ran again, the game did not. - In the game, by hand: about 25 minutes with the rule on at 3840x2160 DLAA through Kokiri Forest, Kakariko Village, Castle Town and Hyrule Field, with many pause menus. No wrong picture seen (nothing stale, black or flickering that `main` does not show), no device loss. The storm after the menu that `main` shows at the same spot did not come once. - In the game, scripted: the routine under Measurements, three runs each way. - `tests/test_direct_buffer_writes.py` (extended) runs the real `gpu_buffer_bind` in every case with the rule off and on, including a binding whose write-back is partly left out. The previously reported 24 of 24 passes were before the latest changes; the updated check still needs to run. - `tests/test_upload_arrived.py` (new) checks notification and dirty-scan ordering, then runs a game-free fixture using the renderer's page scan and write history. The updated check still needs to run. - `python -I -B tests/run_all.py --changed origin/main` on Windows with the pinned llvm-mingw, after the rebase: 133 passed, 14 failed, 36 skipped. All 14 fail on `main` @ 8f02906b on this PC too, run in the same environment: `test_capture_read_once`, `test_crash_dialog`, `test_dev_ab`, `test_frame_overlap`, `test_guest_imported`, `test_input_movie`, `test_intel_prepare_side_by_side`, `test_linux_pressure`, `test_pending_guest_import`, `test_prepare_stall`, `test_readback_timeout_source`, `test_sparse_mirror`, `test_submit_oom`, `test_switch_table` (the table is out of order on `main` since #2093). None is new. The whole suite did not finish on this PC (it stopped making progress after 319 checks): CI runs it. - The renderer builds alone (`RX_VK_ONLY=1`, llvm-mingw clang) at this head, and built in the game at the measured commit. **Still to test:** - Any GPU, several areas: the same picture as `main`, with attention to textures the game fills through buffers (terrain, effects, the HUD's labels) and to anything that updates late or not at all, expecting no difference (the Test Farm, no request posted yet: Kokiri Forest, Castle Town, Hyrule Field and Kakariko with a pause menu opened and closed in each, pictures against main, the log's `uploaded again` and `upload over a drawn image` lines counted on both sides) - AMD and Intel, Linux: the same, expecting fewer of those lines and no picture difference (the Test Farm) - A Steam Deck or another small screen (#750's setup): Castle Town with a camera spin, the `upload over a drawn image ... a GPU buffer wrote it` lines with the rule on and `RX_VK_UPLOAD_ARRIVED=0`, expecting fewer; this is the case that may be unchanged, since the written buffer is small enough there to be copied down (by hand) - A card of 6 GB or less: VRAM over a session with and without the rule, expecting no more held (by hand; on the 12 GB card it is equal, below) - The updated checks at this head, including on Linux and GCC with `--cxx g++`, and the renderer's build on Linux (by hand or CI; only clang on Windows ran here) - Windows, the same PC: the routine under Measurements once more at this head, since the game ran before the rebase, expecting the same (by hand) ## Measurements **Earlier measurements:** the upload storm after the pause menu and its slow frames were gone with the earlier rule: 7.3 GB uploaded in the 15 seconds after three menu closings without the rule (the median of three runs; 7.2 to 13.7), 0.28 GB with it; frames over 16.7 ms in those seconds from 32 % to 1.4 %. VRAM is the same. Setup: Windows 11, RTX 4070 SUPER 12 GB (NVIDIA 621.24), Ryzen 9 9900X, 64 GB. 3840x2160 output, DLAA, frame rate uncapped, V-Sync on (a 180 Hz screen), Threaded rendering on. A scripted routine through the control bridge on one save: Kokiri Forest, teleport to Castle Town and run forward 10 s, teleport to Hyrule Field and run forward 10 s, teleport back to Kokiri Forest, then three times the pause menu open 4 s and closed 5 s. Measured from the frame trace (`RX_VK_FRAME_TRACE=1`): the 5 seconds after each of the three closings, pooled per run. Six runs, alternating: off, on, off, on, on, off. Both sides are this PR's build on `main` @ e7579b0a, a switch against a switch: "before" is `RX_VK_UPLOAD_ARRIVED=0`, which is `main`'s path. No separate build of `main` was run for these numbers, and they are from before the rebase onto 8f02906b. The first four runs were made at an earlier commit of this branch where the rule was off unless set (so "on" was `=1`); the last two with the rule on by default, and with `=0`. The rule's code was the same in both, but has since changed; these measurements need to be repeated at this head. | | Before: main @ e7579b0a (its path in this PR's build: `RX_VK_UPLOAD_ARRIVED=0`) | After: this PR @ 9321ba15 (the head before the rebase) | |---|---|---| | MB uploaded in the 15 s after three closings, by run | 13,710 / 7,153 / 7,304 | 278 / 288 / 244 | | median | 7,304 | 278 | | Frames over 16.7 ms in those seconds, by run | 47 % / 32 % / 21 % | 1.5 % / 1.4 % / 1.1 % | | Frame time p95, by run (ms) | 22.6 / 18.4 / 22.5 | 14.0 / 14.4 / 14.6 | | Frame time p99, by run (ms) | 25.7 / 19.5 / 23.1 | 19.2 / 18.3 / 17.2 | | Frame rate in those seconds, by run | 64.5 / 77.2 / 76.1 | 85.7 / 84.2 / 88.9 | | The worst frame, by run (ms) | 43.0 / 36.6 / 27.2 | 32.1 / 36.2 / 36.4 | | `upload over a drawn image` lines a run | 58 / 36 / 123 | 0 / 1 / 0 | | VRAM used, the run's peak (MB, the log's `vk: VRAM:` line) | 7,965 / 8,127 / 9,432 | 8,061 / 8,146 / 8,189 | | Textures the GPU drew, during the menus (MB) | 1,269 / 1,269 | 1,292 / 1,251 | The worst single frame is not better: those are textures arriving. The one `upload over a drawn image` line with the earlier rule is a 960x540 picture uploaded once for a write that covered all of it. What the earlier rule's own line counted in one run with it on (10,800 presents): 146,979 written bindings not written back and so not told to the textures whole; 787 checks that found only the GPU's own page stamps and uploaded nothing (1,262 MB named as written), none of which had a page the CPU wrote; 14 textures refreshed in part (28 MB of 157 MB); 4 times a drawn picture left as it is (98 MB). By hand, in play with the earlier rule without the routine (one session each way, 3840x2160 DLAA): without the rule one 1920x1080 target in Castle Town was uploaded again every 5 presents, 146 times (1.2 GB), and 4.4 and 4.7 GB were uploaded after two menu closings; with it, over 75,600 presents through four areas, no drawn picture was uploaded over, and the line counted 4,391 echo checks (6.1 GB named), 154 drawn pictures left (4.8 GB) and 46 part refreshes (97 MB of 552 MB). In Hyrule Field the echo alone ran at about 0.8 MB a present. VRAM: the drawn textures' total, which is the only kind the rule could hold longer, is equal within 30 MB across the four runs it was read from, and the log's list of drawn textures unused for 120 presents holds the same three in all of them. Not measured on a card where VRAM is short. ## Risks - **A stale texture.** If the game fills a texture through a buffer and the rule misjudges it, the texture keeps what it had. Writes whose data is written back and CPU writes are taken for upload. Not seen in 25 minutes of play with the earlier rule, and not looked for systematically at this head (Still to test). `RX_VK_UPLOAD_ARRIVED=0` turns the rule off without a rebuild; a revert also undoes it. - **Part uploads** go only to single-level, single-layer, uncompressed, non-sparse textures; every other texture is uploaded whole as before when a write's data reached it. - **After updating:** nothing changes for saves, `longshot_options.ini` or the caches. The renderer is compiled again. ## Checklist ### Ready for review - [x] One topic, on current `main`; the title is a Conventional Commit. - [x] Tested on my own PC, in the game for a change the game runs, as Testing says. - [ ] `mise run test` passes (any failure that also fails on `main` is named). (The checks that read the changed files ran, with the 14 that also fail on `main` named; the whole suite did not finish on this PC: CI.) - [x] Measured against `main`, or Measurements says why there is nothing to measure. (Against this build with the rule off, which is `main`'s path: a switch against a switch, not a build of `main`.) - [x] No game files, build output or hand-edited generated files; `third_party/` untouched; nothing GPL-incompatible. ### When it applies - [ ] A fix: a game-free test that fails on `main` and passes here. (The updated tests still need to run.) - [x] A switch added or changed: it ships at the default the measurements chose, with its row in `gpu/vk/switches.def`. - [ ] A check under `tests/` that reads changed source is updated, not weakened. (The updated tests still need to run.) - [ ] Host code builds on Windows (llvm-mingw) and Linux, with UTF-8 paths. (Windows llvm-mingw: builds. Linux not built.) - [x] AI assistance recorded with `Assisted-by:` / `Co-Authored-By:` trailers. ### Ready to merge - [ ] Everything in Still to test has run, its results in Tested. - [ ] Tier 1 hardware it can affect is tested (docs/dev/standards.md, "Hardware tiers"). - [ ] Approved in review, every comment answered. - [ ] CI passes on the head; on current `main`, measured again or a comment says why the numbers still hold. 🤖 Generated with [Claude Code](https://claude.com/claude-code)
gpu_buffer_bind tells the textures over a written binding of all of it in its first line, before it decides whether
the binding is written back. A binding over 1024 KB the CPU does not read is not (RX_VK_WB_MAX_KB), so a texture over
it is uploaded again from memory the write never reached: the bytes it had, over what the GPU drew into it since.

tests/test_direct_buffer_writes.py runs the real gpu_buffer_bind in every case with RX_VK_UPLOAD_ARRIVED off and on,
and in a new case, a binding whose write-back is left out but for 64 KB in its middle and its edge page: the textures
must be told of the 64 KB alone. tests/test_upload_arrived.py checks the order in gpu_buffer_bind, memory_stamp_gpu and
texture_touch, then cuts the rule's functions out and runs them: a write over a part of a drawn picture against one
over all of it, the ranges for a texture that is not a drawn picture, and a GPU write's own page stamps against a CPU
write, with the switch on and off.

Both fail here: the fix is the next commit.

Co-Authored-By: Claude Fable 5.2 <[email protected]>
fix(vk): no more storms of texture re-uploads, gigabytes in seconds with slow frames (a GPU buffer write no longer re-uploads a texture from memory its data never reached)
Some checks failed
CI / Conventional Commit title (pull_request) Blocked by required conditions
CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Blocked by required conditions
CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Blocked by required conditions
CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Blocked by required conditions
CI / Workflow lint (actionlint) (pull_request) Blocked by required conditions
CI / Website build and checks (pull_request) Blocked by required conditions
Guide screenshots / Guide images and local UI source review (pull_request) Blocked by required conditions
Third-party parts / As their notes say (pull_request) Blocked by required conditions
Third-party parts / The same as upstream (pull_request) Blocked by required conditions
PR description / Description follows the template (pull_request_target) Failing after 12s
Website / Website on Cloudflare (pull_request_target) Has been skipped
beanstalk/approval approved by llun (reviewed)
beanstalk/any 0/1 pass, 2 running, 0 queued
0572e848ad
A buffer the GPU writes over a texture's storage has the texture uploaded again from guest memory. That upload brought
back old bytes over a picture the GPU had drawn since, wherever memory was taken for newer, or for the texture's, when
it was not: one 3840x2160 target on 232 presents in a row in play, 7 to 14 GB in the seconds after the pause menu
closes. A texture is now uploaded from guest memory only where memory holds newer data of its own, checked in the three
places that went wrong (RX_VK_UPLOAD_ARRIVED=0: as before):

- not newer: a GPU write stamps its pages (memory_stamp_gpu), and a texture's check for CPU writes took those stamps
  for one. The stamps that were a GPU write's own are noted, and a page whose newest stamp is one is left out.
- newer only where the write reached memory: the textures are told of a written binding when gpu_buffer_bind returns,
  after its write-back is decided: of the whole binding where it is written back or written in place, of the kept
  ranges alone where the rest is left out (RX_VK_WB_MAX_KB), and not of its edge page. Those ranges are uploaded, not
  the whole texture (single level, single layer, not ASTC or BC; the others whole, as before).
- not its own: the game's frame targets share memory with its culling's buffers, so another object's write lands on a
  part of a drawn picture. The picture is left; a write over all of it is the texture filled through a buffer and is
  taken as before.

The write history notes what a drawn picture's writes cover of it only while the rule is on. The log's
`vk: upload arrived` line counts each place every 600 presents.

The log's lines give a part upload as a part with its size (`re-uploaded`, `slow upload`, `uploaded again (partial)`,
RX_VK_LOG_REWRITES). tests/test_wrote_regions.py follows: a write over a part of a drawn picture is seen and not taken.

Co-Authored-By: Claude Fable 5.2 <[email protected]>
Author
Contributor

Tested in game: Windows 11, RTX 4070 SUPER 12 GB, NVIDIA 621.24. About 25 minutes by hand at 3840x2160 DLAA in Kokiri Forest, Kakariko Village, Castle Town and Hyrule Field with many pause menus: no wrong picture seen, and no upload storm after a menu closes, where the same build with RX_VK_UPLOAD_ARRIVED=0 shows it every time. A scripted routine, three runs each way, is in Measurements. The game ran on main e7579b0a, before the rebase onto 8f02906b.

Tested in game: Windows 11, RTX 4070 SUPER 12 GB, NVIDIA 621.24. About 25 minutes by hand at 3840x2160 DLAA in Kokiri Forest, Kakariko Village, Castle Town and Hyrule Field with many pause menus: no wrong picture seen, and no upload storm after a menu closes, where the same build with RX_VK_UPLOAD_ARRIVED=0 shows it every time. A scripted routine, three runs each way, is in Measurements. The game ran on main e7579b0a, before the rebase onto 8f02906b.
Member

Possibly related, for context:

  • issue #1608 How many texture re-uploads bring the same bytes the image already holds? Nothing in the renderer compares them — Open issue on redundant texture uploads, but investigates identical-content writes rather than skipped GPU write-back
  • PR #944 WIP: fix(vk): a texture whose upload failed is tried again, not left empty or stale — Open upload-retry work touches texture_touch and the same renderer upload path
Possibly related, for context: - issue #1608 How many texture re-uploads bring the same bytes the image already holds? Nothing in the renderer compares them — Open issue on redundant texture uploads, but investigates identical-content writes rather than skipped GPU write-back - PR #944 WIP: fix(vk): a texture whose upload failed is tried again, not left empty or stale — Open upload-retry work touches texture_touch and the same renderer upload path
Author
Contributor

@Beanstalk Renderer change (gpu/vk): textures are no longer uploaded again for GPU buffer writes whose data never reached memory. On by default, RX_VK_UPLOAD_ARRIVED=0 is main's way. The risk is a stale or wrong texture.

Please run it against main on Windows and Linux, NVIDIA and AMD, and a Steam Deck, with no upscaling: Render resolution 100% and the game's own upscaler, at 4K output where the tester's screen has it. From the save kokiri-start: hold the left stick forward 10 s, teleport to Market_Child and hold it 10 s, to HyruleField_Child and hold it 10 s, back to KokiriForest_Child. Then three times: press PLUS, wait 4 s, press PLUS, measure 5 s with no warm-up.

Please count the log lines "upload over a drawn image" and "uploaded again" on both sides without failing on them: main shows dozens after the menus, the PR few or none.

@Beanstalk Renderer change (gpu/vk): textures are no longer uploaded again for GPU buffer writes whose data never reached memory. On by default, RX_VK_UPLOAD_ARRIVED=0 is main's way. The risk is a stale or wrong texture. Please run it against main on Windows and Linux, NVIDIA and AMD, and a Steam Deck, with no upscaling: Render resolution 100% and the game's own upscaler, at 4K output where the tester's screen has it. From the save kokiri-start: hold the left stick forward 10 s, teleport to Market_Child and hold it 10 s, to HyruleField_Child and hold it 10 s, back to KokiriForest_Child. Then three times: press PLUS, wait 4 s, press PLUS, measure 5 s with no warm-up. Please count the log lines "upload over a drawn image" and "uploaded again" on both sides without failing on them: main shows dozens after the menus, the PR few or none.
Member

@Veix only members of BigBeans can start the farm. One of them can approve this request by reacting 👍 to it or to this comment.

@Veix only members of `BigBeans` can start the farm. One of them can approve this request by reacting 👍 to it or to this comment.
Member

@Beanstalk

@Beanstalk
Member
Beanstalk test farm · an earlier run: commit 0572e84 (the newer results are in a comment below)

Commit 0572e84 · approved by llun (reviewed) · dashboard

⚠️ Note: The farm runs at a fixed 1920x1080 output, so the requested 4K output cannot be tested; render resolution is left at the tester's default because its 100% value is not specified.
⚠️ Note: The log counts cover each whole run, not just the post-menu intervals, and the diagnostic lines may be capped; they do not by themselves prove the upload storm is gone.
⚠️ Note: This general renderer change uses one unrestricted target rather than guaranteeing separate Windows, Linux, NVIDIA, AMD and Steam Deck reports.

Started at 00:02 UTC.

Estimate: done in about 45 min, from the queue and the testers online now.

Target Result Reports Frame time p99 (ms), main → PR VRAM peak (MB) Crashes
any 🔄 running 0 of 1 — — —
What was tested

What will be tested · AI plan (gpt-6.1-terra)

The plan compares main and the PR through Kokiri Forest, Market, Hyrule Field and Kakariko, including repeated pause-menu cycles in Kokiri. It captures pictures, measures post-menu frame times and counts texture re-upload log lines on both sides without requiring a particular count. It takes about 25 minutes per tester; crashes, a material screenshot difference or a post-menu p99 frame-time regression over 20% fail the plan.

  • Where: any (any PC, 1-2 reports).
  • texture-updates: from the repository's save kokiri-start (settings dlss=0), then move the stick, teleport to Market, take the picture market to compare with main, teleport to Hyrule Field, take the picture field to compare with main, teleport to Kakariko Village, take the picture kakariko to compare with main, teleport to Kokiri Forest, press buttons, take the picture pause-menu to compare with main, measure after-pause-1 for 5 s (after 0 s warm-up), take the picture kokiri-after-pause to compare with main, measure after-pause-2 for 5 s (after 0 s warm-up), measure after-pause-3 for 5 s (after 0 s warm-up); 2 runs on each side.
  • Measured: crashes, frame times, stutters and hitches, numbers from the game's log.
  • The PR fails if: it crashes; or a picture differs from main by more than 5%; or frame time (p99) is more than 20% above main.
  • Time: about 23 min of game runs per tester (4 game starts), plus the two builds.
<!-- beanstalk:previous --> <details><summary>Beanstalk test farm · an earlier run: commit <code>0572e84</code> (the newer results are in a comment below)</summary> Commit `0572e84` · approved by llun (reviewed) · [dashboard](https://beanstalk.magicbean.dev/prs/2164) ⚠️ **Note:** The farm runs at a fixed 1920x1080 output, so the requested 4K output cannot be tested; render resolution is left at the tester's default because its 100% value is not specified. ⚠️ **Note:** The log counts cover each whole run, not just the post-menu intervals, and the diagnostic lines may be capped; they do not by themselves prove the upload storm is gone. ⚠️ **Note:** This general renderer change uses one unrestricted target rather than guaranteeing separate Windows, Linux, NVIDIA, AMD and Steam Deck reports. Started at 00:02 UTC. **Estimate:** done in about 45 min, from the queue and the testers online now. | Target | Result | Reports | Frame time p99 (ms), main → PR | VRAM peak (MB) | Crashes | | --- | --- | --- | --- | --- | --- | | `any` | 🔄 running | 0 of 1 | — | — | — | <details><summary>What was tested</summary> **What will be tested** · AI plan (`gpt-6.1-terra`) The plan compares main and the PR through Kokiri Forest, Market, Hyrule Field and Kakariko, including repeated pause-menu cycles in Kokiri. It captures pictures, measures post-menu frame times and counts texture re-upload log lines on both sides without requiring a particular count. It takes about 25 minutes per tester; crashes, a material screenshot difference or a post-menu p99 frame-time regression over 20% fail the plan. - **Where:** any (any PC, 1-2 reports). - **texture-updates:** from the repository's save `kokiri-start` (settings dlss=0), then move the stick, teleport to Market, take the picture market to compare with main, teleport to Hyrule Field, take the picture field to compare with main, teleport to Kakariko Village, take the picture kakariko to compare with main, teleport to Kokiri Forest, press buttons, take the picture pause-menu to compare with main, measure after-pause-1 for 5 s (after 0 s warm-up), take the picture kokiri-after-pause to compare with main, measure after-pause-2 for 5 s (after 0 s warm-up), measure after-pause-3 for 5 s (after 0 s warm-up); 2 runs on each side. - **Measured:** crashes, frame times, stutters and hitches, numbers from the game's log. - **The PR fails if:** it crashes; or a picture differs from main by more than 5%; or frame time (p99) is more than 20% above main. - **Time:** about 23 min of game runs per tester (4 game starts), plus the two builds. </details> </details>
Owner
@review
BeanBot left a comment

Recommendation: Request changes. The change can permanently miss real CPU and GPU texture updates, leaving stale pictures. The reported pause-menu improvement is substantial, but it was measured against a switch-off build before the rebase, and the current head still needs comparison with its base.

Avoids texture uploads caused by GPU buffer writes that did not reach guest memory, and limits some refreshes to written ranges. It reduces the reported upload storm at the cost of 8 MB for stamp tracking and new ownership and range-tracking logic; several of those shortcuts can discard real updates.

@fix will do (6)

  1. L1 gpu/vk/vk_texture.cpp:4867: A GPU stamp hides an earlier CPU write on the same page. (more on the line)
  2. L2 gpu/vk/vk_texture.cpp:3679: The empty-dirty path reads memory_generation after scanning and filtering the pages. (more on the line)
  3. L3 gpu/vk/vk_texture.cpp:3650: Coverage does not establish ownership: a compute operation can intentionally update only part of a texture previously drawn by the GPU. (more on the line)
  4. L4 gpu/vk/vk_memory.cpp:5488: A kept range at a binding edge is not necessarily unrelated data. (more on the line)
  5. L5 gpu/vk/gpu_write_history.h:92: The drawn image's range list includes writes as soon as they are recorded, including submissions that have not completed. (more on the line)
  6. L6 (minor) tests/test_upload_arrived.py:140: The new test defaults to g++ instead of honoring CXX or the project's selected compiler. (more on the line)

Before merging: testing (3) (runs on the systems it can change; it doesn't hold the recommendation)

  • L7 (minor): Compare the current head with the actual merge base on the same hardware, save, settings and routine, with interleaved runs. Include the pause-menu frame times and uploads and the heavy areas this hot path also affects; the existing numbers use a switch-off branch build before the rebase.
  • L8 (minor): After the correctness fixes, compare pictures and texture updates against the base on Windows and Linux across NVIDIA and AMD, including Steam Deck, and check Intel when available. Exercise buffer-filled textures, effects, HUD and pause-menu transitions; the existing play test was only on NVIDIA Windows before the rebase.
  • L9 (minor): Measure session VRAM and frame pacing against the base on a low-VRAM card. The rule can retain drawn textures longer, and the reported 12 GB runs do not establish the effect under memory pressure.

Checks: Failing on this head, which is behind its base, where it passes: PR description / Description follows the template (pull_request_target). @rebase (or @fix) brings it up to date and runs it again; not held against it. Still running: CI / Conventional Commit title (pull_request), CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request), CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request), CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request), CI / Workflow lint (actionlint) (pull_request), CI / Website build and checks (pull_request), Guide screenshots / Guide images and local UI source review (pull_request), Third-party parts / As their notes say (pull_request), Third-party parts / The same as upstream (pull_request).

More: does it help, other systems, switches, what was read

Does it help: Unproven against the base at this head. On the author's pre-rebase Windows RTX 4070 SUPER build, three interleaved runs with the switch off versus on reduced median uploads after three menu closings from 7,304 MB to 278 MB and frames over 16.7 ms from 32% to 1.4%. Neither side was a separate base build, and the game was not rerun after the rebase.

Other systems: The texture ownership and page-stamp changes affect all GPUs and both Windows and Linux, including compressed textures and render targets. The reported play test covers Windows and NVIDIA before the rebase; AMD, Intel, Linux, Steam Deck, low-VRAM behavior and systematic image comparisons remain untested.

Switches: RX_VK_UPLOAD_ARRIVED is added as a kill switch, default on; 0 restores the earlier path. It has a switches.def row, but the default is unsafe with the missed-update cases below and has not been validated on the other affected systems.

Reviewed: 0572e848ad against 8f02906b5a: Read the repository rules, PR template, standards, commit messages, production diff, gpu_write_history.h, wb_learn.h, and the changed test scripts and upload-arrived fixture. Read the changed sections and relevant callers in vk_memory.cpp and vk_texture.cpp; the large production files and direct-buffer fixture were not read in full. No recorded runs, translations or generated files were changed. docs/review-checklist.md and the adjacent Tooling checkout were absent.; nothing run (a code review).

Checked and fine:

  • Both commits record AI assistance with Co-Authored-By trailers.
  • The new switch has a switches.def entry and a default-off escape path.
  • The changed file list contains no game-derived output, shader changes, third-party changes or renderer API version change.
  • The reported CI checks are still running; no head-only failure was supplied.

Testing: Read-only review; no builds, tests, scripts or game runs were executed.

Overlaps and merge order: The PR identifies related work in #332, #282 and #1223; no merge order is established by the supplied information.


Automated code review by BeanBot (gpt-6.1-terra), asked for by HumanBean. Nothing was run; a maintainer decides.

**Recommendation: Request changes.** The change can permanently miss real CPU and GPU texture updates, leaving stale pictures. The reported pause-menu improvement is substantial, but it was measured against a switch-off build before the rebase, and the current head still needs comparison with its base. Avoids texture uploads caused by GPU buffer writes that did not reach guest memory, and limits some refreshes to written ranges. It reduces the reported upload storm at the cost of 8 MB for stamp tracking and new ownership and range-tracking logic; several of those shortcuts can discard real updates. **`@fix` will do (6)** 1. **L1** `gpu/vk/vk_texture.cpp:4867`: A GPU stamp hides an earlier CPU write on the same page. (more on the line) 2. **L2** `gpu/vk/vk_texture.cpp:3679`: The empty-dirty path reads memory_generation after scanning and filtering the pages. (more on the line) 3. **L3** `gpu/vk/vk_texture.cpp:3650`: Coverage does not establish ownership: a compute operation can intentionally update only part of a texture previously drawn by the GPU. (more on the line) 4. **L4** `gpu/vk/vk_memory.cpp:5488`: A kept range at a binding edge is not necessarily unrelated data. (more on the line) 5. **L5** `gpu/vk/gpu_write_history.h:92`: The drawn image's range list includes writes as soon as they are recorded, including submissions that have not completed. (more on the line) 6. **L6** (minor) `tests/test_upload_arrived.py:140`: The new test defaults to g++ instead of honoring CXX or the project's selected compiler. (more on the line) **Before merging: testing (3)** (runs on the systems it can change; it doesn't hold the recommendation) - **L7** (minor): Compare the current head with the actual merge base on the same hardware, save, settings and routine, with interleaved runs. Include the pause-menu frame times and uploads and the heavy areas this hot path also affects; the existing numbers use a switch-off branch build before the rebase. - **L8** (minor): After the correctness fixes, compare pictures and texture updates against the base on Windows and Linux across NVIDIA and AMD, including Steam Deck, and check Intel when available. Exercise buffer-filled textures, effects, HUD and pause-menu transitions; the existing play test was only on NVIDIA Windows before the rebase. - **L9** (minor): Measure session VRAM and frame pacing against the base on a low-VRAM card. The rule can retain drawn textures longer, and the reported 12 GB runs do not establish the effect under memory pressure. **Checks:** Failing on this head, which is behind its base, where it passes: `PR description / Description follows the template (pull_request_target)`. `@rebase` (or `@fix`) brings it up to date and runs it again; not held against it. Still running: `CI / Conventional Commit title (pull_request)`, `CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request)`, `CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request)`, `CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request)`, `CI / Workflow lint (actionlint) (pull_request)`, `CI / Website build and checks (pull_request)`, `Guide screenshots / Guide images and local UI source review (pull_request)`, `Third-party parts / As their notes say (pull_request)`, `Third-party parts / The same as upstream (pull_request)`. <details> <summary>More: does it help, other systems, switches, what was read</summary> **Does it help:** Unproven against the base at this head. On the author's pre-rebase Windows RTX 4070 SUPER build, three interleaved runs with the switch off versus on reduced median uploads after three menu closings from 7,304 MB to 278 MB and frames over 16.7 ms from 32% to 1.4%. Neither side was a separate base build, and the game was not rerun after the rebase. **Other systems:** The texture ownership and page-stamp changes affect all GPUs and both Windows and Linux, including compressed textures and render targets. The reported play test covers Windows and NVIDIA before the rebase; AMD, Intel, Linux, Steam Deck, low-VRAM behavior and systematic image comparisons remain untested. **Switches:** RX_VK_UPLOAD_ARRIVED is added as a kill switch, default on; 0 restores the earlier path. It has a switches.def row, but the default is unsafe with the missed-update cases below and has not been validated on the other affected systems. **Reviewed:** 0572e848ad6c against 8f02906b5aa3: Read the repository rules, PR template, standards, commit messages, production diff, gpu_write_history.h, wb_learn.h, and the changed test scripts and upload-arrived fixture. Read the changed sections and relevant callers in vk_memory.cpp and vk_texture.cpp; the large production files and direct-buffer fixture were not read in full. No recorded runs, translations or generated files were changed. docs/review-checklist.md and the adjacent Tooling checkout were absent.; nothing run (a code review). **Checked and fine:** - Both commits record AI assistance with Co-Authored-By trailers. - The new switch has a switches.def entry and a default-off escape path. - The changed file list contains no game-derived output, shader changes, third-party changes or renderer API version change. - The reported CI checks are still running; no head-only failure was supplied. **Testing:** Read-only review; no builds, tests, scripts or game runs were executed. **Overlaps and merge order:** The PR identifies related work in #332, #282 and #1223; no merge order is established by the supplied information. </details> --- <sub>Automated code review by BeanBot (gpt-6.1-terra), asked for by HumanBean. Nothing was run; a maintainer decides.</sub>
@ -64,6 +89,7 @@ class GpuWriteHistory {
const auto value = std::get<0>(write);
if (image.draw < write && (!image.first_write || value < image.first_write))
image.first_write = value;
if (parts_on && image.draw < write) add_part(image.wr, std::max(lo, it->first.first), std::min(hi, image.end));
Member

L5: The drawn image's range list includes writes as soon as they are recorded, including submissions that have not completed. If a completed write covers one half and a later unfinished write covers the other, parts_since reports full coverage and texture_touch can upload the unfinished half from old guest memory, then mark the image no longer GPU-owned. Keep ranges by submission and expose only completed writes, with a test that checks the returned ranges before the second submission completes.

**L5**: The drawn image's range list includes writes as soon as they are recorded, including submissions that have not completed. If a completed write covers one half and a later unfinished write covers the other, parts_since reports full coverage and texture_touch can upload the unfinished half from old guest memory, then mark the image no longer GPU-owned. Keep ranges by submission and expose only completed writes, with a test that checks the returned ranges before the second submission completes.
@ -5443,0 +5485,4 @@
notice_parts.clear();
for (const auto& [klo, khi] : keep) {
// (the binding's edge page, kept for what shares it: not a texture's data)
if (khi - klo <= (1ull << kPageShift) && (klo == wlo || khi == whi)) continue;
Member

L4: A kept range at a binding edge is not necessarily unrelated data. wb_learn also keeps CPU-read or CPU-written ranges, which may be a page or less at that edge; the GPU's bytes within that range are actually written back, yet this skips their texture notification. A texture located there can remain stale. Notify the intersection of every kept range with the binding, including edge ranges, and cover a genuinely used edge range in the fixture.

**L4**: A kept range at a binding edge is not necessarily unrelated data. wb_learn also keeps CPU-read or CPU-written ranges, which may be a page or less at that edge; the GPU's bytes within that range are actually written back, yet this skips their texture notification. A texture located there can remain stale. Notify the intersection of every kept range with the binding, including edge ranges, and cover a genuinely used edge range in the fixture.
@ -3635,6 +3647,13 @@ Texture* texture_touch(Texture* tex, bool for_sampling) {
// and stayed the flat placeholder they were made with)
const uint64_t seen_before = tex->gpu_wrote_seen; // (put back if the upload fails: the write is still to be taken)
uint64_t gpu_dirty_found = tex->uploaded ? gpu_wrote_into(*tex) : 0;
if (upload_arrived_on() && gpu_dirty_found && tex->gpu_written && arrived_over_part(*tex, seen_before)) {
Member

L3: Coverage does not establish ownership: a compute operation can intentionally update only part of a texture previously drawn by the GPU. This branch marks that real write seen without uploading it, and separate partial updates in later submissions are discarded individually, so the texture remains stale. Preserve and apply the written ranges unless ownership can be established from actual resource information; test a drawn texture intentionally updated in parts across submissions.

**L3**: Coverage does not establish ownership: a compute operation can intentionally update only part of a texture previously drawn by the GPU. This branch marks that real write seen without uploading it, and separate partial updates in later submissions are discarded individually, so the texture remains stale. Preserve and apply the written ranges unless ownership can be established from actual resource information; test a drawn texture intentionally updated in parts across submissions.
@ -3653,0 +3676,4 @@
for (const auto& r : dirty) named += r.second - r.first;
const uint64_t dropped = arrived_drop_echo(*tex, dirty);
if (dirty.empty()) {
arrived_stats.echo_n++, arrived_stats.echo_bytes += named;
Member

L2: The empty-dirty path reads memory_generation after scanning and filtering the pages. A CPU write arriving between that scan and this read is included in upload_gen although it was never uploaded, so the next check misses it. Capture a safe generation before the scan and only advance to that generation, with synchronization consistent with the page tracker.

**L2**: The empty-dirty path reads memory_generation after scanning and filtering the pages. A CPU write arriving between that scan and this read is included in upload_gen although it was never uploaded, so the next check misses it. Capture a safe generation before the scan and only advance to that generation, with synchronization consistent with the page tracker.
@ -4756,0 +4864,4 @@
const uint64_t end = std::min<uint64_t>(r.second, off + (0x1000 - ((g ? g : tex.gpu + off) & 0xFFF)));
if (g && memory_page_gpu_last(g)) {
if (in) out.push_back({run, off}), in = false;
dropped += end - off;
Member

L1: A GPU stamp hides an earlier CPU write on the same page. For example, the CPU changes texture bytes, then a GPU buffer write stamps another part of that page before the texture is checked: this removes the entire page from dirty and can advance upload_gen past the CPU change, leaving stale pixels indefinitely. Track CPU and GPU generations separately, or otherwise retain any CPU write since the texture's last upload; add a test for CPU then GPU on one page.

**L1**: A GPU stamp hides an earlier CPU write on the same page. For example, the CPU changes texture bytes, then a GPU buffer write stamps another part of that page before the texture is checked: this removes the entire page from dirty and can advance upload_gen past the CPU change, leaving stale pixels indefinitely. Track CPU and GPU generations separately, or otherwise retain any CPU write since the texture's last upload; add a test for CPU then GPU on one page.
@ -0,0 +137,4 @@
@OVER_PART@
@TAKEN@
#include "@ROOT@/tests/test_upload_arrived.cpp"
"""
Member

L6 (minor): The new test defaults to g++ instead of honoring CXX or the project's selected compiler. A Windows llvm-mingw setup with clang++ but no g++ cannot run it directly. Choose the configured C++ compiler by default while retaining --cxx.

**L6** (minor): The new test defaults to g++ instead of honoring CXX or the project's selected compiler. A Windows llvm-mingw setup with clang++ but no g++ cannot run it directly. Choose the configured C++ compiler by default while retaining --cxx.
Member
@fix
BeanBot force-pushed fix/upload-only-arrived from 0572e848ad
Some checks failed
CI / Conventional Commit title (pull_request) Blocked by required conditions
CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Blocked by required conditions
CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Blocked by required conditions
CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Blocked by required conditions
CI / Workflow lint (actionlint) (pull_request) Blocked by required conditions
CI / Website build and checks (pull_request) Blocked by required conditions
Guide screenshots / Guide images and local UI source review (pull_request) Blocked by required conditions
Third-party parts / As their notes say (pull_request) Blocked by required conditions
Third-party parts / The same as upstream (pull_request) Blocked by required conditions
PR description / Description follows the template (pull_request_target) Failing after 12s
Website / Website on Cloudflare (pull_request_target) Has been skipped
beanstalk/approval approved by llun (reviewed)
beanstalk/any 0/1 pass, 2 running, 0 queued
to c670586ecb
Some checks reported errors
PR description / Description follows the template (pull_request_target) Failing after 8s
Website / Website on Cloudflare (pull_request_target) Has been skipped
beanstalk/approval approved by HumanBean, llun (double-reviewed)
beanstalk/any 0/1 pass, 2 error
2026-11-22 00:31:17 +00:00
Compare
BeanBot left a comment

Recommendation: Approve. The code is ready: only testing is left. The revised rule has not yet been measured against the merge base or tested in play across the affected systems. Earlier results showed a large reduction in uploads and slow frames, but they used an earlier version of the rule. Recommend approval; not approved on the forge.

What @fix changed
I changed this pull request: the new head is c670586ecb. I rebased it onto main (a053f9e121). I edited its description to match the change.

Changed

  • CPU writes hidden by GPU stamps: Removed GPU-stamp filtering so dirty pages remain eligible for upload, and added the CPU-then-GPU fixture in tests/test_upload_arrived.cpp:15. This may restore some redundant uploads.
  • Generation captured after dirty scan: Capture the generation before scanning in gpu/vk/vk_texture.cpp:3654, and added a scan-snapshot fixture.
  • Partial writes to drawn textures: Apply completed buffer writes to drawn textures instead of discarding partial coverage in gpu/vk/vk_texture.cpp:3638; added a multi-submission fixture.
  • Kept binding-edge ranges: Notify textures of every kept range intersecting the binding in gpu/vk/vk_memory.cpp:5498, including the CPU-used edge in tests/direct_buffer_writes_test.cpp:124.
  • Unfinished drawn-image writes: Keep exact drawn-image ranges by submission and expose only completed ranges in gpu/vk/gpu_write_history.h:40; added a fixture checking the unfinished second half.
  • Test compiler default: Honor CXX with a clang++ fallback while retaining --cxx in tests/test_upload_arrived.py:21. No builds or tests were run, as requested.

Nothing was built or run: the checks run again on the new head.

The change limits texture refreshes to GPU buffer writes that reach guest memory and preserves partial updates to drawn textures. It aims to reduce redundant uploads, at the cost of additional write-range tracking and a risk of stale or incorrect textures.

Before merging: testing (3) (runs on the systems it can change; it doesn't hold the recommendation)

  • L7 (minor): Compare the current head with the actual merge base on the same hardware, save, settings and routine, with interleaved runs. Include the pause-menu frame times and uploads and the heavy areas this hot path also affects; the existing numbers use a switch-off branch build before the rebase.
  • L8 (minor): After the correctness fixes, compare pictures and texture updates against the base on Windows and Linux across NVIDIA and AMD, including Steam Deck, and check Intel when available. Exercise buffer-filled textures, effects, HUD and pause-menu transitions; the existing play test was only on NVIDIA Windows before the rebase.
  • L9 (minor): Measure session VRAM and frame pacing against the base on a low-VRAM card. The rule can retain drawn textures longer, and the reported 12 GB runs do not establish the effect under memory pressure.

Checks: they run again on this head (PR description / Description follows the template (pull_request_target) failed on the last one); a failure shows on the pull request.

Also noticed (2; not asked for, not blocking)
  • The pull request description still describes GPU-stamp filtering and discarding partial writes to drawn pictures, both removed by the fixes.
  • The description’s test and measurement results predate the latest correctness fixes.
More: does it help, other systems, switches, what was read

Does it help: Unproven at this head. Earlier switch-on versus switch-off runs reported median uploads falling from 7,304 MB to 278 MB and frames over 16.7 ms falling from 32% to 1.4% after pause-menu closings; the rule has since changed and those runs did not compare separate base and head builds.

Other systems: The renderer change can affect texture updates and frame pacing on Windows, Linux, NVIDIA, AMD, Intel and Steam Deck. Cross-platform picture comparisons and low-VRAM behavior remain unreported.

Switches: RX_VK_UPLOAD_ARRIVED is added, default on; 0 restores the earlier notification behavior. The default's benefit and correctness at this head remain unverified.

Reviewed: c670586ecb against a053f9e121: Read AGENTS.md, CONTRIBUTING.md, gpu/vk/gpu_write_history.h, tests/test_upload_arrived.cpp and tests/test_upload_arrived.py; inspected the fix commit and relevant texture and buffer paths. The remaining changed files and full base diff were not read in full.; nothing run (a code review).

Checked and fine:

  • The updated description identifies the earlier measurements as predating the revised rule.
  • Drawn-image write ranges are filtered by completed submission before partial refresh.

Testing: No tests or builds were run. CI has not yet run on this head; the updated game-free checks and Linux build are also reported as pending.

Overlaps and merge order: The description identifies related work in #332 and #282; neither is presented as a replacement for this change.


Automated code review by BeanBot (gpt-6.1-terra), asked for by BeanBot, after its fix. Nothing was run; a maintainer decides.

**Recommendation: Approve.** The code is ready: only testing is left. The revised rule has not yet been measured against the merge base or tested in play across the affected systems. Earlier results showed a large reduction in uploads and slow frames, but they used an earlier version of the rule. Recommend approval; not approved on the forge. **What `@fix` changed** I changed this pull request: the new head is c670586ecbba. I rebased it onto `main` (a053f9e12182). I edited its description to match the change. **Changed** - **CPU writes hidden by GPU stamps**: Removed GPU-stamp filtering so dirty pages remain eligible for upload, and added the CPU-then-GPU fixture in tests/test_upload_arrived.cpp:15. This may restore some redundant uploads. - **Generation captured after dirty scan**: Capture the generation before scanning in gpu/vk/vk_texture.cpp:3654, and added a scan-snapshot fixture. - **Partial writes to drawn textures**: Apply completed buffer writes to drawn textures instead of discarding partial coverage in gpu/vk/vk_texture.cpp:3638; added a multi-submission fixture. - **Kept binding-edge ranges**: Notify textures of every kept range intersecting the binding in gpu/vk/vk_memory.cpp:5498, including the CPU-used edge in tests/direct_buffer_writes_test.cpp:124. - **Unfinished drawn-image writes**: Keep exact drawn-image ranges by submission and expose only completed ranges in gpu/vk/gpu_write_history.h:40; added a fixture checking the unfinished second half. - **Test compiler default**: Honor CXX with a clang++ fallback while retaining --cxx in tests/test_upload_arrived.py:21. No builds or tests were run, as requested. Nothing was built or run: the checks run again on the new head. The change limits texture refreshes to GPU buffer writes that reach guest memory and preserves partial updates to drawn textures. It aims to reduce redundant uploads, at the cost of additional write-range tracking and a risk of stale or incorrect textures. **Before merging: testing (3)** (runs on the systems it can change; it doesn't hold the recommendation) - **L7** (minor): Compare the current head with the actual merge base on the same hardware, save, settings and routine, with interleaved runs. Include the pause-menu frame times and uploads and the heavy areas this hot path also affects; the existing numbers use a switch-off branch build before the rebase. - **L8** (minor): After the correctness fixes, compare pictures and texture updates against the base on Windows and Linux across NVIDIA and AMD, including Steam Deck, and check Intel when available. Exercise buffer-filled textures, effects, HUD and pause-menu transitions; the existing play test was only on NVIDIA Windows before the rebase. - **L9** (minor): Measure session VRAM and frame pacing against the base on a low-VRAM card. The rule can retain drawn textures longer, and the reported 12 GB runs do not establish the effect under memory pressure. **Checks:** they run again on this head (`PR description / Description follows the template (pull_request_target)` failed on the last one); a failure shows on the pull request. <details> <summary>Also noticed (2; not asked for, not blocking)</summary> - The pull request description still describes GPU-stamp filtering and discarding partial writes to drawn pictures, both removed by the fixes. - The description’s test and measurement results predate the latest correctness fixes. </details> <details> <summary>More: does it help, other systems, switches, what was read</summary> **Does it help:** Unproven at this head. Earlier switch-on versus switch-off runs reported median uploads falling from 7,304 MB to 278 MB and frames over 16.7 ms falling from 32% to 1.4% after pause-menu closings; the rule has since changed and those runs did not compare separate base and head builds. **Other systems:** The renderer change can affect texture updates and frame pacing on Windows, Linux, NVIDIA, AMD, Intel and Steam Deck. Cross-platform picture comparisons and low-VRAM behavior remain unreported. **Switches:** RX_VK_UPLOAD_ARRIVED is added, default on; 0 restores the earlier notification behavior. The default's benefit and correctness at this head remain unverified. **Reviewed:** c670586ecbba against a053f9e12182: Read AGENTS.md, CONTRIBUTING.md, gpu/vk/gpu_write_history.h, tests/test_upload_arrived.cpp and tests/test_upload_arrived.py; inspected the fix commit and relevant texture and buffer paths. The remaining changed files and full base diff were not read in full.; nothing run (a code review). **Checked and fine:** - The updated description identifies the earlier measurements as predating the revised rule. - Drawn-image write ranges are filtered by completed submission before partial refresh. **Testing:** No tests or builds were run. CI has not yet run on this head; the updated game-free checks and Linux build are also reported as pending. **Overlaps and merge order:** The description identifies related work in #332 and #282; neither is presented as a replacement for this change. </details> --- <sub>Automated code review by BeanBot (gpt-6.1-terra), asked for by BeanBot, after its fix. Nothing was run; a maintainer decides.</sub>
Owner

@Beanstalk

@Beanstalk
Member

Beanstalk test farm · ⚠️ couldn't be judged on any

Commit c670586 · approved by HumanBean, llun (double-reviewed) · dashboard

⚠️ Note: Upload log counts are diagnostic, not a required reduction: the messages can be capped and the affected write-back path may not occur on every run.
⚠️ Note: The single general-hardware target does not guarantee coverage of every requested GPU, operating system, Steam Deck, or low-VRAM configuration; the farm plan also cannot run the standalone C++ fixtures.

Summary, written by AI (gpt-6.1-terra) from the numbers below:

The change could not be judged on the any target: both the Windows RTX 4070 and Linux RX 9070 XT reports errored when the game refused the START button step. All four required frame-time p99 comparisons had zero usable starts on both baseline and candidate, so no p99 regression could be assessed.

Done (started 00:32 UTC, last report 01:04 UTC). Comment @Beanstalk to run it again.

Target Result Reports Frame time p99 (ms), main → PR VRAM peak (MB) Crashes
any ⚠️ couldn't be judged (an outlier) 2 1 2 — — none

Couldn't be judged: the tester's PC could not finish the test. The farm tries another PC, or the same one after a fix; it says nothing about the PR on its own.

  • any, report 1: Insufficient frame-time evidence for 4 required comparison(s). See rule results. the game refused a step: bad_request: not a button: START
  • any, report 2: Insufficient frame-time evidence for 4 required comparison(s). See rule results. the game refused a step: bad_request: not a button: START
What was tested

What will be tested · AI plan (gpt-6.1-terra)

Compare the pull request with main over roughly 35 minutes per tester, opening and closing the pause menu in Kokiri Forest, Castle Town, Hyrule Field, and Kakariko Village. The plan compares pictures, frame times, VRAM, and logged texture re-uploads; crashes, substantial picture differences, or frame-time and VRAM regressions fail it.

  • Where: any (any PC, 1-2 reports).
  • texture-refresh: from the repository's save kokiri-start, then teleport to Kokiri Forest, move the stick, press buttons, take the picture kokiri-menu to compare with main, measure kokiri-after-menu for 15 s (after 0 s warm-up), take the picture kokiri-after to compare with main, teleport to Market, measure market-after-menu for 15 s (after 0 s warm-up), take the picture market-after to compare with main, teleport to Hyrule Field, measure field-after-menu for 15 s (after 0 s warm-up), take the picture field-after to compare with main, teleport to Kakariko Village, measure kakariko-after-menu for 15 s (after 0 s warm-up), take the picture kakariko-after to compare with main; 3 runs on each side.
  • Measured: crashes, frame times, stutters and hitches, VRAM use, numbers from the game's log.
  • The PR fails if: it crashes; or a picture differs from main by more than 3%; or peak VRAM is more than 10% above main; or frame time (p99) is more than 20% above main.
  • Time: about 39 min of game runs per tester (6 game starts), plus the two builds.
Measurements on `any`, for the PR description

Verdict: could not be judged on any (2 reports): required frame-time comparison inconclusive (fewer than two usable starts per side).

Hardware: 1: windows+nvidia+nvidia-proprietary+dgpu NVIDIA GeForce RTX 4070, 621.24.0 AMD Ryzen 9 7900X 12-Core Processor, 64 GB RAM; 2: linux+amd+radv+dgpu AMD Radeon RX 9070 XT (RADV GFX1201), 26.1.99 AMD Ryzen 7 5700X3D 8-Core Processor, 32 GB RAM
Scene: texture-refresh, from the repository's save kokiri-start; each cell's settings and switches as the table names them
Method: the Beanstalk test farm: main and this PR built and run on the same PC in one session, game starts alternating (ABBA); texture-refresh: 3 game starts a side, kokiri-after-menu 15 s after 0 s warm-up, market-after-menu 15 s after 0 s warm-up, field-after-menu 15 s after 0 s warm-up, kakariko-after-menu 15 s after 0 s warm-up; each number the median of the runs, then of 2 reports
How to reproduce: the farm's plan above (@Beanstalk on this PR runs it again)

Nothing was measured: no metric came back.

Flags and settings

None: the plan varied no setting, switch or build option.

As Markdown for the PR description:

## Measurements

**Verdict:** could not be judged on any (2 reports): required frame-time comparison inconclusive (fewer than two usable starts per side).

Hardware: 1: windows+nvidia+nvidia-proprietary+dgpu `NVIDIA GeForce RTX 4070, 621.24.0` `AMD Ryzen 9 7900X 12-Core Processor, 64 GB RAM`; 2: linux+amd+radv+dgpu `AMD Radeon RX 9070 XT (RADV GFX1201), 26.1.99` `AMD Ryzen 7 5700X3D 8-Core Processor, 32 GB RAM`  
Scene: texture-refresh, from the repository's save kokiri-start; each cell's settings and switches as the table names them  
Method: the Beanstalk test farm: main and this PR built and run on the same PC in one session, game starts alternating (ABBA); texture-refresh: 3 game starts a side, kokiri-after-menu 15 s after 0 s warm-up, market-after-menu 15 s after 0 s warm-up, field-after-menu 15 s after 0 s warm-up, kakariko-after-menu 15 s after 0 s warm-up; each number the median of the runs, then of 2 reports  
How to reproduce: the farm's plan above (`@Beanstalk` on this PR runs it again)

Nothing was measured: no metric came back.

**Flags and settings**

None: the plan varied no setting, switch or build option.
The testers' hardware
Target Report Ran on
any 1 windows+nvidia+nvidia-proprietary+dgpu NVIDIA GeForce RTX 4070, 621.24.0 AMD Ryzen 9 7900X 12-Core Processor, 64 GB RAM
any 2 linux+amd+radv+dgpu AMD Radeon RX 9070 XT (RADV GFX1201), 26.1.99 AMD Ryzen 7 5700X3D 8-Core Processor, 32 GB RAM
<!-- beanstalk:sticky --> ### Beanstalk test farm · ⚠️ couldn't be judged on any Commit `c670586` · approved by HumanBean, llun (double-reviewed) · [dashboard](https://beanstalk.magicbean.dev/prs/2164) ⚠️ **Note:** Upload log counts are diagnostic, not a required reduction: the messages can be capped and the affected write-back path may not occur on every run. ⚠️ **Note:** The single general-hardware target does not guarantee coverage of every requested GPU, operating system, Steam Deck, or low-VRAM configuration; the farm plan also cannot run the standalone C++ fixtures. > **Summary**, written by AI (`gpt-6.1-terra`) from the numbers below: > > The change could not be judged on the any target: both the Windows RTX 4070 and Linux RX 9070 XT reports errored when the game refused the START button step. All four required frame-time p99 comparisons had zero usable starts on both baseline and candidate, so no p99 regression could be assessed. Done (started 00:32 UTC, last report 01:04 UTC). Comment `@Beanstalk` to run it again. | Target | Result | Reports | Frame time p99 (ms), main → PR | VRAM peak (MB) | Crashes | | --- | --- | --- | --- | --- | --- | | `any` | ⚠️ couldn't be judged (an outlier) | 2 [1](https://beanstalk.magicbean.dev/reports/rp_86f5742f9d3f24cb) [2](https://beanstalk.magicbean.dev/reports/rp_fed0744e0759cd72) | — | — | none | **Couldn't be judged**: the tester's PC could not finish the test. The farm tries another PC, or the same one after a fix; it says nothing about the PR on its own. - `any`, [report 1](https://beanstalk.magicbean.dev/reports/rp_86f5742f9d3f24cb): `Insufficient frame-time evidence for 4 required comparison(s). See rule results. the game refused a step: bad_request: not a button: START` - `any`, [report 2](https://beanstalk.magicbean.dev/reports/rp_fed0744e0759cd72): `Insufficient frame-time evidence for 4 required comparison(s). See rule results. the game refused a step: bad_request: not a button: START` <details><summary>What was tested</summary> **What will be tested** · AI plan (`gpt-6.1-terra`) Compare the pull request with main over roughly 35 minutes per tester, opening and closing the pause menu in Kokiri Forest, Castle Town, Hyrule Field, and Kakariko Village. The plan compares pictures, frame times, VRAM, and logged texture re-uploads; crashes, substantial picture differences, or frame-time and VRAM regressions fail it. - **Where:** any (any PC, 1-2 reports). - **texture-refresh:** from the repository's save `kokiri-start`, then teleport to Kokiri Forest, move the stick, press buttons, take the picture kokiri-menu to compare with main, measure kokiri-after-menu for 15 s (after 0 s warm-up), take the picture kokiri-after to compare with main, teleport to Market, measure market-after-menu for 15 s (after 0 s warm-up), take the picture market-after to compare with main, teleport to Hyrule Field, measure field-after-menu for 15 s (after 0 s warm-up), take the picture field-after to compare with main, teleport to Kakariko Village, measure kakariko-after-menu for 15 s (after 0 s warm-up), take the picture kakariko-after to compare with main; 3 runs on each side. - **Measured:** crashes, frame times, stutters and hitches, VRAM use, numbers from the game's log. - **The PR fails if:** it crashes; or a picture differs from main by more than 3%; or peak VRAM is more than 10% above main; or frame time (p99) is more than 20% above main. - **Time:** about 39 min of game runs per tester (6 game starts), plus the two builds. </details> <details><summary>Measurements on `any`, for the PR description</summary> **Verdict:** could not be judged on any (2 reports): required frame-time comparison inconclusive (fewer than two usable starts per side). Hardware: 1: windows+nvidia+nvidia-proprietary+dgpu `NVIDIA GeForce RTX 4070, 621.24.0` `AMD Ryzen 9 7900X 12-Core Processor, 64 GB RAM`; 2: linux+amd+radv+dgpu `AMD Radeon RX 9070 XT (RADV GFX1201), 26.1.99` `AMD Ryzen 7 5700X3D 8-Core Processor, 32 GB RAM` Scene: texture-refresh, from the repository's save kokiri-start; each cell's settings and switches as the table names them Method: the Beanstalk test farm: main and this PR built and run on the same PC in one session, game starts alternating (ABBA); texture-refresh: 3 game starts a side, kokiri-after-menu 15 s after 0 s warm-up, market-after-menu 15 s after 0 s warm-up, field-after-menu 15 s after 0 s warm-up, kakariko-after-menu 15 s after 0 s warm-up; each number the median of the runs, then of 2 reports How to reproduce: the farm's plan above (`@Beanstalk` on this PR runs it again) Nothing was measured: no metric came back. **Flags and settings** None: the plan varied no setting, switch or build option. As Markdown for the PR description: ```markdown ## Measurements **Verdict:** could not be judged on any (2 reports): required frame-time comparison inconclusive (fewer than two usable starts per side). Hardware: 1: windows+nvidia+nvidia-proprietary+dgpu `NVIDIA GeForce RTX 4070, 621.24.0` `AMD Ryzen 9 7900X 12-Core Processor, 64 GB RAM`; 2: linux+amd+radv+dgpu `AMD Radeon RX 9070 XT (RADV GFX1201), 26.1.99` `AMD Ryzen 7 5700X3D 8-Core Processor, 32 GB RAM` Scene: texture-refresh, from the repository's save kokiri-start; each cell's settings and switches as the table names them Method: the Beanstalk test farm: main and this PR built and run on the same PC in one session, game starts alternating (ABBA); texture-refresh: 3 game starts a side, kokiri-after-menu 15 s after 0 s warm-up, market-after-menu 15 s after 0 s warm-up, field-after-menu 15 s after 0 s warm-up, kakariko-after-menu 15 s after 0 s warm-up; each number the median of the runs, then of 2 reports How to reproduce: the farm's plan above (`@Beanstalk` on this PR runs it again) Nothing was measured: no metric came back. **Flags and settings** None: the plan varied no setting, switch or build option. ``` </details> <details><summary>The testers' hardware</summary> | Target | Report | Ran on | | --- | --- | --- | | `any` | [1](https://beanstalk.magicbean.dev/reports/rp_86f5742f9d3f24cb) | windows+nvidia+nvidia-proprietary+dgpu `NVIDIA GeForce RTX 4070, 621.24.0` `AMD Ryzen 9 7900X 12-Core Processor, 64 GB RAM` | | `any` | [2](https://beanstalk.magicbean.dev/reports/rp_fed0744e0759cd72) | linux+amd+radv+dgpu `AMD Radeon RX 9070 XT (RADV GFX1201), 26.1.99` `AMD Ryzen 7 5700X3D 8-Core Processor, 32 GB RAM` | </details>
Member

@Beanstalk

@Beanstalk
Member

@llun c670586 is already running: see the dashboard.

@llun `c670586` is already running: see the [dashboard](https://beanstalk.magicbean.dev/prs/2164).
Contributor

Write-back overlap with a local shadow-floor branch — two questions, no competing change

We have a local branch (not a PR) for the write-back's remaining bytes: RX_VK_WB_SHADOW_KB, default off, gives a pool whose small runs keep copying back whole a VRAM shadow so wb_diff.comp diffs those runs instead of copying them whole. It changes the p->shadow decision and the path counters in memory_end_submission; it does not touch wb_leave_out() or the keep ranges.

We read #2164's Notice as: keep is decided by wb_leave_out(wlo, whi, keep) before the copy path is chosen, and the textures are told of those kept ranges (less the edge page), whether the copy ends up whole or diffed. If that is right, our branch composes: a diffed run is still a written-back run, and notifying the whole kept range is conservative — it may upload a few unchanged bytes, never stale ones.

Two questions only:

  1. Is that the intended semantics — does a diffed write-back still count as "reached memory" for every range in keep? If not, which ranges should be notified when the write-back is diffed?
  2. If both land, which side should rebase? We are happy to rebase onto #2164 after it merges. If you would rather land ours first, it is local and default-off, and you can rebase onto it. Your call.

We are not opening a competing PR and are not asking for a design change. We will rebase onto yours if it merges first. No machine time or game data is needed for either answer.

**Write-back overlap with a local shadow-floor branch — two questions, no competing change** We have a local branch (not a PR) for the write-back's remaining bytes: `RX_VK_WB_SHADOW_KB`, default off, gives a pool whose small runs keep copying back whole a VRAM shadow so `wb_diff.comp` diffs those runs instead of copying them whole. It changes the `p->shadow` decision and the path counters in `memory_end_submission`; it does not touch `wb_leave_out()` or the `keep` ranges. We read #2164's `Notice` as: `keep` is decided by `wb_leave_out(wlo, whi, keep)` before the copy path is chosen, and the textures are told of those kept ranges (less the edge page), whether the copy ends up whole or diffed. If that is right, our branch composes: a diffed run is still a written-back run, and notifying the whole kept range is conservative — it may upload a few unchanged bytes, never stale ones. Two questions only: 1. Is that the intended semantics — does a diffed write-back still count as "reached memory" for every range in `keep`? If not, which ranges should be notified when the write-back is diffed? 2. If both land, which side should rebase? We are happy to rebase onto #2164 after it merges. If you would rather land ours first, it is local and default-off, and you can rebase onto it. Your call. We are not opening a competing PR and are not asking for a design change. We will rebase onto yours if it merges first. No machine time or game data is needed for either answer.
Some checks reported errors
PR description / Description follows the template (pull_request_target) Failing after 8s
Website / Website on Cloudflare (pull_request_target) Has been skipped
beanstalk/approval approved by HumanBean, llun (double-reviewed)
beanstalk/any 0/1 pass, 2 error
This pull request doesn't have enough approvals yet. 0 of 1 approvals granted.
Some workflows are waiting to be reviewed.
This branch is out-of-date with the base branch
You are not authorized to merge this pull request.
View command line instructions

Checkout

From your project repository, check out a new branch and test the changes.
git fetch -u 'fix/upload-only-arrived:Veix-fix/upload-only-arrived'
git switch 'Veix-fix/upload-only-arrived'
Sign in to join this conversation.
No reviewers
No milestone
No project
No assignees
6 participants
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
MagicBeanDev/Longshot!2164
No description provided.