WIP: perf(vk): re-upload only the part of a texture a GPU write covered (carries on #282) #1223

Draft
TrashPanda wants to merge 13 commits from TrashPanda/Longshot:perf/gpu-write-part into main
Contributor

Why

In Castle Town a compute pipeline binds a 1.75 MB write buffer that lands inside a finished 2560x1440 render target's storage. Every frame of that burst the renderer copies the whole 15 MB image back from guest memory, though the CPU wrote none of it. This uploads only the part the GPU write covered.

This is #282 by @bluetoe, carried on. The PR had a "request changes" review and has had no reply since. The six perf/feat/test/docs commits are bluetoe's, unchanged apart from conflict resolution and authored under their name. Four of them came after mooki's review and already answer most of it. I added two commits on top, described below.

Stacked on #1162. Its first commit (jarynclouatre's) keeps the GPU write records from overlapping. That is the fix for mooki's first point: bluetoe had made the same fix independently, and #1162's version was kept, which already has its own test. Review the last eight commits here; the first four are #1162's.

What changed

bluetoe's commits (see #282 for the full write-up):

  • gpu_wrote_into reports the ranges a GPU write covered, clipped to the texture and in image offsets. texture_touch uploads those, plus whatever the CPU wrote since the last upload, in the same pass. Without those CPU writes they would be lost, since the upload moves upload_gen past them. Sparse images keep the whole path, and RX_VK_GPU_WRITE_WHOLE=1 restores the old behaviour.
  • Each upload path now reports the bytes it really takes from the storage, not the ranges it was handed: whole GOB-block rows on the CPU path, and whole layers for ASTC. The tally line prints the three side by side: vk: gpu-write uploads: N (X MB taken by the upload path, Y MB of written ranges, the whole image would have been Z MB). RX_VK_GPU_WRITE_LOG=1 prints one line per event.
  • The GPU unswizzle dispatches only the block rows a write reached, instead of the whole level. The shader already offsets both its read and its write by the origin/destination push constants.
  • tests/test_gpu_write_part.py runs the recorder, the lookup, the merging of CPU and GPU ranges, the row choice and its byte count. It includes mooki's nested-write case.

Mine:

  • perf(vk): the GPU unswizzle's count. A dispatch covers whole groups of 32 rows, so with GOB blocks of 8 or 16 rows it also redoes rows past the written band. The tally counted the band, not what was dispatched; it now counts rows as dispatched, each once.
  • test: seven checks cut these functions out of the source by their text, and now look for the new signatures. None is weakened.

mooki's review on #282, point by point

Point Where it stands
A nested earlier write is missed, leaving part of the texture stale Fixed by #1162's recorder (records never overlap), tested in test_gpu_write_intervals and in test_gpu_write_part's nested case
The tally sums input ranges, not the bytes the upload processes bluetoe's c2806167 counts what each path takes. My commit counts what the GPU path dispatches
The CPU/GPU union is only checked by looking for strings bluetoe's fe587a3e runs it, with a separate CPU write next to the GPU's part
The measurement labels, run order and raw lines Re-measured on this head with the corrected count: each arm named by its build and switch, nine runs interleaved, run order and per-run figures below
The test-request links Still open: AMD and Windows requests to be opened

Flags and settings

Flag What it does Default now Recommendation Helps on Doesn't help on Untested on
RX_VK_GPU_WRITE_WHOLE=1 Uploads the whole image again on a GPU write, as before off (only the written part is uploaded) keep off; it exists for A/B reproducing the old behaviour anything else AMD, Intel, Windows
RX_VK_GPU_WRITE_LOG=1 One log line per GPU-write re-upload off on only while measuring reading a single burst - -

Measurements

main: ac778751 (its renderer built alone, RX_VK_ONLY)
this PR: perf/gpu-write-part @ 882479a1, built the same way
Hardware: RTX 4070 Ti 12 GB, 595.99.02, Ryzen 7 9800X3D, 30 GB, Ubuntu 26.04.1 (kernel 7.0.0-43), headless
Game: one copy of a v3.0.0-rc.5-26-g71b1edd8 folder, the renderer swapped between runs
Scene: scripts/memtour.py from the kokiri-start repository save: Market, Kakariko Village, Market, Kokiri Forest, Market, 40 s in each. Output 2560x1440, cap 240, RX_VK_GPU_WRITE_LOG=1.
Method: three arms, interleaved three times (part, main, whole, part, main, whole, ...), each a timing job in scripts/job_queue.py, so nothing else used the GPU or ran a build alongside. "whole" is this PR's renderer with RX_VK_GPU_WRITE_WHOLE=1; main's renderer has no per-upload line. Frame times exclude the first 10 s after each arrival.

The copying, from the per-upload lines (what the upload path took from guest memory, the written ranges it was handed, the whole images):

Run GPU-write uploads Taken by the path Written ranges Whole images
c1 this PR 211 240 MB 150 MB 1,134 MB
c4 this PR 133 140 MB 88 MB 679 MB
c7 this PR 191 222 MB 141 MB 1,066 MB
c3 whole 129 636 MB 665 MB 665 MB
c6 whole 204 808 MB 851 MB 851 MB
c9 whole 201 630 MB 670 MB 670 MB

The number of uploads varies a lot between runs of the same arm, so the comparison is per upload. With this PR an upload took 1.12 MB on average (601 MB over 535), against 3.88 MB with the whole path (2,073 MB over 534): 71% less. Within this PR's runs, the path took 21% of what the whole images would have been, and 1.6 times its written ranges, since a written page brings its whole GOB-block row up. The case in Why, the 2560x1440 target: gpu-write upload #631: 2560 KB taken by the path, 288 KB of written ranges, 15360 KB whole (the part).

Frame times (median of the three runs' medians, ms; p99 in brackets):

Market (first stay) Kakariko Village Market (second stay) Kokiri Forest
main 10.55 (24.58) 8.86 (18.81) 10.39 (21.71) 7.60 (12.31)
this PR 10.73 (23.73) 10.05 (17.62) 11.74 (24.59) 8.77 (13.94)
whole 10.25 (21.70) 12.02 (17.41) 10.46 (21.79) 9.54 (13.91)

No fps effect is claimed either way. This PR's slowest run (c7) was slower in every area, Kokiri Forest included, where there are few GPU-write uploads. The game folder's render resolution is on Auto, which lowered it in play by a different amount in each run, and about 19 GB of swap was in use by other work throughout. The teleport out of Kokiri Forest was refused in every run, and in one run per arm the second Market as well, so the second-stay columns have two runs per arm. Recorded in docs/tested/2026-11-16-gpu-write-part-trashpanda-run01.md.

Testing

Tested:

  • In the game, the nine runs above: no device lost, and every run finished its tour.
  • Market screenshots with this PR's renderer and with main's (RX_VK_SHOTS, one every 600 frames, 13 and 12 of them): nothing missing or corrupt with this PR. The walk drifts between runs, so the frames are not the same views.
  • python -I -B tests/run_all.py on Linux: 652 passed, 23 failed, 21 skipped. The same 23 fail on main (ac778751) on this PC: they want packages it doesn't have (numpy, zstandard, a game folder for the build tests). None fails only here.
  • The nine checks this change touches, all green: test_gpu_write_part, test_gpu_write_intervals, test_upload_retry, test_bc_upload_retry, test_wrote_regions, test_bc_scratch_layers, test_texture_staging_publication, test_guest_holes, test_compressed_over_rt.

Not tested: Windows, AMD and Intel. A fixed render resolution, for a cleaner frame-time comparison.

Needs other testers:

  • AMD (RDNA2 or newer): the Market tour with RX_VK_GPU_WRITE_LOG=1, expecting the per-upload drop and no texture corruption.
  • Windows: a build and one Market tour with the same line.

Screenshots

Nothing on screen: the change uploads a subset of what was uploaded before, and the Market screenshots described under Testing show no difference from main's.

Risk and rollback

Partial uploads have a known kind of hazard: an unrelated variable sharing an edge page can re-upload stale bytes over a drawn resource. This change copies a strict subset of what the old path copied. One wider change to note: the GPU unswizzle now uploads only the written rows for CPU-written partial uploads too, the same rows the CPU path has always copied, where before it redid the whole level. RX_VK_NO_PARTIAL=1 turns all partial uploads off, and RX_VK_GPU_WRITE_WHOLE=1 the GPU-write part only.

Assisted-by: Claude Opus 5.5

## Why In Castle Town a compute pipeline binds a 1.75 MB write buffer that lands inside a finished 2560x1440 render target's storage. Every frame of that burst the renderer copies the whole 15 MB image back from guest memory, though the CPU wrote none of it. This uploads only the part the GPU write covered. **This is [#282](https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/282) by @bluetoe, carried on.** The PR had a "request changes" review and has had no reply since. The six `perf`/`feat`/`test`/`docs` commits are bluetoe's, unchanged apart from conflict resolution and authored under their name. Four of them came after mooki's review and already answer most of it. I added two commits on top, described below. **Stacked on [#1162](https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/1162).** Its first commit (jarynclouatre's) keeps the GPU write records from overlapping. That is the fix for mooki's first point: bluetoe had made the same fix independently, and #1162's version was kept, which already has its own test. Review the last eight commits here; the first four are #1162's. ## What changed bluetoe's commits (see #282 for the full write-up): - `gpu_wrote_into` reports the ranges a GPU write covered, clipped to the texture and in image offsets. `texture_touch` uploads those, plus whatever the CPU wrote since the last upload, in the same pass. Without those CPU writes they would be lost, since the upload moves `upload_gen` past them. Sparse images keep the whole path, and `RX_VK_GPU_WRITE_WHOLE=1` restores the old behaviour. - Each upload path now reports the bytes it really takes from the storage, not the ranges it was handed: whole GOB-block rows on the CPU path, and whole layers for ASTC. The tally line prints the three side by side: `vk: gpu-write uploads: N (X MB taken by the upload path, Y MB of written ranges, the whole image would have been Z MB)`. `RX_VK_GPU_WRITE_LOG=1` prints one line per event. - The GPU unswizzle dispatches only the block rows a write reached, instead of the whole level. The shader already offsets both its read and its write by the `origin`/`destination` push constants. - `tests/test_gpu_write_part.py` runs the recorder, the lookup, the merging of CPU and GPU ranges, the row choice and its byte count. It includes mooki's nested-write case. Mine: - `perf(vk)`: the GPU unswizzle's count. A dispatch covers whole groups of 32 rows, so with GOB blocks of 8 or 16 rows it also redoes rows past the written band. The tally counted the band, not what was dispatched; it now counts rows as dispatched, each once. - `test`: seven checks cut these functions out of the source by their text, and now look for the new signatures. None is weakened. ## mooki's review on #282, point by point | Point | Where it stands | |---|---| | A nested earlier write is missed, leaving part of the texture stale | Fixed by #1162's recorder (records never overlap), tested in `test_gpu_write_intervals` and in `test_gpu_write_part`'s nested case | | The tally sums input ranges, not the bytes the upload processes | bluetoe's `c2806167` counts what each path takes. My commit counts what the GPU path dispatches | | The CPU/GPU union is only checked by looking for strings | bluetoe's `fe587a3e` runs it, with a separate CPU write next to the GPU's part | | The measurement labels, run order and raw lines | Re-measured on this head with the corrected count: each arm named by its build and switch, nine runs interleaved, run order and per-run figures below | | The test-request links | **Still open:** AMD and Windows requests to be opened | ## Flags and settings | Flag | What it does | Default now | Recommendation | Helps on | Doesn't help on | Untested on | |---|---|---|---|---|---|---| | `RX_VK_GPU_WRITE_WHOLE=1` | Uploads the whole image again on a GPU write, as before | off (only the written part is uploaded) | keep off; it exists for A/B | reproducing the old behaviour | anything else | AMD, Intel, Windows | | `RX_VK_GPU_WRITE_LOG=1` | One log line per GPU-write re-upload | off | on only while measuring | reading a single burst | - | - | ## Measurements main: ac778751 (its renderer built alone, `RX_VK_ONLY`) this PR: perf/gpu-write-part @ 882479a1, built the same way Hardware: RTX 4070 Ti 12 GB, 595.99.02, Ryzen 7 9800X3D, 30 GB, Ubuntu 26.04.1 (kernel 7.0.0-43), headless Game: one copy of a `v3.0.0-rc.5-26-g71b1edd8` folder, the renderer swapped between runs Scene: `scripts/memtour.py` from the kokiri-start repository save: Market, Kakariko Village, Market, Kokiri Forest, Market, 40 s in each. Output 2560x1440, cap 240, `RX_VK_GPU_WRITE_LOG=1`. Method: three arms, interleaved three times (part, main, whole, part, main, whole, ...), each a `timing` job in `scripts/job_queue.py`, so nothing else used the GPU or ran a build alongside. "whole" is this PR's renderer with `RX_VK_GPU_WRITE_WHOLE=1`; main's renderer has no per-upload line. Frame times exclude the first 10 s after each arrival. **The copying, from the per-upload lines** (what the upload path took from guest memory, the written ranges it was handed, the whole images): | Run | GPU-write uploads | Taken by the path | Written ranges | Whole images | |---|---|---|---|---| | c1 this PR | 211 | 240 MB | 150 MB | 1,134 MB | | c4 this PR | 133 | 140 MB | 88 MB | 679 MB | | c7 this PR | 191 | 222 MB | 141 MB | 1,066 MB | | c3 whole | 129 | 636 MB | 665 MB | 665 MB | | c6 whole | 204 | 808 MB | 851 MB | 851 MB | | c9 whole | 201 | 630 MB | 670 MB | 670 MB | The number of uploads varies a lot between runs of the same arm, so the comparison is per upload. With this PR an upload took **1.12 MB on average (601 MB over 535)**, against **3.88 MB with the whole path (2,073 MB over 534)**: 71% less. Within this PR's runs, the path took 21% of what the whole images would have been, and 1.6 times its written ranges, since a written page brings its whole GOB-block row up. The case in *Why*, the 2560x1440 target: `gpu-write upload #631: 2560 KB taken by the path, 288 KB of written ranges, 15360 KB whole (the part)`. **Frame times** (median of the three runs' medians, ms; p99 in brackets): | | Market (first stay) | Kakariko Village | Market (second stay) | Kokiri Forest | |---|---|---|---|---| | main | 10.55 (24.58) | 8.86 (18.81) | 10.39 (21.71) | 7.60 (12.31) | | this PR | 10.73 (23.73) | 10.05 (17.62) | 11.74 (24.59) | 8.77 (13.94) | | whole | 10.25 (21.70) | 12.02 (17.41) | 10.46 (21.79) | 9.54 (13.91) | **No fps effect is claimed either way.** This PR's slowest run (c7) was slower in every area, Kokiri Forest included, where there are few GPU-write uploads. The game folder's render resolution is on Auto, which lowered it in play by a different amount in each run, and about 19 GB of swap was in use by other work throughout. The teleport out of Kokiri Forest was refused in every run, and in one run per arm the second Market as well, so the second-stay columns have two runs per arm. Recorded in `docs/tested/2026-11-16-gpu-write-part-trashpanda-run01.md`. ## Testing **Tested:** - In the game, the nine runs above: no device lost, and every run finished its tour. - Market screenshots with this PR's renderer and with main's (`RX_VK_SHOTS`, one every 600 frames, 13 and 12 of them): nothing missing or corrupt with this PR. The walk drifts between runs, so the frames are not the same views. - `python -I -B tests/run_all.py` on Linux: 652 passed, 23 failed, 21 skipped. The same 23 fail on `main` (`ac778751`) on this PC: they want packages it doesn't have (numpy, zstandard, a game folder for the build tests). None fails only here. - The nine checks this change touches, all green: `test_gpu_write_part`, `test_gpu_write_intervals`, `test_upload_retry`, `test_bc_upload_retry`, `test_wrote_regions`, `test_bc_scratch_layers`, `test_texture_staging_publication`, `test_guest_holes`, `test_compressed_over_rt`. **Not tested:** Windows, AMD and Intel. A fixed render resolution, for a cleaner frame-time comparison. **Needs other testers:** - AMD (RDNA2 or newer): the Market tour with `RX_VK_GPU_WRITE_LOG=1`, expecting the per-upload drop and no texture corruption. - Windows: a build and one Market tour with the same line. ## Screenshots Nothing on screen: the change uploads a subset of what was uploaded before, and the Market screenshots described under Testing show no difference from main's. ## Risk and rollback Partial uploads have a known kind of hazard: an unrelated variable sharing an edge page can re-upload stale bytes over a drawn resource. This change copies a strict subset of what the old path copied. One wider change to note: the GPU unswizzle now uploads only the written rows for CPU-written partial uploads too, the same rows the CPU path has always copied, where before it redid the whole level. `RX_VK_NO_PARTIAL=1` turns all partial uploads off, and `RX_VK_GPU_WRITE_WHOLE=1` the GPU-write part only. Assisted-by: Claude Opus 5.5
`texture_gpu_wrote` in `gpu/vk/vk_texture.cpp` notes each GPU buffer
write into guest memory as a record, start to end plus the submission
that wrote it. `gpu_wrote_into` finds the finished ones a texture was
not uploaded after. It starts at the one record before the texture's
address and walks forward, which is right only when records do not
overlap. The writer erased only the records a new write covered whole,
so a later, smaller write inside an older one left two records that
overlap.

For example, take `[0x100000, 0x140000)` at submission 10, then
`[0x110000, 0x120000)` at 11. For a texture at `0x130000`, the record
before it is the second one, which ends before the texture, and the
first is never looked at. The texture kept its old texels after the GPU
wrote them.

## Change

The writer keeps the records apart:
- A record the new write lands in keeps its parts before and after it,
each with its own submission. This applies to the one record that starts
before the write, and to one that starts inside it and runs past its
end.
- Then the new record is added.
- The "same start, covers it" shortcut stays.
- The cap of 65536 records is now enforced with a loop, because a write
inside a record adds two records.

`gpu_wrote_into` is unchanged.

## Testing

- New `tests/test_gpu_write_intervals.py`: both functions and the map
are cut from the source and run on such writes:
- The texture past the inner write finds 10, one inside it 11, one
before it 10.
- With only 10 finished, the outer ones are found and the inner one
waits.
- A write that runs past a record's end leaves the record's first part
with the old submission.
  - A write seen once is not found again, and a later one is.
  - After each write no two records overlap.
  - 100 interior writes into a full map keep it at 65536 records.
- On main the overlap cases fail. The fixture also compiles with
llvm-mingw for Windows.
- Linux renderer built (`hookshot_vk.so`).
- Full suite on Linux: 474 passed, 53 skipped, 2 failed, both the same
on main on this machine (`test_bake_portability`: `zstandard` is not
installed; `nx2/test_texture_init_barrier.py`: `clang++` is not
installed).
- No game ran.

Restored-from: https://github.com/NotRexed/Hookshot/pull/3002
When `upload()` fails in `texture_touch` (`gpu/vk/vk_texture.cpp`), for
example for want of staging memory with the VRAM or RAM full, a BC3
texture is tried again 30 frames on (https://github.com/NotRexed/Hookshot/pull/1505). Every other kind was marked
uploaded with `upload_gen` moved on ("not retried every draw; a write to
its memory tries again"):
- A first upload that failed left an empty image, drawn as it was until
the game wrote that memory again, which for most textures is never.
- A re-upload that failed dropped what the CPU had written since the
last upload. It also dropped the GPU buffer write `gpu_wrote_into` had
just marked as seen.

This is the "textures degrade until a restart" kind of report on cards
with little memory (https://github.com/NotRexed/Hookshot/pull/993's family).

## Change

- One rule for every kind. `Texture::bc_retry_frame` becomes
`retry_frame`.
- On a failed upload:
  - `uploaded` and `upload_gen` stay as they were.
  - `gpu_wrote_seen` goes back to its value from before this touch.
  - `retry_frame` is set 30 frames on.
- A line is logged (40 at most). The BC3 line keeps its text, and the
other kinds get one like it.
- The wait moves up, right after the epoch check and before
`gpu_wrote_into` and the dirty scan. A waiting texture costs a compare
and loses nothing.
- A successful upload clears `retry_frame`.
- A texture evicted and made again over storage render targets took has
its `evicted_from_memory` entry taken before the upload. If the upload
fails, the entry is put back. Otherwise the retry would take the texture
for rendered storage and mark it uploaded, empty.
- An image no upload takes (no GPU address, multisampled, empty or too
big, packed depth-stencil) still fails at once, every time. It is still
marked uploaded, so it is not tried forever. Those checks move from the
top of `upload_inner` into `upload_never`, used by both.
- Nothing changes while uploads succeed.

## Testing

- New `tests/test_upload_retry.py` checks the source, because
`texture_touch` cannot run outside the renderer:
- The failed branch marks nothing done, sets the retry, puts
`gpu_wrote_seen` back and logs.
- The wait stands before `gpu_wrote_into` and the dirty scan, and
`gpu_wrote_seen`'s old value is taken before it.
  - A success clears the retry.
- `upload_never` is shared with `upload_inner` and still marks those
images uploaded.
  - A taken `evicted_from_memory` entry is put back on a failure.
- It fails on main.
- `tests/test_bc_upload_retry.py` follows the rename.
- Linux renderer built (`hookshot_vk.so`).
- Full suite on Linux: 474 passed, 53 skipped, 2 failed, both the same
on main on this machine (`test_bake_portability`: `zstandard` is not
installed; `nx2/test_texture_init_barrier.py`: `clang++` is not
installed).
- Not tested: a failed upload in the game. No card here runs out of
memory on demand, and no game ran.
- Not covered: for a texture in a virtual pool (GPU address at or above
1 << 40), the dirty scan has already taken a remap (`page_map`,
`map_sig`) when the upload fails. The retry uploads what the CPU wrote,
but not pages that were only remapped. That is the same as before;
replaying it would mean a whole re-upload of a sparse array while memory
is short.

Restored-from: https://github.com/NotRexed/Hookshot/pull/3003
test: the guest holes check cuts up to upload_never, which now comes before upload_inner
Some checks are pending
CI / Conventional Commit title (pull_request) Blocked by required conditions
CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Blocked by required conditions
CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Blocked by required conditions
CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Blocked by required conditions
CI / Workflow lint (actionlint) (pull_request) Blocked by required conditions
Guide screenshots / Guide images and local UI source review (pull_request) Blocked by required conditions
Third-party parts / As their notes say (pull_request) Blocked by required conditions
Third-party parts / The same as upstream (pull_request) Blocked by required conditions
Website / Website on Cloudflare (pull_request) Blocked by required conditions
PR description / Description follows the template (pull_request_target) Successful in 3s
856036459c
test_guest_holes.py cuts upload_all_zero out of vk_texture.cpp up to the
next function. The failed-upload retry puts upload_never between the two,
and the fixture has no Texture fields it reads: the cut stops at it.

Assisted-by: Claude Opus 5.5
A buffer bound for writing marks its whole bound range written, though the shader may touch less
of it, and the game's scratch lands on the storage of a target the GPU drew and finalized: in
Castle Town a 1.75 MB bind made the renderer re-upload a 15 MB 2560x1440 target every frame of
the burst: gpu_wrote_into now reports the ranges it matched, clipped to the image and
merged, and texture_touch uploads those plus the CPU's writes since: the whole upload used to
cover them, and the upload moves upload_gen past them, so carrying them along is required. Sparse
images and RX_VK_NO_PARTIAL keep the old path; RX_VK_GPU_WRITE_WHOLE=1 asks for all of it again.
The new "gpu-write uploads" line counts the path: over a five-area tour, run twice, 150 events
copied 648 MB where the whole-image rule would have copied 1,462 MB (second sample 632 against
1,434). Per event 15,360 KB to 1,800 KB for a bind-only write-back, and an event where the CPU had
also written still uploads its full 15,360 KB. No fps effect across two samples per side, and the
tour showed up to 19 fps of run-to-run noise for the same build, so none is claimed.
tests/test_gpu_write_part.py runs the marking and the range report against stand-ins.

Assisted-by: Qwen3.8-Flash-Next
Co-Authored-By: Qwen3.8-Flash-Next <[email protected]>
(cherry picked from commit 2e64a2a3fd)
Assisted-by: Qwen3.8-Flash-Next
Co-Authored-By: Qwen3.8-Flash-Next <[email protected]>
(cherry picked from commit 3ab8444a52)
The gpu-write tally added the sizes of the ranges it was given, but no path moves exactly those
bytes: the CPU path works in whole GOB-block rows, the GPU unswizzle redoes a level entire
(vkCmdDispatch over its width, height and every layer) and the ASTC path stages whole layers, so a
small range can be reported as a small copy over work that is not small. The row choice moves into
dirty_runs() with its geometry in row_geom(), and both report the bytes they take; upload() passes
that amount out of the path it chose, and the line now prints the three apart: what the path took,
the written ranges it was handed, and the whole image the old rule would have taken.

tests/test_gpu_write_part.py runs the row choice and its count against a 2560x1440 RGBA8 level: a
1.75 MB write brings up 24 of its 180 block rows, 1,966,080 bytes of the storage, which is neither
the 1,851,392 the range names nor the 14,745,600 of the level. tests/test_bc_upload_retry.py
follows the ASTC path's new parameter. vk_texture.cpp compiled with clang -fsyntax-only over the
renderer's own include set; the suite is 466 passed, 0 failed, 63 skipped here (the skips want
vulkan-headers, which this machine has not).

Assisted-by: Qwen3.8-Flash-Next
Co-Authored-By: Qwen3.8-Flash-Next <[email protected]>
(cherry picked from commit c28061676f)
The union of the two sets of written ranges was checked by a source string, so it said nothing about
what the union does: a CPU page written since the last upload, in a corner of the storage the GPU
write never touched, is exactly the case that would be lost if the part left it out (the upload moves
upload_gen past it and nothing looks for it again). It moves out of texture_touch as join_ranges(),
which the part test now runs: a CPU range beside the GPU's part comes back in the list, and one
reaching into the tail of it merges. Same sort and same merge rule as before, one call instead of a
block.

Assisted-by: Qwen3.8-Flash-Next
Co-Authored-By: Qwen3.8-Flash-Next <[email protected]>
(cherry picked from commit fe587a3ed3)
The tally speaks at the first event and every 50th, which is right in a scene where the path runs
hundreds of times and useless in a short one: a single area change shows nothing to compare.
RX_VK_GPU_WRITE_LOG=1 prints the same three numbers per event - what the upload path took from the
storage, the written ranges it was handed, and the whole image the old rule would have taken - which
is also what the description's per-event row has to be measured against now that the counters say
"taken" rather than "copied". Row added to the switch table in docs/development.md.

Assisted-by: Qwen3.8-Flash-Next
Co-Authored-By: Qwen3.8-Flash-Next <[email protected]>
(cherry picked from commit 3630b0cc10)
The GPU unswizzle used the written ranges to choose which *levels* to redo and then dispatched the
level entire: vkCmdDispatch over its width, its height and every layer. A 1.8 MB write into a
2560x1440 target therefore read and wrote 15 MB, which is what the corrected byte counts showed:
15,360 KB taken for 1,800 KB of ranges, so the partial path saved nothing on the route the Castle
Town events actually take. The shader already offsets both sides - it reads at (invocation + origin)
and writes at (invocation + destination) - so the rows go in as those offsets: per layer, the runs
dirty_runs() names (the same GOB rows the CPU path copies), one dispatch each, via unswizzle_band()
for the first block and how many, stopped at the level's height. A whole-level upload keeps the old
single dispatch with every layer at once. Band cases in tests/test_gpu_write_part.py: a run of 24
rows becomes blocks 96..288, and in a 1437-block-high level the last band stops at 5.

Assisted-by: Qwen3.8-Flash-Next
Co-Authored-By: Qwen3.8-Flash-Next <[email protected]>
(cherry picked from commit 541418349f)
The part upload carries the ranges out of gpu_wrote_into and the bytes a
path took out of upload_inner, upload_astc_gpu and upload_unswizzle_gpu.
The checks that cut these functions by their text look for the new
signatures: test_gpu_write_intervals, test_upload_retry,
test_wrote_regions, test_bc_scratch_layers and
test_texture_staging_publication (whose fixture passes no count).
test_gpu_write_part compiles the reader, which asks wrote_regions
first, with the real header. No check is weakened.

Assisted-by: Claude Opus 5.5
perf(vk): the GPU unswizzle's part counts the rows it dispatches, not the band it was asked for
Some checks failed
PR description / Description follows the template (pull_request_target) Successful in 3s
Website / Website on Cloudflare (pull_request_target) Has been skipped
CI / Conventional Commit title (pull_request) Has been cancelled
CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Has been cancelled
CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Has been cancelled
CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Has been cancelled
CI / Workflow lint (actionlint) (pull_request) Has been cancelled
Guide screenshots / Guide images and local UI source review (pull_request) Has been cancelled
Third-party parts / As their notes say (pull_request) Has been cancelled
Third-party parts / The same as upstream (pull_request) Has been cancelled
Website / Website on Cloudflare (pull_request) Has been cancelled
882479a193
A dispatch covers whole groups of 32 block rows, cut at the level's
height by the shader's bound. With GOB blocks of 8 or 16 rows that is
more than the band dirty_runs chose, rows unswizzled again from memory
nobody wrote, and the gpu-write tally left them out. They are counted
as dispatched now, each row once where two runs' dispatches overlap.

Assisted-by: Claude Opus 5.5
docs(tested): the GPU-write part upload measured against main and the whole path, nine runs
Some checks are pending
CI / Conventional Commit title (pull_request) Blocked by required conditions
CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Blocked by required conditions
CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Blocked by required conditions
CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Blocked by required conditions
CI / Workflow lint (actionlint) (pull_request) Blocked by required conditions
Guide screenshots / Guide images and local UI source review (pull_request) Blocked by required conditions
Third-party parts / As their notes say (pull_request) Blocked by required conditions
Third-party parts / The same as upstream (pull_request) Blocked by required conditions
Website / Website on Cloudflare (pull_request) Blocked by required conditions
PR description / Description follows the template (pull_request_target) Successful in 4s
308ddd874c
Assisted-by: Claude Opus 5.5
Some checks are pending
CI / Conventional Commit title (pull_request) Blocked by required conditions
CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Blocked by required conditions
CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Blocked by required conditions
CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Blocked by required conditions
CI / Workflow lint (actionlint) (pull_request) Blocked by required conditions
Guide screenshots / Guide images and local UI source review (pull_request) Blocked by required conditions
Third-party parts / As their notes say (pull_request) Blocked by required conditions
Third-party parts / The same as upstream (pull_request) Blocked by required conditions
Website / Website on Cloudflare (pull_request) Blocked by required conditions
PR description / Description follows the template (pull_request_target) Successful in 4s
This pull request has changes conflicting with the target branch.
  • docs/development.md
  • gpu/vk/vk_texture.cpp
Some workflows are waiting to be reviewed.
View command line instructions

Manual merge helper

Use this merge commit message when completing the merge manually.

Checkout

From your project repository, check out a new branch and test the changes.
git fetch -u 'perf/gpu-write-part:TrashPanda-perf/gpu-write-part'
git switch 'TrashPanda-perf/gpu-write-part'
Sign in to join this conversation.
No reviewers
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
MagicBeanDev/Longshot!1223
No description provided.