MagicBeanDev/Hookshot

Archived

Watch

Fork 0

perf(vk): with batched submits (RADV), send a batch at once while the GPU is idle (#1993) #1999

Merged

NotRexed merged 5 commits from perf/batch-idle-flush into main

2026-11-11 03:49:17 +00:00

NotRexed commented

2026-11-10 23:11:51 +00:00

(Migrated from github.com)

Implements #1993's proposal by @mrdev023, as a default where batching is on.

With batched submits (on by default with RADV, so Linux AMD including the Steam Deck; RX_VK_BATCH=1 elsewhere), a held batch waited for the next wait, present or the 32-buffer cap. That happened even when the GPU had finished everything it was sent. So the GPU sat idle until the game waited for it, and with serial replay that's mid-frame, at take_capture's late-page wait.

Now, when a submit joins the batch and the GPU has completed every batch sent so far (batch_tl >= batch_flushed), the batch goes out at once. While the GPU is busy, submits are still grouped.

The reporter's numbers (RX 9070 XT, RADV, Kokiri Forest, FSR 4, 5120x1440):

build	fps in play	flashes (framelog, 5+ blocks)
main	33-41	0
idle flush	46-61	0 in 5230 frames

Flushes went from about 5.7 a frame to about 10 of 3.7 command buffers each, and the late-page wait from about 1.4 to 0.64 ms.

RX_VK_BATCH_IDLE=0 keeps the old behaviour.

Not yet checked here: I have no RADV machine. I'll A/B it on Windows with RX_VK_BATCH=1 (NVIDIA) for fps and flashes before merging. Drivers other than RADV are unchanged, since batching stays off there by default.

🤖 Generated with Claude Code

Implements #1993's proposal by @mrdev023, as a default where batching is on. With batched submits (on by default with RADV, so Linux AMD including the Steam Deck; `RX_VK_BATCH=1` elsewhere), a held batch waited for the next wait, present or the 32-buffer cap. That happened even when the GPU had finished everything it was sent. So the GPU sat idle until the game waited for it, and with serial replay that's mid-frame, at `take_capture`'s late-page wait. Now, when a submit joins the batch and the GPU has completed every batch sent so far (`batch_tl` >= `batch_flushed`), the batch goes out at once. While the GPU is busy, submits are still grouped. The reporter's numbers (RX 9070 XT, RADV, Kokiri Forest, FSR 4, 5120x1440): | build | fps in play | flashes (framelog, 5+ blocks) | |---|---|---| | main | 33-41 | 0 | | idle flush | 46-61 | 0 in 5230 frames | Flushes went from about 5.7 a frame to about 10 of 3.7 command buffers each, and the late-page wait from about 1.4 to 0.64 ms. `RX_VK_BATCH_IDLE=0` keeps the old behaviour. **Not yet checked here:** I have no RADV machine. I'll A/B it on Windows with `RX_VK_BATCH=1` (NVIDIA) for fps and flashes before merging. Drivers other than RADV are unchanged, since batching stays off there by default. 🤖 Generated with [Claude Code](https://claude.com/claude-code)

mrdev023 commented

2026-11-10 23:28:04 +00:00

(Migrated from github.com)

Tested this branch as is (35a8aaa on main abd5e5f) on RX 9070 XT, RADV, Ryzen 9 9950X3D, NixOS. Kokiri Forest, FSR 4 at 5120x1440, GPU power level auto, same build both ways:

	fps in play	framelog events of 5+ blocks	flushes a frame	late-page wait a frame
`RX_VK_BATCH_IDLE=0`	29-38	0 / 2068	~4.1 (14.1 cbs each)	~7.9 ms
this PR	41-57	0 / 3044	~14.5 (4.0 cbs each)	~4.6 ms

Same gain as in #1993, no flashes.

Tested this branch as is (`35a8aaa` on `main` abd5e5f) on RX 9070 XT, RADV, Ryzen 9 9950X3D, NixOS. Kokiri Forest, FSR 4 at 5120x1440, GPU power level `auto`, same build both ways: | | fps in play | framelog events of 5+ blocks | flushes a frame | late-page wait a frame | |---|---|---|---|---| | `RX_VK_BATCH_IDLE=0` | 29-38 | 0 / 2068 | ~4.1 (14.1 cbs each) | ~7.9 ms | | this PR | 41-57 | 0 / 3044 | ~14.5 (4.0 cbs each) | ~4.6 ms | Same gain as in #1993, no flashes.

hckrman101 (Migrated from github.com) reviewed

2026-11-11 01:39:06 +00:00

hckrman101 (Migrated from github.com) left a comment

The change looks right to me. One thing should be checked before merge: the hardware this is aimed at.

Correctness. Sending a held batch early can't reorder anything. queue_flush_locked still runs under dev.present_mu, on the one queue, and signals timeline, rec_timeline and batch_tl with the same values it would have used later. batch_flushed is only read or written under that lock (recording_wait, queue_flush_locked, now batch_gpu_idle), so the replay, present and recorder threads gain no new race. Waits that need the batch keep their own flushes (wait/wait_value/vk_timeline_wait -> FR_VALUE/FR_WAIT, wait_recorded -> FR_WAIT, recording_wait -> FR_SLOT, present -> FR_PRESENT), so nothing new depends on this flush. It only makes them find the batch already sent more often. The first submit works: batch_tl and batch_flushed both start at 0, so it goes out at once. A lost device just makes vkGetSemaphoreCounterValue fail and the batch stays held, as before. The second commit only moves the check into batch_gpu_idle(), so mrdev023's run on 35a8aaa still applies to this head.

Gating. batch_gpu_idle() is only called inside the batch_on() branch, so drivers without batching (everything but RADV unless RX_VK_BATCH=1) don't change.

Perf. The numbers are there, and mrdev023 reran this branch (41-57 vs 29-38 fps, 0 flashes in 3044 frames). The catch is that both runs are one machine (9070 XT + 9950X3D at 5120x1440 with FSR 4), and that machine is GPU-bound. #1993 also notes that RX_VK_BATCH=0 was faster than the default there (44-55 vs 41-48). So on that box, much of the win comes from batching less. Batching was turned on for RADV for the opposite case: issue 413's 8060S, where each submit paid for the userptr imports again (~70 -> ~6 ms of queue submit a frame). On a setup where the GPU keeps up with the recorder, done >= batch_flushed holds at almost every submit, so most submits go out alone and we're close to RX_VK_BATCH=0 again. The Steam Deck is Tier 1 and RADV, so this goes default-on there. The NVIDIA RX_VK_BATCH=1 A/B you plan won't show that cost, because NVIDIA doesn't have the per-submit userptr cost. Could someone run it on a Deck or an RADV APU, or on a lighter RADV setup (native res, no FSR) where the GPU isn't the bottleneck? Check that the stats line's flushes a frame and the queue-submit time stay well below what RX_VK_BATCH=0 gives. If it does regress, the fix could be to flush on idle only once the batch is a few buffers big or has been held for a while, like RX_VK_BATCH_POLL_US does.

#1905. No duplicate and no textual conflict. #1905 changes wait_recorded's flush, which main already does unconditionally as FR_WAIT since the load-hang fix, into a conditional FR_READBACK. This PR touches submit_recording and not the FR enum or the stats line. The two complement each other: this one sends a batch early while the GPU is idle, and the readback flush still covers a batch held while the GPU was busy. #1905 is in conflict with main as it stands and also carries unrelated TAAU shader files, so it needs a rebase anyway.

Nit: the description says the late-page wait went from ~1.4 to 0.64 ms. #1993 says ~8.4 -> ~3.8 ms a frame and mrdev023's rerun ~7.9 -> ~4.6 ms, so the description's number is probably per wait or from another build.

CI: the only failure is tests/test_log_batch.py on windows-2022 (GetTickCount64() - t0 < 200), a timing check in logging that has nothing to do with this change. Ubuntu checks, the llvm-mingw Vulkan build and the AppImage pass.

The change looks right to me. One thing should be checked before merge: the hardware this is aimed at. **Correctness.** Sending a held batch early can't reorder anything. `queue_flush_locked` still runs under `dev.present_mu`, on the one queue, and signals `timeline`, `rec_timeline` and `batch_tl` with the same values it would have used later. `batch_flushed` is only read or written under that lock (`recording_wait`, `queue_flush_locked`, now `batch_gpu_idle`), so the replay, present and recorder threads gain no new race. Waits that need the batch keep their own flushes (`wait`/`wait_value`/`vk_timeline_wait` -> FR_VALUE/FR_WAIT, `wait_recorded` -> FR_WAIT, `recording_wait` -> FR_SLOT, present -> FR_PRESENT), so nothing new depends on this flush. It only makes them find the batch already sent more often. The first submit works: `batch_tl` and `batch_flushed` both start at 0, so it goes out at once. A lost device just makes `vkGetSemaphoreCounterValue` fail and the batch stays held, as before. The second commit only moves the check into `batch_gpu_idle()`, so mrdev023's run on `35a8aaa` still applies to this head. **Gating.** `batch_gpu_idle()` is only called inside the `batch_on()` branch, so drivers without batching (everything but RADV unless `RX_VK_BATCH=1`) don't change. **Perf.** The numbers are there, and mrdev023 reran this branch (41-57 vs 29-38 fps, 0 flashes in 3044 frames). The catch is that both runs are one machine (9070 XT + 9950X3D at 5120x1440 with FSR 4), and that machine is GPU-bound. #1993 also notes that `RX_VK_BATCH=0` was *faster* than the default there (44-55 vs 41-48). So on that box, much of the win comes from batching less. Batching was turned on for RADV for the opposite case: issue 413's 8060S, where each submit paid for the userptr imports again (~70 -> ~6 ms of queue submit a frame). On a setup where the GPU keeps up with the recorder, `done >= batch_flushed` holds at almost every submit, so most submits go out alone and we're close to `RX_VK_BATCH=0` again. The Steam Deck is Tier 1 and RADV, so this goes default-on there. The NVIDIA `RX_VK_BATCH=1` A/B you plan won't show that cost, because NVIDIA doesn't have the per-submit userptr cost. Could someone run it on a Deck or an RADV APU, or on a lighter RADV setup (native res, no FSR) where the GPU isn't the bottleneck? Check that the stats line's flushes a frame and the queue-submit time stay well below what `RX_VK_BATCH=0` gives. If it does regress, the fix could be to flush on idle only once the batch is a few buffers big or has been held for a while, like `RX_VK_BATCH_POLL_US` does. **#1905.** No duplicate and no textual conflict. #1905 changes `wait_recorded`'s flush, which main already does unconditionally as FR_WAIT since the load-hang fix, into a conditional FR_READBACK. This PR touches `submit_recording` and not the FR enum or the stats line. The two complement each other: this one sends a batch early while the GPU is idle, and the readback flush still covers a batch held while the GPU was busy. #1905 is in conflict with main as it stands and also carries unrelated TAAU shader files, so it needs a rebase anyway. Nit: the description says the late-page wait went from ~1.4 to 0.64 ms. #1993 says ~8.4 -> ~3.8 ms a frame and mrdev023's rerun ~7.9 -> ~4.6 ms, so the description's number is probably per wait or from another build. CI: the only failure is `tests/test_log_batch.py` on windows-2022 (`GetTickCount64() - t0 < 200`), a timing check in logging that has nothing to do with this change. Ubuntu checks, the llvm-mingw Vulkan build and the AppImage pass.

gpu/vk/vk_replay.cpp

Nit: these flushes are counted as FR_OTHER, so in the stats line they're mixed with the other untagged flushes (a parallel segment's own submit, queue_flush(), device_wait_idle_locked). An FR_IDLE reason, its own column in the stats line (one more stub in the two test fixtures), would show directly how many of the ~14 flushes a frame this causes. That's also the number to watch on a Deck or APU run.

Nit: these flushes are counted as FR_OTHER, so in the stats line they're mixed with the other untagged flushes (a parallel segment's own submit, `queue_flush()`, `device_wait_idle_locked`). An FR_IDLE reason, its own column in the stats line (one more stub in the two test fixtures), would show directly how many of the ~14 flushes a frame this causes. That's also the number to watch on a Deck or APU run.

NotRexed commented

2026-11-11 01:55:22 +00:00

(Migrated from github.com)

A/B on Windows (RTX 4060 Ti, batching forced on with RX_VK_BATCH=1), smooth.ps1's route (Kokiri Forest, Hyrule Field, Kakariko), 2 rounds alternating:

run	median	p99
batching off (Windows' default)	59 / 59 fps	29.1 / 29.5 ms
batching on + idle flush (this PR)	59 / 58 fps	28.9 / 31.5 ms
batching on, held (`RX_VK_BATCH_IDLE=0`, before this PR)	47 / 46 fps	34.1 / 37.3 ms

With the batch held, the GPU sits idle and loses ~12 fps here, matching what #1993 measured on RADV. The idle flush brings batching back to the unbatched frame rate, and Windows' default (batching off) is unchanged.

🤖 Generated with Claude Code

A/B on Windows (RTX 4060 Ti, batching forced on with `RX_VK_BATCH=1`), smooth.ps1's route (Kokiri Forest, Hyrule Field, Kakariko), 2 rounds alternating: | run | median | p99 | crashes | |---|---|---|---| | batching off (Windows' default) | 59 / 59 fps | 29.1 / 29.5 ms | 0 | | batching on + idle flush (this PR) | 59 / 58 fps | 28.9 / 31.5 ms | 0 | | batching on, held (`RX_VK_BATCH_IDLE=0`, before this PR) | 47 / 46 fps | 34.1 / 37.3 ms | 0 | With the batch held, the GPU sits idle and loses ~12 fps here, matching what #1993 measured on RADV. The idle flush brings batching back to the unbatched frame rate, and Windows' default (batching off) is unchanged. 🤖 Generated with [Claude Code](https://claude.com/claude-code)

hckrman101 (Migrated from github.com) reviewed

2026-11-11 02:16:33 +00:00

hckrman101 (Migrated from github.com) left a comment

Re-review at da8a8a2. Since our review the only new commit is the merge of main. The PR's own diff (vk_replay.cpp and the two test stubs) is byte-for-byte what I reviewed at 23c47ad, so the correctness part of that review still holds.

Thanks for the Windows A/B. It's useful: holding the batch costs ~12 fps on the 4060 Ti too, and the idle flush gets batching back to the unbatched rate with no crashes. Windows' default is unchanged.

It also confirms why the RADV APU run is still open. When the GPU keeps up, the idle flush makes batched submits behave about like RX_VK_BATCH=0. That's fine on NVIDIA and on mrdev023's GPU-bound 9070 XT, where unbatched was already as fast or faster. But batching is on for RADV because of issue 413's 8060S, where unbatched was the slow path (each submit paid for the userptr imports again, ~70 ms of queue submit a frame). The Deck is Tier 1 and gets this by default, and neither A/B covers that case. So my request stands: one run on a Deck or an RADV APU (8060S/780M) with RX_VK_BATCH_IDLE=0, this PR, and RX_VK_BATCH=0, comparing fps, flushes a frame and queue-submit ms from the stats line. If the PR comes out close to RX_VK_BATCH=0 there, a minimum batch size or a minimum hold time before the idle flush would keep most of the win without paying that cost.

Still open from before, both nits: the idle flushes are counted as FR_OTHER (inline thread above; an FR_IDLE column would make the APU run easy to read), and the description's late-page wait (~1.4 -> 0.64 ms) doesn't match #1993's or mrdev023's per-frame numbers (~7.9 -> ~4.6 ms).

CI: the failures (test_make_game_ngx.py on Ubuntu and Windows 1/2, test_pipeline_state_portable.py on Windows 2/2, actionlint's SC2012 in linux-appimage.yml) fail the same way on main's run at 21fa1365, so they're not from this PR. The Vulkan build, AppImage and builder pass. Mergeable, no conflicts.

Re-review at `da8a8a2`. Since our review the only new commit is the merge of main. The PR's own diff (`vk_replay.cpp` and the two test stubs) is byte-for-byte what I reviewed at `23c47ad`, so the correctness part of that review still holds. Thanks for the Windows A/B. It's useful: holding the batch costs ~12 fps on the 4060 Ti too, and the idle flush gets batching back to the unbatched rate with no crashes. Windows' default is unchanged. It also confirms why the RADV APU run is still open. When the GPU keeps up, the idle flush makes batched submits behave about like `RX_VK_BATCH=0`. That's fine on NVIDIA and on mrdev023's GPU-bound 9070 XT, where unbatched was already as fast or faster. But batching is on for RADV because of issue 413's 8060S, where unbatched was the slow path (each submit paid for the userptr imports again, ~70 ms of queue submit a frame). The Deck is Tier 1 and gets this by default, and neither A/B covers that case. So my request stands: one run on a Deck or an RADV APU (8060S/780M) with `RX_VK_BATCH_IDLE=0`, this PR, and `RX_VK_BATCH=0`, comparing fps, flushes a frame and queue-submit ms from the stats line. If the PR comes out close to `RX_VK_BATCH=0` there, a minimum batch size or a minimum hold time before the idle flush would keep most of the win without paying that cost. Still open from before, both nits: the idle flushes are counted as FR_OTHER (inline thread above; an FR_IDLE column would make the APU run easy to read), and the description's late-page wait (~1.4 -> 0.64 ms) doesn't match #1993's or mrdev023's per-frame numbers (~7.9 -> ~4.6 ms). CI: the failures (`test_make_game_ngx.py` on Ubuntu and Windows 1/2, `test_pipeline_state_portable.py` on Windows 2/2, actionlint's SC2012 in `linux-appimage.yml`) fail the same way on main's run at `21fa1365`, so they're not from this PR. The Vulkan build, AppImage and builder pass. Mergeable, no conflicts.

mrdev023 commented

2026-11-11 10:40:18 +00:00

(Migrated from github.com)

the description's late-page wait (~1.4 -> 0.64 ms) doesn't match #1993's or mrdev023's per-frame numbers (~7.9 -> ~4.6 ms)

Those came from my first write-up in #1993, and they were my misreading. The log line a capture taken (ms): ... GPU wait for late pages X (100 of 600 takes) averages over all 600 takes. There are 6 takes a frame and the wait falls in one of them, so the per-frame wait is 6 × X: ~8.4 -> ~3.8 ms in #1993, ~7.9 -> ~4.6 ms in my rerun of this branch. I've corrected both. The relative gain is unchanged.

The catch is that both runs are one machine (9070 XT + 9950X3D at 5120x1440 with FSR 4), and that machine is GPU-bound.

I don't think it is, after profiling it today (main abd5e5f + this PR, Kokiri Forest):

Render scale barely matters: FSR 4 Native AA -> Performance (50%) -> Ultra Performance (33%) gave 46 -> 50 fps at best. GPU time stays at ~11-14 ms a frame for a ~19-21 ms frame.
HOS_SAMPLE_ALL + scripts/crit.py: the main thread runs ~18% of the time. It waits ~25% in take_capture's GPU wait and ~35% in event_wait for the Presentation thread, which does the replay. The replay costs ~14-20 ms of CPU a frame (a frame's replay time: draw 6-10, descriptors 2-4, commands 2.5-4).
RX_VK_PASS_TIMING=1: the GPU's work is ~175 passes (~39 us each) and ~143 dispatches (~21 us each), with ~409 full barriers a frame. That looks latency-bound (many small serialized pieces), not throughput-bound, which is why the resolution doesn't change it.

So on this box the frame is the CPU replay plus the mid-frame wait, and the GPU mostly waits for work. That also explains why RX_VK_BATCH=0 was faster than the held batch: the GPU sat idle while the batch was held.

@hckrman101

> the description's late-page wait (~1.4 -> 0.64 ms) doesn't match #1993's or mrdev023's per-frame numbers (~7.9 -> ~4.6 ms) Those came from my first write-up in #1993, and they were my misreading. The log line `a capture taken (ms): ... GPU wait for late pages X (100 of 600 takes)` averages over all 600 takes. There are 6 takes a frame and the wait falls in one of them, so the per-frame wait is 6 × X: ~8.4 -> ~3.8 ms in #1993, ~7.9 -> ~4.6 ms in my rerun of this branch. I've corrected both. The relative gain is unchanged. > The catch is that both runs are one machine (9070 XT + 9950X3D at 5120x1440 with FSR 4), and that machine is GPU-bound. I don't think it is, after profiling it today (`main` abd5e5f + this PR, Kokiri Forest): - Render scale barely matters: FSR 4 Native AA -> Performance (50%) -> Ultra Performance (33%) gave 46 -> 50 fps at best. GPU time stays at ~11-14 ms a frame for a ~19-21 ms frame. - `HOS_SAMPLE_ALL` + `scripts/crit.py`: the main thread runs ~18% of the time. It waits ~25% in `take_capture`'s GPU wait and ~35% in `event_wait` for the Presentation thread, which does the replay. The replay costs ~14-20 ms of CPU a frame (`a frame's replay time`: draw 6-10, descriptors 2-4, commands 2.5-4). - `RX_VK_PASS_TIMING=1`: the GPU's work is ~175 passes (~39 us each) and ~143 dispatches (~21 us each), with ~409 full barriers a frame. That looks latency-bound (many small serialized pieces), not throughput-bound, which is why the resolution doesn't change it. So on this box the frame is the CPU replay plus the mid-frame wait, and the GPU mostly waits for work. That also explains why `RX_VK_BATCH=0` was faster than the held batch: the GPU sat idle while the batch was held. @hckrman101