Batched submits: send the held batch as soon as the GPU is idle (+40% fps in Kokiri Forest on RX 9070 XT / RADV, no flashes) #1993

Closed

opened 2026-11-10 22:59:30 +00:00 by mrdev023 · 3 comments

mrdev023 commented

2026-11-10 22:59:30 +00:00

(Migrated from github.com)

The idea

With batched submits (RX_VK_BATCH, on by default with RADV), the replay's command buffers are held until something needs the GPU: a wait, a present, a readback or the 32-buffer cap. When the GPU has already finished everything it was sent, it sits idle while the batch waits, and only gets the work when the game starts waiting for it. With serial replay that is every frame: the game waits for the GPU mid-frame (take_capture's late-page wait, RX_VK_CAPTURE_WAIT=2), so the GPU starts cold right when it is waited for.

Proposal: when a submit is added to the batch and the GPU has completed every batch sent so far (batch_tl >= batch_flushed), send the batch at once. While the GPU is busy, submits keep being batched as today. #1276 and #1905 flush when someone waits; this flushes before anyone has to, so the GPU is already working when the wait comes.

// gpu/vk/vk_replay.cpp, submit_recording, after the RX_VK_BATCH_MAX check
if (!pend.cbs.empty()) {
    uint64_t done = 0;
    vkGetSemaphoreCounterValue(dev.device, batch_tl, &done);
    if (done >= batch_flushed) flush_reason = FR_OTHER, queue_flush_locked();
}

The patch as tested (opt-in with RX_VK_BATCH_IDLE=1; applies on current main)

diff --git a/gpu/vk/vk_replay.cpp b/gpu/vk/vk_replay.cpp
index 232203c..01ec052 100644
--- a/gpu/vk/vk_replay.cpp
+++ b/gpu/vk/vk_replay.cpp
@@ -352,6 +352,14 @@ void submit_recording(Recording& r, bool signal_value = true, uint64_t signal_at
         // (RX_VK_BATCH_MAX=<command buffers>: held at most this many, default 32)
         static const size_t cap = getenv("RX_VK_BATCH_MAX") ? (size_t)std::max(1, atoi(getenv("RX_VK_BATCH_MAX"))) : 32;
         if (pend.cbs.size() >= cap) flush_reason = FR_CAP, queue_flush_locked();
+        // (RX_VK_BATCH_IDLE=1: sent at once when the GPU has done every batch sent so far: held submits wait for the next
+        // wait or present, and the GPU sat idle until the game waited for it, serial replay's mid-frame wait above all)
+        static const bool idle_flush = getenv("RX_VK_BATCH_IDLE") && atoi(getenv("RX_VK_BATCH_IDLE"));
+        if (idle_flush && !pend.cbs.empty()) {
+            uint64_t done = 0;
+            vkGetSemaphoreCounterValue(dev.device, batch_tl, &done);
+            if (done >= batch_flushed) flush_reason = FR_OTHER, queue_flush_locked();
+        }
         return;
     }
     PROF(PROF_QSUBMIT);

Measured on RX 9070 XT (RADV), Ryzen 9 9950X3D, NixOS, main 51de681 + be8b512 (#1858, the link fix), AMD FSR 4 at 3840x1080 -> 5120x1440. Same Kokiri Forest area each run, default settings (RX_VK_CAPTURE_WAIT=2), GPU power level auto. Flashes counted with RX_VK_FRAMELOG + scripts/framelog.py.

build	fps in play	framelog events of 5+ blocks
as is	33-41	0
send the batch when the GPU is idle	46-61	0 / 5230 frames

Flushes go from ~5.7 a frame (~3.6 of them triggered by a wait) to ~10 a frame of 3.7 command buffers each, so batching still does its job.
take_capture's late-page wait (once a frame) drops from ~8.4 ms to ~3.8 ms a frame; GPU time ~10.4 ms a frame.
For comparison, RX_VK_BATCH=0 gave 44-55 fps, against 41-48 for the default, both with the GPU's power level forced to high instead of auto:
```
echo high | sudo tee /sys/class/drm/card1/device/power_dpm_force_performance_level
echo auto | sudo tee /sys/class/drm/card1/device/power_dpm_force_performance_level   # back to the default
```
card1 is my RX 9070 XT on this machine (card0 is the Ryzen's integrated GPU); the number may differ elsewhere: cat /sys/class/drm/card*/device/device gives each card's PCI device ID (0x7550 here for the 9070 XT).
(high alone took the default from 33-41 to 41-48 fps: the serial path's bursty load keeps the GPU at low clocks under auto.)

Anything else

Not tested: other GPUs or drivers (batching is only on with RADV by default), other areas, Windows.

Claude Code (Anthropic's AI coding assistant) did the analysis and wrote the patch; I checked the idea and ran the tests on my machine.

### The idea With batched submits (`RX_VK_BATCH`, on by default with RADV), the replay's command buffers are held until something needs the GPU: a wait, a present, a readback or the 32-buffer cap. When the GPU has already finished everything it was sent, it sits idle while the batch waits, and only gets the work when the game starts waiting for it. With serial replay that is every frame: the game waits for the GPU mid-frame (`take_capture`'s late-page wait, `RX_VK_CAPTURE_WAIT=2`), so the GPU starts cold right when it is waited for. Proposal: when a submit is added to the batch and the GPU has completed every batch sent so far (`batch_tl` >= `batch_flushed`), send the batch at once. While the GPU is busy, submits keep being batched as today. #1276 and #1905 flush when someone waits; this flushes before anyone has to, so the GPU is already working when the wait comes. ```cpp // gpu/vk/vk_replay.cpp, submit_recording, after the RX_VK_BATCH_MAX check if (!pend.cbs.empty()) { uint64_t done = 0; vkGetSemaphoreCounterValue(dev.device, batch_tl, &done); if (done >= batch_flushed) flush_reason = FR_OTHER, queue_flush_locked(); } ``` <details> <summary>The patch as tested (opt-in with <code>RX_VK_BATCH_IDLE=1</code>; applies on current <code>main</code>)</summary> ```diff diff --git a/gpu/vk/vk_replay.cpp b/gpu/vk/vk_replay.cpp index 232203c..01ec052 100644 --- a/gpu/vk/vk_replay.cpp +++ b/gpu/vk/vk_replay.cpp @@ -352,6 +352,14 @@ void submit_recording(Recording& r, bool signal_value = true, uint64_t signal_at // (RX_VK_BATCH_MAX=<command buffers>: held at most this many, default 32) static const size_t cap = getenv("RX_VK_BATCH_MAX") ? (size_t)std::max(1, atoi(getenv("RX_VK_BATCH_MAX"))) : 32; if (pend.cbs.size() >= cap) flush_reason = FR_CAP, queue_flush_locked(); + // (RX_VK_BATCH_IDLE=1: sent at once when the GPU has done every batch sent so far: held submits wait for the next + // wait or present, and the GPU sat idle until the game waited for it, serial replay's mid-frame wait above all) + static const bool idle_flush = getenv("RX_VK_BATCH_IDLE") && atoi(getenv("RX_VK_BATCH_IDLE")); + if (idle_flush && !pend.cbs.empty()) { + uint64_t done = 0; + vkGetSemaphoreCounterValue(dev.device, batch_tl, &done); + if (done >= batch_flushed) flush_reason = FR_OTHER, queue_flush_locked(); + } return; } PROF(PROF_QSUBMIT); ``` </details> Measured on RX 9070 XT (RADV), Ryzen 9 9950X3D, NixOS, `main` 51de681 + `be8b512` (#1858, the link fix), AMD FSR 4 at 3840x1080 -> 5120x1440. Same Kokiri Forest area each run, default settings (`RX_VK_CAPTURE_WAIT=2`), GPU power level `auto`. Flashes counted with `RX_VK_FRAMELOG` + `scripts/framelog.py`. | build | fps in play | framelog events of 5+ blocks | |---|---|---| | as is | 33-41 | 0 | | send the batch when the GPU is idle | 46-61 | 0 / 5230 frames | - Flushes go from ~5.7 a frame (~3.6 of them triggered by a wait) to ~10 a frame of 3.7 command buffers each, so batching still does its job. - `take_capture`'s late-page wait (once a frame) drops from ~8.4 ms to ~3.8 ms a frame; GPU time ~10.4 ms a frame. - For comparison, `RX_VK_BATCH=0` gave 44-55 fps, against 41-48 for the default, both with the GPU's power level forced to `high` instead of `auto`: ```sh echo high | sudo tee /sys/class/drm/card1/device/power_dpm_force_performance_level echo auto | sudo tee /sys/class/drm/card1/device/power_dpm_force_performance_level # back to the default ``` `card1` is my RX 9070 XT on this machine (`card0` is the Ryzen's integrated GPU); the number may differ elsewhere: `cat /sys/class/drm/card*/device/device` gives each card's PCI device ID (0x7550 here for the 9070 XT). (`high` alone took the default from 33-41 to 41-48 fps: the serial path's bursty load keeps the GPU at low clocks under `auto`.) ### Anything else Not tested: other GPUs or drivers (batching is only on with RADV by default), other areas, Windows. Claude Code (Anthropic's AI coding assistant) did the analysis and wrote the patch; I checked the idea and ran the tests on my machine.

🎉 1

mrdev023 commented

2026-11-10 23:12:04 +00:00

(Migrated from github.com)

I not purpose as PR because I need more feedback. You can try it without any patch

echo high | sudo tee /sys/class/drm/cardX/device/power_dpm_force_performance_level

cardX: You're card number

If it improve the performance, my idea works for you.

echo auto | sudo tee /sys/class/drm/card1/device/power_dpm_force_performance_level

To restore the default value

I not purpose as PR because I need more feedback. You can try it without any patch ```bash echo high | sudo tee /sys/class/drm/cardX/device/power_dpm_force_performance_level ``` > cardX: You're card number If it improve the performance, my idea works for you. ```bash echo auto | sudo tee /sys/class/drm/card1/device/power_dpm_force_performance_level ``` > To restore the default value

addisonbair commented

2026-11-11 16:22:19 +00:00

(Migrated from github.com)

A data point from the other end: on a Steam Deck LCD (Van Gogh, RADV, GPU at 1200 MHz), the idle send costs CPU time. This is current main (ebbba2e with #1375 reverted, plus RX_VK_NO_READBACK_WAIT=1, see #1000), in a deterministic headless run of the standing Kokiri scene (~2,400 draws), alternating A/B:

	fps	replay `submit`	`queue submit`
default (idle send on)	18.8 / 18.6	41.5 / 44.1 ms	11.2 / 12.0 ms
`RX_VK_BATCH_IDLE=0`	20.0 / 20.7	39.0 / 36.8 ms	7.1 / 6.6 ms

The frame logs 6 submits either way. Each vkQueueSubmit already costs the Deck ~1.1 ms on this setup, so I'd guess that sending batches early adds submission work there that the RX 9070 XT's fast CPU doesn't notice. v2.0.14 on the same Deck has queue submit at 6.8-7.6 ms. Maybe the idle send could be held back where submits are this expensive, or measured per device. Happy to rerun a candidate on the Deck.

A data point from the other end: on a **Steam Deck LCD** (Van Gogh, RADV, GPU at 1200 MHz), the idle send costs CPU time. This is current main (ebbba2e with #1375 reverted, plus `RX_VK_NO_READBACK_WAIT=1`, see #1000), in a deterministic headless run of the standing Kokiri scene (~2,400 draws), alternating A/B: | | fps | replay `submit` | `queue submit` | |---|---|---|---| | default (idle send on) | 18.8 / 18.6 | 41.5 / 44.1 ms | 11.2 / 12.0 ms | | `RX_VK_BATCH_IDLE=0` | 20.0 / 20.7 | 39.0 / 36.8 ms | 7.1 / 6.6 ms | The frame logs 6 submits either way. Each `vkQueueSubmit` already costs the Deck ~1.1 ms on this setup, so I'd guess that sending batches early adds submission work there that the RX 9070 XT's fast CPU doesn't notice. v2.0.14 on the same Deck has queue submit at 6.8-7.6 ms. Maybe the idle send could be held back where submits are this expensive, or measured per device. Happy to rerun a candidate on the Deck.

NotRexed commented

2026-11-11 21:24:30 +00:00

(Migrated from github.com)

Thanks for the report. Reports of the same problem are now collected in #995, which lists this one with your details and links back here, so one fix closes them together and the open list stays readable. Follow #995 for the fix, and add new details there (or here). If this turns out to be a separate problem it will be reopened.