perf(vk): back-to-back passes and dispatches ordered by the game's barriers (NVIDIA by default) #2559

Open

NotRexed wants to merge 1 commit from perf/game-barrier-order into main

NotRexed commented

2026-11-11 23:15:07 +00:00

(Migrated from github.com)

Why. Every draw pass begin and every compute dispatch records a full barrier, which makes the GPU drain everything before each of them. The game already records its own barrier (nvnCommandBufferBarrier, which the replay keeps as a full barrier) wherever a pass samples what an earlier one drew or a dispatch reads what another wrote. The extra barriers between back-to-back work are pure GPU idle time: elsewhere it measured 2.3 ms of a 9.7 ms GPU frame in Kokiri Forest, and about a tenth off the GPU's frame time on an RTX 5090.

What (RX_VK_GAME_ORDER):

A draw pass right after another, with nothing recorded between, waits in its fragment stages only. This is the existing RX_VK_PASS_BARRIER=game path, now on with the switch. RX_VK_PASS_BARRIER=game / full still decide that part on their own.
A direct dispatch right after another gets no barrier. "Right after" means the same command buffer with no GPU work recorded between, checked through the work_epoch count taken right after the previous dispatch's command.
Still full barriers: a switch between draws and compute, the replay's own copies, clears, fills and uniform updates (all count as work), indirect dispatches (their grid may come from the dispatch before), a parallel replay, and the game's barriers.
Defaults: on for NVIDIA (vendor 0x10de), since the game's barriers were placed for an NVIDIA GPU. Off on other vendors, where it hasn't run: RX_VK_GAME_ORDER=1 to try it there, =0 to turn it off everywhere.
Logging: the log states the choice once, and counts the dispatches left without a barrier every 50,000.

Checks

tests/test_game_order.py: the rules as written.
The barrier-related tests pass (mesh_fallback, dump_readback, present_aspect, upscale_fallback, vulkan_clear_mip).
vk_texture.cpp compiles. vk_draw.cpp needs Boost, which this machine lacks, so CI compiles it.

Not yet verified in-game: GPU frame time A/B on this NVIDIA card plus the flash check (every-frame shots).

🤖 Generated with Claude Code

**Why.** Every draw pass begin and every compute dispatch records a full barrier, which makes the GPU drain everything before each of them. The game already records its own barrier (`nvnCommandBufferBarrier`, which the replay keeps as a full barrier) wherever a pass samples what an earlier one drew or a dispatch reads what another wrote. The extra barriers between back-to-back work are pure GPU idle time: elsewhere it measured 2.3 ms of a 9.7 ms GPU frame in Kokiri Forest, and about a tenth off the GPU's frame time on an RTX 5090. **What** (`RX_VK_GAME_ORDER`): - **A draw pass right after another**, with nothing recorded between, waits in its fragment stages only. This is the existing `RX_VK_PASS_BARRIER=game` path, now on with the switch. `RX_VK_PASS_BARRIER=game` / `full` still decide that part on their own. - **A direct dispatch right after another** gets no barrier. "Right after" means the same command buffer with no GPU work recorded between, checked through the `work_epoch` count taken right after the previous dispatch's command. - **Still full barriers:** a switch between draws and compute, the replay's own copies, clears, fills and uniform updates (all count as work), indirect dispatches (their grid may come from the dispatch before), a parallel replay, and the game's barriers. - **Defaults:** on for NVIDIA (vendor 0x10de), since the game's barriers were placed for an NVIDIA GPU. Off on other vendors, where it hasn't run: `RX_VK_GAME_ORDER=1` to try it there, `=0` to turn it off everywhere. - **Logging:** the log states the choice once, and counts the dispatches left without a barrier every 50,000. **Checks** - `tests/test_game_order.py`: the rules as written. - The barrier-related tests pass (`mesh_fallback`, `dump_readback`, `present_aspect`, `upscale_fallback`, `vulkan_clear_mip`). - `vk_texture.cpp` compiles. `vk_draw.cpp` needs Boost, which this machine lacks, so CI compiles it. **Not yet verified in-game:** GPU frame time A/B on this NVIDIA card plus the flash check (every-frame shots). 🤖 Generated with [Claude Code](https://claude.com/claude-code)

github-sync changed title from ~~WIP: perf(vk): back-to-back passes and dispatches ordered by the game's barriers (NVIDIA by default)~~ to perf(vk): back-to-back passes and dispatches ordered by the game's barriers (NVIDIA by default)

2026-11-12 14:42:57 +00:00

github-sync commented

2026-11-12 14:42:58 +00:00

Owner

@PedroFlaminio commented on GitHub · 2026-11-12 14:33 UTC · original

Tested in the game on NVIDIA (on by default there): no change in frame rate here; this machine is CPU-bound in this scene.

Hardware: RTX 3060 12 GB, NVIDIA 615.71.09, Manjaro (kernel 6.12), Core Ultra 7 265KF (8 P-cores + 12 E-cores), Wayland. Built on main 51da0464: each PR merged onto it, only hookshot_vk.so built (container build, RX_MARCH=x86-64-v3 RX_RELEASE=1) and swapped into the same game folder; main's hookshot_vk.so built the same way as the baseline.
Settings: 3440x1440 monitor, picture 2576x1080 (Aspect ratio: monitor's), frame interpolation 2x, Reflex on, V-Sync off, no frame rate cap; otherwise unchanged between runs.
Scene: Hyrule Field from the same save, standing, no input, 3.5 to 7 minutes a run; only the 5 s windows after loading (3,370-3,760 draws in every run) are counted. One run each, in the order main, #2633, #2644, #2559.

renderer	fps median (min-max)	replay submit	draw	descriptors	memory sync
main	42.7 (41.4-43.4)	21.4 ms	9.4	3.5	9.4
#2633	42.8 (40.6-43.8)	19.9 ms	7.6	2.7	9.4
#2644	42.6 (40.2-43.5)	21.2 ms	8.9	2.9	9.5
#2559	43.0 (42.3-43.8)	20.1 ms	7.7	2.7	9.6

(replay columns: medians of the last 20 a frame's replay time lines; GPU-bound wait was 0.0 ms in all runs)

The log confirms it was on: passes and dispatches right after others ordered by the game's barriers: on, and dispatches left without a barrier were counted (over 50,000).
GPU-bound wait was 0.0 ms in every run, so a shorter GPU frame would not show in the frame rate here; I did not measure GPU time.
No device lost, and no visual problem seen while playing; I did not check for flashes frame by frame (no every-frame screenshots).

> [@PedroFlaminio](https://github.com/PedroFlaminio) commented on GitHub · 2026-11-12 14:33 UTC · [original](https://github.com/NotRexed/Hookshot/pull/2559#issuecomment-6826489985) Tested in the game on NVIDIA (on by default there): no change in frame rate here; this machine is CPU-bound in this scene. Hardware: RTX 3060 12 GB, NVIDIA 615.71.09, Manjaro (kernel 6.12), Core Ultra 7 265KF (8 P-cores + 12 E-cores), Wayland. Built on main `51da0464`: each PR merged onto it, only `hookshot_vk.so` built (container build, `RX_MARCH=x86-64-v3 RX_RELEASE=1`) and swapped into the same game folder; main's `hookshot_vk.so` built the same way as the baseline. Settings: 3440x1440 monitor, picture 2576x1080 (Aspect ratio: monitor's), frame interpolation 2x, Reflex on, V-Sync off, no frame rate cap; otherwise unchanged between runs. Scene: Hyrule Field from the same save, standing, no input, 3.5 to 7 minutes a run; only the 5 s windows after loading (3,370-3,760 draws in every run) are counted. One run each, in the order main, #2633, #2644, #2559. | renderer | fps median (min-max) | replay submit | draw | descriptors | memory sync | |---|---|---|---|---|---| | main | 42.7 (41.4-43.4) | 21.4 ms | 9.4 | 3.5 | 9.4 | | #2633 | 42.8 (40.6-43.8) | 19.9 ms | 7.6 | 2.7 | 9.4 | | #2644 | 42.6 (40.2-43.5) | 21.2 ms | 8.9 | 2.9 | 9.5 | | #2559 | 43.0 (42.3-43.8) | 20.1 ms | 7.7 | 2.7 | 9.6 | (replay columns: medians of the last 20 `a frame's replay time` lines; `GPU-bound wait` was 0.0 ms in all runs) - The log confirms it was on: `passes and dispatches right after others ordered by the game's barriers: on`, and dispatches left without a barrier were counted (over 50,000). - `GPU-bound wait` was 0.0 ms in every run, so a shorter GPU frame would not show in the frame rate here; I did not measure GPU time. - No device lost, and no visual problem seen while playing; I did not check for flashes frame by frame (no every-frame screenshots).

github-sync referenced this pull request

2026-11-12 14:43:41 +00:00

perf(vk): with frame generation, the replay no longer waits behind each present, and a capture waits for less of the frame before #2633

github-sync referenced this pull request

2026-11-12 14:43:47 +00:00

perf(vk): cheaper replay: uniform-only descriptor rebuilds and flat program picks #2644

github-sync referenced this pull request

2026-11-12 16:54:39 +00:00

[Tracking] Low FPS with low GPU usage (CPU-bound), FPS drops in Castle Town / Hyrule Field, shader stutter #995

github-sync referenced this pull request

2026-11-12 19:01:09 +00:00

[Tracking] Low FPS with low GPU usage (CPU-bound), FPS drops in Castle Town / Hyrule Field, shader stutter #995