WIP: perf(vk): mesh fallback: barriers on its own buffers, not four full barriers a draw #1636

Closed

rakefire-impala wants to merge 3 commits from perf/mesh-fallback-targeted-barriers into main

rakefire-impala commented

2026-11-10 11:45:39 +00:00

(Migrated from github.com)

Builds on #1509 and #1511 (it changes the barrier lines they add), so this PR also shows their commits; merge those first. Draft: it passes the tests and merges cleanly, but it isn't played yet. I'll post the before/after numbers here.

Problem

fallback_record recorded a full barrier (all stages to all stages) at four points of every fallback draw: before the shared records are reset, after the reset, between the task and mesh dispatches, and before the draw. Each one waits for every stage of all the work before it, including the fragment work of the pass the fallback has just ended.

In Hyrule Field the game asks for far more fallback work than in other areas. For example, the far-terrain programs 739acd678ae08655 and a6a688bae3c9b80c start 1,504 task workgroups, about one triangle per mesh workgroup. On an RX 5500M the frame there takes 36 ms of GPU time, and a pass timing (RX_VK_PASS_TIMING=2) showed 8 ms of GPU idle between spans in an earlier capture.

Change

fallback_barrier records one memory barrier from the stages that last touched the fallback's buffers to the stages that touch them next:

Point	From	To
Before the reset	the last fallback draw's indirect read, vertex stage and dispatches	the fills and this draw's dispatches
After the reset	the fills	the dispatches' atomics and the mesh dispatch's indirect read
Task → mesh	the task dispatch	the mesh dispatch and its indirect read
Before the draw	the mesh dispatch	every later stage (the records, the draw's command and anything the game's mesh program wrote)

What isn't relied on any more, and why that's safe:
- The game's own memory: what the fallback reads of it is ordered by the game's barriers, as on the console.
- The converted indirect arguments: they order themselves (mesh_indirect_convert ends with transfer → indirect read).
- The render targets: the pass after the fallback starts with pass_start_barrier, as before.
RX_VK_FALLBACK_FULL_BARRIERS=1 records the full barriers again, for an A/B.
Only GPUs without mesh shaders run fallback_record. Nothing changes for any other GPU, and no shader changes, so there's no re-translation.

Tests

tests/test_mesh_fallback_barriers.py (new): checks that no full barrier is left in fallback_record, that there are four targeted ones, the stages each one covers, the A/B switch, and that the converted arguments order themselves.
tests/test_mesh_fallback.py: its two reset checks accept the targeted barrier as well as a full one.
Compile check of vk_draw.cpp (llvm-mingw); merges cleanly with main and with #1505-#1514 and #1581.

Tested on


Tested on	RX 5500M (RDNA1, 4 GB VRAM, 16 GB RAM), Windows 11, AMD driver
Result	Pending: the same Hyrule Field spot with and without `RX_VK_FALLBACK_FULL_BARRIERS=1`: fps, GPU time, and any flicker or missing geometry

On a GPU without mesh shaders, each fallback draw recorded four full barriers. A dense Hyrule Field view has 116 fallback draws, so that's 464 full barriers a frame, and on an RX 5500M the GPU spends much of the frame idle between the fallback's many small dispatches. This replaces them with barriers that cover only the fallback's own buffers. **Builds on #1509 and #1511** (it changes the barrier lines they add), so this PR also shows their commits; merge those first. **Draft:** it passes the tests and merges cleanly, but it isn't played yet. I'll post the before/after numbers here. ## Problem `fallback_record` recorded a full barrier (all stages to all stages) at four points of every fallback draw: before the shared records are reset, after the reset, between the task and mesh dispatches, and before the draw. Each one waits for every stage of all the work before it, including the fragment work of the pass the fallback has just ended. In Hyrule Field the game asks for far more fallback work than in other areas. For example, the far-terrain programs `739acd678ae08655` and `a6a688bae3c9b80c` start 1,504 task workgroups, about one triangle per mesh workgroup. On an RX 5500M the frame there takes 36 ms of GPU time, and a pass timing (`RX_VK_PASS_TIMING=2`) showed 8 ms of GPU idle between spans in an earlier capture. ## Change - **`fallback_barrier`** records one memory barrier from the stages that last touched the fallback's buffers to the stages that touch them next: | Point | From | To | |---|---|---| | Before the reset | the last fallback draw's indirect read, vertex stage and dispatches | the fills and this draw's dispatches | | After the reset | the fills | the dispatches' atomics and the mesh dispatch's indirect read | | Task → mesh | the task dispatch | the mesh dispatch and its indirect read | | Before the draw | the mesh dispatch | every later stage (the records, the draw's command and anything the game's mesh program wrote) | - **What isn't relied on any more,** and why that's safe: - **The game's own memory:** what the fallback reads of it is ordered by the game's barriers, as on the console. - **The converted indirect arguments:** they order themselves (`mesh_indirect_convert` ends with transfer → indirect read). - **The render targets:** the pass after the fallback starts with `pass_start_barrier`, as before. - **`RX_VK_FALLBACK_FULL_BARRIERS=1`** records the full barriers again, for an A/B. - Only GPUs without mesh shaders run `fallback_record`. Nothing changes for any other GPU, and no shader changes, so there's no re-translation. ## Tests - **`tests/test_mesh_fallback_barriers.py` (new):** checks that no full barrier is left in `fallback_record`, that there are four targeted ones, the stages each one covers, the A/B switch, and that the converted arguments order themselves. - **`tests/test_mesh_fallback.py`:** its two reset checks accept the targeted barrier as well as a full one. - Compile check of `vk_draw.cpp` (llvm-mingw); merges cleanly with main and with #1505-#1514 and #1581. ## Tested on | | | |---|---| | Tested on | RX 5500M (RDNA1, 4 GB VRAM, 16 GB RAM), Windows 11, AMD driver | | Result | *Pending:* the same Hyrule Field spot with and without `RX_VK_FALLBACK_FULL_BARRIERS=1`: fps, GPU time, and any flicker or missing geometry |

rakefire-impala commented

2026-11-11 00:26:13 +00:00

(Migrated from github.com)

Tested on the RX 5500M (RDNA1, Windows 11), and I'm closing this. With the targeted barriers the picture broke in Hyrule Field: flicker, UI drawn in the wrong places, stretched geometry, and the tree-shadow flicker came back. RX_VK_FALLBACK_FULL_BARRIERS=1 on the same build fixed all of it, so the premise above doesn't hold: some of what the fallback's compute stages read is only ordered by those full barriers. Cutting the barrier count would need a different approach, such as letting consecutive fallback draws share them.

Tested on the RX 5500M (RDNA1, Windows 11), and I'm closing this. With the targeted barriers the picture broke in Hyrule Field: flicker, UI drawn in the wrong places, stretched geometry, and the tree-shadow flicker came back. `RX_VK_FALLBACK_FULL_BARRIERS=1` on the same build fixed all of it, so the premise above doesn't hold: some of what the fallback's compute stages read is only ordered by those full barriers. Cutting the barrier count would need a different approach, such as letting consecutive fallback draws share them.

Commenting is not possible because the repository is archived.

No reviewers