Idea: cut the per-draw cost of the compute fallback (the Market at 2,300 draws: GPU 22.7 ms plus 19 ms of replay submit on an RX 5600 XT) #549

New issue

Open

opened 2026-11-14 19:15:26 +00:00 by waryxxful · 1 comment

waryxxful commented

2026-11-14 19:15:26 +00:00

The idea

On a card without mesh shaders, once the mirror retry of #288 is in, what is left under 30 fps is not a memory mode: it is the cost of each draw. I am looking for someone who knows the compute fallback or the replay's submit path to say which part of that cost can go, and for the same numbers from other cards.

The numbers. RX 5600 XT 6 GB (RDNA1, Adrenalin 26.10.1), Ryzen 5 5600X, Windows 11, Render resolution 50% of 1920x1080. Main 9c9d69ed with ea6dc5f5 (the mirror retry) cherry-picked, the same renderer in the four runs. The Market, 25 s while Link walks; the walk ends in a different place each run, which is why the draws differ:

Draws a frame	fps	GPU a frame	Replay submit a frame
1,165	36.1	16.3 ms	14.3 ms
1,221	36.8	16.2 ms	14.4 ms
1,623	28.2	19.3 ms	18.3 ms
2,290	20.8	22.7 ms	19.1 ms

At 2,290 draws the frame is 48 ms, more than the GPU's 22.7 ms and the submit's 19.1 ms added up. The submit there is draw 11.5 ms, commands 3.6 ms, uploads 2.2 ms, memory_sync 1.7 ms, dispatch 1.3 ms. Kokiri Forest on the same renderer, 730 draws: 46-48 fps, GPU 14.5-14.8 ms, submit 9 ms.

Where the GPU's frame goes (RX_VK_PASS_TIMING=2, one block at the Desert Colossus, 842 draws; the shares matter, the profile lengthens the frame):

Kind	ms	Spans a frame
passes	5.89	238 for 842 draws
the fallback's dispatches	3.49	115
write-back	2.84 (60 MB)	7
barriers	1.82	789
FSR 3.1	1.02	7
mirror sync	0.74	57
between spans, not named	7.99

What I tried that did not move the frame, so nobody repeats it:

Not ending the pass for each mesh draw: a prototype that holds the pass and records the fallback's dispatches, barriers and header fills ahead of it. It works (9 of 10 fallback draws get their dispatches ahead of the pass, no device lost), and in the profile the same block falls to 101 pass spans and 273 barriers, 21.6 ms from 25.0. Without the profile the GPU's frame is the same within the noise between runs: 18.1 / 17.1 ms against 18.7 / 18.0 at the Desert Colossus, 14.8 / 14.4 against 14.8 / 14.5 in Kokiri Forest. It also drops primitives for a frame or two when a list grows (56-60 draws in a five-minute run). Not proposed.
#326 on top (the mirror sync in one dispatch), and narrower barriers around the fallback's dispatches: no gain in two runs each.
RX_VK_READBACK_LATE=newest: the long readback waits are gone (0 of 60 log lines against 4 to 19), the Market is the same, 27-28 fps at 1,400-1,600 draws.
RX_VK_ASYNC=1: worse everywhere, 13-25 fps in the six areas.

What does help on this card is RX_VK_WB_MAX_KB=1024: the write-back above falls from 60 to 16 MB a frame and the GPU's frame by 2.5 ms (#547 has the runs).

The questions:

The fallback runs one task dispatch and one mesh dispatch per mesh draw (115-146 a frame here, 26-30 us each on the GPU). Can draws of one program share a dispatch, or can a draw whose task shader emitted nothing the frame before skip its mesh dispatch?
The submit costs 8-12 us a draw on a 5600X, most of it in draw. #402 and #410 report the same wall on other hardware. Is there work in flight on it that a card like this one could test?
Why is the frame longer than GPU plus submit at 2,300 draws: is the game's own thread the third term in the Market?

Who can help without writing code: anyone on the compute fallback (the log says no mesh shaders (the compute fallback)). Add RX_VK_PASS_TIMING=2 to longshot_env.txt, stand in the Market for a minute and paste one vk: GPU pass timing block from longshot_log.txt here with the card and the driver. Two more cards would show whether the passes and the dispatches weigh the same on NVIDIA and on Polaris.

Anything else

The prototype (a switch, off by default, with its game-free test) is not pushed anywhere. I can push the branch to my fork if someone wants to take it further or measure it on another card.
The earlier state of this card, without the retry, is in my comment on #441.

### The idea On a card without mesh shaders, once the mirror retry of https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/288 is in, what is left under 30 fps is not a memory mode: it is the cost of each draw. I am looking for someone who knows the compute fallback or the replay's submit path to say which part of that cost can go, and for the same numbers from other cards. **The numbers.** RX 5600 XT 6 GB (RDNA1, Adrenalin 26.10.1), Ryzen 5 5600X, Windows 11, Render resolution 50% of 1920x1080. Main 9c9d69ed with ea6dc5f5 (the mirror retry) cherry-picked, the same renderer in the four runs. The Market, 25 s while Link walks; the walk ends in a different place each run, which is why the draws differ: | Draws a frame | fps | GPU a frame | Replay submit a frame | |---|---|---|---| | 1,165 | 36.1 | 16.3 ms | 14.3 ms | | 1,221 | 36.8 | 16.2 ms | 14.4 ms | | 1,623 | 28.2 | 19.3 ms | 18.3 ms | | 2,290 | 20.8 | 22.7 ms | 19.1 ms | At 2,290 draws the frame is 48 ms, more than the GPU's 22.7 ms and the submit's 19.1 ms added up. The submit there is `draw` 11.5 ms, `commands` 3.6 ms, `uploads` 2.2 ms, `memory_sync` 1.7 ms, `dispatch` 1.3 ms. Kokiri Forest on the same renderer, 730 draws: 46-48 fps, GPU 14.5-14.8 ms, submit 9 ms. **Where the GPU's frame goes** (`RX_VK_PASS_TIMING=2`, one block at the Desert Colossus, 842 draws; the shares matter, the profile lengthens the frame): | Kind | ms | Spans a frame | |---|---|---| | passes | 5.89 | 238 for 842 draws | | the fallback's dispatches | 3.49 | 115 | | write-back | 2.84 (60 MB) | 7 | | barriers | 1.82 | 789 | | FSR 3.1 | 1.02 | 7 | | mirror sync | 0.74 | 57 | | between spans, not named | 7.99 | | **What I tried that did not move the frame**, so nobody repeats it: - Not ending the pass for each mesh draw: a prototype that holds the pass and records the fallback's dispatches, barriers and header fills ahead of it. It works (9 of 10 fallback draws get their dispatches ahead of the pass, no device lost), and in the profile the same block falls to 101 pass spans and 273 barriers, 21.6 ms from 25.0. Without the profile the GPU's frame is the same within the noise between runs: 18.1 / 17.1 ms against 18.7 / 18.0 at the Desert Colossus, 14.8 / 14.4 against 14.8 / 14.5 in Kokiri Forest. It also drops primitives for a frame or two when a list grows (56-60 draws in a five-minute run). Not proposed. - https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/326 on top (the mirror sync in one dispatch), and narrower barriers around the fallback's dispatches: no gain in two runs each. - `RX_VK_READBACK_LATE=newest`: the long readback waits are gone (0 of 60 log lines against 4 to 19), the Market is the same, 27-28 fps at 1,400-1,600 draws. - `RX_VK_ASYNC=1`: worse everywhere, 13-25 fps in the six areas. **What does help** on this card is `RX_VK_WB_MAX_KB=1024`: the write-back above falls from 60 to 16 MB a frame and the GPU's frame by 2.5 ms (https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/547 has the runs). **The questions:** 1. The fallback runs one task dispatch and one mesh dispatch per mesh draw (115-146 a frame here, 26-30 us each on the GPU). Can draws of one program share a dispatch, or can a draw whose task shader emitted nothing the frame before skip its mesh dispatch? 2. The submit costs 8-12 us a draw on a 5600X, most of it in `draw`. https://git.magicbean.dev/MagicBeanDev/Longshot/issues/402 and https://git.magicbean.dev/MagicBeanDev/Longshot/issues/410 report the same wall on other hardware. Is there work in flight on it that a card like this one could test? 3. Why is the frame longer than GPU plus submit at 2,300 draws: is the game's own thread the third term in the Market? **Who can help without writing code:** anyone on the compute fallback (the log says `no mesh shaders (the compute fallback)`). Add `RX_VK_PASS_TIMING=2` to `longshot_env.txt`, stand in the Market for a minute and paste one `vk: GPU pass timing` block from `longshot_log.txt` here with the card and the driver. Two more cards would show whether the passes and the dispatches weigh the same on NVIDIA and on Polaris. ### Anything else - The prototype (a switch, off by default, with its game-free test) is not pushed anywhere. I can push the branch to my fork if someone wants to take it further or measure it on another card. - The earlier state of this card, without the retry, is in my comment on https://git.magicbean.dev/MagicBeanDev/Longshot/issues/441.

waryxxful referenced this issue

2026-11-14 19:28:29 +00:00

WIP: perf: Hookshot's new defaults and speed-ups merged after Longshot branched, restored #288

waryxxful referenced this issue

2026-11-14 20:42:45 +00:00

docs(tested): RX_VK_WB_MAX_KB=1024 measured on an RX 5600 XT (no mesh shaders) #547

proci referenced this issue

2026-11-14 23:28:02 +00:00

WIP: perf(vk): a mirror sync's pages from the captures go in one compute dispatch, not one copy region each #326

gmourao referenced this issue

2026-11-15 05:51:24 +00:00

Game: Ganondorf initially renders completely black, then recovers after a long delay on RX 9060 XT #715

waryxxful commented

2026-11-15 13:13:37 +00:00

Author

Where the two things that did not help were run, since #326 cites this issue and I had left that out.

Main 9c9d69ed with ea6dc5f5 (the mirror retry) on every side, RX_VK_MIRROR_SPARSE=0, RX 5600 XT 6 GB, Adrenalin 26.10.1, Windows 11, render resolution 50% of 1920x1080. Six areas by teleport as child Link, 10 s to settle and then 25 s while Link walks by scripted input; two runs a side, interleaved with the other sides. The PR 326 side is its commit e089b486 (the head on 2026-11-14, before the tables' fix 9e8545ac and the rebase onto e090cc48) cherry-picked onto that base; the barriers side is a local patch that narrows the compute fallback's barriers, not a PR. fps / GPU ms a frame / replay submit ms a frame:

Area	Base, run a	Base, run b	PR 326, run a	PR 326, run b	Narrow barriers, run a	Narrow barriers, run b
Kokiri Forest	49.2 / 14.4 / 8.6	49.8 / 14.4 / 9.4	49.0 / 14.7 / 8.8	48.0 / 15.8 / 9.8	49.1 / 14.2 / 8.9	50.8 / 14.1 / 8.4
Hyrule Field	43.3 / 16.1 / 13.1	46.2 / 12.6 / 12.5	33.7 / 18.8 / 12.2	42.3 / 16.3 / 13.2	44.7 / 16.0 / 12.4	34.4 / 18.4 / 14.0
Kakariko	25.1 / 22.9 / 16.5	42.0 / 16.1 / 12.0	31.0 / 18.7 / 12.8	34.5 / 18.1 / 11.8	41.9 / 15.1 / 12.4	43.4 / 16.6 / 10.7
Market	36.8 / 16.2 / 14.4	20.8 / 22.7 / 19.1	22.2 / 22.8 / 19.4	21.1 / 21.6 / 18.4	20.9 / 22.6 / 22.0	19.8 / 20.3 / 20.6
Hyrule Castle	41.2 / 16.6 / 13.4	26.4 / 21.1 / 12.7	28.1 / 21.1 / 14.9	37.8 / 18.6 / 15.5	42.2 / 16.3 / 10.9	39.6 / 18.1 / 14.0
Desert Colossus	44.4 / 16.1 / 12.5	39.3 / 18.4 / 14.7	38.3 / 16.9 / 14.3	44.5 / 16.7 / 12.4	45.0 / 17.9 / 12.2	46.0 / 16.9 / 11.5

Only Kokiri Forest is steady enough between runs to compare (the walk ends in a different place in the others, so the draws differ by up to 2-3 times): there neither side moves the GPU's frame or the submit by more than the base's own spread. That is all "no gain" means here: two runs a side, one card, an earlier head of the PR. It is not a result against the PR's own measurement on an RTX 5080.

Where the two things that did not help were run, since https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/326 cites this issue and I had left that out. Main 9c9d69ed with ea6dc5f5 (the mirror retry) on every side, `RX_VK_MIRROR_SPARSE=0`, RX 5600 XT 6 GB, Adrenalin 26.10.1, Windows 11, render resolution 50% of 1920x1080. Six areas by teleport as child Link, 10 s to settle and then 25 s while Link walks by scripted input; two runs a side, interleaved with the other sides. The PR 326 side is its commit e089b486 (the head on 2026-11-14, before the tables' fix 9e8545ac and the rebase onto e090cc48) cherry-picked onto that base; the barriers side is a local patch that narrows the compute fallback's barriers, not a PR. fps / GPU ms a frame / replay submit ms a frame: | Area | Base, run a | Base, run b | PR 326, run a | PR 326, run b | Narrow barriers, run a | Narrow barriers, run b | |---|---|---|---|---|---|---| | Kokiri Forest | 49.2 / 14.4 / 8.6 | 49.8 / 14.4 / 9.4 | 49.0 / 14.7 / 8.8 | 48.0 / 15.8 / 9.8 | 49.1 / 14.2 / 8.9 | 50.8 / 14.1 / 8.4 | | Hyrule Field | 43.3 / 16.1 / 13.1 | 46.2 / 12.6 / 12.5 | 33.7 / 18.8 / 12.2 | 42.3 / 16.3 / 13.2 | 44.7 / 16.0 / 12.4 | 34.4 / 18.4 / 14.0 | | Kakariko | 25.1 / 22.9 / 16.5 | 42.0 / 16.1 / 12.0 | 31.0 / 18.7 / 12.8 | 34.5 / 18.1 / 11.8 | 41.9 / 15.1 / 12.4 | 43.4 / 16.6 / 10.7 | | Market | 36.8 / 16.2 / 14.4 | 20.8 / 22.7 / 19.1 | 22.2 / 22.8 / 19.4 | 21.1 / 21.6 / 18.4 | 20.9 / 22.6 / 22.0 | 19.8 / 20.3 / 20.6 | | Hyrule Castle | 41.2 / 16.6 / 13.4 | 26.4 / 21.1 / 12.7 | 28.1 / 21.1 / 14.9 | 37.8 / 18.6 / 15.5 | 42.2 / 16.3 / 10.9 | 39.6 / 18.1 / 14.0 | | Desert Colossus | 44.4 / 16.1 / 12.5 | 39.3 / 18.4 / 14.7 | 38.3 / 16.9 / 14.3 | 44.5 / 16.7 / 12.4 | 45.0 / 17.9 / 12.2 | 46.0 / 16.9 / 11.5 | Only Kokiri Forest is steady enough between runs to compare (the walk ends in a different place in the others, so the draws differ by up to 2-3 times): there neither side moves the GPU's frame or the submit by more than the base's own spread. That is all "no gain" means here: two runs a side, one card, an earlier head of the PR. It is not a result against the PR's own measurement on an RTX 5080.