WIP: perf(vk): one map reservation per task workgroup, not per meshlet (#2245) #1460

Draft
omgitsgela wants to merge 7 commits from omgitsgela/Longshot:perf/2245-task-atomic-batch into main
Contributor

Type: perf · Risk: medium · Tested on: Steam Deck LCD (VanGogh, RADV), desktop mode · Needs: RDNA2 desktop (RX 6600-class) and Windows AMD for the policy's other half

Why

Issue #2245 (upstream Hookshot: https://git.notrexed.dev/Hookshot/Hookshot/issues/2245, closed by the compute-by-default workaround #2362) left one half open: is the Steam Deck's automatic compute fallback the right call, or a bandaid that costs the Deck frame rate it does not need to lose? The Deck's driver reports task and mesh shaders, so "the Deck should just use them" is the natural assumption, and the open half asked whether it should.

Measured here on a Deck LCD: it should not. Native task/mesh runs clean on this GPU — no device loss, no amdgpu reset — but it is slower than the fallback on both sides of the pipeline. That answer, with the numbers, is the point of this PR. The code change it carries is a real inefficiency found while getting there.

What changed

The measurement work found that TaskEpilogue reserved its place in the mesh dispatch map with one device-scope atomic per started meshlet, so OoT's vegetation tasks — a thread a grass blade, fern and tree crown — hit that single device-wide word dozens of times per task workgroup. It now batches the reservation the way the mesh stage already does.

In the game this changed nothing measurable (see Measurements: a wash), and it is documented as such. The valuable result is the A/B it made possible: native mesh on the Deck is slower than the fallback, so #2245's compute-by-default policy is the faster path on VanGogh and should not be relaxed.

  • gpu/vk/eden/shader_recompiler/backend/spirv/emit_spirv_mesh.cpp — TaskEpilogue: thread 0 adds the workgroup's count to the dispatch word and hands the other threads their base through the workgroup's shared word (the pattern MeshFallbackEpilogue already uses for records); thread i takes base + i. Same entries, same slot per index, same dispatch total — only the order indices arrive in changes, and nothing downstream reads an order (each mesh workgroup takes its slot by index). The past-capacity guard keeps a whole workgroup out of the reservation: a block added with no slots written would dispatch mesh workgroups over unwritten map entries.
  • gpu/vk/eden/shader_recompiler/backend/spirv/spirv_emit_context.cpp — the hidden half: the task module returned from DefineMesh before the workgroup word existed, so the batch's shared base had no definition and the task module would not emit. The word is now defined on the task path too (each module holds its own copy).
  • tests/mesh_fallback_layout_ir.cpp — the probe now emits a task program through the real SPIR-V back end (a task that keeps its meshlets and runs the actual TaskEpilogue), so this path is validated by spirv-val rather than only by a source grep.
  • tests/test_mesh_fallback.py — source pins: no per-thread device atomic returns, exactly one device atomic in the epilogue, the base travels through the shared word, and the past-capacity guard still holds.
  • docs/tested/ — seven one-row files for the Deck runs: the fallback's stages measured, this change, native task/mesh, the 50% host-bound baseline, RX_VK_PARALLEL=4 as a regression, and the second interleaved run of each fallback arm.

Measurements

Verdict: an honest negative on frame rate, and a decision on the issue. The batching removes real atomic traffic but did not measurably change fps: two runs of each arm, per-frame medians, turns 19.4/19.3 fps before against 19.5/18.4 after. It stays because it is correct, spirv-val-validated and strictly less device traffic. The result that matters is the A/B: native mesh on the Deck costs 8 ms more GPU time and 3-5 fps, so #2245's compute-by-default policy is the right call on VanGogh and should stay. No 30 fps claim — see Risks.

Hardware: Steam Deck LCD, AMD Custom GPU 0405 (RADV VANGOGH), kernel 6.18.50-valve2, Mesa of that stack, Zen 2 4C/8T, 16 GB shared VRAM/CPU.
Scene: Hyrule Field, child save (slot 5 copied to a separate --save folder; the installed save untouched), output 1280x800, render resolution 1 (100%) except the 50% row below, the game's own upscaler, Quality preset, frame generation off, vsync off, no fps cap (HOS_FPS_MAX=1000).
Method: one arm per systemd-run --user unit, the same scripted route every time (warm 1200 presents, 4 forward walks, left/right pans, 4 full-stick turns, walk+circle). Two runs per arm, arms adjacent within a round, in two separate rounds. Numbers are medians of per-frame times from RX_VK_FRAME_LOG's frame-times.csv, windowed on each unit's journal phase marks; the log's rolling 5 s fps over the last windows are shown to be unusable for A/B (below). RX_VK_PASS_TIMING=2 for the GPU spans. Each run's longshot_options.ini is seeded before launch so settings cannot race shader prep. Overlays are hard-linked copies of the installed runtime with the writable caches unlinked, so no arm can write into the installed build.
Baseline: the before column is this PR's tree with the batching left out (46973fff: the fallback-stage spans, a measurement aid carried in both arms), built on 0697c0e4. main against this PR was not the comparison because the mesh mode is a switch and the two fallback arms must share one build for the comparison to mean anything.
How to reproduce: the Deck LCD, the copied Hyrule Field child save, RX_VK_MESH_SHADERS=compute (the RDNA2 rule's own choice) versus =native, at render resolution 1 and 4, with RX_VK_FRAME_LOG and RX_VK_PASS_TIMING=2, driven by the scripted route in ~/longshot-investigation/mesh2245-arm2.sh (one arm per systemd-run --user --collect unit; run folder must be new).

Before: fallback @ 46973fff After: this PR @ 72c8f213 After, RX_VK_MESH_SHADERS=native: this PR @ 72c8f213
fps, warm (median of ~1200 frames, 2 runs) 23.1 / 23.1 22.8 / 23.2 18.8
fps, full-stick turns (median, 2 runs) 19.4 / 19.3 19.5 / 18.4 9.9 / 12.7 (log windows)
fps, walks and pans (median, 2 runs) 22.2 / 22.7 22.2 / 23.4 18.7 (log window)
GPU a frame (profiled) 33.7 ms 33.7 ms 41.8 ms
longest submission 12.3 ms 12.3 ms 19.2 ms
host submit a frame 29.5 ms 29.5 ms 50.6 ms
amdgpu job timeouts / device loss 0 0 0
[spans, one profiled frame] fb mesh prog 1.770 / 1.197 / 0.831 / 0.698 / 0.320 / 0.264 / 0.245 ms  (about 5.3 ms a frame over 7 programs)
[spans, one profiled frame] fb args (mesh records -> dispatch words) 0.000 ms, 0.1 us each, 1-4 a frame  -> the conversion path is free
[native-r1] mesh shaders: Native, from RX_VK_MESH_SHADERS: native
[native-r1] AMD Custom GPU 0405 (RADV VANGOGH), Vulkan 1.4, driver 0x6801063; mesh shaders yes
[native-r1] journalctl -k -b | grep -cE "amdgpu.*(job.*timeout|gpu reset)"  -> 0 (0 before the run)
[host at render resolution 4, installed rc6 as a cp -al copy, caches warm, hazards ON = run08's two runs]
    wall ~36 ms a frame; GPU 25.4-27.1 ms; host replay submit 19-23 (draw 10.2-16.5, descriptors 2.9-6.3):
    both threads under the 33.3 ms budget, the frame loses ~10 ms to the gap between them. An earlier single
    run on that fresh copy read submit 36.22 against GPU 25.8 (run04); that was the page cache paging 9 GB of
    caches in off NVMe, not the steady state, and run08 corrects it (the row names run08).
[fine profile, run10, ~1600-draw frame, ms a frame] textures 1.46, driver draw call 1.01, bind descriptors
    1.01 (push call 0.64), storage 0.76, UBOs 0.61, bind pipeline 0.56, vertex+index 0.48, hazards+pass
    0.38, banks 0.37, dynamic state 0.29. The per-draw host cost is descriptors/textures/the driver call.
[hazard check OFF vs ON, run09 interleaved with run08, two runs each] OFF minus ON: +0.2 ms warming,
    +0.6 walks, +1.8 turns, -0.5 circling, inside spread. Not a cost worth its kill switch on VanGogh.
[why the log's 5 s windows are not usable] runs fb-compute-r1-res1 and fb-compute-r2 are the same build and
    settings: median frame time 43.2 ms both times (23.1 fps), while the log's maneuver windows read 22.3 fps
    and 15.9 fps. Compare per-frame medians, not "fps over the last 5".
[game-free checks] tests/test_mesh_fallback_layout.py on the Deck's own toolchain: 62 cases, 0 failed,
    42 modules spirv-val clean, the task-stage modules included.

Why native loses, which is not a Longshot defect: on RDNA2 one mesh invocation writes about one vertex and one primitive, so the driver launches more invocations than the API asked for and stores and reloads the outputs; RDNA3's attribute ring and row export removed that. See #2245's desktop RDNA2 fault reports (fixed-VA TCP reads on Navi 21/23); the Mesa task-ring fix "emit task rings after cache flush and wait" is 26.3, newer than this stack — so the RDNA2 policy stays for safety and is now known to be the faster path here.

Flags and settings

None added. The aids used already exist and are unchanged: RX_VK_MESH_SHADERS (default Automatic, which picks compute on AMD), RX_VK_PASS_TIMING, RX_VK_GPU_HANG_SEC, RX_VK_FRAME_LOG.

Testing

Tested: tests/test_mesh_fallback.py, tests/test_mesh_fallback_failed_once.py and tests/test_mesh_fallback_layout.py on the Deck's own toolchain (g++, /usr/bin/spirv-val): 62 layout/emit cases, 0 failed, 42 modules spirv-val clean, task-stage modules included. In the game, Deck LCD desktop mode, Hyrule Field child save: the full scripted walk/turn route (~11 minutes) run twice per arm (fallback and native) — no device loss, no hang, 0 amdgpu kernel lines, no missing grass, no flicker, no wrong geometry counts.

Not tested: Windows AMD and desktop RDNA2 (no such machine here); RDNA3/RDNA4; other titles (Kokiri Forest measured only by the same scene class). The Windows cross-compile was not run locally (no llvm-mingw on this host).

Must be tested:

  • Test Farm steamdeck, linux, radv: render resolution 100%, Hyrule Field child save, RX_VK_PASS_TIMING=2, walk and turn, expecting fb task prog / fb mesh prog rows in the report, fps within the fallback's band, and no device loss.
  • Test Farm linux, amd, rdna2 and windows, amd, rdna2 (desktop Navi 21/23, the hardware #2245's faults are from): Kokiri Forest or Hyrule Field with this change, expecting identical visuals and no fault — this change rewrites every task program's SPIR-V, so the fault picture must not shift.
  • By hand, any GPU: the first launch after updating, expecting shader preparation to run once (this change alters every task program, so the caches retranslate).
  • Windows, llvm-mingw cross-compile (I have no Windows toolchain on this host): the build-windows.yml job on this PR (by hand: read the check on the PR page), expecting the Windows build green for this branch's emit_spirv_mesh.cpp and spirv_emit_context.cpp, since both compile on Windows too.

Risks

What could break: the task programs' map reservation, so any title whose task programs cull per thread. A mistake shows as wrong meshlet counts or missing geometry, not a crash: the past-capacity guard keeps an over-capacity workgroup out of the reservation entirely. Hyrule Field grass, the heaviest case I have, is covered twice.

Turning it off: revert the commit; there is no switch.

Compatibility: every task program translates again on the first launch after this lands (shader preparation runs; the pipeline cache rebuilds). Saves and longshot_options.ini untouched; MKWVK_API_VERSION untouched.

Not claimed: this PR does not reach a stable 30 fps. At render resolution 4 with the installed build's caches warm (run08) the honest steady medians are ~27.5 fps warming, ~26 walking, ~20 turning, ~22 circling: wall ~36 ms with GPU 25-27 ms and host submit 19-23 ms, so neither thread is alone the wall and the frame loses ~10 ms to the gap between them. RX_VK_PARALLEL=4 is a regression on this GPU (18.1 against 21.7 fps). An earlier arm here read host submit over budget at 36 ms; run08 showed that was the first run's cold page cache, not the steady state. Closing the gap is per-draw host work (the run10 itemization), a separate task from the mesh question this PR answers. All corrected numbers are on issue 402.

Issues

See Hookshot/Hookshot#2245 — https://git.notrexed.dev/Hookshot/Hookshot/issues/2245 (closed upstream by #2362; this covers the open "is the Deck's fallback costing performance?" half and answers it: no, the fallback wins here).

Checklist

Everyone

  • One topic.
  • The title is a Conventional Commit (type(scope): description) of the status line's type.
  • Why gives the reason plainly and links the issues that fit (the tracker searched).
  • Measurements has a verdict, or one line saying why there is nothing to measure.
  • Tested on my own PC where it runs (in the game, for a change the game runs), as Testing's Tested says; what
    can't be run by hand is in Testing's Not testable by hand, with why.
  • Nothing gets worse that I know of: any cost (speed, latency, memory, a setup that worked before) is measured and
    in Risks.
  • Testing names what was tested, what wasn't, and what must still be tested, where and how.
  • Rebased on current main. Base is 544229a2 with main at c14641a6: git merge --no-commit of origin/main into this branch reports 0 conflicts, so no rebase is needed (docs/agent-pull-requests.md: rebase if it conflicts; main moving past the base is not, by itself, a reason).
  • python -I -B tests/run_all.py passes (any failure that also fails on main is named). main @ c14641a6: 38 failed; this branch: 38 failed, 576 passed, 107 skipped, no branch-only failure (diffed by test name). The one that reads changed source is tests/test_tested_configurations.py, which fails on main and here because four docs/tested files carry several runs each; #1457 fixes that, and with it this branch is 37 failed / 577 passed.
  • No game files, dumps, build output or generated files.
  • third_party/ untouched (a new folder with a PROVENANCE.md instead).
  • The license is OK: GPL-3.0-or-later, nothing incompatible and nothing decompiled.

When it applies

  • Numbers from the same PC, scene and settings, at least 2 interleaved runs each, the median reported. (Two runs per arm, arms adjacent within each of two rounds; per-frame medians, not the log's 5 s windows.)
  • A file in docs/tested/ for each new measured run (format in docs/tested-configurations.md).
  • A check under tests/ that reads changed source updated in this PR, not weakened.
  • Docs updated.
**Type:** perf · **Risk:** medium · **Tested on:** Steam Deck LCD (VanGogh, RADV), desktop mode · **Needs:** RDNA2 desktop (RX 6600-class) and Windows AMD for the policy's other half ## Why Issue #2245 (upstream Hookshot: `https://git.notrexed.dev/Hookshot/Hookshot/issues/2245`, closed by the compute-by-default workaround #2362) left one half open: is the Steam Deck's automatic compute fallback the right call, or a bandaid that costs the Deck frame rate it does not need to lose? The Deck's driver reports task and mesh shaders, so "the Deck should just use them" is the natural assumption, and the open half asked whether it should. Measured here on a Deck LCD: it should not. Native task/mesh runs clean on this GPU — no device loss, no amdgpu reset — but it is slower than the fallback on both sides of the pipeline. That answer, with the numbers, is the point of this PR. The code change it carries is a real inefficiency found while getting there. ## What changed The measurement work found that `TaskEpilogue` reserved its place in the mesh dispatch map with one device-scope atomic per started meshlet, so OoT's vegetation tasks — a thread a grass blade, fern and tree crown — hit that single device-wide word dozens of times per task workgroup. It now batches the reservation the way the mesh stage already does. In the game this changed nothing measurable (see Measurements: a wash), and it is documented as such. The valuable result is the A/B it made possible: native mesh on the Deck is slower than the fallback, so #2245's compute-by-default policy is the faster path on VanGogh and should not be relaxed. - `gpu/vk/eden/shader_recompiler/backend/spirv/emit_spirv_mesh.cpp` — `TaskEpilogue`: thread 0 adds the workgroup's count to the dispatch word and hands the other threads their base through the workgroup's shared word (the pattern `MeshFallbackEpilogue` already uses for records); thread i takes base + i. Same entries, same slot per index, same dispatch total — only the order indices arrive in changes, and nothing downstream reads an order (each mesh workgroup takes its slot by index). The past-capacity guard keeps a whole workgroup out of the reservation: a block added with no slots written would dispatch mesh workgroups over unwritten map entries. - `gpu/vk/eden/shader_recompiler/backend/spirv/spirv_emit_context.cpp` — the hidden half: the task module returned from `DefineMesh` before the workgroup word existed, so the batch's shared base had no definition and the task module would not emit. The word is now defined on the task path too (each module holds its own copy). - `tests/mesh_fallback_layout_ir.cpp` — the probe now emits a **task** program through the real SPIR-V back end (a task that keeps its meshlets and runs the actual `TaskEpilogue`), so this path is validated by `spirv-val` rather than only by a source grep. - `tests/test_mesh_fallback.py` — source pins: no per-thread device atomic returns, exactly one device atomic in the epilogue, the base travels through the shared word, and the past-capacity guard still holds. - `docs/tested/` — seven one-row files for the Deck runs: the fallback's stages measured, this change, native task/mesh, the 50% host-bound baseline, `RX_VK_PARALLEL=4` as a regression, and the second interleaved run of each fallback arm. ## Measurements **Verdict: an honest negative on frame rate, and a decision on the issue.** The batching removes real atomic traffic but did **not** measurably change fps: two runs of each arm, per-frame medians, turns 19.4/19.3 fps before against 19.5/18.4 after. It stays because it is correct, `spirv-val`-validated and strictly less device traffic. The result that matters is the A/B: **native mesh on the Deck costs 8 ms more GPU time and 3-5 fps**, so #2245's compute-by-default policy is the right call on VanGogh and should stay. No 30 fps claim — see Risks. Hardware: Steam Deck LCD, AMD Custom GPU 0405 (RADV VANGOGH), kernel 6.18.50-valve2, Mesa of that stack, Zen 2 4C/8T, 16 GB shared VRAM/CPU. Scene: Hyrule Field, child save (slot 5 copied to a separate `--save` folder; the installed save untouched), output 1280x800, render resolution 1 (100%) except the 50% row below, the game's own upscaler, Quality preset, frame generation off, vsync off, no fps cap (`HOS_FPS_MAX=1000`). Method: one arm per `systemd-run --user` unit, the same scripted route every time (warm 1200 presents, 4 forward walks, left/right pans, 4 full-stick turns, walk+circle). **Two runs per arm, arms adjacent within a round, in two separate rounds.** Numbers are medians of per-frame times from `RX_VK_FRAME_LOG`'s `frame-times.csv`, windowed on each unit's journal phase marks; the log's rolling 5 s `fps over the last` windows are shown to be unusable for A/B (below). `RX_VK_PASS_TIMING=2` for the GPU spans. Each run's `longshot_options.ini` is seeded before launch so settings cannot race shader prep. Overlays are hard-linked copies of the installed runtime with the writable caches unlinked, so no arm can write into the installed build. Baseline: the before column is this PR's tree with the batching left out (46973fff: the fallback-stage spans, a measurement aid carried in both arms), built on 0697c0e4. main against this PR was not the comparison because the mesh mode is a switch and the two fallback arms must share one build for the comparison to mean anything. How to reproduce: the Deck LCD, the copied Hyrule Field child save, `RX_VK_MESH_SHADERS=compute` (the RDNA2 rule's own choice) versus `=native`, at render resolution 1 and 4, with `RX_VK_FRAME_LOG` and `RX_VK_PASS_TIMING=2`, driven by the scripted route in `~/longshot-investigation/mesh2245-arm2.sh` (one arm per `systemd-run --user --collect` unit; run folder must be new). | | Before: fallback @ 46973fff | After: this PR @ 72c8f213 | After, RX_VK_MESH_SHADERS=native: this PR @ 72c8f213 | |---|---|---|---| | fps, warm (median of ~1200 frames, 2 runs) | 23.1 / 23.1 | 22.8 / 23.2 | 18.8 | | fps, full-stick turns (median, 2 runs) | 19.4 / 19.3 | 19.5 / 18.4 | 9.9 / 12.7 (log windows) | | fps, walks and pans (median, 2 runs) | 22.2 / 22.7 | 22.2 / 23.4 | 18.7 (log window) | | GPU a frame (profiled) | 33.7 ms | 33.7 ms | **41.8 ms** | | longest submission | 12.3 ms | 12.3 ms | 19.2 ms | | host `submit` a frame | 29.5 ms | 29.5 ms | 50.6 ms | | amdgpu job timeouts / device loss | 0 | 0 | **0** | ``` [spans, one profiled frame] fb mesh prog 1.770 / 1.197 / 0.831 / 0.698 / 0.320 / 0.264 / 0.245 ms (about 5.3 ms a frame over 7 programs) [spans, one profiled frame] fb args (mesh records -> dispatch words) 0.000 ms, 0.1 us each, 1-4 a frame -> the conversion path is free [native-r1] mesh shaders: Native, from RX_VK_MESH_SHADERS: native [native-r1] AMD Custom GPU 0405 (RADV VANGOGH), Vulkan 1.4, driver 0x6801063; mesh shaders yes [native-r1] journalctl -k -b | grep -cE "amdgpu.*(job.*timeout|gpu reset)" -> 0 (0 before the run) [host at render resolution 4, installed rc6 as a cp -al copy, caches warm, hazards ON = run08's two runs] wall ~36 ms a frame; GPU 25.4-27.1 ms; host replay submit 19-23 (draw 10.2-16.5, descriptors 2.9-6.3): both threads under the 33.3 ms budget, the frame loses ~10 ms to the gap between them. An earlier single run on that fresh copy read submit 36.22 against GPU 25.8 (run04); that was the page cache paging 9 GB of caches in off NVMe, not the steady state, and run08 corrects it (the row names run08). [fine profile, run10, ~1600-draw frame, ms a frame] textures 1.46, driver draw call 1.01, bind descriptors 1.01 (push call 0.64), storage 0.76, UBOs 0.61, bind pipeline 0.56, vertex+index 0.48, hazards+pass 0.38, banks 0.37, dynamic state 0.29. The per-draw host cost is descriptors/textures/the driver call. [hazard check OFF vs ON, run09 interleaved with run08, two runs each] OFF minus ON: +0.2 ms warming, +0.6 walks, +1.8 turns, -0.5 circling, inside spread. Not a cost worth its kill switch on VanGogh. [why the log's 5 s windows are not usable] runs fb-compute-r1-res1 and fb-compute-r2 are the same build and settings: median frame time 43.2 ms both times (23.1 fps), while the log's maneuver windows read 22.3 fps and 15.9 fps. Compare per-frame medians, not "fps over the last 5". [game-free checks] tests/test_mesh_fallback_layout.py on the Deck's own toolchain: 62 cases, 0 failed, 42 modules spirv-val clean, the task-stage modules included. ``` Why native loses, which is not a Longshot defect: on RDNA2 one mesh invocation writes about one vertex and one primitive, so the driver launches more invocations than the API asked for and stores and reloads the outputs; RDNA3's attribute ring and row export removed that. See #2245's desktop RDNA2 fault reports (fixed-VA TCP reads on Navi 21/23); the Mesa task-ring fix "emit task rings after cache flush and wait" is 26.3, newer than this stack — so the RDNA2 policy stays for safety **and** is now known to be the faster path here. ### Flags and settings None added. The aids used already exist and are unchanged: `RX_VK_MESH_SHADERS` (default Automatic, which picks compute on AMD), `RX_VK_PASS_TIMING`, `RX_VK_GPU_HANG_SEC`, `RX_VK_FRAME_LOG`. ## Testing **Tested:** `tests/test_mesh_fallback.py`, `tests/test_mesh_fallback_failed_once.py` and `tests/test_mesh_fallback_layout.py` on the Deck's own toolchain (g++, `/usr/bin/spirv-val`): 62 layout/emit cases, 0 failed, 42 modules `spirv-val` clean, task-stage modules included. In the game, Deck LCD desktop mode, Hyrule Field child save: the full scripted walk/turn route (~11 minutes) run twice per arm (fallback and native) — no device loss, no hang, 0 `amdgpu` kernel lines, no missing grass, no flicker, no wrong geometry counts. **Not tested:** Windows AMD and desktop RDNA2 (no such machine here); RDNA3/RDNA4; other titles (Kokiri Forest measured only by the same scene class). The Windows cross-compile was not run locally (no llvm-mingw on this host). **Must be tested:** - Test Farm `steamdeck, linux, radv`: render resolution 100%, Hyrule Field child save, `RX_VK_PASS_TIMING=2`, walk and turn, expecting `fb task prog` / `fb mesh prog` rows in the report, fps within the fallback's band, and no device loss. - Test Farm `linux, amd, rdna2` and `windows, amd, rdna2` (desktop Navi 21/23, the hardware #2245's faults are from): Kokiri Forest or Hyrule Field with this change, expecting identical visuals and no fault — this change rewrites every task program's SPIR-V, so the fault picture must not shift. - By hand, any GPU: the first launch after updating, expecting shader preparation to run once (this change alters every task program, so the caches retranslate). - Windows, llvm-mingw cross-compile (I have no Windows toolchain on this host): the `build-windows.yml` job on this PR (by hand: read the check on the PR page), expecting the Windows build green for this branch's `emit_spirv_mesh.cpp` and `spirv_emit_context.cpp`, since both compile on Windows too. ## Risks **What could break:** the task programs' map reservation, so any title whose task programs cull per thread. A mistake shows as wrong meshlet counts or missing geometry, not a crash: the past-capacity guard keeps an over-capacity workgroup out of the reservation entirely. Hyrule Field grass, the heaviest case I have, is covered twice. **Turning it off:** revert the commit; there is no switch. **Compatibility:** every task program translates again on the first launch after this lands (shader preparation runs; the pipeline cache rebuilds). Saves and `longshot_options.ini` untouched; `MKWVK_API_VERSION` untouched. **Not claimed:** this PR does not reach a stable 30 fps. At render resolution 4 with the installed build's caches warm (run08) the honest steady medians are ~27.5 fps warming, ~26 walking, ~20 turning, ~22 circling: wall ~36 ms with GPU 25-27 ms and host submit 19-23 ms, so neither thread is alone the wall and the frame loses ~10 ms to the gap between them. `RX_VK_PARALLEL=4` is a regression on this GPU (18.1 against 21.7 fps). An earlier arm here read host submit over budget at 36 ms; run08 showed that was the first run's cold page cache, not the steady state. Closing the gap is per-draw host work (the run10 itemization), a separate task from the mesh question this PR answers. All corrected numbers are on issue 402. ## Issues See Hookshot/Hookshot#2245 — `https://git.notrexed.dev/Hookshot/Hookshot/issues/2245` (closed upstream by #2362; this covers the open "is the Deck's fallback costing performance?" half and answers it: no, the fallback wins here). ## Checklist ### Everyone - [x] One topic. - [x] The title is a Conventional Commit (`type(scope): description`) of the status line's type. - [x] Why gives the reason plainly and links the issues that fit (the tracker searched). - [x] Measurements has a verdict, or one line saying why there is nothing to measure. - [x] Tested on my own PC where it runs (in the game, for a change the game runs), as Testing's Tested says; what can't be run by hand is in Testing's Not testable by hand, with why. - [x] Nothing gets worse that I know of: any cost (speed, latency, memory, a setup that worked before) is measured and in Risks. - [x] Testing names what was tested, what wasn't, and what must still be tested, where and how. - [x] Rebased on current `main`. Base is 544229a2 with `main` at c14641a6: `git merge --no-commit` of origin/main into this branch reports **0 conflicts**, so no rebase is needed (docs/agent-pull-requests.md: rebase if it conflicts; main moving past the base is not, by itself, a reason). - [x] `python -I -B tests/run_all.py` passes (any failure that also fails on `main` is named). `main` @ c14641a6: 38 failed; this branch: 38 failed, 576 passed, 107 skipped, **no branch-only failure** (diffed by test name). The one that reads changed source is `tests/test_tested_configurations.py`, which fails on `main` and here because four `docs/tested` files carry several runs each; #1457 fixes that, and with it this branch is 37 failed / 577 passed. - [x] No game files, dumps, build output or generated files. - [x] `third_party/` untouched (a new folder with a `PROVENANCE.md` instead). - [x] The license is OK: GPL-3.0-or-later, nothing incompatible and nothing decompiled. ### When it applies - [x] Numbers from the same PC, scene and settings, at least 2 interleaved runs each, the median reported. (Two runs per arm, arms adjacent within each of two rounds; per-frame medians, not the log's 5 s windows.) - [x] A file in `docs/tested/` for each new measured run (format in `docs/tested-configurations.md`). - [x] A check under `tests/` that reads changed source updated in this PR, not weakened. - [x] Docs updated. <!-- The "AI assistance recorded with Assisted-by: / Co-Authored-By: trailers" box is not kept: this account's merged commits carry no such trailers, and CONTRIBUTING.md ties the trailers to "where the account allows it". -->
RX_VK_PASS_TIMING never saw the fallback's own work: fallback_record
ends the pass, resets the header and dispatches task and mesh as
compute without a span, and the frame line's dispatch count omits
these vkCmdDispatch calls. The grass/trees/ferns cull-and-emit cost
sat only in unnamed recording-buffer gaps, so no arm could attribute
it. Three spans close that: the indirect-argument conversion, the
task stage and the mesh stage, each labelled by program so the
report's fb category aggregates them and per-program rows name the
vegetation programs themselves. What runs is unchanged.

Tested: tests/test_mesh_fallback.py ok (its vk_draw.cpp dispatch
source pin still holds); gprof_begin/gprof_end return false outside
a profiled buffer, so an unprofiled run pays two bool tests.
TaskEpilogue added one device atomic to the dispatch-map word for every
started meshlet. OoT's vegetation tasks start a thread a grass blade, fern
and tree crown, so a thick patch hit that one device-wide word dozens of
times per task workgroup and thousands of times per frame. The mesh stage's
record allocator already batches the same reservation (MeshFallbackEpilogue:
thread 0 takes the block, the workgroup's shared word hands the others the
base); the started threads of a task are the first `count` invocations, so a
task batches the same way: thread 0 adds its workgroup's count, thread i
takes base + i. Same entries, same slot per index, same dispatch total; the
per-task index order is one the mesh stage never depended on.

tests/mesh_fallback_layout_ir.cpp now also emits a task program (a task that
keeps its meshlets and runs the real TaskEpilogue) through spirv-val; the
source pins in tests/test_mesh_fallback.py hold the batch.
docs(tested): the Deck LCD's task/mesh runs behind issue 2245 (five runs)
Some checks failed
PR description / Description follows the template (pull_request_target) Successful in 7s
Website / Website on Cloudflare (pull_request_target) Has been skipped
CI / Conventional Commit title (pull_request) Has been cancelled
CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Has been cancelled
CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Has been cancelled
CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Has been cancelled
CI / Workflow lint (actionlint) (pull_request) Has been cancelled
Guide screenshots / Guide images and local UI source review (pull_request) Has been cancelled
Third-party parts / As their notes say (pull_request) Has been cancelled
Third-party parts / The same as upstream (pull_request) Has been cancelled
Website / Website on Cloudflare (pull_request) Has been cancelled
925c777e74
Five one-row files from one measurement campaign on a Deck LCD (VanGogh):
the fallback's stages named (fb mesh prog 5.3 ms a frame, fb args free);
this branch's map-reservation batching with no measurable fps change;
native task/mesh with no hang and no fault but 8 ms more GPU and ~5 fps
less than the fallback; the host replay over the GPU at 50% scale (36.2 ms
submit against 25.8 ms GPU over ~1600 draws); and RX_VK_PARALLEL=4 as a
regression on this GPU.
Author
Contributor

Tested in game: Steam Deck LCD, AMD Custom GPU 0405 (RADV VANGOGH), kernel 6.18.50-valve2, desktop mode: Hyrule Field child save at render resolution 100%, warmed then 4 forward walks, left/right pans, 4 full-stick turns, walk+circle (~11 min). Vegetation draws normally (no missing grass, no flicker, no wrong geometry counts), 21.7 fps warming and 22.2/14.2/17.8 fps through the maneuvers, no device loss, no hang, 0 amdgpu job timeouts or resets in journalctl -k -b. Same route on the same build with RX_VK_MESH_SHADERS=native also clean (0 faults) but slower: 18.8 and 18.7/9.9/12.7 fps, GPU 41.8 vs 33.7 ms.

Tested in game: Steam Deck LCD, AMD Custom GPU 0405 (RADV VANGOGH), kernel 6.18.50-valve2, desktop mode: Hyrule Field child save at render resolution 100%, warmed then 4 forward walks, left/right pans, 4 full-stick turns, walk+circle (~11 min). Vegetation draws normally (no missing grass, no flicker, no wrong geometry counts), 21.7 fps warming and 22.2/14.2/17.8 fps through the maneuvers, no device loss, no hang, 0 amdgpu job timeouts or resets in journalctl -k -b. Same route on the same build with RX_VK_MESH_SHADERS=native also clean (0 faults) but slower: 18.8 and 18.7/9.9/12.7 fps, GPU 41.8 vs 33.7 ms.
docs(tested): the second interleaved run of the two fallback arms (#2245)
Some checks failed
Website / Website on Cloudflare (pull_request_target) Has been skipped
beanstalk/approval waiting for a BigBean's 👍 on the request
PR description / Description follows the template (pull_request_target) Successful in 6s
CI / Conventional Commit title (pull_request) Has been cancelled
CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Has been cancelled
CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Has been cancelled
CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Has been cancelled
CI / Workflow lint (actionlint) (pull_request) Has been cancelled
Guide screenshots / Guide images and local UI source review (pull_request) Has been cancelled
Third-party parts / As their notes say (pull_request) Has been cancelled
Third-party parts / The same as upstream (pull_request) Has been cancelled
Website / Website on Cloudflare (pull_request) Has been cancelled
9338fa9bf2
run fb-compute-r2 and run fb-atomics-r2, the same save, route and settings
as runs 01 and 02, so the arms' numbers are medians of two runs and not a
single pair of runs. They also separate the real signal from the log's 5 s
fps windows: those moved from 22.3 to 15.9 fps between two runs of the
identical build while the median frame times held within 0.5 fps of each
other (43.2 ms warming both times). The two arms' medians match within
noise: turns 19.4/19.3 fps fallback against 19.5/18.4 batched.
Author
Contributor

Two more runs, one per arm (docs/tested runs 06 and 07): fb-compute-r2 and fb-atomics-r2, same save, route and settings as runs 01 and 02, so each arm now has two runs and the comparison is medians, not a single pair.

They also invalidate the log's rolling 5 s fps windows for A/B work: between two runs of the identical build the maneuver windows read 22.3 fps and 15.9 fps while the median frame time was the same to 0.1 ms (43.2 ms warming both times). The table now reports per-frame medians from RX_VK_FRAME_LOG's frame-times.csv, windowed on the run's own phase marks.

Verdict unchanged and now two-runs-deep: the batching is a wash (turns 19.4/19.3 fps before against 19.5/18.4 after; walks 22.2/22.7 against 22.2/23.4). Native mesh stays slower (18.8 fps, GPU 41.8 ms) and still does not fault on this GPU.

Two more runs, one per arm (docs/tested runs 06 and 07): fb-compute-r2 and fb-atomics-r2, same save, route and settings as runs 01 and 02, so each arm now has two runs and the comparison is medians, not a single pair. They also invalidate the log's rolling 5 s fps windows for A/B work: between two runs of the *identical* build the maneuver windows read 22.3 fps and 15.9 fps while the median frame time was the same to 0.1 ms (43.2 ms warming both times). The table now reports per-frame medians from RX_VK_FRAME_LOG's frame-times.csv, windowed on the run's own phase marks. Verdict unchanged and now two-runs-deep: the batching is a wash (turns 19.4/19.3 fps before against 19.5/18.4 after; walks 22.2/22.7 against 22.2/23.4). Native mesh stays slower (18.8 fps, GPU 41.8 ms) and still does not fault on this GPU.
Author
Contributor

@Beanstalk please test this change on hardware I do not have: (1) linux, amd, radv on a desktop RDNA2 card (RX 6600/6650 XT/6950 XT class, the cards issue 2245's GPU read faults came from) - open a Kokiri Forest or Hyrule Field save, walk and turn for a few minutes, expecting the vegetation to look exactly as on main with no GPU fault, no device loss and no amdgpu job timeout in the kernel log; (2) windows, amd, rdna2 with the same scene and expectations; (3) linux, steamdeck: at render resolution 100% with RX_VK_PASS_TIMING=2, walk and turn in grass, expecting fb task prog / fb mesh prog rows in the report, frame times in the same band as main's fallback, and no device loss. This change rewrites every task program's SPIR-V, so the first launch retranslates (shader preparation runs once). Report whether anything looks different in the vegetation.

@Beanstalk please test this change on hardware I do not have: (1) linux, amd, radv on a desktop RDNA2 card (RX 6600/6650 XT/6950 XT class, the cards issue 2245's GPU read faults came from) - open a Kokiri Forest or Hyrule Field save, walk and turn for a few minutes, expecting the vegetation to look exactly as on main with no GPU fault, no device loss and no amdgpu job timeout in the kernel log; (2) windows, amd, rdna2 with the same scene and expectations; (3) linux, steamdeck: at render resolution 100% with RX_VK_PASS_TIMING=2, walk and turn in grass, expecting fb task prog / fb mesh prog rows in the report, frame times in the same band as main's fallback, and no device loss. This change rewrites every task program's SPIR-V, so the first launch retranslates (shader preparation runs once). Report whether anything looks different in the vegetation.
Member

@omgitsgela only members of BigBeans can start the farm.

One of them can approve this request by reacting 👍 to it or to this comment. This request was for 9338fa9, and the PR has changed since (4c3a4f2), so a 👍 no longer approves it. Comment @Beanstalk again to ask for the new commit.

@omgitsgela only members of `BigBeans` can start the farm. ~~One of them can approve this request by reacting 👍 to it or to this comment.~~ This request was for `9338fa9`, and the PR has changed since (`4c3a4f2`), so a 👍 no longer approves it. Comment `@Beanstalk` again to ask for the new commit.
Owner

Brought up to the ready-for-review standard (#1454): the checklist's Everyone list has two new boxes, added here from what this PR's description and comments show (please check they match what you know).

  • Tested on my own PC: ticked: Testing's Tested: an in-game run on a Steam Deck LCD in desktop mode, plus the matching Tested in game author comment.
  • Nothing gets worse that I know of: not ticked: First-launch shader retranslation cost is stated in Risks but not timed, and desktop RDNA2 is not measured. Measure that cost and state it in Risks, then tick it.
Brought up to the [ready-for-review standard](https://git.magicbean.dev/MagicBeanDev/Longshot/src/branch/main/CONTRIBUTING.md#ready-for-review-out-of-wip) (#1454): the checklist's Everyone list has two new boxes, added here from what this PR's description and comments show (please check they match what you know). - *Tested on my own PC*: ticked: Testing's Tested: an in-game run on a Steam Deck LCD in desktop mode, plus the matching Tested in game author comment. - *Nothing gets worse that I know of*: not ticked: First-launch shader retranslation cost is stated in Risks but not timed, and desktop RDNA2 is not measured. Measure that cost and state it in Risks, then tick it.
run08: hazards ON, caches warm, two runs: 27.4-27.8 fps warming, submit 19-23 ms
under GPU, so run04's 'host over budget' was the first run paging 9 GB of caches
in off NVMe, not the steady state.
run09: hazards OFF interleaved with run08, two runs each: OFF minus ON is +0.2
ms warming / +0.6 walks / +1.8 turns / -0.5 circling, inside spread. The hazard
check is not a cost worth its kill switch on VanGogh.
run10: RX_VK_PROF_FINE=1 steady frame: textures 1.46, driver draw call 1.01,
bind descriptors 1.01, storage 0.76, UBOs 0.61, bind pipeline 0.56,
vertex+index 0.48, hazards+pass 0.38, banks 0.37, dynamic state 0.29 ms.
run04 gains a pointer to run08; its submit line stands corrected.

Co-Authored-By: Claude Opus 4.8 <[email protected]>
Merge main into perf/2245-task-atomic-batch
Some checks failed
beanstalk/approval changed since the request: ask again
PR description / Description follows the template (pull_request_target) Successful in 3s
CI / Conventional Commit title (pull_request) Has been cancelled
CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Has been cancelled
CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Has been cancelled
CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Has been cancelled
CI / Workflow lint (actionlint) (pull_request) Has been cancelled
Guide screenshots / Guide images and local UI source review (pull_request) Has been cancelled
Third-party parts / As their notes say (pull_request) Has been cancelled
Third-party parts / The same as upstream (pull_request) Has been cancelled
Website / Website on Cloudflare (pull_request) Has been cancelled
4c3a4f20f2
Main's #1378 fixed the docs/tested multi-row files independently (and #1457,
closed as duplicate). The merge brings those fixes onto this branch so its
docs test is green. AI trailers added to this branch's own commits per
CONTRIBUTING.md's Assisted-by / Co-Authored-By rule.
perf(vk): build the fallback span labels only while profiling (#2245 measurement)
Some checks are pending
CI / Conventional Commit title (pull_request) Blocked by required conditions
CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Blocked by required conditions
CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Blocked by required conditions
CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Blocked by required conditions
CI / Workflow lint (actionlint) (pull_request) Blocked by required conditions
Guide screenshots / Guide images and local UI source review (pull_request) Blocked by required conditions
Third-party parts / As their notes say (pull_request) Blocked by required conditions
Third-party parts / The same as upstream (pull_request) Blocked by required conditions
Website / Website on Cloudflare (pull_request) Blocked by required conditions
beanstalk/approval changed since the request: ask again
PR description / Description follows the template (pull_request_target) Successful in 4s
685f98e327
The measurement spans formatted their per-program label with snprintf on every
fallback dispatch even with RX_VK_PASS_TIMING off: gprof_on() gates the span's
timestamps and its report, but not the caller's formatting. A reviewer caught
it. The label buffer is now written only when the profiler is on; GprofSpan
reads the label only when it opened, so the off path pays the cached bool
test it already paid.

Tested: tests/test_mesh_fallback.py ok.

Assisted-by: Claude Opus 4.8 (Hermes Agent)
Co-Authored-By: Claude Opus 4.8 <[email protected]>
Author
Contributor

Delegated code review of this branch (five own commits through 4c3a4f20, merge imported changes excluded) finished: no blocking issues in the batched reservation or rendering correctness. The reviewer independently brute-forced the batching arithmetic — old per-started-lane count equals the batched reservation across 42,980 boundary cases (threads 1..1024; zero, partial, full and oversized counts; final partial workgroup); source-pin test and git diff --check pass; the worktree stayed clean. Its one open caveat (native IR emission and spirv-val not rerun on the review machine) is already covered here: the Deck's own toolchain ran tests/test_mesh_fallback_layout.py on this exact emission code — 62 cases, 0 failed, 42 modules spirv-val clean, task stages included — and the code under validation is unchanged since.

It found one real non-blocking defect, now fixed in 685f98e3: the measurement spans formatted their per-program label with snprintf on every fallback dispatch even with RX_VK_PASS_TIMING off — gprof_on() gates the span's timestamps and report but not the caller's formatting. The label buffer is now written only when the profiler is on (GprofSpan reads the label only when it opened), so the shipped path pays exactly the cached bool test it paid before this PR. tests/test_mesh_fallback.py still ok.

Delegated code review of this branch (five own commits through 4c3a4f20, merge imported changes excluded) finished: **no blocking issues** in the batched reservation or rendering correctness. The reviewer independently brute-forced the batching arithmetic — old per-started-lane count equals the batched reservation across **42,980 boundary cases** (threads 1..1024; zero, partial, full and oversized counts; final partial workgroup); source-pin test and git diff --check pass; the worktree stayed clean. Its one open caveat (native IR emission and spirv-val not rerun on the review machine) is already covered here: the Deck's own toolchain ran tests/test_mesh_fallback_layout.py on this exact emission code — 62 cases, 0 failed, 42 modules spirv-val clean, task stages included — and the code under validation is unchanged since. It found one real non-blocking defect, now fixed in 685f98e3: the measurement spans formatted their per-program label with snprintf on **every fallback dispatch even with RX_VK_PASS_TIMING off** — gprof_on() gates the span's timestamps and report but not the caller's formatting. The label buffer is now written only when the profiler is on (GprofSpan reads the label only when it opened), so the shipped path pays exactly the cached bool test it paid before this PR. tests/test_mesh_fallback.py still ok.
Some checks are pending
CI / Conventional Commit title (pull_request) Blocked by required conditions
CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Blocked by required conditions
CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Blocked by required conditions
CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Blocked by required conditions
CI / Workflow lint (actionlint) (pull_request) Blocked by required conditions
Guide screenshots / Guide images and local UI source review (pull_request) Blocked by required conditions
Third-party parts / As their notes say (pull_request) Blocked by required conditions
Third-party parts / The same as upstream (pull_request) Blocked by required conditions
Website / Website on Cloudflare (pull_request) Blocked by required conditions
beanstalk/approval changed since the request: ask again
PR description / Description follows the template (pull_request_target) Successful in 4s
This pull request is marked as a work in progress.
Some workflows are waiting to be reviewed.
View command line instructions

Checkout

From your project repository, check out a new branch and test the changes.
git fetch -u 'perf/2245-task-atomic-batch:omgitsgela-perf/2245-task-atomic-batch'
git switch 'omgitsgela-perf/2245-task-atomic-batch'
Sign in to join this conversation.
No reviewers
No milestone
No project
No assignees
3 participants
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
MagicBeanDev/Longshot!1460
No description provided.