MagicBeanDev/Longshot

Watch

Fork 101

WIP: perf(vk): the mesh fallback's records line by line, or each vertex once and drawn indexed (RX_VK_MESH_FB_LAYOUT, off by default) #906

Draft

zeldaootr wants to merge 4 commits from zeldaootr/Longshot:fallback-records-indexed into main

zeldaootr commented

2026-11-15 20:05:23 +00:00

Contributor

Type: perf · Risk: medium · Tested on: Steam Deck OLED (RADV, Kokiri at handheld settings), Linux game-free checks and lavapipe · Needs: Steam Deck (native settings, Hyrule Field, plain fps), any other GPU on the compute fallback (RX_VK_MESH_SHADERS=compute)

Why

On a Steam Deck the compute fallback that draws the game's mesh programs (the forest floor, the grass, the trees) is the
largest GPU cost that does not shrink with the render resolution: in Kokiri Forest at handheld settings it takes 17 to 20 ms
a frame, a third of the profiled GPU frame, and 86 % of that is its mesh dispatches. Those dispatches write one 22 to 27
word record per triangle corner, 53 MB a frame, which the vertex stage then reads back: every vertex is written two to six
times, and neighbouring threads write words a record apart. This PR lets the fallback write each vertex once, line by
line, and draw the triangles from indices. Off by default until the remaining Deck measurements are in.

What changed

A switch, RX_VK_MESH_FB_LAYOUT, chooses how the fallback's mesh stage stores what it hands to its vertex stage: aos
(unset: as it always was), soa (the same records, line by line) or indexed (each vertex once and the triangles as
32-bit indices, for triangle programs with no per-primitive outputs; other programs as soa). Unset, the translation is
byte for byte main's and the recorded commands are main's.

translation (emit_spirv_mesh.cpp): WriteFallbackLines writes soa's records, line l of record r at
H + l * fit + r, the threads taking the records in turn; MeshFallbackIndexedEpilogue counts a workgroup's
vertices (1 + its largest index, a workgroup OpAtomicUMax), takes its slots from the same allocator (its corners, or
its vertices rounded up to whole triangles when they are more), writes its vertices once and its corners' indices in
corner order (the same provoking vertex and winding); slots past its corners hold a triangle of its first vertex,
which has no area. EmitMeshFallbackVertexBody reads each layout. The layout, fit and the index line:
MeshFallbackLayout, MeshFallbackFit, MeshFallbackIndexed, MeshFallbackIndexWord in emit_spirv.h.
the records' header (emit_spirv.h): unchanged in aos and soa. In indexed it is 16 words: words 0-7 as before
(the plain draw's count stays 0), the VkDrawIndexedIndirectCommand at words 11-15. The allocator and the task
stage's dispatch words do not move, so task programs translate the same in every layout.
overflow: one allocator, so the slots that fit are the first taken, as the records are: every index drawn is written
and names a vertex written; past the area the draw is the first workgroups'.
host (vk_draw.cpp): fallback_header_fills (main's four fills, and the indexed command's three); fallback_draw
draws the header as before, or binds the program's index line through bind_index (so the game's next indexed draw
binds its own again) and draws it with vkCmdDrawIndexedIndirect. With indexed the records buffer is also an index
buffer and a thread's pair keeps a 16-word header for every program, so its programs still share it.
program cache: Program::fallback_indexed in the record head (vk_progcache.cpp). The switch is a name under
RX_VK_MESH, which the cache's stamp holds: a start with it translates again, a start without it keeps its caches.
the switch: mesh_rule::fb_layout (mesh_mode.h), mesh_fallback_layout() (vk_shader.cpp), docs/development.md.

Measurements

Verdict: the fallback's mesh dispatches take half the time with either new layout on a Steam Deck in Kokiri Forest at
handheld settings: 17.0 ms a frame (aos, main's layout) to 8.3 ms (soa, -51%) and 8.5 ms (indexed, -50%), two runs
each. The gain is the line-by-line layout: indexing adds nothing measurable on top in the mesh stage, and the draw of the
records stays at 4.5-5.1 ms in every arm. Pending: native settings and Hyrule Field, the plain (unprofiled) fps arms, and
a frozen-frame picture comparison.

Hardware: Steam Deck OLED, SteamOS, RADV
Scene: Kokiri Forest, the heavy standing view (~2,200 draws), 120 s standing and three walk cycles; handheld: 1280x720 output, 50 % render, the game's own upscaler
Method: a measuring build (this PR's gpu/ change on the Deck dev base 9c9d69ed, plus a probe's RX_VK_PASS_TIMING=2 parts and RX_VK_FALLBACK_STATS=1, not part of this PR); the mesh part of mesh fallback by part, every 600 presents, the first report after a start ignored; arms interleaved (aos, indexed, soa, indexed, aos, soa), two runs each, the mean; every run from empty program and pipeline caches with RX_VK_PRECOMPILE=0, each arm translating in its own layout
Baseline: the Deck dev base, main @ 9c9d69ed, with the switch unset (aos, main's layout), in the same measuring build; the After columns are this PR's gpu/ change applied on that base.
How to reproduce: RX_VK_MESH_FB_LAYOUT=soa or indexed in longshot_env.txt, with RX_VK_PASS_TIMING=2 on a build that has the fallback's parts timed

Kokiri, handheld (two runs each)	Before: main @ `9c9d69ed`	After, RX_VK_MESH_FB_LAYOUT=soa: this PR @ `787025d6`	After, RX_VK_MESH_FB_LAYOUT=indexed: this PR @ `787025d6`
fallback mesh dispatches, ms a frame	17.81, 16.25	7.91, 8.78	8.44, 8.59
the fallback before its draws, ms a frame	20.76, 18.86	10.57, 11.88	11.43, 11.91
draw of the records, ms a frame	5.02, 4.50	4.63, 5.08	4.47, 5.05
hot programs' mesh ms (`43ec86f3` / `d8e59341` / `3b8c07f2` / `89ca750f`), first run	2.38 / 2.27 / 1.50 / 1.31	1.81 / 1.49 / 0.80 / 0.77	1.83 / 1.51 / 0.64 / 0.76
records cut past the area (`over`)	0	0	0
standing fps / GPU ms (profiled, noisy)	10.34 / 59.9, 11.18 / 55.8	12.27 / 48.3, 10.41 / 55.3	12.24 / 46.0, 10.39 / 56.0
native settings, Hyrule Field, plain fps	PENDING	PENDING	PENDING

vk: mesh fallback records: each vertex once and the triangles as indices, other programs line after line (indexed) (RX_VK_MESH_FB_LAYOUT)

Pictures: the same moments of each arm, looked at full size, show the same ground, grass, bushes, ferns, foliage and
shadows, with no missing, black or stretched geometry. A per-pixel difference cannot prove identity there, as the scene
animates (falling leaves, fairies, swaying grass): across arms it is the same as between two runs of one arm (mean largest
channel difference 9.2-11.9 across, 9.9-10.7 within). Pending: a frozen-frame (pause menu) comparison.

Game-free, Linux, lavapipe (llvmpipe, LLVM 20): the fixture's modules (below) run as the fallback runs them, one
workgroup a dispatch so that the records' order is the workgroups', then drawn into two RGBA32F targets (a smooth and a
flat input, back faces culled):

tri-pos-first soa vs aos: triangles equal, images equal
tri-pos-first indexed vs aos: triangles equal, images equal          (18,812 triangles; 64,007 px lit)
tri-pos-later-prim soa vs aos: triangles equal, images equal
tri-pos-later-prim indexed vs aos: triangles equal, images equal     (a per-primitive line: indexed is soa here)
tri-wide soa vs aos: triangles equal, images equal                   (27-word records)
tri-wide indexed vs aos: triangles equal, images equal
tri-task soa vs aos: triangles equal, images equal                   (a task stage before it)
tri-task indexed vs aos: triangles equal, images equal
tri-wide, 6,000 workgroups, past the area (fit 621,378): soa draws aos's 621,315 records exactly; indexed draws 621,267
slots, its 188,946 triangles the first of aos's 189,741, nothing read past what was written

Flags and settings

One new switch, off by default.

Flag	What it is	Before this PR	After this PR	Effect	Helps / doesn't help / untested	Recommendation
`RX_VK_MESH_FB_LAYOUT=aos`, `soa` or `indexed`	How the compute fallback's mesh stage stores its vertices	not there (`aos`)	unset (`aos`)	`soa`: line by line; `indexed`: each vertex once, triangles as indices	helps on a Steam Deck in Kokiri at handheld (mesh dispatches -50%); untested elsewhere	off until the native, Field and picture checks are in

Testing

Tested:

Steam Deck OLED: Kokiri Forest at handheld settings, aos, soa and indexed, two runs each (Measurements).
python -I -B tests/run_all.py on Linux, before the last rebase: 571 passed, 16 failed, 24 skipped; the 16 fail the same on main on this PC (test_options_budget_once, test_readback_late, test_vk_pgo, test_bake_portability, test_crash_dialog, test_walk_key, test_missing_nup, test_appimage_player_build, test_lift_cache, test_env_on, test_lift_cleanup_error, test_pipeline_tour, test_make_game_cache, test_linux_preflight, test_mods_size_table, test_vk_occlusion_queries). After it, the 81 checks that read the changed files, once with g++ 13 and once with clang 23: 77 passed, and the 4 that failed (test_bake_portability, test_crash_dialog, test_pipeline_tour, test_vk_occlusion_queries) fail the same on current main.
test_vk_occlusion_queries fails on main before it reaches draw() (the replay's recording-end barrier, its line 809), so its run does not test this PR's change to its draw-command pattern. Checked apart instead: with that one replay check left out, the check passes on this branch with the new pattern, and fails with the old one (occ.issue() with a way out before its draw command).
New: tests/test_mesh_fallback_layout.py (the fixture tests/mesh_fallback_layout_ir.cpp through the real SPIR-V back
end: every module spirv-val clean, SPIR-V 1.6, the vertex stages Vulkan 1.3 too; unset translates byte for byte as
aos; and the model tests/mesh_fallback_layout_model.cpp: random meshlets, up to 256 vertices and primitives,
counts past the most, index bytes past the vertices, degenerate triangles, and past the area); tests/test_mesh_fallback_draw.py
(the fills, the draw of each layout, the index binding it leaves). Updated: test_fallback_buffer_cleanup,
test_program_record_head, test_mesh_fallback, test_vk_occlusion_queries. With g++ 13 and clang 23; the back end is compiled
as C++23 (C++2b where the compiler has no -std=c++23), as the bake's builds do.
The fixture's aos modules are byte for byte main's (twelve modules: six shapes, mesh and vertex stage).
Mutations caught: indices rotated (the provoking vertex), workgroup-local indices (the index base), the indexed draw
counting slots that did not fit (the overflow rule), slots of the corners only, soa's vertex stage line stride: by
the model, and by the lavapipe run of the real modules.
Whole-file syntax checks of the changed files for Linux and Windows (llvm-mingw): the same warnings as main.

Not tested: native settings and Hyrule Field on the Deck; unprofiled fps; a pixel-exact picture comparison; any other GPU.

Must be tested:

Steam Deck, native settings and Hyrule Field: aos, soa, indexed, expecting the mesh dispatch time to fall
as at handheld; plain fps arms without the profiling switches; and a frozen-frame (pause menu) screenshot of each arm at
the same spot, expecting identical pictures (by hand).
Linux, AMD RX 5000 series (RDNA 1, which takes the compute fallback), Mesa: Kokiri Forest and Hyrule Field with
RX_VK_MESH_FB_LAYOUT=indexed and without it, expecting the same picture (by hand: same-spot screenshots).

Risks

What could break: with soa or indexed, the fallback's terrain and vegetation (wrong or missing triangles, wrong
flat-shaded values) on GPUs using the compute fallback. Unset, nothing: the translation and the commands are main's.
Past the records area, indexed can draw a few triangles fewer than aos (a workgroup with more vertices than corners
takes more slots); the area is the same 64 MB.

Turning it off: unset RX_VK_MESH_FB_LAYOUT (the default).

Compatibility: nothing changes for a player who does not set it. Setting it, or changing it, translates the programs
again on the next start (the program cache's stamp holds the switch), as RX_VK_MESH_NOCULL does.

Also: in indexed, a workgroup's padding triangles (its slots past its corners, which have no area) shift
gl_PrimitiveID for the triangles after them; that only matters to a fragment program that reads the primitive ID.
RX_VK_MESH_FB_LAYOUT=aos set explicitly translates as unset does, but its text still changes the cache's stamp, so it
translates again once. And past the records area, indexed can draw a few triangles fewer than aos (a workgroup with
more vertices than corners takes more slots).

Works with / conflicts with

#707 (mesh count draws, a records area that grows on demand): it takes header words 8-10 (the census address and
the area's words) and a 16-word header; this PR leaves words 8-10 unused and puts its indexed command at 11-15, so the
headers fit together. But #707's fit is read at run time from the area word, while here fit is a compile-time
constant (MeshFallbackFit) in three places: soa's line stride in the mesh and vertex stages, the indexed epilogue,
and the host's index-line offset (MeshFallbackIndexWord in fallback_draw); all three would take the area's word
instead. #707's count draws, a command per record, would need a VkDrawIndexedIndirectCommand per record for an
indexed program.
#703 / #708 (macOS: each triangle's last vertex first, through a static rotation list): they draw the fallback with
vkCmdDrawIndexedIndirect at the same draw site. With indexed the rotation {2,0,1} would be written into the
generated indices instead of a static list; with aos and soa their list works as it is.
#783 (batching a pass's fallback draws on MoltenVK): each member keeps its own header; no conflict expected.

Whichever lands second adapts to the other; I can do that rebase for any of them.

Checklist

Everyone

One topic.
The title is a Conventional Commit (type(scope): description) of the status line's type.
Why gives the reason plainly and links the issues that fit (the tracker searched).
Measurements has a verdict, or one line saying why there is nothing to measure.
Testing names what was tested, what wasn't, and what must still be tested, where and how.
Rebased on current main.
python -I -B tests/run_all.py passes (any failure that also fails on main is named).
No game files, dumps, build output or generated files.
third_party/ untouched (a new folder with a PROVENANCE.md instead).
The license is OK: GPL-3.0-or-later, nothing incompatible and nothing decompiled.

When it applies

AI assistance recorded with Assisted-by: / Co-Authored-By: trailers.
Numbers from the same PC, scene and settings, at least 2 interleaved runs each, the median reported. (Kokiri handheld done, two runs each, the mean; native and the Field pending)
Switches documented (docs/development.md; the README for a menu row) and shipped at their recommended value.
Screenshots of the same spot, captioned like the Measurements columns, for a change that shows on screen. (frozen-frame comparison pending)
A file in docs/tested/ for each new measured run (format in docs/tested-configurations.md). (with the remaining runs)
A check under tests/ that reads changed source updated in this PR, not weakened.
Host code builds on both Windows (llvm-mingw) and Linux, with UTF-8 paths.
Docs updated.

**Type:** perf · **Risk:** medium · **Tested on:** Steam Deck OLED (RADV, Kokiri at handheld settings), Linux game-free checks and lavapipe · **Needs:** Steam Deck (native settings, Hyrule Field, plain fps), any other GPU on the compute fallback (`RX_VK_MESH_SHADERS=compute`) ## Why On a Steam Deck the compute fallback that draws the game's mesh programs (the forest floor, the grass, the trees) is the largest GPU cost that does not shrink with the render resolution: in Kokiri Forest at handheld settings it takes 17 to 20 ms a frame, a third of the profiled GPU frame, and 86 % of that is its mesh dispatches. Those dispatches write one 22 to 27 word record per triangle corner, 53 MB a frame, which the vertex stage then reads back: every vertex is written two to six times, and neighbouring threads write words a record apart. This PR lets the fallback write each vertex once, line by line, and draw the triangles from indices. Off by default until the remaining Deck measurements are in. ## What changed A switch, `RX_VK_MESH_FB_LAYOUT`, chooses how the fallback's mesh stage stores what it hands to its vertex stage: `aos` (unset: as it always was), `soa` (the same records, line by line) or `indexed` (each vertex once and the triangles as 32-bit indices, for triangle programs with no per-primitive outputs; other programs as `soa`). Unset, the translation is byte for byte main's and the recorded commands are main's. - translation (`emit_spirv_mesh.cpp`): `WriteFallbackLines` writes `soa`'s records, line l of record r at `H + l * fit + r`, the threads taking the records in turn; `MeshFallbackIndexedEpilogue` counts a workgroup's vertices (1 + its largest index, a workgroup `OpAtomicUMax`), takes its slots from the same allocator (its corners, or its vertices rounded up to whole triangles when they are more), writes its vertices once and its corners' indices in corner order (the same provoking vertex and winding); slots past its corners hold a triangle of its first vertex, which has no area. `EmitMeshFallbackVertexBody` reads each layout. The layout, `fit` and the index line: `MeshFallbackLayout`, `MeshFallbackFit`, `MeshFallbackIndexed`, `MeshFallbackIndexWord` in `emit_spirv.h`. - the records' header (`emit_spirv.h`): unchanged in `aos` and `soa`. In `indexed` it is 16 words: words 0-7 as before (the plain draw's count stays 0), the `VkDrawIndexedIndirectCommand` at words 11-15. The allocator and the task stage's dispatch words do not move, so task programs translate the same in every layout. - overflow: one allocator, so the slots that fit are the first taken, as the records are: every index drawn is written and names a vertex written; past the area the draw is the first workgroups'. - host (`vk_draw.cpp`): `fallback_header_fills` (main's four fills, and the indexed command's three); `fallback_draw` draws the header as before, or binds the program's index line through `bind_index` (so the game's next indexed draw binds its own again) and draws it with `vkCmdDrawIndexedIndirect`. With `indexed` the records buffer is also an index buffer and a thread's pair keeps a 16-word header for every program, so its programs still share it. - program cache: `Program::fallback_indexed` in the record head (`vk_progcache.cpp`). The switch is a name under `RX_VK_MESH`, which the cache's stamp holds: a start with it translates again, a start without it keeps its caches. - the switch: `mesh_rule::fb_layout` (`mesh_mode.h`), `mesh_fallback_layout()` (`vk_shader.cpp`), `docs/development.md`. ## Measurements **Verdict:** the fallback's mesh dispatches take half the time with either new layout on a Steam Deck in Kokiri Forest at handheld settings: 17.0 ms a frame (`aos`, main's layout) to 8.3 ms (`soa`, -51%) and 8.5 ms (`indexed`, -50%), two runs each. The gain is the line-by-line layout: indexing adds nothing measurable on top in the mesh stage, and the draw of the records stays at 4.5-5.1 ms in every arm. Pending: native settings and Hyrule Field, the plain (unprofiled) fps arms, and a frozen-frame picture comparison. Hardware: Steam Deck OLED, SteamOS, RADV Scene: Kokiri Forest, the heavy standing view (~2,200 draws), 120 s standing and three walk cycles; handheld: 1280x720 output, 50 % render, the game's own upscaler Method: a measuring build (this PR's `gpu/` change on the Deck dev base 9c9d69ed, plus a probe's `RX_VK_PASS_TIMING=2` parts and `RX_VK_FALLBACK_STATS=1`, not part of this PR); the `mesh` part of `mesh fallback by part`, every 600 presents, the first report after a start ignored; arms interleaved (aos, indexed, soa, indexed, aos, soa), two runs each, the mean; every run from empty program and pipeline caches with `RX_VK_PRECOMPILE=0`, each arm translating in its own layout Baseline: the Deck dev base, main @ 9c9d69ed, with the switch unset (`aos`, main's layout), in the same measuring build; the After columns are this PR's `gpu/` change applied on that base. How to reproduce: `RX_VK_MESH_FB_LAYOUT=soa` or `indexed` in `longshot_env.txt`, with `RX_VK_PASS_TIMING=2` on a build that has the fallback's parts timed | Kokiri, handheld (two runs each) | Before: main @ 9c9d69ed | After, RX_VK_MESH_FB_LAYOUT=soa: this PR @ 787025d6 | After, RX_VK_MESH_FB_LAYOUT=indexed: this PR @ 787025d6 | |---|---|---|---| | fallback mesh dispatches, ms a frame | 17.81, 16.25 | 7.91, 8.78 | 8.44, 8.59 | | the fallback before its draws, ms a frame | 20.76, 18.86 | 10.57, 11.88 | 11.43, 11.91 | | draw of the records, ms a frame | 5.02, 4.50 | 4.63, 5.08 | 4.47, 5.05 | | hot programs' mesh ms (`43ec86f3` / `d8e59341` / `3b8c07f2` / `89ca750f`), first run | 2.38 / 2.27 / 1.50 / 1.31 | 1.81 / 1.49 / 0.80 / 0.77 | 1.83 / 1.51 / 0.64 / 0.76 | | records cut past the area (`over`) | 0 | 0 | 0 | | standing fps / GPU ms (profiled, noisy) | 10.34 / 59.9, 11.18 / 55.8 | 12.27 / 48.3, 10.41 / 55.3 | 12.24 / 46.0, 10.39 / 56.0 | | native settings, Hyrule Field, plain fps | PENDING | PENDING | PENDING | ``` vk: mesh fallback records: each vertex once and the triangles as indices, other programs line after line (indexed) (RX_VK_MESH_FB_LAYOUT) ``` Pictures: the same moments of each arm, looked at full size, show the same ground, grass, bushes, ferns, foliage and shadows, with no missing, black or stretched geometry. A per-pixel difference cannot prove identity there, as the scene animates (falling leaves, fairies, swaying grass): across arms it is the same as between two runs of one arm (mean largest channel difference 9.2-11.9 across, 9.9-10.7 within). Pending: a frozen-frame (pause menu) comparison. Game-free, Linux, lavapipe (llvmpipe, LLVM 20): the fixture's modules (below) run as the fallback runs them, one workgroup a dispatch so that the records' order is the workgroups', then drawn into two RGBA32F targets (a smooth and a flat input, back faces culled): ``` tri-pos-first soa vs aos: triangles equal, images equal tri-pos-first indexed vs aos: triangles equal, images equal (18,812 triangles; 64,007 px lit) tri-pos-later-prim soa vs aos: triangles equal, images equal tri-pos-later-prim indexed vs aos: triangles equal, images equal (a per-primitive line: indexed is soa here) tri-wide soa vs aos: triangles equal, images equal (27-word records) tri-wide indexed vs aos: triangles equal, images equal tri-task soa vs aos: triangles equal, images equal (a task stage before it) tri-task indexed vs aos: triangles equal, images equal tri-wide, 6,000 workgroups, past the area (fit 621,378): soa draws aos's 621,315 records exactly; indexed draws 621,267 slots, its 188,946 triangles the first of aos's 189,741, nothing read past what was written ``` ### Flags and settings One new switch, off by default. | Flag | What it is | Before this PR | After this PR | Effect | Helps / doesn't help / untested | Recommendation | |---|---|---|---|---|---|---| | `RX_VK_MESH_FB_LAYOUT=aos`, `soa` or `indexed` | How the compute fallback's mesh stage stores its vertices | not there (`aos`) | unset (`aos`) | `soa`: line by line; `indexed`: each vertex once, triangles as indices | helps on a Steam Deck in Kokiri at handheld (mesh dispatches -50%); untested elsewhere | off until the native, Field and picture checks are in | ## Testing **Tested:** - Steam Deck OLED: Kokiri Forest at handheld settings, `aos`, `soa` and `indexed`, two runs each (Measurements). - `python -I -B tests/run_all.py` on Linux, before the last rebase: 571 passed, 16 failed, 24 skipped; the 16 fail the same on `main` on this PC (`test_options_budget_once`, `test_readback_late`, `test_vk_pgo`, `test_bake_portability`, `test_crash_dialog`, `test_walk_key`, `test_missing_nup`, `test_appimage_player_build`, `test_lift_cache`, `test_env_on`, `test_lift_cleanup_error`, `test_pipeline_tour`, `test_make_game_cache`, `test_linux_preflight`, `test_mods_size_table`, `test_vk_occlusion_queries`). After it, the 81 checks that read the changed files, once with g++ 13 and once with clang 23: 77 passed, and the 4 that failed (`test_bake_portability`, `test_crash_dialog`, `test_pipeline_tour`, `test_vk_occlusion_queries`) fail the same on current `main`. - `test_vk_occlusion_queries` fails on `main` before it reaches `draw()` (the replay's recording-end barrier, its line 809), so its run does not test this PR's change to its draw-command pattern. Checked apart instead: with that one replay check left out, the check passes on this branch with the new pattern, and fails with the old one (`occ.issue() with a way out before its draw command`). - New: `tests/test_mesh_fallback_layout.py` (the fixture `tests/mesh_fallback_layout_ir.cpp` through the real SPIR-V back end: every module spirv-val clean, SPIR-V 1.6, the vertex stages Vulkan 1.3 too; unset translates byte for byte as `aos`; and the model `tests/mesh_fallback_layout_model.cpp`: random meshlets, up to 256 vertices and primitives, counts past the most, index bytes past the vertices, degenerate triangles, and past the area); `tests/test_mesh_fallback_draw.py` (the fills, the draw of each layout, the index binding it leaves). Updated: `test_fallback_buffer_cleanup`, `test_program_record_head`, `test_mesh_fallback`, `test_vk_occlusion_queries`. With g++ 13 and clang 23; the back end is compiled as C++23 (C++2b where the compiler has no `-std=c++23`), as the bake's builds do. - The fixture's `aos` modules are byte for byte main's (twelve modules: six shapes, mesh and vertex stage). - Mutations caught: indices rotated (the provoking vertex), workgroup-local indices (the index base), the indexed draw counting slots that did not fit (the overflow rule), slots of the corners only, `soa`'s vertex stage line stride: by the model, and by the lavapipe run of the real modules. - Whole-file syntax checks of the changed files for Linux and Windows (llvm-mingw): the same warnings as `main`. **Not tested:** native settings and Hyrule Field on the Deck; unprofiled fps; a pixel-exact picture comparison; any other GPU. **Must be tested:** - Steam Deck, native settings and Hyrule Field: `aos`, `soa`, `indexed`, expecting the mesh dispatch time to fall as at handheld; plain fps arms without the profiling switches; and a frozen-frame (pause menu) screenshot of each arm at the same spot, expecting identical pictures (by hand). - Linux, AMD RX 5000 series (RDNA 1, which takes the compute fallback), Mesa: Kokiri Forest and Hyrule Field with `RX_VK_MESH_FB_LAYOUT=indexed` and without it, expecting the same picture (by hand: same-spot screenshots). ## Risks **What could break:** with `soa` or `indexed`, the fallback's terrain and vegetation (wrong or missing triangles, wrong flat-shaded values) on GPUs using the compute fallback. Unset, nothing: the translation and the commands are main's. Past the records area, `indexed` can draw a few triangles fewer than `aos` (a workgroup with more vertices than corners takes more slots); the area is the same 64 MB. **Turning it off:** unset `RX_VK_MESH_FB_LAYOUT` (the default). **Compatibility:** nothing changes for a player who does not set it. Setting it, or changing it, translates the programs again on the next start (the program cache's stamp holds the switch), as `RX_VK_MESH_NOCULL` does. Also: in `indexed`, a workgroup's padding triangles (its slots past its corners, which have no area) shift `gl_PrimitiveID` for the triangles after them; that only matters to a fragment program that reads the primitive ID. `RX_VK_MESH_FB_LAYOUT=aos` set explicitly translates as unset does, but its text still changes the cache's stamp, so it translates again once. And past the records area, `indexed` can draw a few triangles fewer than `aos` (a workgroup with more vertices than corners takes more slots). ### Works with / conflicts with - **#707** (mesh count draws, a records area that grows on demand): it takes header words 8-10 (the census address and the area's words) and a 16-word header; this PR leaves words 8-10 unused and puts its indexed command at 11-15, so the headers fit together. But #707's `fit` is read at run time from the area word, while here `fit` is a compile-time constant (`MeshFallbackFit`) in three places: `soa`'s line stride in the mesh and vertex stages, the indexed epilogue, and the host's index-line offset (`MeshFallbackIndexWord` in `fallback_draw`); all three would take the area's word instead. #707's count draws, a command per record, would need a `VkDrawIndexedIndirectCommand` per record for an indexed program. - **#703 / #708** (macOS: each triangle's last vertex first, through a static rotation list): they draw the fallback with `vkCmdDrawIndexedIndirect` at the same draw site. With `indexed` the rotation {2,0,1} would be written into the generated indices instead of a static list; with `aos` and `soa` their list works as it is. - **#783** (batching a pass's fallback draws on MoltenVK): each member keeps its own header; no conflict expected. Whichever lands second adapts to the other; I can do that rebase for any of them. ## Checklist ### Everyone - [x] One topic. - [x] The title is a Conventional Commit (`type(scope): description`) of the status line's type. - [x] Why gives the reason plainly and links the issues that fit (the tracker searched). - [x] Measurements has a verdict, or one line saying why there is nothing to measure. - [x] Testing names what was tested, what wasn't, and what must still be tested, where and how. - [x] Rebased on current `main`. - [x] `python -I -B tests/run_all.py` passes (any failure that also fails on `main` is named). - [x] No game files, dumps, build output or generated files. - [x] `third_party/` untouched (a new folder with a `PROVENANCE.md` instead). - [x] The license is OK: GPL-3.0-or-later, nothing incompatible and nothing decompiled. ### When it applies - [x] AI assistance recorded with `Assisted-by:` / `Co-Authored-By:` trailers. - [ ] Numbers from the same PC, scene and settings, at least 2 interleaved runs each, the median reported. (Kokiri handheld done, two runs each, the mean; native and the Field pending) - [x] Switches documented (docs/development.md; the README for a menu row) and shipped at their recommended value. - [ ] Screenshots of the same spot, captioned like the Measurements columns, for a change that shows on screen. (frozen-frame comparison pending) - [ ] A file in `docs/tested/` for each new measured run (format in `docs/tested-configurations.md`). (with the remaining runs) - [x] A check under `tests/` that reads changed source updated in this PR, not weakened. - [x] Host code builds on both Windows (llvm-mingw) and Linux, with UTF-8 paths. - [x] Docs updated.

zeldaootr added 4 commits

2026-11-15 20:05:23 +00:00

test(vk): the mesh fallback's record layouts through the real SPIR-V back end, and a model of what each draws 61a8d7109d

`tests/mesh_fallback_layout_ir.cpp` emits the compute fallback's mesh stage (a program that writes its stage buffer as a game's
does) and its generated vertex stage for each `RX_VK_MESH_FB_LAYOUT` value and the shapes the fallback's programs take:
the position first or not, a per-primitive line, a 27-word record, a task stage, lines and points.
`tests/test_mesh_fallback_layout.py` runs spirv-val on every module, checks that the switch unset translates byte for
byte as `aos` and that `soa` changes every program, and runs `tests/mesh_fallback_layout_model.cpp`: random meshlets
through a model of each layout's writes and its draw's reads, which must give the meshlets' own primitives in order, the
same corner first, and past the area the first workgroups that fit. The back end is compiled as C++23 (C++2b on older
compilers), as the bake's does: `common/assert.h` needs it.

Fails until the switch exists: `RuntimeInfo` has no `mesh_fallback_layout`.

Assisted-by: Claude

perf(vk): RX_VK_MESH_FB_LAYOUT=soa stores the mesh fallback's records line after line (off by default) 6762329adf

The compute fallback's mesh dispatches are most of its cost on the Steam Deck (86% of 17-20 ms a frame in Kokiri Forest
at handheld settings), and they write a record a primitive corner, its 22-27 words together: neighbouring threads write
words a record apart, so every store of a workgroup touches as many cache lines as it has threads.

With `RX_VK_MESH_FB_LAYOUT=soa` the area is cut into the record's lines (`MeshFallbackFit` words each) and the threads
take the workgroup's records in turn: line l of record r at H + l * fit + r, so neighbouring threads write neighbouring
words, and the generated vertex stage reads its record's words a line apart. The same records, count and draw; no host
change. Unset or `aos`, the translation is byte for byte as before. The switch is a name under `RX_VK_MESH`, which the
program cache's stamp holds: a start with it translates again, a start without it keeps its caches.

`RuntimeInfo::mesh_fallback_layout` carries it to the mesh stage and its vertex stage; `mesh_rule::fb_layout` reads it.

Assisted-by: Claude

test(vk): the mesh fallback's indexed layout, its header, its draw and the index binding it leaves 0cc3ec3cc6

For `RX_VK_MESH_FB_LAYOUT=indexed`:

- `tests/test_mesh_fallback_layout.py` and its fixture emit the indexed layout too: it changes the triangle programs with
  no per-primitive lines (their workgroups count their vertices) and leaves the others as `soa`. The model
  (`mesh_fallback_layout_model.cpp`) writes each vertex once and the corners as indices from the same allocator, slots
  past the corners holding a degenerate triangle of the workgroup's first vertex: its draw must give the workgroups'
  triangles in order with corner 0 first, read only words written, and past the area draw the records' first triangles.
- `tests/test_mesh_fallback_draw.py` runs `fallback_header_fills`, `bind_index` and `fallback_draw`, cut from
  `vk_draw.cpp`, on recorded Vulkan calls: the records layouts keep main's four fills and plain indirect draw; the indexed
  one adds its command's fills without overlap, binds the program's index line and draws the indexed command, and the
  game's next indexed draw binds its own buffer again.
- `fallback_buffer_cleanup_test.cpp`: the indexed layout's 16-word header and index usage, the pair still shared;
  `test_program_record_head.py`: the record head keeps `fallback_indexed`; `test_mesh_fallback.py` and
  `test_vk_occlusion_queries.py` follow the draw into `fallback_draw`.

Fails until the indexed layout exists.

Assisted-by: Claude

perf(vk): RX_VK_MESH_FB_LAYOUT=indexed writes each fallback vertex once and draws its triangles indexed (off by default)

CI / Conventional Commit title (pull_request) Blocked by required conditions

Details

CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Blocked by required conditions

Details

CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Blocked by required conditions

Details

CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Blocked by required conditions

Details

CI / Workflow lint (actionlint) (pull_request) Blocked by required conditions

Details

Guide screenshots / Guide images and local UI source review (pull_request) Blocked by required conditions

Details

Third-party parts / As their notes say (pull_request) Blocked by required conditions

Details

Third-party parts / The same as upstream (pull_request) Blocked by required conditions

Details

PR description / Description follows the template (pull_request_target) Successful in 4s

Details

Website / Website on Cloudflare (pull_request_target) Has been skipped

Details

787025d627

The mesh fallback wrote a record a primitive corner: each vertex of a meshlet 2 to 6 times over, the 22-27 words of each
read back by the vertex stage (Kokiri Forest on a Steam Deck: 1,037,124 records, 53.2 MB written and read a frame, at
least 1.9 to 3.0 corners a vertex on the four hot programs). With `indexed`, a triangle program with no per-primitive
lines (`MeshFallbackIndexed`; other programs as `soa`) writes each vertex its triangles use once, as `soa`'s lines, and
its corners as 32-bit indices, in corner order (the same provoking vertex and winding), drawn by
`vkCmdDrawIndexedIndirect`.

- Allocation: one allocator, as the records'. A workgroup takes slots, its corners or, when its vertices (1 + its
largest index, counted in workgroup memory) are more, those rounded up to whole triangles; its vertices go to its first
slots of the vertex lines and its indices to its slots of the index line, slots past its corners holding its first slot
(no area). Taking only grows, so the slots that fit are the first taken: every index drawn is written and names a vertex
written, and past the area the draw is the first workgroups', as the records' is.
- Header (16 words in this layout): words 0-7 as before (the plain draw's count stays 0), the
`VkDrawIndexedIndirectCommand` at words 11-15, its index count the slots drawn. The allocator and the dispatch words do
not move, so task programs translate the same in every layout.
- Host: `Program::fallback_indexed` (kept in the program cache's record head); `fallback_header_fills` adds the indexed
command's fills; `fallback_draw` binds the program's index line through `bind_index` (so the next indexed draw binds
its own again) and draws it indexed. The records buffer is an index buffer too, and a recording thread's pair keeps the
16-word header for every program, so they still share it. Unset, the fills and the draw are main's.

Assisted-by: Claude

BeanIssues added the

kind

performance

area/renderer

labels

2026-11-15 20:05:28 +00:00

BeanIssues added the

risk

medium

label

2026-11-15 20:27:07 +00:00

CI / Conventional Commit title (pull_request) Blocked by required conditions

Details

CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Blocked by required conditions

Details

CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Blocked by required conditions

Details

CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Blocked by required conditions

Details

CI / Workflow lint (actionlint) (pull_request) Blocked by required conditions

Details

Guide screenshots / Guide images and local UI source review (pull_request) Blocked by required conditions

Details

Third-party parts / As their notes say (pull_request) Blocked by required conditions

Details

Third-party parts / The same as upstream (pull_request) Blocked by required conditions

Details

PR description / Description follows the template (pull_request_target) Successful in 4s

Details

Website / Website on Cloudflare (pull_request_target) Has been skipped

Details

This pull request has changes conflicting with the target branch.

gpu/vk/vk_draw.cpp
tests/test_fallback_buffer_cleanup.py

Some workflows are waiting to be reviewed.

View command line instructions

Manual merge helper

Use this merge commit message when completing the merge manually.

Merge commit title

WIP: perf(vk): the mesh fallback's records line by line, or each vertex once and drawn indexed (RX_VK_MESH_FB_LAYOUT, off by default) (#906)

Merge commit body

Reviewed-on: https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/906

WIP: perf(vk): the mesh fallback's records line by line, or each vertex once and drawn indexed (RX_VK_MESH_FB_LAYOUT, off by default) (#906)

Reviewed-on: https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/906