fix(vk): size the mesh index map in whole 32-primitive blocks #2312

Merged

Kolkka merged 1 commit from fix/mesh-index-map-blocks into main

2026-11-11 16:37:50 +00:00

Kolkka commented

2026-11-11 14:27:16 +00:00

(Migrated from github.com)

Flashing triangles over the foliage and ground in Kokiri Forest: a stray vertex from mesh programs with 126 primitives, which are most of OoT's.

What changed: a mesh program writes the index bytes of every 32-primitive block, including lanes past max_primitives (a count of 0 wraps min(count - 1, lane) onto every lane). The index map was sized to max_primitives, so those writes were clamped onto its last byte, the last primitive's last corner. DefineMesh now sizes the map like the attributes, in whole blocks of 32 primitives. nxnative found the same thing ("primitive index bytes for every primitive block"). tests/test_mesh_point_topology.py checks the new size.

Tested: Steam Deck LCD, Mesa RADV, Kokiri Forest. I compared 150–200 screenshots per run, each against the previous one: no flashes remain, and the biggest changes are Navi moving and the lighting fading in. tests/run_all.py: no new failures against main.

🤖 Generated with Claude Code

Flashing triangles over the foliage and ground in Kokiri Forest: a stray vertex from mesh programs with 126 primitives, which are most of OoT's. **What changed:** a mesh program writes the index bytes of every 32-primitive block, including lanes past `max_primitives` (a count of 0 wraps `min(count - 1, lane)` onto every lane). The index map was sized to `max_primitives`, so those writes were clamped onto its last byte, the last primitive's last corner. `DefineMesh` now sizes the map like the attributes, in whole blocks of 32 primitives. nxnative found the same thing ("primitive index bytes for every primitive block"). `tests/test_mesh_point_topology.py` checks the new size. **Tested:** Steam Deck LCD, Mesa RADV, Kokiri Forest. I compared 150–200 screenshots per run, each against the previous one: no flashes remain, and the biggest changes are Navi moving and the lighting fading in. `tests/run_all.py`: no new failures against `main`. <img width="1269" height="712" alt="image" src="https://github.com/user-attachments/assets/9b159590-ee90-4b68-bf39-ef0373c56ad4" /> 🤖 Generated with [Claude Code](https://claude.com/claude-code)

hckrman101 (Migrated from github.com) approved these changes

2026-11-11 16:29:02 +00:00

hckrman101 (Migrated from github.com) left a comment

Looks right to me. The map is laid out linearly (byte 4 + corners * primitive + corner, both in the epilogue and in the fallback's record copy), so a program that writes index bytes for a whole 32-lane primitive block reaches up to 4 + corners * DivCeil(max_primitives, 32) * 32 - 1. With the old size, EmitIsbeWriteMapU8's Clamped(address, mesh_map_bytes) folded every out-of-range byte onto the last one, so for 126 primitives the 6 bytes of lanes 126/127 raced with primitive 125's last corner. That fits the flashing-triangle symptom well, and sizing the map in whole blocks matches how mesh_attribute_words already sizes the primitive attributes.

Other things I checked:

Readers only look at primitives < count (bounded by max_primitives), so the extra bytes are scratch and never read back. The prologue zeroing walks the new size too.
mesh_map is defined before the if (mesh_fallback) return;, so the compute fallback path gets the fix as well, not only native mesh shaders.
Cost: at most 31 * 3 extra workgroup words (~372 bytes) per mesh workgroup, which is nothing next to the attribute region.
max_primitives == 0 still gives 4 bytes, same as before.
The test change tracks the new expression, nothing weakened.

No overlap with #2311 (vk_draw.cpp) or #2362 (vk_main.cpp / vk_menu.cpp); they touch different files and this fix helps whichever mesh path ends up the default on RADV.

CI: only the title check has run; the build and game-free regression workflows are waiting for approval to run on this fork, so those should go green before merging.

Looks right to me. The map is laid out linearly (byte `4 + corners * primitive + corner`, both in the epilogue and in the fallback's record copy), so a program that writes index bytes for a whole 32-lane primitive block reaches up to `4 + corners * DivCeil(max_primitives, 32) * 32 - 1`. With the old size, `EmitIsbeWriteMapU8`'s `Clamped(address, mesh_map_bytes)` folded every out-of-range byte onto the last one, so for 126 primitives the 6 bytes of lanes 126/127 raced with primitive 125's last corner. That fits the flashing-triangle symptom well, and sizing the map in whole blocks matches how `mesh_attribute_words` already sizes the primitive attributes. Other things I checked: - Readers only look at primitives `< count` (bounded by `max_primitives`), so the extra bytes are scratch and never read back. The prologue zeroing walks the new size too. - `mesh_map` is defined before the `if (mesh_fallback) return;`, so the compute fallback path gets the fix as well, not only native mesh shaders. - Cost: at most 31 * 3 extra workgroup words (~372 bytes) per mesh workgroup, which is nothing next to the attribute region. - `max_primitives == 0` still gives 4 bytes, same as before. - The test change tracks the new expression, nothing weakened. No overlap with #2311 (`vk_draw.cpp`) or #2362 (`vk_main.cpp` / `vk_menu.cpp`); they touch different files and this fix helps whichever mesh path ends up the default on RADV. CI: only the title check has run; the build and game-free regression workflows are waiting for approval to run on this fork, so those should go green before merging.