vk: a mesh program's stage buffer starts at zero (foliage shadow lines on AMD) #475

Merged

ZurielAtr merged 1 commit from mesh-stage-buffer-zero into main

2026-11-08 08:19:57 +00:00

ZurielAtr commented

2026-11-08 04:37:23 +00:00

(Migrated from github.com)

Why

MeshEpilogue hands Vulkan every vertex up to the program's maximum and the index bytes of every primitive below its count, from the map and attribute regions in workgroup memory. Workgroup memory holds whatever the last workgroup on that unit left, so a vertex or an index byte the game's program leaves unwritten is a leftover from another meshlet. On NVIDIA that memory comes out clean in practice; on AMD it does not.

The change

EmitMeshPrologue zeroes both regions in mesh programs, before the primitive count and its barrier. An unwritten primitive is then (0,0,0), with no area. The stores are straight-line and clamped into their arrays, because the prologue's entry block is a phi predecessor and must not branch. RX_VK_MESH_NO_ZERO=1 restores the old behaviour. It changes the shader stamp, so the program cache starts again once.

Result

The triangles across the whole shadow map are gone (same spot, before and after dumps). Not fixed by this: inside the foliage patch the same map still has long streaks on this GPU, so a striped pattern around Link remains. It goes away with RX_VK_SKIP_PROG=53509044401d78d2,02ee95fbf9e0050d,b0b3b40244c71ec7, which points at those two programs; that is a separate problem, reported in #302.

Tested

In game on the RX 7700S, on device-lost-fix with this change: the shadow map dumps above; all programs translate as before (same 57 failures, none of them mesh programs).
tests/test_mesh_stage_buffer_zero.py (new, wired into source-regressions.yml) checks the prologue: both regions zeroed before the count and the barrier, no branch, clamped stores, the switch. It fails on main without this change.

🤖 Generated with Claude Code

On an AMD laptop (Radeon RX 7700S 8 GB, Windows 11, Adrenalin 26.10.1) the ground, the water and Link showed dark lines in a square around Link in Hyrule Field, changing every frame. They were in the shadow mask (`3427a1dbefd5e02c`), and an `RX_VK_DUMP_SAMPLED` dump of the shadow map it samples (2400 x 2400 D32F, drawn only by the foliage's mesh programs `02ee95fbf9e0050d` and `b0b3b40244c71ec7`) showed **hundreds of long thin triangles across the whole map**. ### Why `MeshEpilogue` hands Vulkan every vertex up to the program's maximum and the index bytes of every primitive below its count, from the map and attribute regions in workgroup memory. Workgroup memory holds whatever the last workgroup on that unit left, so a vertex or an index byte the game's program leaves unwritten is a leftover from another meshlet. On NVIDIA that memory comes out clean in practice; on AMD it does not. ### The change `EmitMeshPrologue` zeroes both regions in mesh programs, before the primitive count and its barrier. An unwritten primitive is then (0,0,0), with no area. The stores are straight-line and clamped into their arrays, because the prologue's entry block is a phi predecessor and must not branch. `RX_VK_MESH_NO_ZERO=1` restores the old behaviour. It changes the shader stamp, so the program cache starts again once. ### Result The triangles across the whole shadow map are gone (same spot, before and after dumps). **Not fixed by this:** inside the foliage patch the same map still has long streaks on this GPU, so a striped pattern around Link remains. It goes away with `RX_VK_SKIP_PROG=53509044401d78d2,02ee95fbf9e0050d,b0b3b40244c71ec7`, which points at those two programs; that is a separate problem, reported in #302. ### Tested - In game on the RX 7700S, on `device-lost-fix` with this change: the shadow map dumps above; all programs translate as before (same 57 failures, none of them mesh programs). - `tests/test_mesh_stage_buffer_zero.py` (new, wired into `source-regressions.yml`) checks the prologue: both regions zeroed before the count and the barrier, no branch, clamped stores, the switch. It fails on `main` without this change. 🤖 Generated with [Claude Code](https://claude.com/claude-code)

Commenting is not possible because the repository is archived.

No reviewers