vk: mesh programs with control topology 1 draw points, with the point sprite coordinate (grass shadow mesh on AMD, #302) #482

Merged

ZurielAtr merged 1 commit from fix/mesh-point-topology into main

2026-11-08 09:44:11 +00:00

ZurielAtr commented

2026-11-08 05:20:02 +00:00

(Migrated from github.com)

What was wrong

NVN2's mesh control section (+0x5f) says 3 for triangles. Four mesh stages in OoT say 1 (ef6e1a5fde0988f0,
df6cf52500d013a4, ed01e3e8ff1c102b, 644dba7ae7b8bced). They draw the grass into the 2400 x 2400 shadow map that
follows Link, in programs 02ee95fbf9e0050d and b0b3b40244c71ec7 among others. mesh_registers_of passed that 1 on
as GL's 1, lines, but these programs draw points. Their native code:

each lane takes its compacted index popc(ballot & lanemask_lt) and writes one map byte, at 4 + i, with the value i;
the primitive count at map byte 3 is popc(ballot), the number of vertices;
the vertex lines are PointSize and the position (1b 1c 1d 1e 1f).

Read as lines, primitive p joined vertex 2p to 2p + 1, and the second half of the index bytes was never written.
The grass patch around Link came out in the shadow map as long thin streaks spanning the patch, and the shadow mask
drew them on everything near him: a flickering dark diagonal mesh on the ground, the water, rocks and Link.

Their fragment program then reads generic 5 (s, t, r, perspective), which the mesh program never writes. It samples
an alpha mask with it and discards: that is the hardware's point sprite coordinate, which gives each point its shape.
Left at (0, 0, 0, 1), every point was a filled square.

Change

vk_shader.cpp: control topology 1 maps to GL 0 (points); 3 stays 4 (triangles).
MeshInfo::Corners(): index bytes per primitive (points 1, lines 2, triangles 3). It is used for the map region's size,
the index builtin and the epilogue.
Points: OutputPoints, PrimitivePointIndicesEXT (one uint per primitive, stored as a scalar), one index byte at
4 + primitive. The lines path stays for a real GL 1.
RuntimeInfo::point_coord_generics: when the mesh program draws points, the generics its fragment program reads and
the mesh program never writes are gl_PointCoord (s, t, 0, 1). The perspective factor Eden's IPA applies to
generics cancels out as it does for the others.
tests/test_mesh_point_topology.py (game-free, wired into source-regressions.yml): evaluates the control-to-GL
mapping and Corners(), and checks that the translator accepts points, the SPIR-V side declares and stores them, and
the point sprite coordinate replaces only the unwritten generics of point mesh programs. It fails on main, and
without either half of the change.

This touches the translator, so the shader stamp changes and caches are rebuilt on the first start.

Tested

AMD Radeon RX 7700S 8 GB (laptop), Adrenalin 26.10.1, Windows 11, Hyrule Field, in the sun with grass around Link.

Before: a dark striped mesh in a large square around Link. It went away only with
RX_VK_SKIP_PROG=53509044401d78d2,02ee95fbf9e0050d,b0b3b40244c71ec7, which also drops the grass shadow.
After: no mesh, and the grass casts its shadow again. Full-resolution F9 dumps of the 2400 x 2400 map (sampled by the
shadow mask pass 3427a1dbefd5e02c) show the patch as separate rounded points instead of streaks; with points alone
and no point coordinate they were squares.
The two programs translate without errors, and the startup preparation has the same 57 failures as before.

The tested build also carried #475 (the mesh stage buffer zeroed in the prologue). This change does not depend on it:
with points, the epilogue reads only the index bytes the program writes. Not tested on NVIDIA.

Addresses the grass shadow mesh in #302. The crops and bushes that vanish when Link gets close are a separate problem
and are not fixed by this.

🤖 Generated with Claude Code

## What was wrong NVN2's mesh control section (`+0x5f`) says 3 for triangles. Four mesh stages in OoT say 1 (`ef6e1a5fde0988f0`, `df6cf52500d013a4`, `ed01e3e8ff1c102b`, `644dba7ae7b8bced`). They draw the grass into the 2400 x 2400 shadow map that follows Link, in programs `02ee95fbf9e0050d` and `b0b3b40244c71ec7` among others. `mesh_registers_of` passed that 1 on as GL's 1, lines, but these programs draw **points**. Their native code: - each lane takes its compacted index `popc(ballot & lanemask_lt)` and writes **one** map byte, at `4 + i`, with the value `i`; - the primitive count at map byte 3 is `popc(ballot)`, the number of vertices; - the vertex lines are PointSize and the position (`1b 1c 1d 1e 1f`). Read as lines, primitive `p` joined vertex `2p` to `2p + 1`, and the second half of the index bytes was never written. The grass patch around Link came out in the shadow map as long thin streaks spanning the patch, and the shadow mask drew them on everything near him: a flickering dark diagonal mesh on the ground, the water, rocks and Link. Their fragment program then reads generic 5 (`s, t, r`, perspective), which the mesh program never writes. It samples an alpha mask with it and discards: that is the hardware's point sprite coordinate, which gives each point its shape. Left at `(0, 0, 0, 1)`, every point was a filled square. ## Change - `vk_shader.cpp`: control topology 1 maps to GL 0 (points); 3 stays 4 (triangles). - `MeshInfo::Corners()`: index bytes per primitive (points 1, lines 2, triangles 3). It is used for the map region's size, the index builtin and the epilogue. - Points: `OutputPoints`, `PrimitivePointIndicesEXT` (one `uint` per primitive, stored as a scalar), one index byte at `4 + primitive`. The lines path stays for a real GL 1. - `RuntimeInfo::point_coord_generics`: when the mesh program draws points, the generics its fragment program reads and the mesh program never writes are `gl_PointCoord` (`s, t, 0, 1`). The perspective factor Eden's IPA applies to generics cancels out as it does for the others. - `tests/test_mesh_point_topology.py` (game-free, wired into `source-regressions.yml`): evaluates the control-to-GL mapping and `Corners()`, and checks that the translator accepts points, the SPIR-V side declares and stores them, and the point sprite coordinate replaces only the unwritten generics of point mesh programs. It fails on `main`, and without either half of the change. This touches the translator, so the shader stamp changes and caches are rebuilt on the first start. ## Tested AMD Radeon RX 7700S 8 GB (laptop), Adrenalin 26.10.1, Windows 11, Hyrule Field, in the sun with grass around Link. - Before: a dark striped mesh in a large square around Link. It went away only with `RX_VK_SKIP_PROG=53509044401d78d2,02ee95fbf9e0050d,b0b3b40244c71ec7`, which also drops the grass shadow. - After: no mesh, and the grass casts its shadow again. Full-resolution F9 dumps of the 2400 x 2400 map (sampled by the shadow mask pass `3427a1dbefd5e02c`) show the patch as separate rounded points instead of streaks; with points alone and no point coordinate they were squares. - The two programs translate without errors, and the startup preparation has the same 57 failures as before. The tested build also carried #475 (the mesh stage buffer zeroed in the prologue). This change does not depend on it: with points, the epilogue reads only the index bytes the program writes. Not tested on NVIDIA. Addresses the grass shadow mesh in #302. The crops and bushes that vanish when Link gets close are a separate problem and are not fixed by this. 🤖 Generated with [Claude Code](https://claude.com/claude-code)

NotRexed commented

2026-11-08 09:56:51 +00:00

(Migrated from github.com)