vk: inline the global-load helper in task and mesh stages (the floating trees, #994) #1334

Merged

blushOFF merged 1 commit from fix/task-mesh-load-inline into main

2026-11-09 22:29:34 +00:00

blushOFF commented

2026-11-09 22:00:53 +00:00

(Migrated from github.com)

Fixes the wall of floating trees from #994 at the spots I could test (Kakariko's entrance, the Hyrule Castle cliff), on an RTX 3070 / Windows 11.

Cause. The tree task stage 3cb7682790143836 computes each visible cell's tree count as min(offs[e+1] - offs[e], 256) from the placement file. Traced per load (details in #994): the offs[e+1] load went to file + (cells[e] + 9 + e) * 4 instead of file + (n + 9 + e) * 4, the result of the previous call to the global-load helper where the one before it belongs. The address is outside every buffer, the load returns 0, and every cell is drawn with 256 trees, the following cells' trees placed in it. The IR and the emitted SPIR-V are both right; the compiled shader is not, while the helper is DontInline.

Change. The global-load helper is emitted Inline for the task and mesh stages. The compute stages keep DontInline: their culling programs have hundreds of global loads, and inlining was what made NVIDIA's compiler take 41 s (the comment above define_load). EDEN_NX2_GM_INLINE=0 brings the old behaviour back; as an EDEN_ switch it is part of the program cache's stamp.

Tested (RTX 3070, Windows 11, NVIDIA driver 610.88 as Vulkan reports it):

Kakariko's entrance, same scripted camera: the pair's mesh stage went from 1184 threads / 2082 primitives to 160 / 78 (the file holds 81 trees for the frame's 10 cells), 0 global-memory misses instead of 3200 a frame, no tree left in the sky. Also with an empty program cache.
Hyrule Castle cliff view, with and without the switch: the trees above the cliff are gone with it.
A teleport tour of 15 areas (Kokiri Forest to Zora's Fountain): no device lost, no pipeline refused.

Not tested: AMD, Intel, Linux; the open-field view toward Death Mountain; frame rate in a proper A/B (a first single run at Kakariko gave about 7 fps more, which I would not rely on). @Nitnek offered to run it at the castle cliff on another driver version.

No game data in this change.

Fixes the wall of floating trees from #994 at the spots I could test (Kakariko's entrance, the Hyrule Castle cliff), on an RTX 3070 / Windows 11. **Cause.** The tree task stage `3cb7682790143836` computes each visible cell's tree count as `min(offs[e+1] - offs[e], 256)` from the placement file. Traced per load (details in #994): the `offs[e+1]` load went to `file + (cells[e] + 9 + e) * 4` instead of `file + (n + 9 + e) * 4`, the result of the previous call to the global-load helper where the one before it belongs. The address is outside every buffer, the load returns 0, and every cell is drawn with 256 trees, the following cells' trees placed in it. The IR and the emitted SPIR-V are both right; the compiled shader is not, while the helper is `DontInline`. **Change.** The global-load helper is emitted `Inline` for the task and mesh stages. The compute stages keep `DontInline`: their culling programs have hundreds of global loads, and inlining was what made NVIDIA's compiler take 41 s (the comment above `define_load`). `EDEN_NX2_GM_INLINE=0` brings the old behaviour back; as an `EDEN_` switch it is part of the program cache's stamp. **Tested** (RTX 3070, Windows 11, NVIDIA driver 610.88 as Vulkan reports it): - Kakariko's entrance, same scripted camera: the pair's mesh stage went from 1184 threads / 2082 primitives to 160 / 78 (the file holds 81 trees for the frame's 10 cells), 0 global-memory misses instead of 3200 a frame, no tree left in the sky. Also with an empty program cache. - Hyrule Castle cliff view, with and without the switch: the trees above the cliff are gone with it. - A teleport tour of 15 areas (Kokiri Forest to Zora's Fountain): no device lost, no pipeline refused. **Not tested:** AMD, Intel, Linux; the open-field view toward Death Mountain; frame rate in a proper A/B (a first single run at Kakariko gave about 7 fps more, which I would not rely on). @Nitnek offered to run it at the castle cliff on another driver version. No game data in this change.

❤️ 2

Nitnek commented

2026-11-09 22:28:16 +00:00

(Migrated from github.com)

Tested at the Hyrule Castle cliff view, as offered: RTX 3070 8 GB, Windows 11, main 7dbdcdc + this PR. The setup:

a scripted run loads the Hyrule Castle autosave and turns the camera to the cliff;
both runs reach the same view;
EDEN_NX2_GM_INLINE=0 is the "before".

	`3cb7682790143836` mesh threads	its fragments	on screen
`EDEN_NX2_GM_INLINE=0`	39392	2.1M	the wall of spruces in the sky over the cliff
this PR	6944	267k	a few spruces standing on the hill, no wall

The pair's task groups barely change (164 vs 208 from b612216258832b9c's list); what each group emits drops ~6x. That matches the per-cell count no longer coming out at the 256 clamp.

No device lost, no failed programs. Frame rate is the same in both runs (~50 fps over the load and the turn).
The floating-tree wall is gone at this spot too, a second spot on a different driver than yours.

Edit: also checked by hand in Kakariko with this PR and #1338 on the same build (the latter for the foliage that went bare at mid distance, #1151). Looking from the village entrance toward Hyrule Field, the render is as expected: no floating trees, and the spruces keep their needles.

Tested at the Hyrule Castle cliff view, as offered: RTX 3070 8 GB, Windows 11, main 7dbdcdc + this PR. The setup: - a scripted run loads the Hyrule Castle autosave and turns the camera to the cliff; - both runs reach the same view; - `EDEN_NX2_GM_INLINE=0` is the "before". | | `3cb7682790143836` mesh threads | its fragments | on screen | |---|---|---|---| | `EDEN_NX2_GM_INLINE=0` | 39392 | 2.1M | the wall of spruces in the sky over the cliff | | this PR | 6944 | 267k | a few spruces standing on the hill, no wall | The pair's task groups barely change (164 vs 208 from `b612216258832b9c`'s list); what each group emits drops ~6x. That matches the per-cell count no longer coming out at the 256 clamp. - No device lost, no failed programs. Frame rate is the same in both runs (~50 fps over the load and the turn). - The floating-tree wall is gone at this spot too, a second spot on a different driver than yours. **Edit:** also checked by hand in Kakariko with this PR and #1338 on the same build (the latter for the foliage that went bare at mid distance, #1151). Looking from the village entrance toward Hyrule Field, the render is as expected: no floating trees, and the spruces keep their needles.

shayderrr commented

2026-11-09 22:29:28 +00:00

(Migrated from github.com)

Thank you!

msxdan commented

2026-11-09 22:38:38 +00:00

(Migrated from github.com)

Tested on Linux: the wall of trees is gone, same numbers with and without the switch as on Windows

System: CachyOS (kernel 7.3), NVIDIA RTX 5070, driver 615.78.08; Ryzen 9 9900X.
Build: main at 7dbdcdc plus this PR's commit (f78c175) and #1338's (9928d74), built from source; the A/B below is this PR's switch alone on that one build. Headless runs from one save, same frame after the load, RX_VK_DRAW_STATS.
Where: Hyrule Field, two saves: one in the western canyon looking toward the castle, and the view from #1306 (the castle wall across the field).

The tree pair 3cb7682790143836 / 4f0c0b40ab3a32cd, canyon save:

	mesh threads	primitives	fragments (`3cb76827`)
`EDEN_NX2_GM_INLINE=0`	32608	64878	5.51M
this PR (default)	9760	3712	0.31M

With the PR the conifer wall on the plateau is gone and the far mountains show. From the #1306 spot the view now matches the reporter's reference picture: the castle, the hills, single trees.
No device lost in 5 cold starts. Preparing 38809 programs with an empty program cache: 33.7 s with the PR, 32.9 s with the switch at 0.
60 fps in both (frame-limited, so no frame rate comparison).

Not tested: AMD, a long play session, Kakariko and the castle cliff.

No game data attached.

**Tested on Linux: the wall of trees is gone, same numbers with and without the switch as on Windows** - **System:** CachyOS (kernel 7.3), NVIDIA RTX 5070, driver 615.78.08; Ryzen 9 9900X. - **Build:** `main` at 7dbdcdc plus this PR's commit (f78c175) and #1338's (9928d74), built from source; the A/B below is this PR's switch alone on that one build. Headless runs from one save, same frame after the load, `RX_VK_DRAW_STATS`. - **Where:** Hyrule Field, two saves: one in the western canyon looking toward the castle, and the view from #1306 (the castle wall across the field). The tree pair `3cb7682790143836` / `4f0c0b40ab3a32cd`, canyon save: | | mesh threads | primitives | fragments (`3cb76827`) | |---|---|---|---| | `EDEN_NX2_GM_INLINE=0` | 32608 | 64878 | 5.51M | | this PR (default) | 9760 | 3712 | 0.31M | - With the PR the conifer wall on the plateau is gone and the far mountains show. From the #1306 spot the view now matches the reporter's reference picture: the castle, the hills, single trees. - No device lost in 5 cold starts. Preparing 38809 programs with an empty program cache: 33.7 s with the PR, 32.9 s with the switch at 0. - 60 fps in both (frame-limited, so no frame rate comparison). Not tested: AMD, a long play session, Kakariko and the castle cliff. No game data attached.

renannpaiva commented

2026-11-10 11:26:38 +00:00

(Migrated from github.com)

Ryzen 7 9800x3d RTX 5070 32gb

Everything is perfect where there used to be issues; however, it gives the impression that it’s on the verge of freezing whenever VRAM usage hits a high percentage. The game feels like it’s running twice as fast, haha! I’ll play a bit more so I can provide some precise feedback!

@NotRexed

<img width="1919" height="1079" alt="image" src="https://github.com/user-attachments/assets/c3a6605c-2f13-4b61-9ef2-a14d4e0f94f7" /> Ryzen 7 9800x3d RTX 5070 32gb Everything is perfect where there used to be issues; however, it gives the impression that it’s on the verge of freezing whenever VRAM usage hits a high percentage. The game feels like it’s running twice as fast, haha! I’ll play a bit more so I can provide some precise feedback! @NotRexed