fix(vk): use the mesh fallback for mesh-only draws on Windows RDNA 2 #1650

Merged
HumanBean merged 2 commits from futondestroyer/Longshot:fix/rdna2-mesh-only-1239 into main 2026-11-18 20:40:53 +00:00
Member

Type: fix · Risk: medium · Tested on: Windows 11, RTX 4070 and game-free LLVM-MinGW routing test · Needs: Windows, Radeon RX 6800 XT

TL;DR

RC7 can lose the GPU near Saria on a Windows RX 6800 XT. This is a stable-release blocker, so we're putting the targeted fix up for review and affected-hardware testing as soon as possible. The default now takes the compute path for mesh-only draws on Windows Navi 21–23, and the game-free regression fails on main and passes here.

Breakdown

  • Who it helps: Players on Windows with affected Radeon RX 6000 GPUs.
  • What was wrong: The draw near Saria has a mesh stage without a task stage, so it missed the existing driver workaround.
  • What changes: The existing Windows Navi 21–23 policy also routes mesh-only draws to compute.
  • What does not change: Other devices keep their existing defaults, and players can still select Native or explicitly turn off the workaround.
  • How it was checked: A game-free routing and cache-key test and a Windows NVIDIA gameplay smoke test.
  • What is still open: The exact patched build needs the default-settings retest on a Windows RX 6800 XT.
  • Risk: Medium. More draws use compute on the affected GPUs, which can affect frame time or VRAM.

Why

Stable-release blocker. RC7 hung twice on a Windows RX 6800 XT in ordinary Kokiri Forest gameplay. This is a targeted change so the next candidate can get the affected-hardware retest promptly.

Credit to GrimVisage for the diagnosis and validation that made this fix possible. GrimVisage reproduced the device loss twice, used GPU checkpoints to isolate the exact mesh-only draw, ruled out removing that draw as a viable workaround because it dropped foliage, and then ran the same scene for 20 minutes with the compute fallback and the foliage intact. This patch implements the policy change GrimVisage proposed. Reproduction and diagnosis

How to reproduce

Seen on: Windows, Radeon RX 6800 XT (Navi 21), Longshot v3.0.0-rc.7.

Settings and switches: Default mesh settings. The reporter used checkpoints for diagnosis and forced compute only for the workaround control.

Steps: Reach Kokiri Forest and hold the camera on Saria on her stump for several minutes.

Expected: The game continues drawing the scene, including foliage.

Actual: The driver hangs during the mesh-only draw and the game loses the device.

What changed

  • Device setup and renderer routing extend the existing Windows Navi 21–23 fallback to mesh-only programs. The existing device list and override order still govern it.
  • The program cache key encodes the new routing choice so a mesh-only program compiled on the native path does not get reused on these devices.
  • The game-free fixture compiles the production device setup, program selector, and cache-key expression. It covers the three named Navi chips, unaffected controls, and opt-outs.
  • The switch table, startup log and development guide describe which draws use the fallback.

Measurements

Verdict: The game-free check now routes the repro's mesh-only program to compute on the affected policy. Affected-GPU stability and performance are pending.

Before: main @ da91e770 After: this PR @ e89db34e
Windows RDNA 2 mesh-only routing fixture Fails, expected compute and got native Passes, including the native controls and cache separation

Flags and settings

RX_VK_TASK_MESH_FALLBACK retains its existing default device set and opt-out.

Flag What it is Before this PR After this PR Effect Helps / doesn't help / untested Recommendation
RX_VK_TASK_MESH_FALLBACK Existing driver workaround Task + mesh on by default on Windows Navi 21–23 Mesh-only and task + mesh on the same devices Use compute for both draw types on those devices Mesh-only crash workaround reproduced by GrimVisage; exact-head retest pending Default on for the affected devices

Testing

Tested: Windows 11, NVIDIA RTX 4070, driver 621.01. The patched renderer built and linked with LLVM-MinGW. With a compatible existing host (49089983, renderer API 33), a disposable Kokiri save and full startup precompile disabled, it reached Kokiri Forest, rendered 180 presents and exited cleanly. The RTX 4070 kept native mesh routing. The focused device-policy test passed with Vulkan headers. The switch-table check reported the same nine pre-existing findings on exact main and the patch. The new game-free fixture failed on main with RDNA 2 mesh-only draw: expected 1, got 0 and passed on the patch with LLVM-MinGW.

Not testable by hand: Local PC has an RTX 4070, so it cannot reproduce the AMD driver timeout. The source-based fixture checks the exact routing decision and cache separation.

Not tested: RX 6800 XT runtime behavior, affected-GPU frame time and VRAM, and Linux gameplay on this branch.

Must be tested:

  • Windows RX 6800 XT, by hand. Run the exact patched build with default mesh settings at the same Kokiri Forest camera position for at least 20 minutes. Verify foliage, no device loss, and the startup log's fallback choice. Repeat with a warm program cache and record frame time and VRAM if available.

Risks

What could break: Extra compute-path draws can affect visuals, frame time or VRAM on the affected AMD devices. The regression fixture covers the choice of route. The linked hardware test is needed for driver and image behavior.

Turning it off: Set RX_VK_TASK_MESH_FALLBACK=0 or select Native via RX_VK_MESH_SHADERS=native.

Compatibility: The renderer's updated translator stamp can refresh existing program caches once. The mesh route also has its own cache-key bit. The host-renderer interface and save format are unchanged.

Issues

See #1239. Leave the issue open for the exact-head retest.

Checklist

Everyone

  • One topic.
  • The title is a Conventional Commit (type(scope): description) of the status line's type.
  • Why gives the reason plainly and links the issues that fit (the tracker searched).
  • Tested on my own PC where it runs (in the game, for a change the game runs), as Testing's Tested says; what can't be run by hand is in Testing's Not testable by hand, with why.
  • Measurements has a verdict, or one line saying why there is nothing to measure.
  • Nothing gets worse that I know of: any cost (speed, latency, memory, a setup that worked before) is measured and in Risks. Affected AMD GPU comparison pending.
  • Testing names what was tested, what wasn't, and what must still be tested, where and how.
  • Rebased on current main. Pinned base da91e770; check for newer main before ready.
  • python -I -B tests/run_all.py passes (any failure that also fails on main is named). Full suite pending.
  • No game files, dumps, build output or generated files.
  • third_party/ untouched (a new folder with a PROVENANCE.md instead).
  • The license is OK: GPL-3.0-or-later, nothing incompatible and nothing decompiled.

When it applies

  • AI assistance recorded with Assisted-by: / Co-Authored-By: trailers.
  • Switches documented (docs/development.md; the README for a menu row) and shipped at their recommended value.
  • A game-free test added for the fix, failing on main and passing here.
  • Docs updated.
**Type:** fix · **Risk:** medium · **Tested on:** Windows 11, RTX 4070 and game-free LLVM-MinGW routing test · **Needs:** Windows, Radeon RX 6800 XT ## TL;DR RC7 can lose the GPU near Saria on a Windows RX 6800 XT. This is a stable-release blocker, so we're putting the targeted fix up for review and affected-hardware testing as soon as possible. The default now takes the compute path for mesh-only draws on Windows Navi 21–23, and the game-free regression fails on main and passes here. ## Breakdown - **Who it helps:** Players on Windows with affected Radeon RX 6000 GPUs. - **What was wrong:** The draw near Saria has a mesh stage without a task stage, so it missed the existing driver workaround. - **What changes:** The existing Windows Navi 21–23 policy also routes mesh-only draws to compute. - **What does not change:** Other devices keep their existing defaults, and players can still select Native or explicitly turn off the workaround. - **How it was checked:** A game-free routing and cache-key test and a Windows NVIDIA gameplay smoke test. - **What is still open:** The exact patched build needs the default-settings retest on a Windows RX 6800 XT. - **Risk:** Medium. More draws use compute on the affected GPUs, which can affect frame time or VRAM. ## Why **Stable-release blocker.** RC7 hung twice on a Windows RX 6800 XT in ordinary Kokiri Forest gameplay. This is a targeted change so the next candidate can get the affected-hardware retest promptly. **Credit to GrimVisage for the diagnosis and validation that made this fix possible.** GrimVisage reproduced the device loss twice, used GPU checkpoints to isolate the exact mesh-only draw, ruled out removing that draw as a viable workaround because it dropped foliage, and then ran the same scene for 20 minutes with the compute fallback and the foliage intact. This patch implements the policy change GrimVisage proposed. [Reproduction and diagnosis](https://git.magicbean.dev/MagicBeanDev/Longshot/issues/1239#issuecomment-47895) ## How to reproduce **Seen on:** Windows, Radeon RX 6800 XT (Navi 21), Longshot v3.0.0-rc.7. **Settings and switches:** Default mesh settings. The reporter used checkpoints for diagnosis and forced compute only for the workaround control. **Steps:** Reach Kokiri Forest and hold the camera on Saria on her stump for several minutes. **Expected:** The game continues drawing the scene, including foliage. **Actual:** The driver hangs during the mesh-only draw and the game loses the device. ## What changed - Device setup and renderer routing extend the existing Windows Navi 21–23 fallback to mesh-only programs. The existing device list and override order still govern it. - The program cache key encodes the new routing choice so a mesh-only program compiled on the native path does not get reused on these devices. - The game-free fixture compiles the production device setup, program selector, and cache-key expression. It covers the three named Navi chips, unaffected controls, and opt-outs. - The switch table, startup log and development guide describe which draws use the fallback. ## Measurements **Verdict:** The game-free check now routes the repro's mesh-only program to compute on the affected policy. Affected-GPU stability and performance are pending. | | Before: main @ `da91e770` | After: this PR @ `e89db34e` | |---|---|---| | Windows RDNA 2 mesh-only routing fixture | Fails, expected compute and got native | Passes, including the native controls and cache separation | ### Flags and settings `RX_VK_TASK_MESH_FALLBACK` retains its existing default device set and opt-out. | Flag | What it is | Before this PR | After this PR | Effect | Helps / doesn't help / untested | Recommendation | |---|---|---|---|---|---|---| | `RX_VK_TASK_MESH_FALLBACK` | Existing driver workaround | Task + mesh on by default on Windows Navi 21–23 | Mesh-only and task + mesh on the same devices | Use compute for both draw types on those devices | Mesh-only crash workaround reproduced by GrimVisage; exact-head retest pending | Default on for the affected devices | ## Testing **Tested:** Windows 11, NVIDIA RTX 4070, driver 621.01. The patched renderer built and linked with LLVM-MinGW. With a compatible existing host (`49089983`, renderer API 33), a disposable Kokiri save and full startup precompile disabled, it reached Kokiri Forest, rendered 180 presents and exited cleanly. The RTX 4070 kept native mesh routing. The focused device-policy test passed with Vulkan headers. The switch-table check reported the same nine pre-existing findings on exact main and the patch. The new game-free fixture failed on main with `RDNA 2 mesh-only draw: expected 1, got 0` and passed on the patch with LLVM-MinGW. **Not testable by hand:** Local PC has an RTX 4070, so it cannot reproduce the AMD driver timeout. The source-based fixture checks the exact routing decision and cache separation. **Not tested:** RX 6800 XT runtime behavior, affected-GPU frame time and VRAM, and Linux gameplay on this branch. **Must be tested:** - Windows RX 6800 XT, by hand. Run the exact patched build with default mesh settings at the same Kokiri Forest camera position for at least 20 minutes. Verify foliage, no device loss, and the startup log's fallback choice. Repeat with a warm program cache and record frame time and VRAM if available. ## Risks **What could break:** Extra compute-path draws can affect visuals, frame time or VRAM on the affected AMD devices. The regression fixture covers the choice of route. The linked hardware test is needed for driver and image behavior. **Turning it off:** Set `RX_VK_TASK_MESH_FALLBACK=0` or select Native via `RX_VK_MESH_SHADERS=native`. **Compatibility:** The renderer's updated translator stamp can refresh existing program caches once. The mesh route also has its own cache-key bit. The host-renderer interface and save format are unchanged. ## Issues See https://git.magicbean.dev/MagicBeanDev/Longshot/issues/1239. Leave the issue open for the exact-head retest. ## Checklist ### Everyone - [x] One topic. - [x] The title is a Conventional Commit (`type(scope): description`) of the status line's type. - [x] Why gives the reason plainly and links the issues that fit (the tracker searched). - [x] Tested on my own PC where it runs (in the game, for a change the game runs), as Testing's Tested says; what can't be run by hand is in Testing's Not testable by hand, with why. - [x] Measurements has a verdict, or one line saying why there is nothing to measure. - [ ] Nothing gets worse that I know of: any cost (speed, latency, memory, a setup that worked before) is measured and in Risks. Affected AMD GPU comparison pending. - [x] Testing names what was tested, what wasn't, and what must still be tested, where and how. - [ ] Rebased on current `main`. Pinned base `da91e770`; check for newer main before ready. - [ ] `python -I -B tests/run_all.py` passes (any failure that also fails on `main` is named). Full suite pending. - [x] No game files, dumps, build output or generated files. - [x] `third_party/` untouched (a new folder with a `PROVENANCE.md` instead). - [x] The license is OK: GPL-3.0-or-later, nothing incompatible and nothing decompiled. ### When it applies - [x] AI assistance recorded with `Assisted-by:` / `Co-Authored-By:` trailers. - [x] Switches documented (docs/development.md; the README for a menu row) and shipped at their recommended value. - [x] A game-free test added for the fix, failing on `main` and passing here. - [x] Docs updated.
GrimVisage isolated the mesh-only draw on an RX 6800 XT in issue #1239.

Assisted-by: Codex
fix(vk): use the mesh fallback for mesh-only draws on Windows RDNA 2
Some checks failed
CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Has been skipped
CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Has been skipped
CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Has been skipped
Third-party parts / The same as upstream (pull_request) Has been skipped
Website / Website on Cloudflare (pull_request) Has been skipped
CI / Conventional Commit title (pull_request) Successful in 10s
Website / Website on Cloudflare (pull_request_target) Has been skipped
Guide screenshots / Guide images and local UI source review (pull_request) Failing after 18s
CI / Workflow lint (actionlint) (pull_request) Successful in 18s
Third-party parts / As their notes say (pull_request) Successful in 21s
PR description / Description follows the template (pull_request_target) Failing after 6s
e89db34ead
GrimVisage reproduced the RX 6800 XT device loss twice, isolated the mesh-only draw with checkpoints, and proved the compute path retained foliage and ran for 20 minutes. The policy change follows that diagnosis. See #1239 (comment).

Assisted-by: Codex
Author
Member

Tested in game: Windows 11, NVIDIA RTX 4070, driver 621.01: renderer from e89db34e on a compatible host, disposable Kokiri save, on-demand shader loading; 180 presents in Kokiri Forest with native mesh routing and a clean exit. Windows RX 6800 XT verification is required.

Tested in game: Windows 11, NVIDIA RTX 4070, driver 621.01: renderer from e89db34e on a compatible host, disposable Kokiri save, on-demand shader loading; 180 presents in Kokiri Forest with native mesh routing and a clean exit. Windows RX 6800 XT verification is required.
Member

Possibly related, for context:

  • issue #1239 Game: Longshot closed unexpectedly while you were playing — Exact RX 6800 XT device-loss report this PR targets.
  • issue #583 RX 6600 (Navi 23, Windows): device lost in Dodongo's Cavern with native mesh shaders, compute fixes it — Related Windows RDNA 2 mesh-shader device-loss report; prior task+mesh fallback policy.
  • PR #219 fix(vk): task + mesh compute fallback by default for Navi 23 on Windows, one rule for the IDs — Merged precedent: introduced Windows Navi 21–23 task+mesh fallback policy; this extends it to mesh-only draws.
Possibly related, for context: - issue #1239 Game: Longshot closed unexpectedly while you were playing — Exact RX 6800 XT device-loss report this PR targets. - issue #583 RX 6600 (Navi 23, Windows): device lost in Dodongo's Cavern with native mesh shaders, compute fixes it — Related Windows RDNA 2 mesh-shader device-loss report; prior task+mesh fallback policy. - PR #219 fix(vk): task + mesh compute fallback by default for Navi 23 on Windows, one rule for the IDs — Merged precedent: introduced Windows Navi 21–23 task+mesh fallback policy; this extends it to mesh-only draws.
HumanBean changed title from WIP: fix(vk): use the mesh fallback for mesh-only draws on Windows RDNA 2 to fix(vk): use the mesh fallback for mesh-only draws on Windows RDNA 2 2026-11-18 20:40:43 +00:00
HumanBean deleted branch fix/rdna2-mesh-only-1239 2026-11-18 20:40:54 +00:00
Contributor

Retest of the Must be tested item: Windows RX 6800 XT, v3.0.0 from the launcher (tag 09cbdd97, which contains this PR as 3228cb5f; the log's first line is longshot v3.0.0 (source 87cb36aa), where source is the source fingerprint from nx2/version.py, not a commit).

PC: RX 6800 XT 16 GB, AMD proprietary driver 26.10.1, Ryzen 5 5600X, 16 GB, Windows 11 build 26300. No RX_ switches set (the old longshot_env.txt was deleted), no mesh setting in the options file, no overlays. Output 2048x1152, FSR 4, Quality preset 0. Stood still facing Saria on her stump in Kokiri Forest, the camera position where rc.7 hung 2-4 minutes in.

Startup log, both runs:

vk: mesh-only and task + mesh programs use the compute fallback (RX_VK_TASK_MESH_FALLBACK=0: native)
vk: mesh shaders: Automatic, from the default: mesh-only and task + mesh draws on the compute path
Run Program cache Time at Saria Device lost / GPU hang Foliage fps (last ~9 min) GPU time a frame VRAM by the driver
1 cold (preparation 139 s, precompile 340 s) ~10 min no / no leaf cards drawn median 55.9, low 47.3 median 11.1 ms (9.4-13.0) up to 8,835 of 15,569 MB allowed
2 warm (precompile 4.5 s) ~10 min no / no leaf cards drawn median 59.1, low 44.0 median 10.9 ms (9.3-14.0) up to 9,411 of 15,569 MB allowed

Both runs closed cleanly (display: window closed, exiting). fps and GPU time are from the last 107 log samples of each run (vk: <n> presents, <fps> fps and vk: GPU time lines). The log has no frame-time median or p99 line.

Not as asked: run 1 was about 10 minutes at Saria, not 20 (the tester shortened both runs). That is still well past the 2-4 minutes the hang took on rc.7. There's no before/after on the same build here: on rc.7 the default hung, and forcing RX_VK_MESH_SHADERS=compute ran 20 minutes clean.

Run 1 at Saria, near its start and near its end (leaf cards drawn):

run 1, start
run 1, end

The full logs are kept here; I can send any part of them. #1239's reporter has a different PC, so I've asked them there to check v3.0.0.

Retest of the Must be tested item: Windows RX 6800 XT, v3.0.0 from the launcher (tag `09cbdd97`, which contains this PR as `3228cb5f`; the log's first line is `longshot v3.0.0 (source 87cb36aa)`, where source is the source fingerprint from `nx2/version.py`, not a commit). PC: RX 6800 XT 16 GB, AMD proprietary driver 26.10.1, Ryzen 5 5600X, 16 GB, Windows 11 build 26300. No `RX_` switches set (the old `longshot_env.txt` was deleted), no mesh setting in the options file, no overlays. Output 2048x1152, FSR 4, Quality preset 0. Stood still facing Saria on her stump in Kokiri Forest, the camera position where rc.7 hung 2-4 minutes in. Startup log, both runs: ``` vk: mesh-only and task + mesh programs use the compute fallback (RX_VK_TASK_MESH_FALLBACK=0: native) vk: mesh shaders: Automatic, from the default: mesh-only and task + mesh draws on the compute path ``` | Run | Program cache | Time at Saria | Device lost / GPU hang | Foliage | fps (last ~9 min) | GPU time a frame | VRAM by the driver | |---|---|---|---|---|---|---|---| | 1 | cold (preparation 139 s, precompile 340 s) | ~10 min | no / no | leaf cards drawn | median 55.9, low 47.3 | median 11.1 ms (9.4-13.0) | up to 8,835 of 15,569 MB allowed | | 2 | warm (precompile 4.5 s) | ~10 min | no / no | leaf cards drawn | median 59.1, low 44.0 | median 10.9 ms (9.3-14.0) | up to 9,411 of 15,569 MB allowed | Both runs closed cleanly (`display: window closed, exiting`). fps and GPU time are from the last 107 log samples of each run (`vk: <n> presents, <fps> fps` and `vk: GPU time` lines). The log has no frame-time median or p99 line. Not as asked: run 1 was about 10 minutes at Saria, not 20 (the tester shortened both runs). That is still well past the 2-4 minutes the hang took on rc.7. There's no before/after on the same build here: on rc.7 the default hung, and forcing `RX_VK_MESH_SHADERS=compute` ran 20 minutes clean. Run 1 at Saria, near its start and near its end (leaf cards drawn): ![run 1, start](https://git.magicbean.dev/attachments/636e2f61-7d37-4c09-bec7-fe76611dea9a) ![run 1, end](https://git.magicbean.dev/attachments/6d70bf80-2564-4da3-93b1-d71c7ed34b5d) The full logs are kept here; I can send any part of them. #1239's reporter has a different PC, so I've asked them there to check v3.0.0.
Sign in to join this conversation.
No reviewers
No milestone
No project
No assignees
3 participants
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
MagicBeanDev/Longshot!1650
No description provided.