MagicBeanDev/Hookshot

Archived

Watch

Fork 0

Steam Deck (RADV VANGOGH): process exits leaving the starting room through the door, no device lost, log ends in the finalized-texture upload path #862

Closed

opened 2026-11-08 23:29:07 +00:00 by RufflezAU · 6 comments

RufflezAU commented

2026-11-08 23:29:07 +00:00

(Migrated from github.com)

Steam Deck OLED, RADV VANGOGH (14 GB unified), Linux AppImage toolchain.

The game loads a save, runs the starting room fine, and the process exits when I walk out of the room through the door. No CRASH: line, no device lost, no OOM — and Steam reports exit code 0. The log stops in the finalized-texture upload path:

vk: texture #1387 ... the GPU wrote its storage (submission 18141): uploaded again
vk: upload over a drawn image: #1387 ... at present 2592 ... finalized, 0 KB dirty in 0 ranges, a GPU buffer wrote it
vk: texture ... re-uploaded (its memory was written; flags 0x8): a GPU write into its storage ... over what the GPU wrote (finalized since)
vk: slow upload 630.5 ms: #400 ... 768 of 768 KB again
vk: NVN2 program 4728b3dc6c9df19c (stage 7) lowered with extended registers (350 REGMAP words)

Settings at the time: 50 % resolution, Custom preset, 6x frame generation, RX_VK_VRAM_MB=1200, RX_VK_TEX_MAX_MB=128, RX_VK_TEX_MAX_DIM=1024, RX_VK_EVICT_MIN_MS=1500, RX_VK_CHECKPOINTS=1. VRAM was only 3840 of 6027 MB, so this is not the paging case from #519.

What I tried:

RX_VK_KEEP_FINALIZED_MB=0 — same exit at the door.
Headless (HOS_HEADLESS=1) with the same save runs 7 minutes without the exit, so it needs the window/present path or it is a race.

The same save and route are stable on the older base (the PR #448 branch plus the retired-import giveback), so this looks like a regression between that base and current main.

Build: main at 58af2c0 plus a 10-line local patch that counts the texture eviction's age in milliseconds instead of presents (branch deck/steamdeck on RufflezAU/Hookshot, gpu/vk/vk_texture.cpp only). I can test a vanilla build or any branch on the Deck, and I have the full log.

Is this the same thing as #646 / #708, or a different one? Those report a driver/device lost; here there is neither, the process just goes away with status 0.

**Steam Deck OLED, RADV VANGOGH (14 GB unified), Linux AppImage toolchain.** The game loads a save, runs the starting room fine, and the process exits when I walk out of the room through the door. No `CRASH:` line, no `device lost`, no OOM — and Steam reports **exit code 0**. The log stops in the finalized-texture upload path: ``` vk: texture #1387 ... the GPU wrote its storage (submission 18141): uploaded again vk: upload over a drawn image: #1387 ... at present 2592 ... finalized, 0 KB dirty in 0 ranges, a GPU buffer wrote it vk: texture ... re-uploaded (its memory was written; flags 0x8): a GPU write into its storage ... over what the GPU wrote (finalized since) vk: slow upload 630.5 ms: #400 ... 768 of 768 KB again vk: NVN2 program 4728b3dc6c9df19c (stage 7) lowered with extended registers (350 REGMAP words) ``` Settings at the time: 50 % resolution, Custom preset, 6x frame generation, `RX_VK_VRAM_MB=1200`, `RX_VK_TEX_MAX_MB=128`, `RX_VK_TEX_MAX_DIM=1024`, `RX_VK_EVICT_MIN_MS=1500`, `RX_VK_CHECKPOINTS=1`. VRAM was only 3840 of 6027 MB, so this is not the paging case from #519. What I tried: - `RX_VK_KEEP_FINALIZED_MB=0` — same exit at the door. - Headless (`HOS_HEADLESS=1`) with the same save runs 7 minutes without the exit, so it needs the window/present path or it is a race. The same save and route are stable on the older base (the PR #448 branch plus the retired-import giveback), so this looks like a regression between that base and current `main`. Build: `main` at `58af2c0` plus a 10-line local patch that counts the texture eviction's age in milliseconds instead of presents (branch `deck/steamdeck` on RufflezAU/Hookshot, `gpu/vk/vk_texture.cpp` only). I can test a vanilla build or any branch on the Deck, and I have the full log. Is this the same thing as #646 / #708, or a different one? Those report a driver/device lost; here there is neither, the process just goes away with status 0.

RufflezAU commented

2026-11-08 23:45:33 +00:00

(Migrated from github.com)

Update: I built vanilla latest main (15ec28d, no local patches except the libstdc++ include fix the AppImage needs) and it does not reach the door at all — it crashes at startup, headless, inside the RADV driver:

vk: mesh 12b394ab0742e14d: threads 32 max v 128 p 128 payload 288, 4 vertex lines: 1c 1d 1e 1f; 0 primitive
...
CRASH: exception 0xc0000005 (write 0x40, guest 0xfffff00000000040) at /usr/lib/x86_64-linux-gnu/GL/default/lib/libvulkan_radeon.so+0x19cc52
CRASH: host backtrace: hookshot+0x62658b9 hookshot+0x1bfb0511 libc.so.6+0x42770
       libvulkan_radeon.so+0x19cc52 libvulkan_radeon.so+0x18214c libvulkan_radeon.so+0x182654
       libvulkan_radeon.so+0x1828fa libvulkan_radeon.so+0x182b1b
       hookshot_vk.so+0x3176bb hookshot_vk.so+0x390868 hookshot_vk.so+0x3174e1 hookshot_vk.so+0x46b0f4

About 37 s in, right after the mesh programs are set up. Same machine (Steam Deck OLED, RADV VANGOGH, Mesa 26.x). So on this GPU:

main at 58af2c0 + the millisecond eviction patch: loads and runs the starting room, exits when leaving the room through the door (no device lost, exit 0);
main at 15ec28d, vanilla: segfault inside libvulkan_radeon.so during mesh setup at startup;
the older base (PR #448 branch + giveback): stable through the door, but slower and with the lighting/texture flicker.

The mesh shader path on VANGOGH looks like the common thread. Is there a switch to take the mesh path out on this GPU (the RX_VK_NO_MESH fallback was reverted in #640), or should I bisect between 58af2c0 and 15ec28d to find which of the 38 commits introduced the driver crash? I can build and test any commit on the Deck.

Update: I built **vanilla latest main** (`15ec28d`, no local patches except the libstdc++ include fix the AppImage needs) and it does not reach the door at all — it crashes at startup, headless, **inside the RADV driver**: ``` vk: mesh 12b394ab0742e14d: threads 32 max v 128 p 128 payload 288, 4 vertex lines: 1c 1d 1e 1f; 0 primitive ... CRASH: exception 0xc0000005 (write 0x40, guest 0xfffff00000000040) at /usr/lib/x86_64-linux-gnu/GL/default/lib/libvulkan_radeon.so+0x19cc52 CRASH: host backtrace: hookshot+0x62658b9 hookshot+0x1bfb0511 libc.so.6+0x42770 libvulkan_radeon.so+0x19cc52 libvulkan_radeon.so+0x18214c libvulkan_radeon.so+0x182654 libvulkan_radeon.so+0x1828fa libvulkan_radeon.so+0x182b1b hookshot_vk.so+0x3176bb hookshot_vk.so+0x390868 hookshot_vk.so+0x3174e1 hookshot_vk.so+0x46b0f4 ``` About 37 s in, right after the mesh programs are set up. Same machine (Steam Deck OLED, RADV VANGOGH, Mesa 26.x). So on this GPU: - `main` at `58af2c0` + the millisecond eviction patch: loads and runs the starting room, exits when leaving the room through the door (no device lost, exit 0); - `main` at `15ec28d`, vanilla: segfault inside `libvulkan_radeon.so` during mesh setup at startup; - the older base (PR #448 branch + giveback): stable through the door, but slower and with the lighting/texture flicker. The mesh shader path on VANGOGH looks like the common thread. Is there a switch to take the mesh path out on this GPU (the `RX_VK_NO_MESH` fallback was reverted in #640), or should I bisect between `58af2c0` and `15ec28d` to find which of the 38 commits introduced the driver crash? I can build and test any commit on the Deck.

RufflezAU commented

2026-11-09 00:09:08 +00:00

(Migrated from github.com)

More data on the startup crash with 15ec28d, and a diagnostic that points at the mesh path:

Vanilla 15ec28d, headless: CRASH: exception 0xc0000005 (write 0x40, guest 0xfffff00000000040) at libvulkan_radeon.so+0x19cc52, ~37 s in, right after a run of vk: mesh ... program translations.
The same build with RX_VK_NO_MESH=1 (task/mesh draws left out) ran the full 70 s test with zero CRASH lines. The mesh programs are still translated; only their draws are skipped. So the fault needs a mesh draw, not just the pipeline.

That makes me think this is the mesh path on VANGOGH, and a compute fallback is the way around it. I am building PR #792 (hckrman101:reland-640-mesh-stages) and will run it with RX_VK_FORCE_MESH_FALLBACK=1 on the Deck — if the fallback is stable here, it gets us the newer fixes (#705, #772) at the cost of the fallback.

For reference, the older base (4d7d2f1 + udmabuf + giveback) uses mesh shaders on this same GPU and does not fault, so something between 4d7d2f1 and 15ec28d changed how mesh draws are made. If a bisect would help you more than the fallback test, tell me and I will run it on the Deck.

More data on the startup crash with `15ec28d`, and a diagnostic that points at the mesh path: - Vanilla `15ec28d`, headless: `CRASH: exception 0xc0000005 (write 0x40, guest 0xfffff00000000040) at libvulkan_radeon.so+0x19cc52`, ~37 s in, right after a run of `vk: mesh ...` program translations. - The **same build with `RX_VK_NO_MESH=1`** (task/mesh draws left out) ran the full 70 s test with **zero CRASH lines**. The mesh programs are still translated; only their draws are skipped. So the fault needs a mesh *draw*, not just the pipeline. That makes me think this is the mesh path on VANGOGH, and a compute fallback is the way around it. I am building PR #792 (`hckrman101:reland-640-mesh-stages`) and will run it with `RX_VK_FORCE_MESH_FALLBACK=1` on the Deck — if the fallback is stable here, it gets us the newer fixes (#705, #772) at the cost of the fallback. For reference, the older base (`4d7d2f1` + udmabuf + giveback) uses mesh shaders on this same GPU and does not fault, so something between `4d7d2f1` and `15ec28d` changed how mesh draws are made. If a bisect would help you more than the fallback test, tell me and I will run it on the Deck.

DistantThunder commented

2026-11-09 00:12:42 +00:00

(Migrated from github.com)

Don't know if it's the same issue but on Desktop 7900 XTX:

Mesa RADV : 26.2.6

[...]
[hos host] vk: AMD Radeon RX 7900 XTX (RADV NAVI31), Vulkan 1.4, driver 0x6802006; mesh shaders yes, host memory import yes, pipeline libraries yes
[...]
[hos host] vk: NVN2 program 642659cde652fa7f (stage 1) does not lower at +0x140: unsupported native opcode (native op 0x905, 00017905 7fffffff 0021f000 000e2200)
[hos host] vk: NVN2 program 8d6b3ea391d0841e (stage 1) does not lower at +0xb0: unsupported native opcode (native op 0x905, 000b7905 7fffffff 0021f000 000fe200)
[hos host] vk: NVN2 program fd6730f10031b8d0 (stage 1) does not lower at +0x1c0: unsupported native opcode (native op 0x905, 00017905 7fffffff 0021f000 000e2400)
[hos host] vk: NVN2 program a09bdc185d560567 (stage 5) does not lower at +0x80: native global load uniform addressing mode (native op 0x981, ff087981 00002004 0c1eed00 000ea200)
[hos host] vk: NVN2 program a7a54ff1af0cfa19 (stage 5) does not lower at +0x10: native global load uniform addressing mode (native op 0x981, ff017981 00002c04 0c1ee900 000ea800)
[hos host] vk: NVN2 program f24a3ebf686232fb (stage 5) does not lower at +0x7200: native program has no unconditional EXIT (native op 0xffffffff, 00000000 00000000 00000000 00000000)
[hos host] vk: prepared 38809 shader programs (45658 new, 165 failed, 0 stage libraries; 10042 compute variants, 744 programs with texture variants, 65 at the limit) in 48.6 s
[hos host] CRASH: exception 0xc0000005 (write 0x40, guest 0xfffff00000000040) at /usr/lib/libvulkan_radeon.so+0x15b67d
[hos host] CRASH: host backtrace: hookshot+0x62658b9 hookshot+0x1bfb0511 libc.so.6+0x3e6f0 libvulkan_radeon.so+0x15b67d libvulkan_radeon.so+0x14084d libvulkan_radeon.so+0x140d5a libvulkan_radeon.so+0x140ffa libvulkan_radeon.so+0x14123b hookshot_vk.so+0x30e32d hookshot_vk.so+0x4631e3 libc.so.6+0x98042 libc.so.6+0x12072c

Don't know if it's the same issue but on Desktop 7900 XTX: Mesa RADV : 26.2.6 ``` [...] [hos host] vk: AMD Radeon RX 7900 XTX (RADV NAVI31), Vulkan 1.4, driver 0x6802006; mesh shaders yes, host memory import yes, pipeline libraries yes [...] [hos host] vk: NVN2 program 642659cde652fa7f (stage 1) does not lower at +0x140: unsupported native opcode (native op 0x905, 00017905 7fffffff 0021f000 000e2200) [hos host] vk: NVN2 program 8d6b3ea391d0841e (stage 1) does not lower at +0xb0: unsupported native opcode (native op 0x905, 000b7905 7fffffff 0021f000 000fe200) [hos host] vk: NVN2 program fd6730f10031b8d0 (stage 1) does not lower at +0x1c0: unsupported native opcode (native op 0x905, 00017905 7fffffff 0021f000 000e2400) [hos host] vk: NVN2 program a09bdc185d560567 (stage 5) does not lower at +0x80: native global load uniform addressing mode (native op 0x981, ff087981 00002004 0c1eed00 000ea200) [hos host] vk: NVN2 program a7a54ff1af0cfa19 (stage 5) does not lower at +0x10: native global load uniform addressing mode (native op 0x981, ff017981 00002c04 0c1ee900 000ea800) [hos host] vk: NVN2 program f24a3ebf686232fb (stage 5) does not lower at +0x7200: native program has no unconditional EXIT (native op 0xffffffff, 00000000 00000000 00000000 00000000) [hos host] vk: prepared 38809 shader programs (45658 new, 165 failed, 0 stage libraries; 10042 compute variants, 744 programs with texture variants, 65 at the limit) in 48.6 s [hos host] CRASH: exception 0xc0000005 (write 0x40, guest 0xfffff00000000040) at /usr/lib/libvulkan_radeon.so+0x15b67d [hos host] CRASH: host backtrace: hookshot+0x62658b9 hookshot+0x1bfb0511 libc.so.6+0x3e6f0 libvulkan_radeon.so+0x15b67d libvulkan_radeon.so+0x14084d libvulkan_radeon.so+0x140d5a libvulkan_radeon.so+0x140ffa libvulkan_radeon.so+0x14123b hookshot_vk.so+0x30e32d hookshot_vk.so+0x4631e3 libc.so.6+0x98042 libc.so.6+0x12072c ```

RufflezAU commented

2026-11-09 02:13:50 +00:00

(Migrated from github.com)

Bisect result: the startup fault is not tied to a single commit — it looks like a race in the driver during the prepare.

I bisected 58af2c0 (boots) … 15ec28d (crashed once at 37 s) on the Deck, cold program cache before every run, two runs per commit:

66ea810  no crash x2
c39a418  no crash x2
b533fa5  no crash x2
9607db0  no crash x2
731a056  no crash x2   (the #793 commit alone)
1355765  no crash x2
15ec28d  no crash x3   (rebuilt, same config)

So the endpoint itself does not reproduce reliably. What the runs do show is where the time goes:

vk: prepared 38809 shader programs (35797 new, 165 failed, 0 stage libraries;
    7918 compute variants, 588 programs with texture variants, 43 at the limit) in 166.0 s

The prepare takes 166 s on the Deck (38,809 programs), and the one crash happened 37 s into that prepare, in libvulkan_radeon.so while mesh/compute variants were being made. The other runs got through the prepare and then the process was killed by my 240 s test before it presented.

Two separate Deck problems, then:

the prepare is far too long here (166 s, 38k programs) — every start pays it until the cache is warm;
an intermittent RADV segfault during that prepare, which I cannot pin to a commit.

If it helps, I can run a longer soak of one build (say 20 cold starts) to get a crash rate, or test with RX_VK_PRECOMPILE=0 to see whether taking the prepare out removes the fault.

Bisect result: the startup fault is **not tied to a single commit** — it looks like a race in the driver during the prepare. I bisected `58af2c0` (boots) … `15ec28d` (crashed once at 37 s) on the Deck, cold program cache before every run, two runs per commit: ``` 66ea810 no crash x2 c39a418 no crash x2 b533fa5 no crash x2 9607db0 no crash x2 731a056 no crash x2 (the #793 commit alone) 1355765 no crash x2 15ec28d no crash x3 (rebuilt, same config) ``` So the endpoint itself does not reproduce reliably. What the runs do show is where the time goes: ``` vk: prepared 38809 shader programs (35797 new, 165 failed, 0 stage libraries; 7918 compute variants, 588 programs with texture variants, 43 at the limit) in 166.0 s ``` The prepare takes **166 s** on the Deck (38,809 programs), and the one crash happened 37 s into that prepare, in `libvulkan_radeon.so` while mesh/compute variants were being made. The other runs got through the prepare and then the process was killed by my 240 s test before it presented. Two separate Deck problems, then: 1. the prepare is far too long here (166 s, 38k programs) — every start pays it until the cache is warm; 2. an intermittent RADV segfault during that prepare, which I cannot pin to a commit. If it helps, I can run a longer soak of one build (say 20 cold starts) to get a crash rate, or test with `RX_VK_PRECOMPILE=0` to see whether taking the prepare out removes the fault.

RufflezAU commented

2026-11-09 03:51:29 +00:00

(Migrated from github.com)

One more data point that may help other Deck users: RX_VK_PRECOMPILE=0 avoids the startup fault and the 166 s prepare.

With precompile off, 15ec28d boots, loads a save and runs the starting room at 40 fps headless (GPU 26.5 ms, VRAM 3116/5867 MB), and it survives walking out into Kokiri Forest. No driver fault in that run. The cost is the expected one: a new shader compiles at first use, so there is a 112 ms submit spike when a material first appears, and the first entry to a new area built 889 pipelines in 120 s (that cache then persists, so later entries are fast).

So on this machine the choice is:

precompile on: 166 s prepare, intermittent libvulkan_radeon segfault during it;
precompile off: boots, but the first visit to each area stalls while its pipelines compile.

Separately, the newer build does survive the door but the GPU then hangs a few seconds into Kokiri Forest — that is #817 (hard recovery, markers in my comment there), so it is not the same fault as this one.

One more data point that may help other Deck users: **`RX_VK_PRECOMPILE=0` avoids the startup fault and the 166 s prepare.** With precompile off, `15ec28d` boots, loads a save and runs the starting room at 40 fps headless (GPU 26.5 ms, VRAM 3116/5867 MB), and it survives walking out into Kokiri Forest. No driver fault in that run. The cost is the expected one: a new shader compiles at first use, so there is a **112 ms submit spike** when a material first appears, and the first entry to a new area built **889 pipelines in 120 s** (that cache then persists, so later entries are fast). So on this machine the choice is: - precompile on: 166 s prepare, intermittent `libvulkan_radeon` segfault during it; - precompile off: boots, but the first visit to each area stalls while its pipelines compile. Separately, the newer build does survive the door but the GPU then hangs a few seconds into Kokiri Forest — that is #817 (hard recovery, markers in my comment there), so it is not the same fault as this one.

NotRexed commented

2026-11-09 05:59:24 +00:00

(Migrated from github.com)

Tracked in #1000, follow there. That issue collects every report of Linux / Steam Deck GPU hangs, crashes, low fps and build failures, and this one is linked from it with its GPU, version and log details. If it still happens on v2.0.6 (https://github.com/NotRexed/Hookshot/releases/tag/v2.0.6; Update next to Play in the launcher), please add a comment there with your hookshot_log.txt.

Commenting is not possible because the repository is archived.