Add RX_VK_SKIP_CODE to avoid the Steam Deck GPU hang #1165

2026-11-09T15:53:03Z

hardicd commented

2026-11-09 15:53:03 +00:00

(Migrated from github.com)

Edited after review: the switch hides the models these programs draw (see my comment below), and both v2.0.7 test spots were inside the Great Deku Tree.

What

A new renderer switch, RX_VK_SKIP_CODE=<code key prefix,...>, leaves out the draws of every program that has a stage whose native code key starts with one of the prefixes. It is decided once per program (memoized by program key) and costs nothing while unset. RX_VK_SKIP_PROG already leaves out whole programs; this one targets a stage that many programs share.

gpu/vk/vk_draw.cpp: the switch, right after RX_VK_SKIP_PROG. It logs the first 32 programs it leaves out, with their code keys.
gpu/vk/skip_code.h: the prefix match (comma-separated, either case, only at the start of a stage's key).
tests/test_skip_code.py + .cpp: a game-free check of the match (picked up by tests/run_all.py).
readme.md: a troubleshooting row for the Deck, and the switch under Switches for testing.

Why

On the Steam Deck (RADV VANGOGH) the GPU hangs in Kokiri Forest and the Great Deku Tree: ring gfx_0.0.0 timeout, then a GPU reset (#817, tracked in #1000). Buffer markers read while the GPU was still stuck showed that every stalled draw belongs to a task + mesh program sharing one task shader, code 2c56dc1006942f74: two nested loops over a 2D grid (a culling pass), used by about 30 programs. On RDNA 2 some of these draws run for seconds (one took 9 s and then finished); past 10 s the kernel resets the ring. RX_VK_MESH_OPT=1 does not change it. RX_VK_NO_MESH=1 avoids it, but doors and foliage go missing (#1000).

RX_VK_SKIP_CODE=2c56dc1006942f74 leaves out only those programs, and with them the geometry they draw: key models go missing while it is set. Why that shader runs so long on RDNA 2 is still open; this is an opt-in workaround to get past the hang, not a fix.

Tested

Steam Deck OLED, SteamOS 3.9.5, RADV VANGOGH (Mesa 26.2), Game Mode, 720p handheld, with the same change applied to v2.0.4 and to v2.0.7 (RX_VK_LOW_SPEC=1, HOS_HANDHELD=1, RX_VK_MESH_OPT=1, the memory caps from #817):

v2.0.4 without it: a GPU hang within 30 s of loading a save in the Great Deku Tree. With it: no stall in several 4-minute runs there and 20 min in Kokiri Forest. I saw nothing missing in screenshots, but a player watching the Deck later saw key 3D models missing.
v2.0.7 with it: 0 device lost and 0 stalls in five runs at two spots inside the Great Deku Tree, an early room and B1 (fresh launch, 90 s camera turn, 2x frame generation): 36.1-37.2 fps game rate, 20-27 programs left out.
A second Deck confirmed it in #817 (RufflezAU: 5 min in Kokiri Forest without a device lost, where it was lost within seconds before).
On this branch (main 61d452d): the Linux Vulkan back end builds with the AppImage's toolchain, and python -I -B tests/run_all.py skip_code passes. I have not run main itself on the Deck.

The RX_VK_CHECKPOINTS watchdog that found the shader (it reads the AMD buffer markers while the GPU is still stuck, because the reset kills the process before device lost can read them) is a separate change; I can send it as its own PR.

Refs #817, #1000

🤖 Generated with Claude Code

_Edited after review: the switch hides the models these programs draw (see my comment below), and both v2.0.7 test spots were inside the Great Deku Tree._ **What** A new renderer switch, `RX_VK_SKIP_CODE=<code key prefix,...>`, leaves out the draws of every program that has a stage whose native code key starts with one of the prefixes. It is decided once per program (memoized by program key) and costs nothing while unset. `RX_VK_SKIP_PROG` already leaves out whole programs; this one targets a stage that many programs share. - `gpu/vk/vk_draw.cpp`: the switch, right after `RX_VK_SKIP_PROG`. It logs the first 32 programs it leaves out, with their code keys. - `gpu/vk/skip_code.h`: the prefix match (comma-separated, either case, only at the start of a stage's key). - `tests/test_skip_code.py` + `.cpp`: a game-free check of the match (picked up by `tests/run_all.py`). - `readme.md`: a troubleshooting row for the Deck, and the switch under *Switches for testing*. **Why** On the Steam Deck (RADV VANGOGH) the GPU hangs in Kokiri Forest and the Great Deku Tree: `ring gfx_0.0.0 timeout`, then a GPU reset (#817, tracked in #1000). Buffer markers read while the GPU was still stuck showed that every stalled draw belongs to a task + mesh program sharing one task shader, code `2c56dc1006942f74`: two nested loops over a 2D grid (a culling pass), used by about 30 programs. On RDNA 2 some of these draws run for seconds (one took 9 s and then finished); past 10 s the kernel resets the ring. `RX_VK_MESH_OPT=1` does not change it. `RX_VK_NO_MESH=1` avoids it, but doors and foliage go missing (#1000). `RX_VK_SKIP_CODE=2c56dc1006942f74` leaves out only those programs, and with them the geometry they draw: key models go missing while it is set. Why that shader runs so long on RDNA 2 is still open; this is an opt-in workaround to get past the hang, not a fix. **Tested** Steam Deck OLED, SteamOS 3.9.5, RADV VANGOGH (Mesa 26.2), Game Mode, 720p handheld, with the same change applied to v2.0.4 and to v2.0.7 (`RX_VK_LOW_SPEC=1`, `HOS_HANDHELD=1`, `RX_VK_MESH_OPT=1`, the memory caps from #817): - v2.0.4 without it: a GPU hang within 30 s of loading a save in the Great Deku Tree. With it: no stall in several 4-minute runs there and 20 min in Kokiri Forest. I saw nothing missing in screenshots, but a player watching the Deck later saw key 3D models missing. - v2.0.7 with it: 0 device lost and 0 stalls in five runs at two spots inside the Great Deku Tree, an early room and B1 (fresh launch, 90 s camera turn, 2x frame generation): 36.1-37.2 fps game rate, 20-27 programs left out. - A second Deck confirmed it in #817 (RufflezAU: 5 min in Kokiri Forest without a device lost, where it was lost within seconds before). - On this branch (main 61d452d): the Linux Vulkan back end builds with the AppImage's toolchain, and `python -I -B tests/run_all.py skip_code` passes. I have not run main itself on the Deck. The `RX_VK_CHECKPOINTS` watchdog that found the shader (it reads the AMD buffer markers while the GPU is still stuck, because the reset kills the process before `device lost` can read them) is a separate change; I can send it as its own PR. Refs #817, #1000 🤖 Generated with [Claude Code](https://claude.com/claude-code)

hckrman101 (Migrated from github.com) reviewed

2026-11-09 16:03:52 +00:00

hckrman101 (Migrated from github.com) left a comment

Great detective work on the Deck hang. Reading the AMD markers while the GPU is still stuck is a neat trick, and the RX_VK_CHECKPOINTS watchdog sounds worth its own PR. The switch is small and well contained: it costs nothing while unset, the prefix match is right (start of a key only, either case, empty entries skipped), and memoizing by cp->key is sound since program keys are hashed from the stage code keys. A couple of thoughts:

Great detective work on the Deck hang. Reading the AMD markers while the GPU is still stuck is a neat trick, and the `RX_VK_CHECKPOINTS` watchdog sounds worth its own PR. The switch is small and well contained: it costs nothing while unset, the prefix match is right (start of a key only, either case, empty entries skipped), and memoizing by `cp->key` is sound since program keys are hashed from the stage code keys. A couple of thoughts:

gpu/vk/vk_draw.cpp

While the switch is set, every draw takes this global mutex and does a hash lookup, including on the parallel segment workers, so draws from different recording threads serialize here. On the Deck (4 cores) that's probably measurable. Since the answer is fixed per program, caching it on the Program itself (e.g. a small atomic tri-state: unknown / skip / keep, computed once) would make the check lock-free, like the RX_VK_SKIP_PROG mask test just above. A thread_local memo would also do.

While the switch is set, every draw takes this global mutex and does a hash lookup, including on the parallel segment workers, so draws from different recording threads serialize here. On the Deck (4 cores) that's probably measurable. Since the answer is fixed per program, caching it on the `Program` itself (e.g. a small atomic tri-state: unknown / skip / keep, computed once) would make the check lock-free, like the `RX_VK_SKIP_PROG` mask test just above. A `thread_local` memo would also do.

readme.md

					
				@ -170,6 +170,7 @@ with `--update <game folder>`.

				| The game opens and closes, or the log says "device lost" | Update your graphics driver. On AMD cards there are known crashes ([#708](https://github.com/NotRexed/Hookshot/issues/708), [#614](https://github.com/NotRexed/Hookshot/issues/614)) |

Since every Deck player hits this in two of the first areas, would it be worth turning it on by default when the device is RADV VANGOGH (the way RX_VK_SKIP_PROG has a default entry), with RX_VK_SKIP_CODE=- to opt out? Most players won't find this row before the game has frozen on them a few times. That depends on whether leaving these ~30 programs out really hides nothing. Do you know what they draw? If it's culled geometry that ends up empty on the Deck anyway, a default seems safe. If it's something like distant foliage, opt-in is the right call.

Since every Deck player hits this in two of the first areas, would it be worth turning it on by default when the device is RADV VANGOGH (the way `RX_VK_SKIP_PROG` has a default entry), with `RX_VK_SKIP_CODE=-` to opt out? Most players won't find this row before the game has frozen on them a few times. That depends on whether leaving these ~30 programs out really hides nothing. Do you know what they draw? If it's culled geometry that ends up empty on the Deck anyway, a default seems safe. If it's something like distant foliage, opt-in is the right call.

hardicd commented

2026-11-09 16:37:08 +00:00

(Migrated from github.com)

Thanks for the review. Two corrections first, then your points.

What the skipped programs draw: visible models. With RX_VK_SKIP_CODE=2c56dc1006942f74 set on v2.0.9, the player watching the Deck in the Great Deku Tree saw key 3D models missing, enough to stop playing natively. My "nothing seen missing" came from screenshots only and was wrong: these are task + mesh programs, so skipping them takes their geometry with them, not just a culling pass. So it shouldn't be a default on VANGOGH. It stays opt-in: a way past the hang, and a way to find the shader behind it, at that cost. 02feff6 says so in the readme row, the switch note and the code comment, and I've corrected the description.

Test spots: what I called a Kokiri Forest cave is an early room inside the Great Deku Tree (its save's caption says so). Both v2.0.7 spots were in the Deku Tree.

The lock: agreed, while it's set every draw serializes on that mutex. A cached per-program result (an atomic tri-state on the Program) would make it lock-free like the RX_VK_SKIP_PROG mask. Given the missing models, though, the real fix is probably in that task shader itself (why its grid loops run for seconds on RDNA 2), so the programs can stay in. I don't have that yet.

Thanks for the review. Two corrections first, then your points. **What the skipped programs draw:** visible models. With `RX_VK_SKIP_CODE=2c56dc1006942f74` set on v2.0.9, the player watching the Deck in the Great Deku Tree saw key 3D models missing, enough to stop playing natively. My "nothing seen missing" came from screenshots only and was wrong: these are task + mesh programs, so skipping them takes their geometry with them, not just a culling pass. So it shouldn't be a default on VANGOGH. It stays opt-in: a way past the hang, and a way to find the shader behind it, at that cost. 02feff6 says so in the readme row, the switch note and the code comment, and I've corrected the description. **Test spots:** what I called a Kokiri Forest cave is an early room inside the Great Deku Tree (its save's caption says so). Both v2.0.7 spots were in the Deku Tree. **The lock:** agreed, while it's set every draw serializes on that mutex. A cached per-program result (an atomic tri-state on the `Program`) would make it lock-free like the `RX_VK_SKIP_PROG` mask. Given the missing models, though, the real fix is probably in that task shader itself (why its grid loops run for seconds on RDNA 2), so the programs can stay in. I don't have that yet.

hckrman101 (Migrated from github.com) reviewed

2026-11-09 16:53:12 +00:00

gpu/vk/vk_draw.cpp

Fair enough. Since it's an opt-in workaround that only Deck players hitting the hang will set, the lock isn't worth churning on here, and fixing the task shader so these programs can stay in is the better use of time. Fine to leave as is.

hckrman101 (Migrated from github.com) reviewed

2026-11-09 16:53:14 +00:00

readme.md

					
				@ -170,6 +170,7 @@ with `--update <game folder>`.

				| The game opens and closes, or the log says "device lost" | Update your graphics driver. On AMD cards there are known crashes ([#708](https://github.com/NotRexed/Hookshot/issues/708), [#614](https://github.com/NotRexed/Hookshot/issues/614)) |

Thanks for checking on the Deck. Missing key models settles it: opt-in is right, and the new readme wording makes the trade-off clear.

hckrman101 (Migrated from github.com) approved these changes

2026-11-09 16:53:16 +00:00

hckrman101 (Migrated from github.com) left a comment

LGTM. The docs now say plainly what the switch costs, and it stays off unless set. Small nit, not blocking: since both test spots turned out to be in the Deku Tree, the readme row and switch note still mention Kokiri Forest. Drop it if you haven't actually seen the hang there.

Commenting is not possible because the repository is archived.

No reviewers

hckrman101

No labels

No milestone

No assignees

1 participant

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference

MagicBeanDev/Hookshot!1165

				@ -170,6 +170,7 @@ with `--update <game folder>`.
				`\| The game opens and closes, or the log says "device lost" \| Update your graphics driver. On AMD cards there are known crashes ([#708](https://github.com/NotRexed/Hookshot/issues/708), [#614](https://github.com/NotRexed/Hookshot/issues/614)) \|`

Rows
Columns