Linux AMD (RADV): VK_ERROR_DEVICE_LOST after the opening cutscene and before the Great Deku Tree appears (RX 6600 XT) #2245
Labels
No labels
accessibility
autorelease: pending
autorelease: tagged
bug
dependencies
documentation
duplicate
enhancement
github_actions
good first issue
help wanted
invalid
question
testing wanted
wontfix
No milestone
No assignees
1 participant
Due date
No due date set.
Dependencies
No dependencies set
Reference
MagicBeanDev/Hookshot#2245
Loading…
Reference in a new issue
No description provided.
Delete branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Hookshot version
v2.1.0
What kind of problem
Crash, freeze or black screen
What happened
Subtitle
Reproducible GPU reset/device loss at a specific scene transition on Linux + AMD RADV; occurs after the opening cutscene and before the Great Deku Tree appears.
Description
Summary
On Linux with an AMD GPU using the RADV driver, the application (v2.1.0, a decompiled/recompiled project, not an emulator) crashes after the Vulkan device is lost with VK_ERROR_DEVICE_LOST. Immediately before the device loss, the log shows a GPU sync wait of about 2580 ms, several very slow memory-sync submissions, capture write-back timeouts, and heavy texture re-upload activity. The RADV/amdgpu driver then reports that the command stream was cancelled because the context was lost.
Timing / reproducibility
The GPU reset / device loss occurs consistently after the opening cutscene and before the Great Deku Tree appears. This suggests the crash may be triggered by a particular texture upload, memory-sync, or shader/resource burst during that scene transition rather than by random load.
Environment
OS: Linux (please fill in distro and kernel version)
CPU: AMD Ryzen 7 7840HS
GPU: AMD Radeon RX 6600 XT (8 GB VRAM)
RAM: 32 GB
GPU Driver: RADV/Mesa (please fill in version), amdgpu kernel driver
Application: Decompiled/recompiled project, version v2.1.0
API: Vulkan
Steps to reproduce
Launch the application (v2.1.0) with the Vulkan backend on Linux + AMD RADV using an RX 6600 XT.
Start a new game and let the opening cutscene play.
Continue until the scene transitions to the point where the Great Deku Tree is about to appear.
At this point, the application stalls, the GPU is reset, and the application closes.
The following fatal error appears in the log:
text
vk: FATAL: the GPU device was lost (VK_ERROR_DEVICE_LOST): the driver stopped or reset the GPU, and nothing sent to it runs any more
radv/amdgpu: The CS has been cancelled because the context is lost. This context is innocent.
vk: batched submit 32875 failed (-4)
vk: device lost: reading the fault info
Expected behavior
The application should continue rendering without losing the Vulkan device.
Actual behavior
The GPU device is lost, Vulkan submissions fail with -4 (VK_ERROR_DEVICE_LOST), and the application closes.
Relevant log evidence
Immediately before the device loss:
text
[hos sead::MainThread] nvn: GPU waits in 5 s: 1 sync waits (2580 ms), 0 polls timed out (0 ms), 0 queue finishes (0 ms)
Multiple slow memory-sync submissions:
text
[hos sead::MainThread] vk: slow submit: 176.4 ms: memory sync 176.3
[hos sead::MainThread] vk: slow submit: 252.8 ms: memory sync 252.8
[hos sead::MainThread] vk: slow submit: 258.4 ms: memory sync 258.2
[hos sead::MainThread] vk: slow submit: 251.7 ms: memory sync 251.6
Capture write-back waits timing out:
text
[hos sead::MainThread] vk: a capture's wait for write-back 156638 timed out (done 156634)
[hos sead::MainThread] vk: a capture's wait for write-back 156645 timed out (done 156641)
[hos sead::MainThread] vk: a capture's wait for write-back 156652 timed out (done 156641)
Textures re-uploaded because the GPU wrote to their storage:
text
[hos host] vk: texture 0x10003200e00 2048x1152x1 x1 levels 1 NVN 0x25 (9216 KB of storage) re-uploaded (its memory was written; flags 0x8)
[hos host] vk: texture 0x100030a1a00 1600x1600x1 x3 levels 1 NVN 0x32 (15600 KB of storage) re-uploaded (its memory was written; flags 0x8)
Then the device is lost:
text
radv/amdgpu: The CS has been cancelled because the context is lost. This context is innocent.
[hos ModuleSystemWorker1] vk: batched submit 32875 failed (-4)
[hos ModuleSystemWorker1] vk: FATAL: the GPU device was lost (VK_ERROR_DEVICE_LOST)
Additional information
The crash occurs after a GPU sync wait of ~2580 ms. On some kernels the amdgpu lockup_timeout default may be shorter than this, causing the kernel to reset the GPU before the submission completes.
The log shows heavy texture re-upload activity caused by GPU writes into texture storage, which can amplify memory-sync pressure.
Where in the game
occurs after the opening cutscene and before the Great Deku Tree appears.
How often
Every time
Graphics card and driver version
AMD Radeon RX 6600 XT (8 GB VRAM) mesa:26.2.6
CPU, RAM and operating system
AMD Ryzen 7 7840HS ,32G,LINUX:AOSC OS
Changes from the defaults
No response
Other software hooked into the game
No, nothing hooked in
hookshot_log.txt
hookshot_log.txt
Screenshots or video
No response
Before you send it
The same happens on my Steam Deck Oled!
Root Cause Analysis (Unbound Storage Buffer)
Based on the logs, the GPU hang is not caused by memory exhaustion (VRAM usage is only ~1.3 GB of 7.6 GB). Instead, it is caused by a fatal resource binding issue that triggers a hardware/driver deadlock.
Immediately before the crash, the engine attempts to dispatch a graphics/compute program (program
da7a7ab28b8e73d9) that accesses a storage buffer (SSBO) at address0x4180000000000200which is NOT bound.Because the buffer is unbound, the system falls back to reading zeros. If the shader relies on this buffer's data for loop conditions or calculations, this undefined behavior can lead to an infinite loop on the GPU. On the RADV driver, this infinite loop triggers the amdgpu kernel watchdog, which eventually resets the GPU and causes the
VK_ERROR_DEVICE_LOST.This specific error occurs exactly at the scene transition before the Great Deku Tree appears. Fixing the resource binding for this shader program or correcting the data passed to this SSBO should resolve the device loss.
similar issue with my Rog Xbox Ally X
16gb RAM
8gb VRAM
APU 35W (turbo)
hookshot_log.txt
Same crash here on Linux (CachyOS, kernel 7.3.3): RX 6800 (Navi 21), Mesa 26.2.6 RADV, stock clocks.
VK_ERROR_DEVICE_LOST after the opening cutscene, then an amdgpu ring timeout and GPU reset.
Workaround: set the Mesh shaders option to Compute (mesh_shaders=2 in hookshot_options.ini). With that, I get no crash at all. It's a workaround, not a fix: Native still crashes, and Compute costs some performance.
In-game tests:
amdgpu devcoredumps (no VM page fault in either):
Conclusions:
RX 6800 XT (RADV, Linux): the device loss after the opening cutscene is gone with mesh shaders off
Hardware and versions: RX 6800 XT (RADV NAVI21), Vulkan 1.4, Mesa 25.0.7, kernel 7.0.0-42-generic, running inside a container (games-on-whales), build v2.0.11-871-g9447e1d from main. The dump is my own, supported build.
Before. Title and the whole opening movie ran at 60.0 fps (AV1 1920x1080, 2 draws a frame). Then the fps series went 60.0 for about 78 reports, 10.6, 15.6, and:
Start to device lost: 4 min 34 s.
After RX_VK_FORCE_MESH_FALLBACK=1 in hookshot_env.txt:
Later in that run it became CPU-bound in a dense scene (1,100–1,600 draws: GPU 10.3 ms a frame, queue submit ~34 ms). That part was ours, not the port: the container had no /dev/udmabuf, so it logged guest memory stays host-pointer imports for the GPU (slow submissions on AMD). With the device exposed the same build reports guest memory as dma-bufs: yes.
So this matches #2245 and the fix in #2362. I can attach the full run log and the kernel block.
Thanks for the report. Reports of the same problem are now collected in #1000, which lists this one with your details and links back here, so one fix closes them together and the open list stays readable. Follow #1000 for the fix, and add new details there (or here). If this turns out to be a separate problem it will be reopened. Pull request #2362, #2311 works on this.