[Linux/AMD] GPU hang (ring timeout → MODE1 reset) on scene change, RX 6800 XT / RADV — buffer-marker logs attached #1275
Labels
No labels
accessibility
autorelease: pending
autorelease: tagged
bug
dependencies
documentation
duplicate
enhancement
github_actions
good first issue
help wanted
invalid
question
testing wanted
wontfix
No milestone
No assignees
1 participant
Due date
No due date set.
Dependencies
No dependencies set
Reference
MagicBeanDev/Hookshot#1275
Loading…
Reference in a new issue
No description provided.
Delete branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Hookshot version
v2.0.11
What kind of problem
Graphics (missing, black or wrong-coloured things, lighting, flicker)
What happened
Summary
On an RX 6800 XT with RADV, the GPU hangs on area transitions (leaving Link's house / first frames after a load).
amdgpu times out the ring after ~10 s, the ring reset fails and the kernel does a full MODE1 reset (VRAM lost).
The game then shows a black window and its RAM use keeps growing until it is closed.
System
What happens
Title, intro movie and the first area run fine at ~60 fps.
On the first scene change / first gameplay frames after a load, one submission never completes:
slow submit: 10982.2 ms: ... slot wait 10705.2radv/amdgpu: The CS has been cancelled because the context is lost. This context is innocent.then
batched submit ... failed (-4)/device lost.Kernel log (every hang is owned by hookshot):
ring gfx_0.0.0 timeout ... Process hookshot pid 32219 thread OptimizeChangeB
ring comp_1.2.0 timeout ... Process hookshot pid 32219 thread OptimizeChangeB
Ring comp_1.2.0 reset failed -> MODE1 reset -> VRAM is lost due to GPU reset!
Threads owning the hung job across 7 runs: OptimizeChangeB (x3), Presentation Th (x2), mkw_vk order, sead::MainThrea.
No
page faultlines in the kernel log.Buffer markers (RX_VK_CHECKPOINTS=1, VK_AMD_buffer_marker)
The point where the GPU stops differs between runs:
Possible lead (a guess, not verified): both write-back hangs are on small dma-buf pools created at boot,
and 0x100d000000 is the pool holding the NVN events (
event 0x100d6c8000in CommandBufferWait/SignalEvent).If a write-back can land stale data over an event value a queue is waiting on, the wait would never be met,
which would fit a ~10 s hang whose exact position moves between runs.
Switches tried (hookshot_env.txt)
batched submits offconfirmed)not imported (VkResult -1000072003)), 0 draws, black pictureSecond issue: memory grows after the device is lost
After
device lostthe replay keeps running and nothing is retired:captures keptgoes from 25 to 2948 (and 6527 in another run), and staging from 256 MB in 4 arenas to 1408 MB in 22 arenas.System RAM fills up until the window is closed. Exiting (or at least stopping the replay) on device loss would avoid this.
Attached
Possibly related: #708, #614 (device lost on RX 6600), #604, #528 (Linux AMD).
hookshot_log.txt
Where in the game
Link's House, after load save
How often
Every time
Graphics card and driver version
6X6800XT
CPU, RAM and operating system
R7 5700X3D, 64GB RAM, Arch Linux
Changes from the defaults
No response
Other software hooked into the game
No, nothing hooked in
hookshot_log.txt
hookshot_log.txt
Screenshots or video
No response
Before you send it
Thanks for the buffer-marker logs. Linking this to the Linux tracker #1000, with the same RADV ring timeout and MODE1 reset as #1065 and #1227.
Fixed in the v2.0.12 branch, closing when it ships. After a device lost the game now ends cleanly instead of showing a black window while its RAM keeps growing. The RADV hang itself is still being looked at.
Sorry, I'm writing to you here. But where can I find all the values that can be given in env.txt?
The
hookshot_env.txtswitches are listed in the development guide, under "Switches for testing and debugging": https://git.magicbean.dev/MagicBeanDev/Hookshot/src/branch/main/docs/development.md#switches-for-testing-and-debugging (oneNAME=VALUEa line, besideimage.bin).Same GPU (RX 6800 XT / NAVI21, RADV). Here the ring timeout and the MODE1 reset stop with RX_VK_FORCE_MESH_FALLBACK=1: the run that died at 4 min 34 s now holds 60 fps past that point. Full details in #2245
Thanks for the report. Reports of the same problem are now collected in #1000, which lists this one with your details and links back here, so one fix closes them together and the open list stays readable. Follow #1000 for the fix, and add new details there (or here). If this turns out to be a separate problem it will be reopened.