[Crash] VK_ERROR_DEVICE_LOST on RX 6900 XT, always right after "pool 0x7fde000000..0x7fe0000000 took in 2 pools" #150
Labels
No labels
accessibility
autorelease: pending
autorelease: tagged
bug
dependencies
documentation
duplicate
enhancement
github_actions
good first issue
help wanted
invalid
question
testing wanted
wontfix
No milestone
No assignees
1 participant
Due date
No due date set.
Dependencies
No dependencies set
Reference
MagicBeanDev/Hookshot#150
Loading…
Reference in a new issue
No description provided.
Delete branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Setup: hookshot.exe, AMD Radeon RX 6900 XT, driver 32.0.21045.20006 (Adrenalin 26.10.2), Intel Xeon E5-2697 v3, 64 GB RAM, Windows 11. The game runs at a steady 60 fps, then crashes.
What happens:
The AMD driver crashes (TDR) and the game freezes. The log shows
submit N failed (-4)followed bydevice lost. After that, every pipeline fails with -4 and I close the window manually.Pattern I found across 3 runs (logs attached):
In all three logs, the lines right before the failure are a series of
pool ... remade larger/took in 2 poolsmessages, and the last memory line is always the same one:vk: pool 0x7fde000000..0x7fe0000000 took in 2 poolsThe very next thing in the log is
submit failed (-4).Things I tried (no change):
Happy to test any build or flag you suggest.
built-in log
hookshot_log.txt
my log
log-partida.txt
Linked here from closed duplicates, with log data: #233 (RTX 4060 Ti, checkpoint dispatch 4b03b3040d7cf076, frame 4053), #266 (GTX 1660 SUPER, unbound storage buffers in 53caac5f38279735 / 7984161db3e053b5, RX_VK_TAAU_KEYS=- reported to avoid it), #108 and #273 (RX 6600: device lost as new computes come in, "pool ... not imported" retries, mirrors given back), #243 (RX 6600, Kakariko cemetery), plus reports without logs: #29, #170, #178, #181, #202 (Legion Go S / ROG Ally Z1E), #223, #253, #258, #264.
hookshot_log.txt
still getting hang right after the first cutscene, heres my log, rx 6600.
Managed to show a couple frames at varu bad framerate then the screen turned black again
hookshot_log.txt
hi, now I got through the big deku tree but then it just crashed
hookshot_log.txt
hi crashed after entering the mojo tree my logs :
hookshot_log.txt
Found and fixed (
9bb52a9), in the release https://github.com/NotRexed/Hookshot/releases/tag/device-lost-fix. The renderer keeps a table of where the game's memory lives on the GPU, and the CPU rewrote it while earlier GPU work was still running: right after the game's memory pools merged, the foliage culling shader (53caac5f) could follow an entry into memory that wasn't ready, and the GPU faulted (it stopped for ~2 s, then 'device lost'). Table changes now reach the GPU in submission order. Here that took starts from ~1 crash in 7 to 0 in 14, with the same fps. Please rebuild with this release and tell me if you still get a device lost; if so, attach hookshot_log.txt (the lines after 'device lost: reading' say where).Tested
mainat4cfa08f(includes9bb52a9) on an RTX 4080 (Windows 11, Ryzen 9 7900X): 6 New Game runs, 1 with no caches and 5 with warm caches, each through the opening, the Deku Tree cutscene and into Kokiri Forest. No device lost, no failed submits, and no53caac5flines in any log.For comparison, 3 earlier runs of
e40306e(before9bb52a9) on this PC also had no device lost, so this card may not hit the bug often. Treat this as "no regression and no crash in 6 runs", not proof of the fix. I can run more, or the old ordering (RX_VK_TABLE_NOW=1) as a control, if that helps.Current
mainhas9bb52a9, the guest page-table update ordering in the foliage culling. The last retest here was six clean runs on an RTX 4080, so this remains the canonical device-lost tracker rather than a new bug.If you still see
device loston a build with9bb52a9, attachhookshot_log.txt; the lines afterdevice lost: readingsay where it happened. Otherwise no action is needed on this issue.Still
device loston a build with9bb52a9: Linux, Steam Deck (RADV VANGOGH), reproducibleYou asked for the lines after
device lost: readingfrom builds that already include9bb52a9. Two runs on a Steam Deck, both ending in a GPU hang at the same point.Setup
AMD Custom GPU 0405 (RADV VANGOGH), Vulkan 1.4.350, SteamOS build 20261023.100, kernel 7.2.7-valve2.mainat3601726(source 50b3813b), built on the Deck itself, game v131072. Program cache warm (39,918 keys).hookshot_options.ini: quality preset Custom (no VRAM budget), resolution Auto, frame generation off, game speed Real time, handheld profile on.What happens
Intro and title run at 53 to 60 fps. When the save loads (Kokiri Forest), textures go from about 130 MB to about 3.5 GB within seconds (run 1: 1500 textures, 3496 MB), then the GPU hangs. Kernel:
Hookshot (run 2):
Run 1 ended the same way at frame 3324, after the same table entry (
c0+0x880, addr0x107f7e8000, size0x9596000, programs10dc355939390bdcand918adef1afaa39d4) was unbound and bound again.What I tried
RX_VK_BATCH=0: same hang (a plainsubmit ... failed (-4), no longer a batched one), so batching is not the cause.RX_VK_CHECKPOINTS=1: no marker lines afterdevice lost: reading the fault info. I do not know if RADV supports them.53caac5flines in either log, so it may be another shader than the one in your fix.I can run
RX_VK_TABLE_NOW=1as a control, or any other switch or build you want tested, and can attach the full logs. The measurements were taken on the device; I used an AI assistant (Claude Code) to read the logs and write this up.Release https://github.com/NotRexed/Hookshot/releases/tag/fast-updates no longer gives back memory the GPU used, on Windows (it was meant for a Linux AMD driver; fuxau traced the Arc device lost to it in #17, and the AMD logs show device lost right after it). If you had device lost, could you try this release and tell us? On Linux/Steam Deck the old behaviour stays; there, please try
RX_VK_KEEP_RETIRED=1and say whether the device lost goes away.Steam Deck,
RX_VK_KEEP_RETIRED=1: the GPU hang is gone, but it runs out of memory at the same pointTried as you asked, on the build from my earlier comment (main at
3601726, which is 60 commits afterf2203e4, so it has the switch).Result
RX_VK_KEEP_RETIRED=1the log showsimports: 50 live (1296 MB), 37 retired still imported (880 MB), 0 retired given back (0 MB). Before it was38 retired given back (1088 MB)right before the hang.ring gfx_0.0.0 timeoutand no GPU reset any more (both earlier runs had them).radv/amdgpu: Not enough memory for command submission(kernel:amdgpu [drm] *ERROR* Not enough memory for command submission!), followed bybatched submit ... failed (-4)anddevice lost: reading the fault info.Two runs
textures 132 MB (budget 2200 MB)): ends the same way at 1500 textures, 3444 MB, so the 2200 MB texture budget does not hold while the save loads.Machine: Steam Deck, RADV VANGOGH, 15.2 GB RAM visible (BIOS UMA frame buffer 1 GB, GTT 8 GB). The log says
boot: image 96 MB ..., 9809 MB of memory for the game, so the guest plus 3.4 GB of textures plus about 2.2 GB of imports does not seem to fit.So with the switch the failure changes from a GPU hang to an out-of-memory. I can try anything that caps the textures during the load (a different budget switch, handheld profile settings, or a build you want tested) and can attach the full logs.
(Measured on the device; the write-up was prepared with an AI assistant, Claude Code.)
Same as wimw85, but on Windows + NVIDIA: the Minimum (4GB) texture budget
is not held after loading a save.
GPU: GTX 1650 Super 4 GB (driver 621.14), Ryzen 5 5500, 16 GB RAM, Windows 11
Build: fast-updates (4187feb5), preset Minimum (4GB), Handheld profile On
Title screen: 60 fps, textures ~115 MB.
After loading a save in Kokiri Forest:
"VRAM: 3567 MB used of 3116 MB allowed ... textures 1589 MB (budget 1044 MB)"
-> 4 fps. Same on low-vram (textures up to 2.7 GB).
No device lost. On the older foliage build I had ~30 fps here.
Log attached.
hookshot_log.txt
Update on RX 6900 XT with the latest release (fast-updates, source 4187feb5). This time the crash happened while loading a save, and the game closed by itself about 30 seconds after the driver failure.
Setup: RX 6900 XT, driver 32.0.21045.20006 (Adrenalin 26.10.2), Xeon E5-2697 v3, 64 GB RAM, Windows 11 25h2.
What the log shows (attached, line numbers refer to it):
view of texture 0x1133ccb000 (vk 139) as vk 95 ... not made, 16 times (layers 0 to 15).mesh draw: program 1a70. Line 42 saystask and mesh programs built without the driver's optimizer (AMD; ...).slow submit: 362.7 msand a 171.7 ms dispatch.storage buffer NOT bound (reads zeros)with values that look like garbage (size 0x3f800000, which is the bit pattern of 1.0f, andaddr 0x3e0000003ba478e8 size 0x44ca2618).graphics pipeline ... not made (-4)(VK_ERROR_DEVICE_LOST), followed by five compute pipelines failing with -4 (0ba2f076e4a94acf, 53fcaee4dc45c6ee, 26fe0c70ab9fc547, 685adb41edc19de7, 6637ac1d052d22d4).pool 0x7fff4bda00+0x8 not imported (VkResult -1000072003).the TAAU network fd029a317a9a4b94 is stood in for ... its pipeline was not made.slow submit: 30423.3 ms(draw 30327.7 ms), a ~30 s stall.CRASH: exception 0xc0000005 (write 0x28d004b1000, guest 0xfffff28d004b1000) at amdvlk64.dll+0x2f1c0fbhost backtrace:
amdvlk64.dll+0x2f1c0fb amdvlk64.dll+0x4953eb amdvlk64.dll+0x459901 amdvlk64.dll+0x1abe07 hookshot_vk.dll+0x14efe2 hookshot_vk.dll+0x1c001b ..., on the Presentation Thread. Guest backtrace:lr 0x80590358 0x80df26c0 0x834a2ec8.So after the device is lost, the GPU stalls for about 30 s and then the emulator crashes with an access violation inside the AMD Vulkan driver (amdvlk64.dll). The game closing on its own matches this.
It does not look like VRAM pressure: the last
VRAM:line says 2648 MB used of 15569 MB allowed.hookshot_log.txt
Same flickering here on an RTX 3050 (release fast-updates, source 4187feb5, Windows).
Log attached.
hookshot_log.txt
still black screen for me .
im on windows 11 on ROG Ally Z1 Extreme. tried vram set to auto and at 4GB.
hookshot_log.1.txt
hookshot_log.txt
with vram set to 6gb the game makes the whole system reboot.
hookshot_log.txt
Tracked in #996, follow there. That issue collects every report of AMD driver crashes, device lost and freezes on Windows, and this one is linked from it with its GPU, version and log details. The tracker summarises the device-lost causes found here and what was fixed since. If it still happens on v2.0.6 (https://github.com/NotRexed/Hookshot/releases/tag/v2.0.6; Update next to Play in the launcher), please add a comment there with your
hookshot_log.txt.Steam Deck LCD / SteamOS — Vulkan crashes and rendering issues
Hi! I've been testing Hookshot on my Steam Deck LCD running SteamOS, using the AMD RADV VANGOGH Vulkan driver.
I successfully built and launched the game natively on Linux.
Results:
Configurations tested:
The logs include "storage buffer NOT bound", "batched submit failed (-4)", and "device lost".
I'm happy to provide screenshots, full logs, and test experimental Linux builds or Vulkan configurations.
Is there a recommended fix or branch specifically for Steam Deck / RADV VANGOGH?
Thanks for your work on Linux support!