fix(vk): host-visible buffers stay within their heap's budget, and a submit refused for want of memory is made again #2081

Merged
HumanBean merged 2 commits from HumanBean/Longshot:fix/uma-heap-budget-submit-retry into main 2026-11-21 17:01:45 +00:00
Owner

Summary

On AMD APUs and handhelds on Windows (Legion Go Z1E/780M with 8 GB UMA in #1879 and #1878, the ROG Ally in #668, the save-load reports in #947), the screen goes black while the sound keeps playing. It happens on a save load, entering Hyrule Castle or the Deku Tree, or leaving the Deku Tree. The log ends in repeated vk: submit N failed (-2), which is VK_ERROR_OUT_OF_DEVICE_MEMORY.

Why. All five #1879 logs show the same thing. VRAM as a whole is fine: VRAM: 1877 MB used of 7782 MB allowed. But AMD's driver has a second device-local heap: the 256 MB window of the carve-out that the CPU can write. That heap is over its own budget: heap 2 (device): 55 blocks 252 MB ... used 252 of 243 MB. Each failure comes right after new staging arenas are made (a staging arena of 16/25 MB took ... to make) during the load's texture uploads.

create_buffer asks VMA for a host-visible buffer that the GPU reads (the staging arenas, STORAGE for the ASTC decoder). VMA puts it in DEVICE_LOCAL | HOST_VISIBLE, which is that 256 MB heap, without checking the heap's budget. The allowance (vram_usage) adds that heap's 243 MB into the 7782 MB total, so no rule saw it fill. Once it was past its budget, the driver refused submits. The replay logged each refusal and dropped the work: the frame's work never ran, and the next submit's signal covered its timeline value. The result is a black screen that does not recover. The total allowance on UMA (the carve-out's heaps summed) is right. What went wrong is one heap's budget.

What this changes

  • gpu/vk/vk_vma.cpp create_buffer: a host-visible buffer is first made within its heap's budget (VMA_ALLOCATION_CREATE_WITHIN_BUDGET_BIT). When that heap is full, VMA takes the next memory type (system memory, which on an APU is the same RAM). Where no heap has room, it is made as before. RX_VK_BUFFER_BUDGET=0 (kill switch) restores the old behaviour. Device-only buffers and create_buffer_cached are unchanged.
  • gpu/vk/submit_retry.h and vk_replay.cpp submit_or_retry: every vkQueueSubmit2 of the replay goes through it. That covers submit_recording, part_submit, the batched flush, the present's copy and the upload chunk. After VK_ERROR_OUT_OF_DEVICE_MEMORY or OUT_OF_HOST_MEMORY, it waits for the GPU's earlier work (vkQueueWaitIdle, under the queue's lock that the caller already holds) and submits again, up to 3 tries. The Vulkan spec leaves a refused submit's command buffers, fence and semaphores unchanged, so the same submit can be made again. The next present (texture_frame) then frees every idle staging arena and the image reaper's whole queue. Nothing is destroyed under the queue's lock: a destroy's rec::boundary can wait for the recorder, whose part submit waits for that lock. RX_VK_SUBMIT_RETRY=0 (kill switch) drops the submit as before. RX_VK_TEST_SUBMIT_REFUSE=<n> (diag) refuses the first try of every n-th submit, so any card can run this path.
  • Docs: architecture.md, "Each heap's budget".

Read first: create_buffer (vk_vma.cpp), submit_or_retry and submit_give_back (vk_replay.cpp), and the memory_refused block in texture_frame (vk_texture.cpp).

To see it on main: a Legion Go or ROG Ally (8 GB UMA, Windows, Adrenalin 25.30.44), load a save or enter the Deku Tree. The log shows heap 2 ... used 252 of 243 MB, then submit N failed (-2), then a black screen. Without the hardware: tests/test_submit_oom.py fails on main.

Fixes #1879. See #1878 (duplicate), #947, #668.

Testing

Tested: Linux, RTX 3090, NVIDIA 620.70.06, this PR @ 038d85ca as a pack over main 9dd41326's base, through the job queue. A memory tour from kokiri-start through Hyrule Castle (teleport 4) and Inside the Deku Tree (19), 40 s each, with RX_VK_TEST_SUBMIT_REFUSE=200: the log has vk: the refused submit was taken on try 2 16 times, no failed (-2), no crash or device loss, and the tour finished with a clean quit. Peak game total 11.5 GB (resident + swapped), peak private 4.1 GB. On this PC (resizable BAR) the placement path has room in heap 0, so it changes nothing here; the APU heap's overrun can only be shown on the Test Farm.

The game-free test tests/test_submit_oom.py is written but has not been run on this PC (no tests run locally). CI runs it. It is written to fail on main for the bug's reason: a 16 MB staging arena with heap 2 at 236 of 243 MB lands in heap 2, at 252 of 243 MB, and a refused submit is made only once.

Still to test:

  • This PC (Linux, RTX 3090): build the pack, then do a memtour from kokiri-start through Hyrule Castle (teleport 4) and the Deku Tree (19). Run three ways: default; with RX_VK_TEST_SUBMIT_REFUSE=200, expecting vk: the refused submit was taken on try 2, a normal picture and no failed (-2); and with RX_VK_TEST_SUBMIT_REFUSE=200 RX_VK_SUBMIT_RETRY=0, expecting dropped submits and black or missing frames as on main (by hand).
  • Windows, AMD APU (780M/Z1E, 8 GB UMA): load a save, enter Hyrule Castle and the Deku Tree. Expect no submit ... failed (-2), a picture throughout, and heap 2 (device) ... used X of 243 MB staying at or under 243 (@Beanstalk below).
  • Steam Deck: the same route, expecting no regression in load time or fps (@Beanstalk below).
@Beanstalk test Windows AMD iGPUs (windows+amd+igpu): from the save kokiri-start, load the save, teleport to Hyrule Castle and then Inside the Deku Tree, 30 s in each with the camera turning. Fail on any "vk: submit" or "batched submit" line with "failed (-2)", a black screen, or a "memory heaps" line where a device heap's "used" is over its budget. Compare load times and peak VRAM with main.
@Beanstalk test Steam Deck (steamdeck): from the save kokiri-start, load the save, teleport to Hyrule Castle and Inside the Deku Tree, 30 s each. Compare load time, frame time p99 and peak VRAM with main. Fail on any "failed (-2)" or "failed (-4)" submit line, or a crash.

Measurements

Verdict: not measured yet. The build was stopped by memory pressure before anything ran.

Setup (planned): Linux, RTX 3090, NVIDIA 620.70.06; memtour kokiri-start through teleports 4 and 19; peak VRAM, RAM and the vk: memory heaps: / vk: budget: lines, main against this PR, 2 runs each. Expected: no change on this PC. With resizable BAR, heap 0 has room, so placement is unchanged, and the retry only runs on a refused submit.

Before: main @ 9dd41326 After: this PR @ 038d85ca
peak VRAM / RAM (memtour) not run not run
tests/test_submit_oom.py expected to fail (CI) expected to pass (CI)

Evidence from #1879's logs (v3.0.0, the same 5 attachments): memory heaps: ... heap 2 (device): 55 blocks 252 MB, 58 allocations 240 MB; used 252 of 243 MB next to VRAM: 1877 MB used of 7782 MB allowed, and submit 7687 / 78536 / 109249 / 194971 failed (-2) in each log after new staging arenas were made.

Risks

  • A host-visible buffer can now land in system memory instead of a full device-local heap. That only happens while the heap it would have used is past its budget, where today the driver either pages or refuses. A dGPU with resizable BAR close to its VRAM budget may stage through system memory there, which is slightly slower to read on the GPU. RX_VK_BUFFER_BUDGET=0 turns it off.
  • A refused submit now waits for the GPU to go idle, under the queue's lock, before trying again. That is a hitch where there used to be a dropped frame. It only happens on a refusal. RX_VK_SUBMIT_RETRY=0 turns it off.
  • Not covered: the VK_ERROR_DEVICE_LOST (-4) and GPU-hang reports in #947 (nightly-394, rc.7). Those are a different failure.
  • Nothing changes in saves, options or caches.

Checklist

Ready for review

  • One topic, on current main; the title is a Conventional Commit.
  • Tested on my own PC, in the game for a change the game runs, as Testing says.
  • mise run test passes (any failure that also fails on main is named). (not run on this PC by rule; CI runs it)
  • Measured against main, or Measurements says why there is nothing to measure. (not yet)
  • No game files, build output or hand-edited generated files; third_party/ untouched; nothing GPL-incompatible.

When it applies

  • A fix: a game-free test that fails on main and passes here. (written as test then fix; not run here, CI runs it)
  • A switch added or changed: it ships at the default the measurements chose, with its row in gpu/vk/switches.def. (two kill switches, on by default, and a diag)
  • Host code builds on Windows (llvm-mingw) and Linux, with UTF-8 paths. (Linux built here; Windows: CI)
  • Docs updated.
  • AI assistance recorded with Assisted-by: / Co-Authored-By: trailers.

Ready to merge

  • Everything in Still to test has run, its results in Tested.
  • Tier 1 hardware it can affect is tested (docs/dev/standards.md, "Hardware tiers").
  • Approved in review, every comment answered.
  • CI passes on the head; on current main, measured again or a comment says why the numbers still hold.

🤖 Generated with Claude Code

https://claude.ai/code/session_017vw8e2uxGKQxzDeQ15jjNK

## Summary On AMD APUs and handhelds on Windows (Legion Go Z1E/780M with 8 GB UMA in #1879 and #1878, the ROG Ally in #668, the save-load reports in #947), the screen goes black while the sound keeps playing. It happens on a save load, entering Hyrule Castle or the Deku Tree, or leaving the Deku Tree. The log ends in repeated `vk: submit N failed (-2)`, which is VK_ERROR_OUT_OF_DEVICE_MEMORY. **Why.** All five #1879 logs show the same thing. VRAM as a whole is fine: `VRAM: 1877 MB used of 7782 MB allowed`. But AMD's driver has a second device-local heap: the 256 MB window of the carve-out that the CPU can write. That heap is over its own budget: `heap 2 (device): 55 blocks 252 MB ... used 252 of 243 MB`. Each failure comes right after new staging arenas are made (`a staging arena of 16/25 MB took ... to make`) during the load's texture uploads. `create_buffer` asks VMA for a host-visible buffer that the GPU reads (the staging arenas, STORAGE for the ASTC decoder). VMA puts it in DEVICE_LOCAL | HOST_VISIBLE, which is that 256 MB heap, without checking the heap's budget. The allowance (`vram_usage`) adds that heap's 243 MB into the 7782 MB total, so no rule saw it fill. Once it was past its budget, the driver refused submits. The replay logged each refusal and dropped the work: the frame's work never ran, and the next submit's signal covered its timeline value. The result is a black screen that does not recover. The total allowance on UMA (the carve-out's heaps summed) is right. What went wrong is one heap's budget. **What this changes** - `gpu/vk/vk_vma.cpp` `create_buffer`: a host-visible buffer is first made within its heap's budget (`VMA_ALLOCATION_CREATE_WITHIN_BUDGET_BIT`). When that heap is full, VMA takes the next memory type (system memory, which on an APU is the same RAM). Where no heap has room, it is made as before. `RX_VK_BUFFER_BUDGET=0` (kill switch) restores the old behaviour. Device-only buffers and `create_buffer_cached` are unchanged. - `gpu/vk/submit_retry.h` and `vk_replay.cpp` `submit_or_retry`: every `vkQueueSubmit2` of the replay goes through it. That covers submit_recording, part_submit, the batched flush, the present's copy and the upload chunk. After VK_ERROR_OUT_OF_DEVICE_MEMORY or OUT_OF_HOST_MEMORY, it waits for the GPU's earlier work (`vkQueueWaitIdle`, under the queue's lock that the caller already holds) and submits again, up to 3 tries. The Vulkan spec leaves a refused submit's command buffers, fence and semaphores unchanged, so the same submit can be made again. The next present (`texture_frame`) then frees every idle staging arena and the image reaper's whole queue. Nothing is destroyed under the queue's lock: a destroy's `rec::boundary` can wait for the recorder, whose part submit waits for that lock. `RX_VK_SUBMIT_RETRY=0` (kill switch) drops the submit as before. `RX_VK_TEST_SUBMIT_REFUSE=<n>` (diag) refuses the first try of every n-th submit, so any card can run this path. - Docs: architecture.md, "Each heap's budget". **Read first:** `create_buffer` (vk_vma.cpp), `submit_or_retry` and `submit_give_back` (vk_replay.cpp), and the `memory_refused` block in `texture_frame` (vk_texture.cpp). **To see it on main:** a Legion Go or ROG Ally (8 GB UMA, Windows, Adrenalin 25.30.44), load a save or enter the Deku Tree. The log shows `heap 2 ... used 252 of 243 MB`, then `submit N failed (-2)`, then a black screen. Without the hardware: `tests/test_submit_oom.py` fails on main. Fixes #1879. See #1878 (duplicate), #947, #668. ## Testing **Tested:** Linux, RTX 3090, NVIDIA 620.70.06, this PR @ 038d85ca as a pack over main 9dd41326's base, through the job queue. A memory tour from `kokiri-start` through Hyrule Castle (teleport 4) and Inside the Deku Tree (19), 40 s each, with `RX_VK_TEST_SUBMIT_REFUSE=200`: the log has `vk: the refused submit was taken on try 2` 16 times, no `failed (-2)`, no crash or device loss, and the tour finished with a clean quit. Peak game total 11.5 GB (resident + swapped), peak private 4.1 GB. On this PC (resizable BAR) the placement path has room in heap 0, so it changes nothing here; the APU heap's overrun can only be shown on the Test Farm. The game-free test `tests/test_submit_oom.py` is written but has not been run on this PC (no tests run locally). CI runs it. It is written to fail on main for the bug's reason: a 16 MB staging arena with heap 2 at 236 of 243 MB lands in heap 2, at 252 of 243 MB, and a refused submit is made only once. **Still to test:** - This PC (Linux, RTX 3090): build the pack, then do a memtour from `kokiri-start` through Hyrule Castle (teleport 4) and the Deku Tree (19). Run three ways: default; with `RX_VK_TEST_SUBMIT_REFUSE=200`, expecting `vk: the refused submit was taken on try 2`, a normal picture and no `failed (-2)`; and with `RX_VK_TEST_SUBMIT_REFUSE=200 RX_VK_SUBMIT_RETRY=0`, expecting dropped submits and black or missing frames as on main (by hand). - Windows, AMD APU (780M/Z1E, 8 GB UMA): load a save, enter Hyrule Castle and the Deku Tree. Expect no `submit ... failed (-2)`, a picture throughout, and `heap 2 (device) ... used X of 243 MB` staying at or under 243 (`@Beanstalk` below). - Steam Deck: the same route, expecting no regression in load time or fps (`@Beanstalk` below). ``` @Beanstalk test Windows AMD iGPUs (windows+amd+igpu): from the save kokiri-start, load the save, teleport to Hyrule Castle and then Inside the Deku Tree, 30 s in each with the camera turning. Fail on any "vk: submit" or "batched submit" line with "failed (-2)", a black screen, or a "memory heaps" line where a device heap's "used" is over its budget. Compare load times and peak VRAM with main. ``` ``` @Beanstalk test Steam Deck (steamdeck): from the save kokiri-start, load the save, teleport to Hyrule Castle and Inside the Deku Tree, 30 s each. Compare load time, frame time p99 and peak VRAM with main. Fail on any "failed (-2)" or "failed (-4)" submit line, or a crash. ``` ## Measurements **Verdict:** not measured yet. The build was stopped by memory pressure before anything ran. Setup (planned): Linux, RTX 3090, NVIDIA 620.70.06; memtour `kokiri-start` through teleports 4 and 19; peak VRAM, RAM and the `vk: memory heaps:` / `vk: budget:` lines, main against this PR, 2 runs each. Expected: no change on this PC. With resizable BAR, heap 0 has room, so placement is unchanged, and the retry only runs on a refused submit. | | Before: main @ 9dd41326 | After: this PR @ 038d85ca | |---|---|---| | peak VRAM / RAM (memtour) | not run | not run | | `tests/test_submit_oom.py` | expected to fail (CI) | expected to pass (CI) | Evidence from #1879's logs (v3.0.0, the same 5 attachments): `memory heaps: ... heap 2 (device): 55 blocks 252 MB, 58 allocations 240 MB; used 252 of 243 MB` next to `VRAM: 1877 MB used of 7782 MB allowed`, and `submit 7687 / 78536 / 109249 / 194971 failed (-2)` in each log after new staging arenas were made. ## Risks - A host-visible buffer can now land in system memory instead of a full device-local heap. That only happens while the heap it would have used is past its budget, where today the driver either pages or refuses. A dGPU with resizable BAR close to its VRAM budget may stage through system memory there, which is slightly slower to read on the GPU. `RX_VK_BUFFER_BUDGET=0` turns it off. - A refused submit now waits for the GPU to go idle, under the queue's lock, before trying again. That is a hitch where there used to be a dropped frame. It only happens on a refusal. `RX_VK_SUBMIT_RETRY=0` turns it off. - Not covered: the VK_ERROR_DEVICE_LOST (-4) and GPU-hang reports in #947 (nightly-394, rc.7). Those are a different failure. - Nothing changes in saves, options or caches. ## Checklist ### Ready for review - [x] One topic, on current `main`; the title is a Conventional Commit. - [x] Tested on my own PC, in the game for a change the game runs, as Testing says. - [ ] `mise run test` passes (any failure that also fails on `main` is named). (not run on this PC by rule; CI runs it) - [ ] Measured against `main`, or Measurements says why there is nothing to measure. (not yet) - [x] No game files, build output or hand-edited generated files; `third_party/` untouched; nothing GPL-incompatible. ### When it applies - [ ] A fix: a game-free test that fails on `main` and passes here. (written as test then fix; not run here, CI runs it) - [x] A switch added or changed: it ships at the default the measurements chose, with its row in `gpu/vk/switches.def`. (two kill switches, on by default, and a diag) - [ ] Host code builds on Windows (llvm-mingw) and Linux, with UTF-8 paths. (Linux built here; Windows: CI) - [x] Docs updated. - [x] AI assistance recorded with `Assisted-by:` / `Co-Authored-By:` trailers. ### Ready to merge - [ ] Everything in Still to test has run, its results in Tested. - [ ] Tier 1 hardware it can affect is tested (docs/dev/standards.md, "Hardware tiers"). - [ ] Approved in review, every comment answered. - [ ] CI passes on the head; on current `main`, measured again or a comment says why the numbers still hold. 🤖 Generated with [Claude Code](https://claude.com/claude-code) https://claude.ai/code/session_017vw8e2uxGKQxzDeQ15jjNK
Co-Authored-By: Claude Opus 5.5 <[email protected]>
fix(vk): host-visible buffers within their heap's budget, and a submit refused for want of memory made again (#1879)
Some checks failed
CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Has been skipped
CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Has been skipped
CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Has been skipped
Third-party parts / The same as upstream (pull_request) Has been skipped
Website / Website on Cloudflare (pull_request) Has been skipped
CI / Conventional Commit title (pull_request) Successful in 8s
CI / Workflow lint (actionlint) (pull_request) Successful in 13s
Guide screenshots / Guide images and local UI source review (pull_request) Failing after 14s
Third-party parts / As their notes say (pull_request) Successful in 15s
Website / Website on Cloudflare (pull_request_target) Has been skipped
CI / Website build and checks (pull_request) Successful in 3m18s
PR description / Description follows the template (pull_request_target) Failing after 9s
beanstalk/approval approved by HumanBean (reviewed)
beanstalk/windows+amd+igpu 0/1 pass, 0 running, 0 queued
beanstalk/steamdeck 2/1 fail
038d85ca03
On AMD APUs on Windows (Legion Go, ROG Ally, 780M) the 256 MB DEVICE_LOCAL |
HOST_VISIBLE heap took the staging arenas past its 243 MB budget on a save
load or an area's entrance, and the driver refused every submit after
(VK_ERROR_OUT_OF_DEVICE_MEMORY), which the replay dropped: a black screen.

- create_buffer: a host-visible buffer within its heap's budget first (VMA
  takes the next memory type), as before where no heap has room
  (RX_VK_BUFFER_BUDGET=0: as before)
- submit_or_retry: a refused submit is made again after the GPU's earlier
  work, up to 3 tries; the next present frees the idle staging arenas and the
  reaper's queue (RX_VK_SUBMIT_RETRY=0: dropped, as before;
  RX_VK_TEST_SUBMIT_REFUSE=<n> plays it on any card)

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Author
Owner

Tested in game: Linux, RTX 3090, NVIDIA 620.70.06, this PR @ 038d85ca over main 9dd41326's base, through the job queue: memory tour from kokiri-start through Hyrule Castle and Inside the Deku Tree with RX_VK_TEST_SUBMIT_REFUSE=200. 16 'the refused submit was taken on try 2' lines, no failed (-2), no crash, clean quit. The APU heap overrun itself needs the Test Farm.

Tested in game: Linux, RTX 3090, NVIDIA 620.70.06, this PR @ 038d85ca over main 9dd41326's base, through the job queue: memory tour from kokiri-start through Hyrule Castle and Inside the Deku Tree with RX_VK_TEST_SUBMIT_REFUSE=200. 16 'the refused submit was taken on try 2' lines, no failed (-2), no crash, clean quit. The APU heap overrun itself needs the Test Farm.
Author
Owner

@Beanstalk test Windows AMD iGPUs (windows+amd+igpu): from the save kokiri-start, load the save, teleport to Hyrule Castle and then Inside the Deku Tree, 30 s in each with the camera turning. Fail on any "vk: submit" or "batched submit" line with "failed (-2)", a black screen, or a "memory heaps" line where a device heap's "used" is over its budget. Compare load times and peak VRAM with main.

@Beanstalk test Windows AMD iGPUs (windows+amd+igpu): from the save kokiri-start, load the save, teleport to Hyrule Castle and then Inside the Deku Tree, 30 s in each with the camera turning. Fail on any "vk: submit" or "batched submit" line with "failed (-2)", a black screen, or a "memory heaps" line where a device heap's "used" is over its budget. Compare load times and peak VRAM with main.
Author
Owner

@Beanstalk test Steam Deck (steamdeck): from the save kokiri-start, load the save, teleport to Hyrule Castle and Inside the Deku Tree, 30 s each. Compare load time, frame time p99 and peak VRAM with main. Fail on any "failed (-2)" or "failed (-4)" submit line, or a crash.

@Beanstalk test Steam Deck (steamdeck): from the save kokiri-start, load the save, teleport to Hyrule Castle and Inside the Deku Tree, 30 s each. Compare load time, frame time p99 and peak VRAM with main. Fail on any "failed (-2)" or "failed (-4)" submit line, or a crash.
Member

Beanstalk test farm · 🔄 testing (2 of 2 reports so far, a failure already)

Commit 038d85c · approved by HumanBean (reviewed) · dashboard

⚠️ Note: The farm cannot compare the two numbers in each heap log line, so it cannot enforce the requested per-device-heap budget limit.
⚠️ Note: Pictures are compared at two sampled moments; this can catch a black-screen regression against main but cannot guarantee detection throughout the tour or when main is also black.
⚠️ Note: Camera turns occur before each 30-second measurement, not during it; the shared submit rule also rejects device-lost submits on Windows.

Done (started 15:49 UTC, last report 16:04 UTC). Run 2: re-planned by HumanBean. Comment @Beanstalk to run it again.

Target Result Reports Frame time p99 (ms), main → PR VRAM peak (MB) Crashes
windows+amd+igpu ⏳ queued 0 of 1 — — —
steamdeck ❌ fail (an outlier) 2 1 2 36.8 → 37.9 (+3%) 2,359 → 2,188 (−7%) none

Why it failed

  • steamdeck: castle-deku-tour screenshot.max_diff: limit 0.1
What was tested

What will be tested · AI plan (gpt-6.1-terra)

Windows AMD iGPUs and Steam Decks will load kokiri-start and visit Hyrule Castle and the Deku Tree, with camera turns and 30-second measurements at each stop. The roughly 13-minute run per tester compares load time, frame-time p99, peak VRAM and pictures with main. Crashes, failed memory or device-lost submits, substantial performance or VRAM regressions, and sufficiently different pictures fail the plan. No tester online now matches windows+amd+igpu: that target waits for one.

  • Where: windows+amd+igpu (1-2 reports), steamdeck (1-2 reports).
  • castle-deku-tour: from the repository's save kokiri-start, then teleport to Hyrule Castle, move the stick, take the picture castle to compare with main, measure castle for 30 s (after 0 s warm-up), teleport to Inside the Deku Tree, take the picture deku to compare with main, measure deku for 30 s (after 0 s warm-up); 1 run on each side.
  • Measured: crashes, load times, frame times, stutters and hitches, VRAM use, the game's log.
  • The PR fails if: it crashes; or a picture differs from main by more than 5%; or peak VRAM is more than 10% above main; or longest load is more than 20% above main; or frame time (p99) is more than 15% above main; or forbidden log lines is above 0.
  • Time: about 12 min of game runs per tester (2 game starts), plus the two builds.
Measurements on `steamdeck`, for the PR description

Verdict: fail on steamdeck (2 reports): castle-deku-tour screenshot.max_diff failed.

Hardware: 1: linux+amd+radv+igpu+handheld AMD Custom GPU 0405 (RADV VANGOGH), 26.1.99 AMD Custom APU 0405, 12 GB RAM; 2: linux+amd+radv+igpu+handheld AMD Custom GPU 0405 (RADV VANGOGH), 26.1.99 AMD Custom APU 0405, 12 GB RAM
Scene: castle-deku-tour, from the repository's save kokiri-start; each cell's settings and switches as the table names them
Method: the Beanstalk test farm: main and this PR built and run on the same PC in one session, game starts alternating (ABBA); castle-deku-tour: 1 game start a side, castle 30 s after 0 s warm-up, deku 30 s after 0 s warm-up; each number the median of the runs, then of 2 reports
How to reproduce: the farm's plan above (@Beanstalk on this PR runs it again)

Before: main @ 9dd4132 After: this PR @ 038d85c
castle-deku-tour · deku · peak VRAM (MB) 2,359 2,188 (−7%)
castle-deku-tour · castle · peak VRAM (MB) 2,379 2,352 (−1%)
castle-deku-tour · longest load (s) 8.7 8.4 (−3%)
castle-deku-tour · deku · frame time (p99) (ms) 36.8 37.9 (+3%)
castle-deku-tour · castle · frame time (p99) (ms) 43.0 43.4 (+1%)
castle-deku-tour · forbidden log lines 0.0 0.0

Flags and settings

None: the plan varied no setting, switch or build option.

As Markdown for the PR description:

## Measurements

**Verdict:** fail on steamdeck (2 reports): `castle-deku-tour screenshot.max_diff` failed.

Hardware: 1: linux+amd+radv+igpu+handheld `AMD Custom GPU 0405 (RADV VANGOGH), 26.1.99` `AMD Custom APU 0405, 12 GB RAM`; 2: linux+amd+radv+igpu+handheld `AMD Custom GPU 0405 (RADV VANGOGH), 26.1.99` `AMD Custom APU 0405, 12 GB RAM`  
Scene: castle-deku-tour, from the repository's save kokiri-start; each cell's settings and switches as the table names them  
Method: the Beanstalk test farm: main and this PR built and run on the same PC in one session, game starts alternating (ABBA); castle-deku-tour: 1 game start a side, castle 30 s after 0 s warm-up, deku 30 s after 0 s warm-up; each number the median of the runs, then of 2 reports  
How to reproduce: the farm's plan above (`@Beanstalk` on this PR runs it again)

| | Before: main @ 9dd4132 | After: this PR @ 038d85c |
|---|---|---|
| castle-deku-tour · deku · peak VRAM (MB) | 2,359 | 2,188 (−7%) |
| castle-deku-tour · castle · peak VRAM (MB) | 2,379 | 2,352 (−1%) |
| castle-deku-tour · longest load (s) | 8.7 | 8.4 (−3%) |
| castle-deku-tour · deku · frame time (p99) (ms) | 36.8 | 37.9 (+3%) |
| castle-deku-tour · castle · frame time (p99) (ms) | 43.0 | 43.4 (+1%) |
| castle-deku-tour · forbidden log lines | 0.0 | 0.0 |

**Flags and settings**

None: the plan varied no setting, switch or build option.
The testers' hardware
Target Report Ran on
steamdeck 1 linux+amd+radv+igpu+handheld AMD Custom GPU 0405 (RADV VANGOGH), 26.1.99 AMD Custom APU 0405, 12 GB RAM
steamdeck 2 linux+amd+radv+igpu+handheld AMD Custom GPU 0405 (RADV VANGOGH), 26.1.99 AMD Custom APU 0405, 12 GB RAM
<!-- beanstalk:sticky --> ### Beanstalk test farm · 🔄 testing (2 of 2 reports so far, a failure already) Commit `038d85c` · approved by HumanBean (reviewed) · [dashboard](https://beanstalk.magicbean.dev/prs/2081) ⚠️ **Note:** The farm cannot compare the two numbers in each heap log line, so it cannot enforce the requested per-device-heap budget limit. ⚠️ **Note:** Pictures are compared at two sampled moments; this can catch a black-screen regression against main but cannot guarantee detection throughout the tour or when main is also black. ⚠️ **Note:** Camera turns occur before each 30-second measurement, not during it; the shared submit rule also rejects device-lost submits on Windows. Done (started 15:49 UTC, last report 16:04 UTC). Run 2: re-planned by HumanBean. Comment `@Beanstalk` to run it again. | Target | Result | Reports | Frame time p99 (ms), main → PR | VRAM peak (MB) | Crashes | | --- | --- | --- | --- | --- | --- | | `windows+amd+igpu` | ⏳ queued | 0 of 1 | — | — | — | | `steamdeck` | ❌ fail (an outlier) | 2 [1](https://beanstalk.magicbean.dev/reports/rp_249da7a74fa6dd41) [2](https://beanstalk.magicbean.dev/reports/rp_01e0fef21f03c9ae) | 36.8 → 37.9 (+3%) | 2,359 → 2,188 (−7%) | none | **Why it failed** - `steamdeck`: `castle-deku-tour` `screenshot.max_diff`: `limit 0.1` <details><summary>What was tested</summary> **What will be tested** · AI plan (`gpt-6.1-terra`) Windows AMD iGPUs and Steam Decks will load kokiri-start and visit Hyrule Castle and the Deku Tree, with camera turns and 30-second measurements at each stop. The roughly 13-minute run per tester compares load time, frame-time p99, peak VRAM and pictures with main. Crashes, failed memory or device-lost submits, substantial performance or VRAM regressions, and sufficiently different pictures fail the plan. No tester online now matches windows+amd+igpu: that target waits for one. - **Where:** windows+amd+igpu (1-2 reports), steamdeck (1-2 reports). - **castle-deku-tour:** from the repository's save `kokiri-start`, then teleport to Hyrule Castle, move the stick, take the picture castle to compare with main, measure castle for 30 s (after 0 s warm-up), teleport to Inside the Deku Tree, take the picture deku to compare with main, measure deku for 30 s (after 0 s warm-up); 1 run on each side. - **Measured:** crashes, load times, frame times, stutters and hitches, VRAM use, the game's log. - **The PR fails if:** it crashes; or a picture differs from main by more than 5%; or peak VRAM is more than 10% above main; or longest load is more than 20% above main; or frame time (p99) is more than 15% above main; or forbidden log lines is above 0. - **Time:** about 12 min of game runs per tester (2 game starts), plus the two builds. </details> <details><summary>Measurements on `steamdeck`, for the PR description</summary> **Verdict:** fail on steamdeck (2 reports): `castle-deku-tour screenshot.max_diff` failed. Hardware: 1: linux+amd+radv+igpu+handheld `AMD Custom GPU 0405 (RADV VANGOGH), 26.1.99` `AMD Custom APU 0405, 12 GB RAM`; 2: linux+amd+radv+igpu+handheld `AMD Custom GPU 0405 (RADV VANGOGH), 26.1.99` `AMD Custom APU 0405, 12 GB RAM` Scene: castle-deku-tour, from the repository's save kokiri-start; each cell's settings and switches as the table names them Method: the Beanstalk test farm: main and this PR built and run on the same PC in one session, game starts alternating (ABBA); castle-deku-tour: 1 game start a side, castle 30 s after 0 s warm-up, deku 30 s after 0 s warm-up; each number the median of the runs, then of 2 reports How to reproduce: the farm's plan above (`@Beanstalk` on this PR runs it again) | | Before: main @ 9dd4132 | After: this PR @ 038d85c | |---|---|---| | castle-deku-tour · deku · peak VRAM (MB) | 2,359 | 2,188 (−7%) | | castle-deku-tour · castle · peak VRAM (MB) | 2,379 | 2,352 (−1%) | | castle-deku-tour · longest load (s) | 8.7 | 8.4 (−3%) | | castle-deku-tour · deku · frame time (p99) (ms) | 36.8 | 37.9 (+3%) | | castle-deku-tour · castle · frame time (p99) (ms) | 43.0 | 43.4 (+1%) | | castle-deku-tour · forbidden log lines | 0.0 | 0.0 | **Flags and settings** None: the plan varied no setting, switch or build option. As Markdown for the PR description: ```markdown ## Measurements **Verdict:** fail on steamdeck (2 reports): `castle-deku-tour screenshot.max_diff` failed. Hardware: 1: linux+amd+radv+igpu+handheld `AMD Custom GPU 0405 (RADV VANGOGH), 26.1.99` `AMD Custom APU 0405, 12 GB RAM`; 2: linux+amd+radv+igpu+handheld `AMD Custom GPU 0405 (RADV VANGOGH), 26.1.99` `AMD Custom APU 0405, 12 GB RAM` Scene: castle-deku-tour, from the repository's save kokiri-start; each cell's settings and switches as the table names them Method: the Beanstalk test farm: main and this PR built and run on the same PC in one session, game starts alternating (ABBA); castle-deku-tour: 1 game start a side, castle 30 s after 0 s warm-up, deku 30 s after 0 s warm-up; each number the median of the runs, then of 2 reports How to reproduce: the farm's plan above (`@Beanstalk` on this PR runs it again) | | Before: main @ 9dd4132 | After: this PR @ 038d85c | |---|---|---| | castle-deku-tour · deku · peak VRAM (MB) | 2,359 | 2,188 (−7%) | | castle-deku-tour · castle · peak VRAM (MB) | 2,379 | 2,352 (−1%) | | castle-deku-tour · longest load (s) | 8.7 | 8.4 (−3%) | | castle-deku-tour · deku · frame time (p99) (ms) | 36.8 | 37.9 (+3%) | | castle-deku-tour · castle · frame time (p99) (ms) | 43.0 | 43.4 (+1%) | | castle-deku-tour · forbidden log lines | 0.0 | 0.0 | **Flags and settings** None: the plan varied no setting, switch or build option. ``` </details> <details><summary>The testers' hardware</summary> | Target | Report | Ran on | | --- | --- | --- | | `steamdeck` | [1](https://beanstalk.magicbean.dev/reports/rp_249da7a74fa6dd41) | linux+amd+radv+igpu+handheld `AMD Custom GPU 0405 (RADV VANGOGH), 26.1.99` `AMD Custom APU 0405, 12 GB RAM` | | `steamdeck` | [2](https://beanstalk.magicbean.dev/reports/rp_01e0fef21f03c9ae) | linux+amd+radv+igpu+handheld `AMD Custom GPU 0405 (RADV VANGOGH), 26.1.99` `AMD Custom APU 0405, 12 GB RAM` | </details>
Member

Possibly related, for context:

  • issue #1769 Longshot 3.0.0 - RX 5500 XT - 100% RAM usage, black screen and system freeze — Possible separate AMD memory-pressure black screen; investigate overlap
  • PR #1164 WIP: fix(vk): fall back to host pointers on udmabuf rejection and retry mirror allocation — Open Vulkan memory-allocation retry/fallback work; may overlap allocator changes
Possibly related, for context: - issue #1769 Longshot 3.0.0 - RX 5500 XT - 100% RAM usage, black screen and system freeze — Possible separate AMD memory-pressure black screen; investigate overlap - PR #1164 WIP: fix(vk): fall back to host pointers on udmabuf rejection and retry mirror allocation — Open Vulkan memory-allocation retry/fallback work; may overlap allocator changes
HumanBean changed title from WIP: fix(vk): host-visible buffers stay within their heap's budget, and a submit refused for want of memory is made again to fix(vk): host-visible buffers stay within their heap's budget, and a submit refused for want of memory is made again 2026-11-21 17:01:33 +00:00
HumanBean deleted branch fix/uma-heap-budget-submit-retry 2026-11-21 17:01:47 +00:00
Sign in to join this conversation.
No reviewers
No milestone
No project
No assignees
3 participants
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
MagicBeanDev/Longshot!2081
No description provided.