WIP: fix(vk): memory is added up and given back under pressure, host writes are marked, and late pages are found when a capture is taken #1339

Draft
HumanBean wants to merge 10 commits from HumanBean/Longshot:feat/mem-account into main
Owner

Type: fix · Risk: medium · Tested on: Linux, RTX 3090 · Needs: a card of 8 GB or less; an integrated GPU; a Windows run

Why

Tracks A and B of the renderer rework plan: memory. Nine of 55 grouped crash reports end at a memory limit and ten more are VRAM-full during play, yet nothing added up what the renderer holds and only the texture cache answered to pressure; and 'stale data after the GPU writes' kept being fixed one case at a time. This is the one pull request for memory; it took in #1360. Stacked on #1309: review the commits after it. See #1034, which fixes the same host writes with its measurement tools.

What changed

Read the commits in order; each stands alone.

  1. One log line adds up every pool (vk: budget:), against the driver's count, in two ledgers.
  2. Pools kept for reuse give memory back before textures in use are evicted (an arbiter on the texture cache's own 90% and 97% lines), every allocator buffer is counted, and the mirrors' rule is one rule again: cards of 4.5 GB or less get 256 MB of mirrors where they had a third of the card.
  3. Host writes that zero imported guest memory mark it (three at a free, two more found by an audit), with a check that fails on a new unmarked one. Windows now zeroes imported pages in place, as Linux does, and no longer discards them under the GPU's mapping. RX_VK_MEM_CHECK=1 counts where the sync breaks its rules.
  4. A capture's pages that a write-back wrote since are found late when the capture is taken, and merged; the generation counter is 64 bits (32 wrapped after about 52 hours).
  5. The checker says what it cannot tell. A sync that kept every word the CPU's copy left alone doubled the GPU's time (the game resets counters the GPU fills by writing zero): it never shipped, and the docs say so.

Measurements

Verdict: no change in fps without pressure; under a simulated 6.5 GB allowance the arbiter gave back 192 to 232 MB at a time and textures settled at 3 GB.

  • 'Unaccounted' VRAM on the RTX 3090 fell from 426 MB to about 280 MB once buffers were counted.
  • The late-page class goes from 2,640 candidates a session to 0; 2,600 to 5,700 pages a session are merged. No certain loss was found (0 of 145,000 syncs on a page the CPU had not written), and no visible bug is shown fixed.
  • This head, one session: Kokiri Forest 63.2 fps, Hyrule Field 91.1, the Market 69.0 (main 63.2 to 64.9, 90.3 to 90.7, 68.0 to 72.7).

Testing

Tested:

  • python -I -B tests/run_all.py: 663 passed, 44 failed, 19 skipped. tests/test_mirror_budget.py, which fails on main, passes here. On main at 8e9cfc3a this PC has 46 checks failing (648 pass, 19 skipped); none of them is new here.
  • In the game (Linux, RTX 3090, NVIDIA 620.70, 2560x1440, no upscaler): one session of this head through the three areas, no error line; earlier: sessions with RX_VK_TEST_ALLOWED_MB=8000 and 6500, with RX_VK_MEM_CHECK=1, and with RX_VK_LATE_AT_REPLAY=0 against on.

Not tested:

  • A real small card: the mirror change cannot be seen on a 24 GB card at all, and the arbiter ran only under a simulated allowance.
  • Windows: not compiled; the discard change there is argued, not run.
  • An integrated GPU (imports are charged to VRAM there).
  • The Market's long frames under pressure against main under the same allowance.

Must be tested:

  • A card of 8 GB or less, two laps of scripts/memtour.py: VRAM and fps as on main or better, unaccounted not growing.
  • Windows, any GPU: a start, three area changes, ten minutes (Test Farm: @Beanstalk windows nvidia and windows amd: main against this PR, Kokiri Forest, Hyrule Field, the Market, three area changes).
  • A Steam Deck: the same.

Risks

What could break: a small card with 256 MB of mirrors reads pools over PCIe that fitted before; staging freed on entering pressure is made again at the next load; on Windows freed imported pages cost a memset.

Turning it off: RX_VK_LATE_AT_REPLAY=0 for the late-page fix; RX_VK_MIRROR_MB for the mirrors; the arbiter has no switch.

Compatibility: none: no cache is rebuilt, MKWVK_API_VERSION stays.

Issues

See #1034

Checklist

Everyone

  • One topic.
  • The title is a Conventional Commit (type(scope): description) of the status line's type.
  • Why gives the reason plainly and links the issues that fit (the tracker searched).
  • Measurements has a verdict, or one line saying why there is nothing to measure.
  • Testing names what was tested, what wasn't, and what must still be tested, where and how.
  • Rebased on current main.
  • python -I -B tests/run_all.py passes (any failure that also fails on main is named).
  • No game files, dumps, build output or generated files.
  • third_party/ untouched (a new folder with a PROVENANCE.md instead).
  • The license is OK: GPL-3.0-or-later, nothing incompatible and nothing decompiled.

When it applies

  • AI assistance recorded with Assisted-by: / Co-Authored-By: trailers.
  • Numbers from the same PC, scene and settings, at least 2 interleaved runs each, the median reported.
  • Switches documented (docs/development.md; the README for a menu row) and shipped at their recommended value.
  • A check under tests/ that reads changed source updated in this PR, not weakened.
  • Host code builds on both Windows (llvm-mingw) and Linux, with UTF-8 paths. Built on Linux only here.
  • Docs updated.

🤖 Generated with Claude Code

https://claude.ai/code/session_017vw8dzCsAL3UdW7ft8ptL9

**Type:** fix · **Risk:** medium · **Tested on:** Linux, RTX 3090 · **Needs:** a card of 8 GB or less; an integrated GPU; a Windows run ## Why Tracks A and B of the [renderer rework plan](https://git.magicbean.dev/MagicBeanDev/Longshot/wiki/Renderer-rework-plan): memory. Nine of 55 grouped crash reports end at a memory limit and ten more are VRAM-full during play, yet nothing added up what the renderer holds and only the texture cache answered to pressure; and 'stale data after the GPU writes' kept being fixed one case at a time. This is the one pull request for memory; it took in #1360. Stacked on #1309: review the commits after it. See #1034, which fixes the same host writes with its measurement tools. ## What changed Read the commits in order; each stands alone. 1. **One log line adds up every pool** (`vk: budget:`), against the driver's count, in two ledgers. 2. **Pools kept for reuse give memory back before textures in use are evicted** (an arbiter on the texture cache's own 90% and 97% lines), every allocator buffer is counted, and **the mirrors' rule is one rule again: cards of 4.5 GB or less get 256 MB of mirrors** where they had a third of the card. 3. **Host writes that zero imported guest memory mark it** (three at a free, two more found by an audit), with a check that fails on a new unmarked one. **Windows now zeroes imported pages in place**, as Linux does, and no longer discards them under the GPU's mapping. `RX_VK_MEM_CHECK=1` counts where the sync breaks its rules. 4. **A capture's pages that a write-back wrote since are found late when the capture is taken**, and merged; the generation counter is 64 bits (32 wrapped after about 52 hours). 5. **The checker says what it cannot tell.** A sync that kept every word the CPU's copy left alone doubled the GPU's time (the game resets counters the GPU fills by writing zero): it never shipped, and the docs say so. ## Measurements **Verdict:** no change in fps without pressure; under a simulated 6.5 GB allowance the arbiter gave back 192 to 232 MB at a time and textures settled at 3 GB. - 'Unaccounted' VRAM on the RTX 3090 fell from 426 MB to about 280 MB once buffers were counted. - The late-page class goes from 2,640 candidates a session to 0; 2,600 to 5,700 pages a session are merged. **No certain loss was found** (0 of 145,000 syncs on a page the CPU had not written), and no visible bug is shown fixed. - This head, one session: Kokiri Forest 63.2 fps, Hyrule Field 91.1, the Market 69.0 (main 63.2 to 64.9, 90.3 to 90.7, 68.0 to 72.7). ## Testing **Tested:** - `python -I -B tests/run_all.py`: 663 passed, 44 failed, 19 skipped. `tests/test_mirror_budget.py`, which fails on `main`, passes here. On `main` at 8e9cfc3a this PC has 46 checks failing (648 pass, 19 skipped); none of them is new here. - In the game (Linux, RTX 3090, NVIDIA 620.70, 2560x1440, no upscaler): one session of this head through the three areas, no error line; earlier: sessions with `RX_VK_TEST_ALLOWED_MB=8000` and `6500`, with `RX_VK_MEM_CHECK=1`, and with `RX_VK_LATE_AT_REPLAY=0` against on. **Not tested:** - **A real small card**: the mirror change cannot be seen on a 24 GB card at all, and the arbiter ran only under a simulated allowance. - Windows: not compiled; the discard change there is argued, not run. - An integrated GPU (imports are charged to VRAM there). - The Market's long frames under pressure against `main` under the same allowance. **Must be tested:** - A card of 8 GB or less, two laps of `scripts/memtour.py`: VRAM and fps as on main or better, `unaccounted` not growing. - Windows, any GPU: a start, three area changes, ten minutes (Test Farm: `@Beanstalk windows nvidia and windows amd: main against this PR, Kokiri Forest, Hyrule Field, the Market, three area changes`). - A Steam Deck: the same. ## Risks **What could break:** a small card with 256 MB of mirrors reads pools over PCIe that fitted before; staging freed on entering pressure is made again at the next load; on Windows freed imported pages cost a memset. **Turning it off:** `RX_VK_LATE_AT_REPLAY=0` for the late-page fix; `RX_VK_MIRROR_MB` for the mirrors; the arbiter has no switch. **Compatibility:** none: no cache is rebuilt, `MKWVK_API_VERSION` stays. ## Issues See #1034 ## Checklist ### Everyone - [x] One topic. - [x] The title is a Conventional Commit (`type(scope): description`) of the status line's type. - [x] Why gives the reason plainly and links the issues that fit (the tracker searched). - [x] Measurements has a verdict, or one line saying why there is nothing to measure. - [x] Testing names what was tested, what wasn't, and what must still be tested, where and how. - [x] Rebased on current `main`. - [x] `python -I -B tests/run_all.py` passes (any failure that also fails on `main` is named). - [x] No game files, dumps, build output or generated files. - [x] `third_party/` untouched (a new folder with a `PROVENANCE.md` instead). - [x] The license is OK: GPL-3.0-or-later, nothing incompatible and nothing decompiled. ### When it applies - [x] AI assistance recorded with `Assisted-by:` / `Co-Authored-By:` trailers. - [x] Numbers from the same PC, scene and settings, at least 2 interleaved runs each, the median reported. - [x] Switches documented (docs/development.md; the README for a menu row) and shipped at their recommended value. - [x] A check under `tests/` that reads changed source updated in this PR, not weakened. - [ ] Host code builds on both Windows (llvm-mingw) and Linux, with UTF-8 paths. Built on Linux only here. - [x] Docs updated. 🤖 Generated with [Claude Code](https://claude.com/claude-code) https://claude.ai/code/session_017vw8dzCsAL3UdW7ft8ptL9
refactor(vk): 93 kill switches and the older paths they kept are deleted from the four large renderer files
All checks were successful
CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Has been skipped
CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Has been skipped
CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Has been skipped
Third-party parts / The same as upstream (pull_request) Has been skipped
Website / Website on Cloudflare (pull_request) Has been skipped
CI / Conventional Commit title (pull_request) Successful in 7s
Guide screenshots / Guide images and local UI source review (pull_request) Successful in 11s
Third-party parts / As their notes say (pull_request) Successful in 14s
CI / Workflow lint (actionlint) (pull_request) Successful in 19s
PR description / Description follows the template (pull_request_target) Successful in 5s
Website / Website on Cloudflare (pull_request_target) Has been skipped
b9805ef295
The renderer rework plan's track F. A kill switch turned off, or back to an older way, something that is on for
every PC; nobody ships with one set, and each kept an old path alive. vk_draw.cpp 32, vk_texture.cpp 21,
vk_replay.cpp 20, vk_memory.cpp 20: the read and the path are gone, the default is the only behaviour, and each name
is a GONE row of gpu/vk/switches.def, so a run that still sets one is told. 26 are kept, each for a reason (a value
set by a tool or named in the README, a default that follows the PC, a young fix's way out).

Co-Authored-By: Claude Fable 5.2 <[email protected]>
Claude-Session: https://claude.ai/code/session_017vw8dzCsAL3UdW7ft8ptL9
feat(vk): one log line adds up what every pool of the renderer holds, against VRAM and RAM
Some checks failed
CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Has been skipped
CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Has been skipped
CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Has been skipped
Third-party parts / The same as upstream (pull_request) Has been skipped
Website / Website on Cloudflare (pull_request) Has been skipped
CI / Conventional Commit title (pull_request) Successful in 7s
PR description / Description follows the template (pull_request_target) Successful in 14s
CI / Workflow lint (actionlint) (pull_request) Successful in 19s
Guide screenshots / Guide images and local UI source review (pull_request) Failing after 20s
Third-party parts / As their notes say (pull_request) Successful in 30s
Website / Website on Cloudflare (pull_request_target) Has been skipped
e33b4731f5
The renderer rework plan's track A, step 0. gpu/vk/mem_account.h is a registry of pools in two ledgers; every 600
presents "vk: budget:" says each pool's bytes, what is kept for reuse or waiting to be freed, and what no pool
reports. scripts/memtour.py records it per stop. Nothing decides by these numbers yet.

Co-Authored-By: Claude Fable 5.2 <[email protected]>
Claude-Session: https://claude.ai/code/session_017vw8dzCsAL3UdW7ft8ptL9
Author
Owner

Tested in game: Linux, RTX 3090, NVIDIA 620.70: one session through Kokiri Forest and Hyrule Field at 2560x1440; seven vk: budget: lines, sums within 427 MB of the driver's count, scenes drew as main.

Tested in game: Linux, RTX 3090, NVIDIA 620.70: one session through Kokiri Forest and Hyrule Field at 2560x1440; seven vk: budget: lines, sums within 427 MB of the driver's count, scenes drew as main.
Author
Owner

A second commit is on this branch: the budget's next step. It changes behaviour under VRAM pressure and on small cards.

What it adds

  • Every buffer made through the allocator is counted (other buffers). On the RTX 3090 unaccounted falls from 426 MB to about 280 MB: what the driver and the upscalers' libraries hold themselves.
  • An arbiter: when pressure begins (the texture cache's own lines, 90% and 97% of the driver's allowance), the pools that hold memory for reuse are asked first, in order: capture spares, staging, free blocks (compaction is brought forward), the render target pool. What they give is credited against the textures' excess. Each calls the trim it already had. Without pressure nothing is asked.
  • The mirrors' rule is whole in mirror_rule::auto_bytes again. Cards of 4.5 GB or less get 256 MB of mirrors, as mirror_budget.h argues, where they had a third of the card (1365 MB on 4 GB). 5 to 6.5 GB cards keep 1700 MB. tests/test_mirror_budget.py, which could not pass on main, is back in line with the source.
  • docs/development/memory-budget.md (new).

Run in the game (RTX 3090, 1440p; RX_VK_TEST_ALLOWED_MB stands in for a smaller card's allowance, not its heap):

Kokiri Forest Hyrule Field the Market textures held
nothing set 62.7 fps 91.0 66.7 8.5 GB
allowance 8000 MB 64.0 92.5 71.3 (17 frames over 25 ms) 4.0 GB
allowance 6500 MB 65.3 91.2 65.6 (15 over 25 ms) 3.0 GB
vk: budget: gave back 192 MB from staging at 96% of the allowance
vk: budget: gave back 37 MB from target pool at 96% of the allowance
vk: budget: gave back 224 MB from staging at 98% of the allowance
vk: budget: gave back 232 MB from target pool at 98% of the allowance

No give-back line without pressure, no device lost. Not tested: a real small card (the mirror change cannot be seen here at all), an integrated GPU, Windows, long play, and tests/run_all.py. The Market's long frames under pressure were not compared with main under the same allowance.

A second commit is on this branch: the budget's next step. It changes behaviour under VRAM pressure and on small cards. **What it adds** - Every buffer made through the allocator is counted (`other buffers`). On the RTX 3090 `unaccounted` falls from 426 MB to about 280 MB: what the driver and the upscalers' libraries hold themselves. - An arbiter: when pressure begins (the texture cache's own lines, 90% and 97% of the driver's allowance), the pools that hold memory for reuse are asked first, in order: capture spares, staging, free blocks (compaction is brought forward), the render target pool. What they give is credited against the textures' excess. Each calls the trim it already had. Without pressure nothing is asked. - The mirrors' rule is whole in `mirror_rule::auto_bytes` again. **Cards of 4.5 GB or less get 256 MB of mirrors, as `mirror_budget.h` argues, where they had a third of the card (1365 MB on 4 GB).** 5 to 6.5 GB cards keep 1700 MB. `tests/test_mirror_budget.py`, which could not pass on main, is back in line with the source. - `docs/development/memory-budget.md` (new). **Run in the game** (RTX 3090, 1440p; `RX_VK_TEST_ALLOWED_MB` stands in for a smaller card's allowance, not its heap): | | Kokiri Forest | Hyrule Field | the Market | textures held | |---|---|---|---|---| | nothing set | 62.7 fps | 91.0 | 66.7 | 8.5 GB | | allowance 8000 MB | 64.0 | 92.5 | 71.3 (17 frames over 25 ms) | 4.0 GB | | allowance 6500 MB | 65.3 | 91.2 | 65.6 (15 over 25 ms) | 3.0 GB | ``` vk: budget: gave back 192 MB from staging at 96% of the allowance vk: budget: gave back 37 MB from target pool at 96% of the allowance vk: budget: gave back 224 MB from staging at 98% of the allowance vk: budget: gave back 232 MB from target pool at 98% of the allowance ``` No give-back line without pressure, no device lost. **Not tested:** a real small card (the mirror change cannot be seen here at all), an integrated GPU, Windows, long play, and `tests/run_all.py`. The Market's long frames under pressure were not compared with main under the same allowance.
feat(vk): pools kept for reuse give memory back before textures in use are evicted, and the mirrors' rule is one rule again
Some checks failed
CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Has been skipped
CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Has been skipped
CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Has been skipped
Third-party parts / The same as upstream (pull_request) Has been skipped
Website / Website on Cloudflare (pull_request) Has been skipped
CI / Conventional Commit title (pull_request) Successful in 6s
PR description / Description follows the template (pull_request_target) Successful in 9s
Guide screenshots / Guide images and local UI source review (pull_request) Failing after 10s
CI / Workflow lint (actionlint) (pull_request) Successful in 11s
Third-party parts / As their notes say (pull_request) Successful in 14s
26b6746940
The renderer rework plan's track A, step 1. Every buffer made through the allocator is counted ('other buffers'), so
'unaccounted' is what the driver and the upscalers' libraries hold themselves (426 -> 280 MB on an RTX 3090). When
VRAM pressure begins, by the texture cache's own 90% and 97% lines, an arbiter asks the pools that hold memory for
reuse in a stated order (capture spares, staging, free blocks, the target pool) and credits what they give against
the textures' excess. Nothing is asked without pressure. mirror_rule::auto_bytes holds the whole mirror rule: cards
of 4.5 GB or less get 256 MB again (they had up to 1536), 5 to 6.5 GB keep 1700.

Co-Authored-By: Claude Fable 5.2 <[email protected]>
Claude-Session: https://claude.ai/code/session_017vw8dzCsAL3UdW7ft8ptL9
HumanBean force-pushed feat/mem-account from 26b6746940
Some checks failed
CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Has been skipped
CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Has been skipped
CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Has been skipped
Third-party parts / The same as upstream (pull_request) Has been skipped
Website / Website on Cloudflare (pull_request) Has been skipped
CI / Conventional Commit title (pull_request) Successful in 6s
PR description / Description follows the template (pull_request_target) Successful in 9s
Guide screenshots / Guide images and local UI source review (pull_request) Failing after 10s
CI / Workflow lint (actionlint) (pull_request) Successful in 11s
Third-party parts / As their notes say (pull_request) Successful in 14s
to 8d13734772
Some checks failed
CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Has been skipped
CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Has been skipped
CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Has been skipped
Third-party parts / The same as upstream (pull_request) Has been skipped
Website / Website on Cloudflare (pull_request) Has been skipped
CI / Conventional Commit title (pull_request) Successful in 12s
Guide screenshots / Guide images and local UI source review (pull_request) Failing after 21s
CI / Workflow lint (actionlint) (pull_request) Successful in 23s
Third-party parts / As their notes say (pull_request) Successful in 38s
PR description / Description follows the template (pull_request_target) Successful in 14s
2026-11-17 15:48:17 +00:00
Compare
HumanBean changed title from WIP: feat(vk): one log line adds up what every pool of the renderer holds, against VRAM and RAM to WIP: fix(vk): memory is added up and given back under pressure, host writes are marked, and late pages are found when a capture is taken 2026-11-17 15:49:41 +00:00
Some checks failed
CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Has been skipped
CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Has been skipped
CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Has been skipped
Third-party parts / The same as upstream (pull_request) Has been skipped
Website / Website on Cloudflare (pull_request) Has been skipped
CI / Conventional Commit title (pull_request) Successful in 12s
Guide screenshots / Guide images and local UI source review (pull_request) Failing after 21s
CI / Workflow lint (actionlint) (pull_request) Successful in 23s
Third-party parts / As their notes say (pull_request) Successful in 38s
PR description / Description follows the template (pull_request_target) Successful in 14s
This pull request has changes conflicting with the target branch.
  • gpu/vk/switches.def
  • gpu/vk/vk_memory.cpp
  • gpu/vk/vk_replay.cpp
  • gpu/vk/vk_texture.cpp
  • tests/test_compressed_over_rt.py
  • tests/test_device_policy.py
  • tests/test_mirror_inherit.py
View command line instructions

Manual merge helper

Use this merge commit message when completing the merge manually.

Checkout

From your project repository, check out a new branch and test the changes.
git fetch -u 'feat/mem-account:HumanBean-feat/mem-account'
git switch 'HumanBean-feat/mem-account'
Sign in to join this conversation.
No reviewers
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
MagicBeanDev/Longshot!1339
No description provided.