WIP: fix(vk): cut a texture cache far over the VRAM rules to 7/8 of their budget #519

Draft

Amateurwizrd wants to merge 1 commit from Amateurwizrd/Longshot:fix/vram-excess-cut into main

pull from: Amateurwizrd/Longshot:fix/vram-excess-cut

merge into: MagicBeanDev:main

MagicBeanDev:main

Conversation 1 Commits 1 Files changed 3 +51 -7

Amateurwizrd commented 2026-11-14 17:36:55 +00:00

Contributor

• Copy link
• Copy Markdown

Why

When an area keeps the texture cache far over the 90% rule's or the driver's limit, the rules let go of only the excess
and 64 MB more, so the cache sits at the line and every pass is another eviction, with what went uploaded again: a
stutter that does not stop while the player stays there.

What changed

• vk: vram_compact::rule_target in gpu/vk/vram_compact.h keeps the trim for an excess of up to a sixteenth of the
  rule's budget (kExcessSmall), so a cache a few MB over the line still does not lose an eighth of its textures; a
  larger excess is cut to 7/8 of that budget, as before the trim. The budget's own size still asks for its 7/8, and
  RX_VK_EVICT_EXCESS=0 (the cut on any excess) is unchanged.
• tests: tests/test_vram_compact.cpp checks the large excess, a grid of budgets and excesses around the edge, and a
  zero budget.
• docs: docs/development.md's RX_VK_EVICT_EXCESS row says what the default does now.

Flags and settings

Flag	What it does	Default now	Recommendation	Helps on	Doesn't help on	Untested on
RX_VK_EVICT_EXCESS=0	The rules cut to 7/8 on any excess, as before the trim (unchanged; its documented default updated)	unset (excess rule)	keep unset	caches far over the line on short cards	caches a few MB over the line (unchanged)	8 and 10 GB cards in the game

Measurements

main: 9c9d69ed (2026-11-14)
this PR: fix/vram-excess-cut @ bf3637f2
Hardware: RTX 5070 Ti 16 GB, driver 621.01, Ryzen 7 7800X3D, 32 GB, Windows 11 Pro 26200
Scene: none (game-free check)
Method: the new test cases, run 2 times on each side, interleaved (main, this PR, main, this PR); main gets this PR's tests/test_vram_compact.cpp with kExcessSmall written as its value, 16, since main has no such constant
How to reproduce: python -I -B tests/run_all.py vram_compact on this branch, and on main with the test file copied in and b / kExcessSmall replaced by b / 16

	main @ 9c9d69ed	this PR @ bf3637f2
tests/test_vram_compact.py (run 1 / run 2)	FAIL / FAIL	pass / pass
rule_target(4022 MB, 4096 MB, 3639 MB): a 383 MB excess over a 3639 MB rule budget	the excess and 64 MB more go: the cache stays at the line	cut to 7/8 of 3639 MB

main @ 9c9d69ed:
FAIL line 67: rule_target(4022 * MB, 4096 * MB, 3639 * MB) == 3639 * MB / 8 * 7
FAIL         0.9s  tests/test_vram_compact.py
this PR @ bf3637f2:
pass         1.0s  tests/test_vram_compact.py
1 passed, 0 failed, 0 skipped

Testing

Tested: python -I -B tests/run_all.py on Windows 11 (llvm-mingw): main @ 9c9d69ed 531 passed, 5 failed; this PR
530 passed and the same 5 failing on main (test_oot_state_tap, test_portable_builder_home,
test_portable_builder_startup, test_tested_configurations, test_walk_key), plus test_gpu_hang_watch, which
fails and passes on reruns of main @ 9c9d69ed itself (a 4 s timing bound) and passed alone on this branch. The renderer
builds.

Not tested: the game on a card whose cache goes far over the line (a 16 GB card never got there in the scenes
tried); a Linux build (no Linux machine here).

Needs other testers:

• 10 GB NVIDIA or AMD card, Windows: Lake Hylia for a few minutes, expecting the eviction lines to stop repeating each pass once the cache is cut, against main where each pass evicts again (request to open)
• Linux (x86_64, gcc or clang): the renderer builds with this PR, expecting no error in gpu/vk/vram_compact.h (request to open)

Screenshots

Nothing on screen

Risk and rollback

A cache far over the line loses an eighth of its textures at once again, as before the trim, which is one larger burst
of uploads instead of an eviction every pass. RX_VK_EVICT_EXCESS=0 cuts on any excess; reverting the commit restores
the trim for every excess.

Reviewer notes

The rule was first seen in Hookshot-old's 030b4e9a, reported there for Lake Hylia on a 10 GB card (383 MB over at each
pass) and not reproduced here; the code and the edge cases are written for Longshot.

Issues

None

Checklist

For everyone

• One topic.
• The title is a Conventional Commit (type(scope): description).
• Rebased on current main.
• python -I -B tests/run_all.py run and passing (any failure that also fails on main is named above).
• Tested in the game, or what wasn't is named under Testing.
• Testing filled in: what was tested, what wasn't, and what needs other testers (with test requests).
• Measured: main and this PR at named commits, on the same PC, scene and settings, at least 2 interleaved runs each, the median reported.
• No game files, dumps, build output or generated files.
• third_party/ untouched (a new folder with a PROVENANCE.md instead).
• The license is OK: GPL-3.0-or-later, nothing incompatible and nothing decompiled.

If applicable

• AI assistance recorded with Assisted-by: / Co-Authored-By: trailers.
• Flags noted, and the switch reference updated (docs/development.md, the README).
• A game-free test added for the fix, failing on main and passing here.
• A check under tests/ that reads changed source updated in this PR, not weakened.
• Host code builds on both Windows (llvm-mingw) and Linux, with UTF-8 paths. (Windows only so far: Linux is a test request to open)
• Docs updated.

## Why When an area keeps the texture cache far over the 90% rule's or the driver's limit, the rules let go of only the excess and 64 MB more, so the cache sits at the line and every pass is another eviction, with what went uploaded again: a stutter that does not stop while the player stays there. ## What changed - vk: `vram_compact::rule_target` in `gpu/vk/vram_compact.h` keeps the trim for an excess of up to a sixteenth of the rule's budget (`kExcessSmall`), so a cache a few MB over the line still does not lose an eighth of its textures; a larger excess is cut to 7/8 of that budget, as before the trim. The budget's own size still asks for its 7/8, and `RX_VK_EVICT_EXCESS=0` (the cut on any excess) is unchanged. - tests: `tests/test_vram_compact.cpp` checks the large excess, a grid of budgets and excesses around the edge, and a zero budget. - docs: `docs/development.md`'s `RX_VK_EVICT_EXCESS` row says what the default does now. ## Flags and settings | Flag | What it does | Default now | Recommendation | Helps on | Doesn't help on | Untested on | |---|---|---|---|---|---|---| | `RX_VK_EVICT_EXCESS=0` | The rules cut to 7/8 on any excess, as before the trim (unchanged; its documented default updated) | unset (excess rule) | keep unset | caches far over the line on short cards | caches a few MB over the line (unchanged) | 8 and 10 GB cards in the game | ## Measurements main: 9c9d69ed (2026-11-14) this PR: fix/vram-excess-cut @ bf3637f2 Hardware: RTX 5070 Ti 16 GB, driver 621.01, Ryzen 7 7800X3D, 32 GB, Windows 11 Pro 26200 Scene: none (game-free check) Method: the new test cases, run 2 times on each side, interleaved (main, this PR, main, this PR); main gets this PR's `tests/test_vram_compact.cpp` with `kExcessSmall` written as its value, 16, since main has no such constant How to reproduce: `python -I -B tests/run_all.py vram_compact` on this branch, and on main with the test file copied in and `b / kExcessSmall` replaced by `b / 16` | | main @ 9c9d69ed | this PR @ bf3637f2 | |---|---|---| | tests/test_vram_compact.py (run 1 / run 2) | FAIL / FAIL | pass / pass | | `rule_target(4022 MB, 4096 MB, 3639 MB)`: a 383 MB excess over a 3639 MB rule budget | the excess and 64 MB more go: the cache stays at the line | cut to 7/8 of 3639 MB | ``` main @ 9c9d69ed: FAIL line 67: rule_target(4022 * MB, 4096 * MB, 3639 * MB) == 3639 * MB / 8 * 7 FAIL 0.9s tests/test_vram_compact.py this PR @ bf3637f2: pass 1.0s tests/test_vram_compact.py 1 passed, 0 failed, 0 skipped ``` ## Testing **Tested:** `python -I -B tests/run_all.py` on Windows 11 (llvm-mingw): main @ 9c9d69ed 531 passed, 5 failed; this PR 530 passed and the same 5 failing on main (`test_oot_state_tap`, `test_portable_builder_home`, `test_portable_builder_startup`, `test_tested_configurations`, `test_walk_key`), plus `test_gpu_hang_watch`, which fails and passes on reruns of main @ 9c9d69ed itself (a 4 s timing bound) and passed alone on this branch. The renderer builds. **Not tested:** the game on a card whose cache goes far over the line (a 16 GB card never got there in the scenes tried); a Linux build (no Linux machine here). **Needs other testers:** - 10 GB NVIDIA or AMD card, Windows: Lake Hylia for a few minutes, expecting the eviction lines to stop repeating each pass once the cache is cut, against main where each pass evicts again (request to open) - Linux (x86_64, gcc or clang): the renderer builds with this PR, expecting no error in `gpu/vk/vram_compact.h` (request to open) ## Screenshots Nothing on screen ## Risk and rollback A cache far over the line loses an eighth of its textures at once again, as before the trim, which is one larger burst of uploads instead of an eviction every pass. `RX_VK_EVICT_EXCESS=0` cuts on any excess; reverting the commit restores the trim for every excess. ## Reviewer notes The rule was first seen in Hookshot-old's 030b4e9a, reported there for Lake Hylia on a 10 GB card (383 MB over at each pass) and not reproduced here; the code and the edge cases are written for Longshot. ## Issues None ## Checklist ### For everyone - [x] One topic. - [x] The title is a Conventional Commit (`type(scope): description`). - [x] Rebased on current `main`. - [x] `python -I -B tests/run_all.py` run and passing (any failure that also fails on `main` is named above). - [x] Tested in the game, or what wasn't is named under Testing. - [x] Testing filled in: what was tested, what wasn't, and what needs other testers (with test requests). - [x] Measured: main and this PR at named commits, on the same PC, scene and settings, at least 2 interleaved runs each, the median reported. - [x] No game files, dumps, build output or generated files. - [x] `third_party/` untouched (a new folder with a `PROVENANCE.md` instead). - [x] The license is OK: GPL-3.0-or-later, nothing incompatible and nothing decompiled. ### If applicable - [x] AI assistance recorded with `Assisted-by:` / `Co-Authored-By:` trailers. - [x] Flags noted, and the switch reference updated (docs/development.md, the README). - [x] A game-free test added for the fix, failing on `main` and passing here. - [x] A check under `tests/` that reads changed source updated in this PR, not weakened. - [ ] Host code builds on both Windows (llvm-mingw) and Linux, with UTF-8 paths. (Windows only so far: Linux is a test request to open) - [x] Docs updated.

Amateurwizrd added 1 commit 2026-11-14 17:36:56 +00:00

fix(vk): a large excess over the 90% rule or the driver's limit is cut to 7/8, a small one still trimmed ... bf3637f22e

Since the rules let go only of their excess and 64 MB more, a cache that an
area keeps well over the line is trimmed to the line at every pass and never
gets under it: each pass is an eviction, and what went is uploaded again. An
excess of up to a sixteenth of the rule's budget keeps the trim (a cache a few
MB over the line does not lose an eighth of its textures); a larger one is cut
to 7/8 of that budget, as before the trim. The budget's size still asks for
its 7/8, and `RX_VK_EVICT_EXCESS=0` is the cut on any excess, both unchanged.

The rule is Hookshot-old's 030b4e9a (PR 3020), reported there for Lake Hylia on
a 10 GB card (383 MB over at each pass) and not reproduced here; written anew
for Longshot with the edge cases tested.

Tested: tests/test_vram_compact.py (the large excess, a grid of budgets and
excesses around the edge, a zero budget).
Not tested: in the game.

Assisted-by: Claude

mooki added the

area/renderer

kind

bug

priority

high

gpu/nvidia

platform/linux

platform/windows

labels 2026-11-14 18:15:16 +00:00

waryxxful referenced this pull request 2026-11-14 21:07:50 +00:00

Idea: sparse mirrors cost 1-4 ms of GPU a frame on an RX 5600 XT; whole mirrors are faster until their budget fills (runs wanted on other 6 GB cards) #585

HumanBean force-pushed fix/vram-excess-cut from bf3637f22e to 4a39da2cde 2026-11-15 18:43:35 +00:00 Compare

HumanBean commented 2026-11-15 18:46:11 +00:00

Owner

• Copy link
• Copy Markdown

Rebased onto main @ bbe6fdf2. No tests were run here; the test boxes stay yours. One conflict in docs/development.md: main rewrote the commit-reserve rows next to the RX_VK_EVICT_EXCESS row, so main's rows are kept and only the RX_VK_EVICT_EXCESS text is this PR's.

Rebased onto `main` @ bbe6fdf2. No tests were run here; the test boxes stay yours. One conflict in docs/development.md: main rewrote the commit-reserve rows next to the `RX_VK_EVICT_EXCESS` row, so main's rows are kept and only the `RX_VK_EVICT_EXCESS` text is this PR's.

Some checks failed

Hide checks

CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Has been skipped

Details

CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Has been skipped

Details

CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Has been skipped

Details

Third-party parts / The same as upstream (pull_request) Has been skipped

Details

Website / Website on Cloudflare (pull_request) Has been skipped

Details

PR description / Description follows the template (pull_request_target) Successful in 4s

Details

CI / Conventional Commit title (pull_request) Successful in 2s

Details

CI / Workflow lint (actionlint) (pull_request) Successful in 5s

Details

Guide screenshots / Guide images and local UI source review (pull_request) Failing after 6s

Details

Third-party parts / As their notes say (pull_request) Successful in 8s

Details

This pull request has changes conflicting with the target branch.

• docs/development.md

View command line instructions

Manual merge helper

Use this merge commit message when completing the merge manually.

Merge commit title WIP: fix(vk): cut a texture cache far over the VRAM rules to 7/8 of their budget (#519)

Merge commit body Reviewed-on: https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/519

WIP: fix(vk): cut a texture cache far over the VRAM rules to 7/8 of their budget (#519) Reviewed-on: https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/519 Copy merge commit message

Checkout

From your project repository, check out a new branch and test the changes.

git fetch -u 'fix/vram-excess-cut:Amateurwizrd-fix/vram-excess-cut'

git switch 'Amateurwizrd-fix/vram-excess-cut'

Sign in to join this conversation.

Reviewers

No reviewers

Labels

Clear labels   

area/accessibility

Barriers for people with disabilities

  

area/audio

Sound and music

  

area/input

Controllers, keyboard, mouse

  

area/install

Installing, updating, building from your game

  

area/menu

In-game menu and settings

  

area/renderer

Vulkan renderer, shaders, effects

  

area/saves

Save files

  

area/upscaling

DLSS, FSR, frame generation

  

beanstalk: approved @ 1697603

Set by Beanstalk for show; the farm's database is what counts

  

beanstalk: approved @ 2c5a62e

Set by Beanstalk for show; the farm's database is what counts

  

beanstalk: approved @ 4fd82b7

Set by Beanstalk for show; the farm's database is what counts

  

beanstalk: approved @ 64d6d09

Set by Beanstalk for show; the farm's database is what counts

  

beanstalk: approved @ 6f91fae

Set by Beanstalk for show; the farm's database is what counts

  

beanstalk: approved @ 70a78f9

Set by Beanstalk for show; the farm's database is what counts

  

beanstalk: approved @ 8d361dc

Set by Beanstalk for show; the farm's database is what counts

  

beanstalk: approved @ 9d8356e

Set by Beanstalk for show; the farm's database is what counts

  

beanstalk: approved @ abf2227

Set by Beanstalk for show; the farm's database is what counts

  

beanstalk: approved @ d82f202

Set by Beanstalk for show; the farm's database is what counts

  

beanstalk: approved @ e5f1e40

Set by Beanstalk for show; the farm's database is what counts

  

beanstalk: approved @ ee8ab5f

Set by Beanstalk for show; the farm's database is what counts

  

ci: approved

A maintainer approved CircleCI for this fork PR (sticky; removed after 14 quiet days)

  

good first issue

A good start for a new contributor

  

gpu/amd

AMD GPUs

  

gpu/igpu

Integrated GPUs

  

gpu/intel

Intel GPUs

  

gpu/nvidia

NVIDIA GPUs

  

help wanted

Extra hands or knowledge wanted

  

kind

bug

Something doesn't work as it should

  

kind

chore

Build, CI, dependencies, tidying

  

kind

docs

Documentation

  

kind

feature

A new feature or an improvement

  

kind

performance

Frame rate, frame time, loading, memory

  

kind

question

A question rather than a change

  

needs measurements

Waiting for before/after measurements

  

platform/linux

Affects Linux (a change for every system carries both Windows and Linux)

  

platform/macos

macOS (Apple silicon or Intel Macs)

  

platform/steam-deck

Steam Deck

  

platform/windows

Affects Windows (a change for every system carries both Windows and Linux)

  

preview

Maintainers: deploy a preview of this PR's website (read its changes first). Removed after the run.

  

priority

critical

Crashes, data loss, or blocks most players

  

priority

high

Important: next in line

  

priority

low

Nice to have

  

regression

Worked in an earlier build, broken in a newer one: highest priority, above other bugs

  

status

blocked

Waits on something else first

  

status

confirmed

Reproduced or agreed on

  

status

duplicate

Already reported or proposed

  

status

invalid

Not a problem in this project

  

status

needs-info

Waiting for more information from the reporter

  

status

wontfix

Will not be worked on

  

testing wanted

Wanted: a test on other hardware (posts a test request in #test-requests)

  

ci: approved

No labels

area/accessibility

area/audio

area/input

area/install

area/menu

area/renderer

area/saves

area/upscaling

beanstalk: approved @ 1697603

beanstalk: approved @ 2c5a62e

beanstalk: approved @ 4fd82b7

beanstalk: approved @ 64d6d09

beanstalk: approved @ 6f91fae

beanstalk: approved @ 70a78f9

beanstalk: approved @ 8d361dc

beanstalk: approved @ 9d8356e

beanstalk: approved @ abf2227

beanstalk: approved @ d82f202

beanstalk: approved @ e5f1e40

beanstalk: approved @ ee8ab5f

ci: approved

good first issue

gpu/amd

gpu/igpu

gpu/intel

gpu/nvidia

help wanted

kind

bug

kind

chore

kind

docs

kind

feature

kind

performance

kind

question

needs measurements

platform/linux

platform/macos

platform/steam-deck

platform/windows

preview

priority

critical

priority

high

priority

low

regression

status

blocked

status

confirmed

status

duplicate

status

invalid

status

needs-info

status

wontfix

testing wanted

ci: approved

Milestone

Clear milestone

No items

No milestone

Projects

Clear projects

No items

No project

Assignees

Clear assignees

No assignees

2 participants

Notifications

Subscribe

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference

MagicBeanDev/Longshot!519

No description provided.