WIP: perf(vk): give the VRAM mirrors to the pools the GPU reads most on 4-6 GB cards #1141

Draft
mage_92 wants to merge 1 commit from mage_92/Longshot:perf/mirrors-by-heat into main
First-time contributor

Why

On 4-6 GB cards the VRAM mirror budget (256-512 MB) goes to the pools bound first. On an RTX 3050 Ti Laptop (4 GB), dozens of small pools bound every frame held the whole budget, while one or two large pools the GPU reads most (160 MB and 132 MB, 96% of the 21.9 GB read over PCIe in 600 submissions) stayed in system memory for good. The GPU read them over PCIe every frame: slower frames and more stutter on small cards.

What changed

  • vk: Pool::heat in gpu/vk/vk_memory.cpp: the bytes bound from each pool (at most 1 MB a binding, RX_VK_MIRROR_HEAT_CAP_KB), decayed by a quarter every 16 submissions, added without the lock (__atomic), read under it.
  • vk: mirrors_by_heat(): every 64 submissions, the hottest wanted pool that does not fit takes the room of mirrored pools bound at least four times less, coldest per MB first. GPU-written and GPU-only pools are never taken, and pools with an inheritance pending are left alone. Pools only go if their combined heat is at most half of the newcomer's, so they cannot take the room back.
  • vk: small pools (under 32 MB) hold at most RX_VK_MIRROR_SMALL_PCT (70) % of the budget, so a big hot pool always finds room.
  • vk: a mirror whose VRAM the driver refused is tried again after 256, 512, … submissions (8 times at most), as the texture cache gives VRAM back, instead of never.
  • vk: mirror_budget() adds mirror_heat_extra: room past the budget for the one hottest pool the full mirrors left out, while VRAM has it (RX_VK_MIRROR_HEAT_EXTRA).
  • mirror_budget() keeps reading mirror_budget.h alone (tests/test_mirror_budget.py passes).

Flags and settings

Flag What it does Default now Recommendation Helps on Doesn't help on Untested on
RX_VK_MIRROR_HEAT Mirrors by heat (everything in this PR) on on 4 GB NVIDIA laptop (measured) cards where every pool fits AMD, Intel, 6-8 GB cards
RX_VK_MIRROR_HEAT_CAP_KB Bytes counted per binding 1024 keep — — —
RX_VK_MIRROR_SMALL_PCT Share of the budget small pools may hold 70 keep 4 GB card — larger cards
RX_VK_MIRROR_HEAT_EXTRA Room past the budget for the hottest left-out pool on off until measured more (=0) — — most cards
RX_VK_MIRROR_HOT_MB Threshold for a "big" pool 64 keep — — —

Measurements

Not measured on Longshot main yet (this PR stays WIP until it is). The numbers below are from the same change applied to an earlier Longshot nightly and to Hookshot 2.3.1, which shares this renderer.

Hardware: Dell XPS 15 9520, NVIDIA RTX 3050 Ti Laptop 4 GB (driver 580.196.07), Intel i9-12900HK (PL1 held at 25 W), 16 GB DDR5-4800, Fedora 43, kernel 7.2.15, KDE Plasma 6 Wayland.
Scene: Hyrule Field, child Link, daytime. Handheld profile (1536×864 output), render resolution 75%, the game's own upscaler, built-in frame generation 2× with HOS_FPS_MAX=30, Reflex on, fullscreen.
Method: a scripted run. Link runs 9 s straight, looks left and right, turns around and repeats. There are 30 s of settling, then 60 s measured from RX_VK_FRAME_LOG (the frames shown), runs interleaved.
How to reproduce: a save in Hyrule Field, the settings above, HOS_INPUT_LIVE driving the same inputs. The scripts are available on request.

Baseline: Hookshot 2.3.1 with the same environment switches but without this change. This change was applied together with two small unrelated patches (range-list versioning, a diagnostic). main wasn't the comparison because the A/B was run on Hookshot during a night of tests; a main A/B is the next step.

Hookshot 2.3.1, Hyrule Field without (5 runs) with this change (3 runs)
frames shown / s 57.9 59.5
frames > 20 ms / min 194 72
frames > 50 ms / min 14.0 5.7
frames > 100 ms / min 3.4 0.3
worst frame 556 ms 240 ms
1% low (fps) 19.8 25.0
Without frame generation (60 cap): game fps 33.9 44.3

Earlier, on Longshot (Kokiri Forest, standing): with the 132 MB pool mirrored, the game's indirect dispatches took 2.8 ms a frame instead of 4.6, and the frame rate was 59.5 instead of 55. The GPU took 12.3-13.6 ms per frame instead of 15.0.

Testing

Tested: On the laptop above, many hours of play and scripted runs in Kokiri Forest, Hyrule Field and Castle Town market on earlier nightlies and on Hookshot 2.3.1. No visual problem was noticed and there were no device losses. On this branch: tests/run_all.py gives 586 passed, 17 failed, 74 skipped. The 17 failures are the same 17 that fail on main @ 71b1edd8 on this PC (environment: missing Vulkan/SPIR-V headers and packaging tests). gpu/vk/vk_memory.cpp compiles with the AppImage's clang.
Not tested: This exact commit in the game, on main (rebased today). Cards above 4 GB, AMD, Intel and Windows.
Needs other testers:

  • 6-8 GB NVIDIA and AMD cards: fps and stutter in Hyrule Field and Castle Town market, RX_VK_MIRROR_HEAT=0 against default, expecting equal or better (request to open).
  • Windows: same check (request to open).

Screenshots

Nothing on screen.

Risk and rollback

A pool read from system memory instead of VRAM is slower, not wrong, so a bad choice costs speed, not correctness. GPU-written pools are never taken. RX_VK_MIRROR_HEAT=0 restores the previous first-come-first-served behaviour; reverting the commit is enough.

Reviewer notes

Start with mirrors_by_heat() and the Pool::heat accounting in memory_bind. RX_VK_MIRROR_HEAT_EXTRA is the part I trust least: it is off-worthy until measured on more cards.

Checklist

For everyone

  • One topic.
  • The title is a Conventional Commit (type(scope): description).
  • Rebased on current main.
  • python -I -B tests/run_all.py run and passing (any failure that also fails on main is named above).
  • Tested in the game, or what wasn't is named under Testing. (not yet on this exact commit; named under Testing)
  • Testing filled in: what was tested, what wasn't, and what needs other testers (with test requests).
  • Measured: main and this PR at named commits, on the same PC, scene and settings, at least 2 interleaved runs each, the median reported. (pending: why this stays WIP)
  • No game files, dumps, build output or generated files.
  • third_party/ untouched (a new folder with a PROVENANCE.md instead).
  • The license is OK: GPL-3.0-or-later, nothing incompatible and nothing decompiled.

If applicable

  • AI assistance recorded with Assisted-by: / Co-Authored-By: trailers.
  • Flags noted, and the switch reference updated (docs/development.md, the README). (flags noted above; docs to update)

🤖 Generated with Claude Code

## Why On 4-6 GB cards the VRAM mirror budget (256-512 MB) goes to the pools bound first. On an RTX 3050 Ti Laptop (4 GB), dozens of small pools bound every frame held the whole budget, while one or two large pools the GPU reads most (160 MB and 132 MB, 96% of the 21.9 GB read over PCIe in 600 submissions) stayed in system memory for good. The GPU read them over PCIe every frame: slower frames and more stutter on small cards. ## What changed - vk: `Pool::heat` in `gpu/vk/vk_memory.cpp`: the bytes bound from each pool (at most 1 MB a binding, `RX_VK_MIRROR_HEAT_CAP_KB`), decayed by a quarter every 16 submissions, added without the lock (`__atomic`), read under it. - vk: `mirrors_by_heat()`: every 64 submissions, the hottest wanted pool that does not fit takes the room of mirrored pools bound at least four times less, coldest per MB first. GPU-written and GPU-only pools are never taken, and pools with an inheritance pending are left alone. Pools only go if their combined heat is at most half of the newcomer's, so they cannot take the room back. - vk: small pools (under 32 MB) hold at most `RX_VK_MIRROR_SMALL_PCT` (70) % of the budget, so a big hot pool always finds room. - vk: a mirror whose VRAM the driver refused is tried again after 256, 512, … submissions (8 times at most), as the texture cache gives VRAM back, instead of never. - vk: `mirror_budget()` adds `mirror_heat_extra`: room past the budget for the one hottest pool the full mirrors left out, while VRAM has it (`RX_VK_MIRROR_HEAT_EXTRA`). - `mirror_budget()` keeps reading `mirror_budget.h` alone (`tests/test_mirror_budget.py` passes). ## Flags and settings | Flag | What it does | Default now | Recommendation | Helps on | Doesn't help on | Untested on | |---|---|---|---|---|---|---| | `RX_VK_MIRROR_HEAT` | Mirrors by heat (everything in this PR) | on | on | 4 GB NVIDIA laptop (measured) | cards where every pool fits | AMD, Intel, 6-8 GB cards | | `RX_VK_MIRROR_HEAT_CAP_KB` | Bytes counted per binding | 1024 | keep | — | — | — | | `RX_VK_MIRROR_SMALL_PCT` | Share of the budget small pools may hold | 70 | keep | 4 GB card | — | larger cards | | `RX_VK_MIRROR_HEAT_EXTRA` | Room past the budget for the hottest left-out pool | on | off until measured more (`=0`) | — | — | most cards | | `RX_VK_MIRROR_HOT_MB` | Threshold for a "big" pool | 64 | keep | — | — | — | ## Measurements **Not measured on Longshot `main` yet (this PR stays WIP until it is).** The numbers below are from the same change applied to an earlier Longshot nightly and to Hookshot 2.3.1, which shares this renderer. Hardware: Dell XPS 15 9520, NVIDIA RTX 3050 Ti Laptop 4 GB (driver 580.196.07), Intel i9-12900HK (PL1 held at 25 W), 16 GB DDR5-4800, Fedora 43, kernel 7.2.15, KDE Plasma 6 Wayland. Scene: Hyrule Field, child Link, daytime. Handheld profile (1536×864 output), render resolution 75%, the game's own upscaler, built-in frame generation 2× with `HOS_FPS_MAX=30`, Reflex on, fullscreen. Method: a scripted run. Link runs 9 s straight, looks left and right, turns around and repeats. There are 30 s of settling, then 60 s measured from `RX_VK_FRAME_LOG` (the frames shown), runs interleaved. How to reproduce: a save in Hyrule Field, the settings above, `HOS_INPUT_LIVE` driving the same inputs. The scripts are available on request. Baseline: Hookshot 2.3.1 with the same environment switches but without this change. This change was applied together with two small unrelated patches (range-list versioning, a diagnostic). main wasn't the comparison because the A/B was run on Hookshot during a night of tests; a main A/B is the next step. | Hookshot 2.3.1, Hyrule Field | without (5 runs) | with this change (3 runs) | |---|---|---| | frames shown / s | 57.9 | 59.5 | | frames > 20 ms / min | 194 | 72 | | frames > 50 ms / min | 14.0 | 5.7 | | frames > 100 ms / min | 3.4 | 0.3 | | worst frame | 556 ms | 240 ms | | 1% low (fps) | 19.8 | 25.0 | | Without frame generation (60 cap): game fps | 33.9 | 44.3 | Earlier, on Longshot (Kokiri Forest, standing): with the 132 MB pool mirrored, the game's indirect dispatches took 2.8 ms a frame instead of 4.6, and the frame rate was 59.5 instead of 55. The GPU took 12.3-13.6 ms per frame instead of 15.0. ## Testing **Tested:** On the laptop above, many hours of play and scripted runs in Kokiri Forest, Hyrule Field and Castle Town market on earlier nightlies and on Hookshot 2.3.1. No visual problem was noticed and there were no device losses. On this branch: `tests/run_all.py` gives 586 passed, 17 failed, 74 skipped. The 17 failures are the same 17 that fail on `main` @ 71b1edd8 on this PC (environment: missing Vulkan/SPIR-V headers and packaging tests). `gpu/vk/vk_memory.cpp` compiles with the AppImage's clang. **Not tested:** This exact commit in the game, on `main` (rebased today). Cards above 4 GB, AMD, Intel and Windows. **Needs other testers:** - 6-8 GB NVIDIA and AMD cards: fps and stutter in Hyrule Field and Castle Town market, `RX_VK_MIRROR_HEAT=0` against default, expecting equal or better (request to open). - Windows: same check (request to open). ## Screenshots Nothing on screen. ## Risk and rollback A pool read from system memory instead of VRAM is slower, not wrong, so a bad choice costs speed, not correctness. GPU-written pools are never taken. `RX_VK_MIRROR_HEAT=0` restores the previous first-come-first-served behaviour; reverting the commit is enough. ## Reviewer notes Start with `mirrors_by_heat()` and the `Pool::heat` accounting in `memory_bind`. `RX_VK_MIRROR_HEAT_EXTRA` is the part I trust least: it is off-worthy until measured on more cards. ## Checklist ### For everyone - [x] One topic. - [x] The title is a Conventional Commit (`type(scope): description`). - [x] Rebased on current `main`. - [x] `python -I -B tests/run_all.py` run and passing (any failure that also fails on `main` is named above). - [ ] Tested in the game, or what wasn't is named under Testing. *(not yet on this exact commit; named under Testing)* - [x] Testing filled in: what was tested, what wasn't, and what needs other testers (with test requests). - [ ] Measured: main and this PR at named commits, on the same PC, scene and settings, at least 2 interleaved runs each, the median reported. *(pending: why this stays WIP)* - [x] No game files, dumps, build output or generated files. - [x] `third_party/` untouched (a new folder with a `PROVENANCE.md` instead). - [x] The license is OK: GPL-3.0-or-later, nothing incompatible and nothing decompiled. ### If applicable - [x] AI assistance recorded with `Assisted-by:` / `Co-Authored-By:` trailers. - [ ] Flags noted, and the switch reference updated (docs/development.md, the README). *(flags noted above; docs to update)* 🤖 Generated with [Claude Code](https://claude.com/claude-code)
perf(vk): give the VRAM mirrors to the pools the GPU reads most on 4-6 GB cards
Some checks are pending
CI / Conventional Commit title (pull_request) Blocked by required conditions
CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Blocked by required conditions
CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Blocked by required conditions
CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Blocked by required conditions
CI / Workflow lint (actionlint) (pull_request) Blocked by required conditions
Guide screenshots / Guide images and local UI source review (pull_request) Blocked by required conditions
Third-party parts / As their notes say (pull_request) Blocked by required conditions
Third-party parts / The same as upstream (pull_request) Blocked by required conditions
PR description / Description follows the template (pull_request_target) Successful in 4s
Website / Website on Cloudflare (pull_request_target) Has been skipped
5649eb641e
On cards of 4-6 GB the mirror budget (256-512 MB) went to the pools bound
first: dozens of small pools bound every frame held it, while one or two
large pools the GPU reads most stayed in system memory and were read over
PCIe every frame.

- Pool::heat: bytes bound from a pool (at most 1 MB a binding,
  RX_VK_MIRROR_HEAT_CAP_KB), decayed every 16 submissions.
- mirrors_by_heat: every 64 submissions, the hottest wanted pool that does
  not fit takes the room of mirrored pools bound at least four times less,
  coldest first (GPU-written and GPU-only pools are never taken).
- Small pools (< 32 MB) hold at most RX_VK_MIRROR_SMALL_PCT (70) % of the
  budget; a refused mirror is tried again later (8 times at most).
- RX_VK_MIRROR_HEAT=0 turns all of it off (the old behaviour).

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Collaborator

Possibly related, for context:

  • issue #585 Idea: sparse mirrors cost 1-4 ms of GPU a frame on an RX 5600 XT; whole mirrors are faster until their budget fills (runs wanted on other 6 GB cards) — Related VRAM mirror budget pressure; proposes prioritizing pool mirrors over texture budget.
  • PR #1073 perf(vk): use smaller VMA block sizes on < 7 GB GPUs — Overlapping constrained-VRAM tuning for small NVIDIA cards.
Possibly related, for context: - issue #585 Idea: sparse mirrors cost 1-4 ms of GPU a frame on an RX 5600 XT; whole mirrors are faster until their budget fills (runs wanted on other 6 GB cards) — Related VRAM mirror budget pressure; proposes prioritizing pool mirrors over texture budget. - PR #1073 perf(vk): use smaller VMA block sizes on < 7 GB GPUs — Overlapping constrained-VRAM tuning for small NVIDIA cards.
Some checks are pending
CI / Conventional Commit title (pull_request) Blocked by required conditions
CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Blocked by required conditions
CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Blocked by required conditions
CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Blocked by required conditions
CI / Workflow lint (actionlint) (pull_request) Blocked by required conditions
Guide screenshots / Guide images and local UI source review (pull_request) Blocked by required conditions
Third-party parts / As their notes say (pull_request) Blocked by required conditions
Third-party parts / The same as upstream (pull_request) Blocked by required conditions
PR description / Description follows the template (pull_request_target) Successful in 4s
Website / Website on Cloudflare (pull_request_target) Has been skipped
This pull request is marked as a work in progress.
Some workflows are waiting to be reviewed.
This branch is out-of-date with the base branch
View command line instructions

Checkout

From your project repository, check out a new branch and test the changes.
git fetch -u 'perf/mirrors-by-heat:mage_92-perf/mirrors-by-heat'
git switch 'mage_92-perf/mirrors-by-heat'
Sign in to join this conversation.
No reviewers
No milestone
No project
No assignees
2 participants
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
MagicBeanDev/Longshot!1141
No description provided.