WIP: fix(vk): cap the render-target image pool and count it in the texture budget #518

Draft
Amateurwizrd wants to merge 1 commit from Amateurwizrd/Longshot:fix/target-pool-cap into main
Contributor

Type: fix · Risk: medium · Tested on: Windows 11, RTX 5070 Ti 16 GB (game-free checks; an earlier version in the game) · Needs: an 8 GB card (NVIDIA or AMD), Windows; Linux build

Why

The render targets' image pool took every evicted render target's image whatever its size and gave them back only 16 a
present, outside the texture budget: a sweep of several hundred targets could leave gigabytes of images in VRAM that the
budget never saw, and an 8 GB card pays for that with evicted textures and uploads.

How to reproduce

Seen on: by reading the code (found beside Hookshot-old's 7bba12ad); the new game-free check shows it on main
Settings and switches: defaults (RX_VK_TARGET_POOL_MB 256)
Steps: on an 8 GB card, play through an area load that sweeps many render targets (Hyrule Field, Castle Town), with the log's target pool line in view; or run python -I -B tests/test_target_pool_cap.py with this PR's test on main
Expected: the pool never holds more than 256 MB, and what it holds counts against the texture budget
Actual: on main the pool takes every gone target whatever its size and is trimmed only 16 images a present, outside the budget

What changed

The render targets' image pool now has a hard cap at the moment an image is offered, and what it holds comes off the texture budget, so pooled targets can no longer push live textures out unseen. Whether VRAM is short is worked out every present, before the pool takes anything.

  • vk: pool_offer in gpu/vk/vk_texture.cpp refuses an image that would take the pool past its cap
    (RX_VK_TARGET_POOL_MB, 256 MB by default), and any image while VRAM is short; a refused image is destroyed with its
    views as before the pool. The pool's log line counts the images it refused.
  • vk: the pool's bytes come off the texture budget, and texture_frame works out whether VRAM is short each present,
    before the dead list's offers to the pool and its trim, not every tenth present.
  • vk: on the tenth present the budget's rules look at VRAM again after the dead images are freed and the pool trimmed,
    so what that freed is not taken for an excess and no live texture is evicted for it.
  • tests: tests/test_target_pool_cap.py and .cpp (new) run pool_offer, pool_trim and the frame's slice from the
    first look to the budget's rules with a stand-in driver count; tests/test_target_image_pool.py's source checks
    follow the new order (offer first, then destroy) and still pin the per-present trim.
  • docs: docs/development.md documents RX_VK_TARGET_POOL_MB.

Measurements

Verdict: the new check fails on main and passes here (2 interleaved runs each); no in-game numbers yet: the case it fixes needs a card short of VRAM, which this PC is not.

Hardware: RTX 5070 Ti 16 GB, driver 621.01, Ryzen 7 7800X3D, 32 GB, Windows 11 Pro 26200
Scene: none (game-free check)
Method: the new test, run 2 times on each side, interleaved (main, this PR, main, this PR); main gets this PR's test files only
How to reproduce: python -I -B tests/run_all.py target_pool_cap on this branch, and on main with tests/test_target_pool_cap.py and .cpp copied in

Before: main @ 9c9d69ed After: this PR @ 159def68
tests/test_target_pool_cap.py (run 1 / run 2) FAIL / FAIL pass / pass
main @ 9c9d69ed:
texture_frame: whether VRAM is short is no longer worked out each present, before the dead list's offers to the pool and the pool's trim
FAIL         0.1s  tests/test_target_pool_cap.py
this PR @ 159def68:
pass         1.0s  tests/test_target_pool_cap.py
1 passed, 0 failed, 0 skipped

On main the check stops at the frame's order (VRAM pressure is looked at every tenth present, after the offers), before
it reaches the cap: main has no cap at the offer, so a pool past 256 MB is only trimmed later, 16 images a present.

Flags and settings

No new switch; RX_VK_TARGET_POOL_MB becomes a hard cap and is documented.

Flag What it is Before this PR After this PR Effect Helps / doesn't help / untested Recommendation
RX_VK_TARGET_POOL_MB=<MB> the render-target pool's cap (0: no pool) 256, a trim target only 256, a hard cap, counted in the texture budget the pool never exceeds the cap helps: cards short of VRAM, scenes sweeping many targets; doesn't: a 16 GB card in the measured scenes; untested: 8 GB cards, AMD, Intel keep 256

Testing

Tested: on Windows 11, RTX 5070 Ti 16 GB, driver 621.01.

  • Rebased onto main 9e6f231d (now 17376190); the conflicts in gpu/vk/vk_texture.cpp were with main's new texture-budget rule (texture_rule::), kept as main has it, with this PR's second VRAM look and budget_less_pool on top. clangd reports no error in the merged file.
  • python -I -B tests/test_target_pool_cap.py and tests/test_target_image_pool.py pass.
  • python -I -B tests/run_all.py with the pinned llvm-mingw 20261104 (tools.lock): 541 passed, 18 failed, 35 skipped; main @ 9e6f231d on the same PC: 540 passed and the same 18 failing (environment checks: portable builder, game font, a few that need tools this PC lacks). None new. The renderer (longshot_vk.dll) builds with llvm-mingw at 17376190.
  • Before the rebase (159def68): run_all with llvm-mingw, 532 passed and the 5 failing on main @ 9c9d69ed; the renderer built. In the game, an earlier version of this change (built with other changes not in this PR) ran 3 rounds at Kokiri Forest and on a Hyrule Field walk with no device loss; that is not a measurement of this PR.

Not tested: a card short of VRAM, where the cap and the refusals matter; a Linux build.

Must be tested:

  • Windows, an 8 GB NVIDIA or AMD card: Hyrule Field and Castle Town for a few minutes, expecting the log's target pool line under 256 MB and no more texture evictions than main (Test Farm: @Beanstalk windows 8gb, Hyrule Field and Castle Town, compare texture evictions and the target pool line with main)
  • Linux (x86_64, gcc or clang): the renderer builds with this PR, expecting no error in gpu/vk/vk_texture.cpp (Test Farm: @Beanstalk linux, build only)

Risks

What could break: more render-target images are destroyed and made again instead of pooled when the pool is full or VRAM is short, which could add a little CPU time in a scene that sweeps many targets; the budget's rules now look at VRAM after the dead list is freed, so eviction decisions on the tenth present can differ from main.

Turning it off: RX_VK_TARGET_POOL_MB=0 turns the pool off; reverting the commit restores the old pool.

Compatibility: Nothing changes (no save, options, cache or ABI change).

Checklist

Everyone

  • One topic.
  • The title is a Conventional Commit (type(scope): description) of the status line's type.
  • Why gives the reason plainly and links the issues that fit (the tracker searched).
  • Measurements has a verdict, or one line saying why there is nothing to measure.
  • Testing names what was tested, what wasn't, and what must still be tested, where and how.
  • Rebased on current main.
  • python -I -B tests/run_all.py passes (any failure that also fails on main is named).
  • No game files, dumps, build output or generated files.
  • third_party/ untouched (a new folder with a PROVENANCE.md instead).
  • The license is OK: GPL-3.0-or-later, nothing incompatible and nothing decompiled.

When it applies

  • AI assistance recorded with Assisted-by: / Co-Authored-By: trailers.
  • Switches documented (docs/development.md; the README for a menu row) and shipped at their recommended value.
  • A game-free test added for the fix, failing on main and passing here.
  • A check under tests/ that reads changed source updated in this PR, not weakened.
  • Host code builds on both Windows (llvm-mingw) and Linux, with UTF-8 paths. (Windows llvm-mingw at 17376190: builds; Linux is under Must be tested)
  • Docs updated.
**Type:** fix · **Risk:** medium · **Tested on:** Windows 11, RTX 5070 Ti 16 GB (game-free checks; an earlier version in the game) · **Needs:** an 8 GB card (NVIDIA or AMD), Windows; Linux build ## Why The render targets' image pool took every evicted render target's image whatever its size and gave them back only 16 a present, outside the texture budget: a sweep of several hundred targets could leave gigabytes of images in VRAM that the budget never saw, and an 8 GB card pays for that with evicted textures and uploads. ## How to reproduce **Seen on:** by reading the code (found beside Hookshot-old's 7bba12ad); the new game-free check shows it on main **Settings and switches:** defaults (`RX_VK_TARGET_POOL_MB` 256) **Steps:** on an 8 GB card, play through an area load that sweeps many render targets (Hyrule Field, Castle Town), with the log's target pool line in view; or run `python -I -B tests/test_target_pool_cap.py` with this PR's test on main **Expected:** the pool never holds more than 256 MB, and what it holds counts against the texture budget **Actual:** on main the pool takes every gone target whatever its size and is trimmed only 16 images a present, outside the budget ## What changed The render targets' image pool now has a hard cap at the moment an image is offered, and what it holds comes off the texture budget, so pooled targets can no longer push live textures out unseen. Whether VRAM is short is worked out every present, before the pool takes anything. - vk: `pool_offer` in `gpu/vk/vk_texture.cpp` refuses an image that would take the pool past its cap (`RX_VK_TARGET_POOL_MB`, 256 MB by default), and any image while VRAM is short; a refused image is destroyed with its views as before the pool. The pool's log line counts the images it refused. - vk: the pool's bytes come off the texture budget, and `texture_frame` works out whether VRAM is short each present, before the dead list's offers to the pool and its trim, not every tenth present. - vk: on the tenth present the budget's rules look at VRAM again after the dead images are freed and the pool trimmed, so what that freed is not taken for an excess and no live texture is evicted for it. - tests: `tests/test_target_pool_cap.py` and `.cpp` (new) run `pool_offer`, `pool_trim` and the frame's slice from the first look to the budget's rules with a stand-in driver count; `tests/test_target_image_pool.py`'s source checks follow the new order (offer first, then destroy) and still pin the per-present trim. - docs: `docs/development.md` documents `RX_VK_TARGET_POOL_MB`. ## Measurements **Verdict:** the new check fails on main and passes here (2 interleaved runs each); no in-game numbers yet: the case it fixes needs a card short of VRAM, which this PC is not. Hardware: RTX 5070 Ti 16 GB, driver 621.01, Ryzen 7 7800X3D, 32 GB, Windows 11 Pro 26200 Scene: none (game-free check) Method: the new test, run 2 times on each side, interleaved (main, this PR, main, this PR); main gets this PR's test files only How to reproduce: `python -I -B tests/run_all.py target_pool_cap` on this branch, and on main with `tests/test_target_pool_cap.py` and `.cpp` copied in | | Before: main @ 9c9d69ed | After: this PR @ 159def68 | |---|---|---| | tests/test_target_pool_cap.py (run 1 / run 2) | FAIL / FAIL | pass / pass | ``` main @ 9c9d69ed: texture_frame: whether VRAM is short is no longer worked out each present, before the dead list's offers to the pool and the pool's trim FAIL 0.1s tests/test_target_pool_cap.py this PR @ 159def68: pass 1.0s tests/test_target_pool_cap.py 1 passed, 0 failed, 0 skipped ``` On main the check stops at the frame's order (VRAM pressure is looked at every tenth present, after the offers), before it reaches the cap: main has no cap at the offer, so a pool past 256 MB is only trimmed later, 16 images a present. ### Flags and settings No new switch; `RX_VK_TARGET_POOL_MB` becomes a hard cap and is documented. | Flag | What it is | Before this PR | After this PR | Effect | Helps / doesn't help / untested | Recommendation | |---|---|---|---|---|---|---| | `RX_VK_TARGET_POOL_MB=<MB>` | the render-target pool's cap (`0`: no pool) | 256, a trim target only | 256, a hard cap, counted in the texture budget | the pool never exceeds the cap | helps: cards short of VRAM, scenes sweeping many targets; doesn't: a 16 GB card in the measured scenes; untested: 8 GB cards, AMD, Intel | keep 256 | ## Testing **Tested:** on Windows 11, RTX 5070 Ti 16 GB, driver 621.01. - Rebased onto main 9e6f231d (now 17376190); the conflicts in `gpu/vk/vk_texture.cpp` were with main's new texture-budget rule (`texture_rule::`), kept as main has it, with this PR's second VRAM look and `budget_less_pool` on top. clangd reports no error in the merged file. - `python -I -B tests/test_target_pool_cap.py` and `tests/test_target_image_pool.py` pass. - `python -I -B tests/run_all.py` with the pinned llvm-mingw 20261104 (`tools.lock`): 541 passed, 18 failed, 35 skipped; main @ 9e6f231d on the same PC: 540 passed and the same 18 failing (environment checks: portable builder, game font, a few that need tools this PC lacks). None new. The renderer (`longshot_vk.dll`) builds with llvm-mingw at 17376190. - Before the rebase (159def68): `run_all` with llvm-mingw, 532 passed and the 5 failing on main @ 9c9d69ed; the renderer built. In the game, an earlier version of this change (built with other changes not in this PR) ran 3 rounds at Kokiri Forest and on a Hyrule Field walk with no device loss; that is not a measurement of this PR. **Not tested:** a card short of VRAM, where the cap and the refusals matter; a Linux build. **Must be tested:** - Windows, an 8 GB NVIDIA or AMD card: Hyrule Field and Castle Town for a few minutes, expecting the log's target pool line under 256 MB and no more texture evictions than main (Test Farm: @Beanstalk windows 8gb, Hyrule Field and Castle Town, compare texture evictions and the target pool line with main) - Linux (x86_64, gcc or clang): the renderer builds with this PR, expecting no error in `gpu/vk/vk_texture.cpp` (Test Farm: @Beanstalk linux, build only) ## Risks **What could break:** more render-target images are destroyed and made again instead of pooled when the pool is full or VRAM is short, which could add a little CPU time in a scene that sweeps many targets; the budget's rules now look at VRAM after the dead list is freed, so eviction decisions on the tenth present can differ from main. **Turning it off:** `RX_VK_TARGET_POOL_MB=0` turns the pool off; reverting the commit restores the old pool. **Compatibility:** Nothing changes (no save, options, cache or ABI change). ## Checklist ### Everyone - [x] One topic. - [x] The title is a Conventional Commit (`type(scope): description`) of the status line's type. - [x] Why gives the reason plainly and links the issues that fit (the tracker searched). - [x] Measurements has a verdict, or one line saying why there is nothing to measure. - [x] Testing names what was tested, what wasn't, and what must still be tested, where and how. - [x] Rebased on current `main`. - [x] `python -I -B tests/run_all.py` passes (any failure that also fails on `main` is named). - [x] No game files, dumps, build output or generated files. - [x] `third_party/` untouched (a new folder with a `PROVENANCE.md` instead). - [x] The license is OK: GPL-3.0-or-later, nothing incompatible and nothing decompiled. ### When it applies - [x] AI assistance recorded with `Assisted-by:` / `Co-Authored-By:` trailers. - [x] Switches documented (docs/development.md; the README for a menu row) and shipped at their recommended value. - [x] A game-free test added for the fix, failing on `main` and passing here. - [x] A check under `tests/` that reads changed source updated in this PR, not weakened. - [ ] Host code builds on both Windows (llvm-mingw) and Linux, with UTF-8 paths. (Windows llvm-mingw at 17376190: builds; Linux is under Must be tested) - [x] Docs updated.
fix(vk): the render targets' image pool never holds more than its cap, and what it holds comes off the texture budget
Some checks failed
PR description / Description follows the template (pull_request_target) Successful in 3s
Website / Website on Cloudflare (pull_request_target) Has been skipped
CI / Conventional Commit title (pull_request) Has been cancelled
CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Has been cancelled
CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Has been cancelled
CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Has been cancelled
CI / Workflow lint (actionlint) (pull_request) Has been cancelled
Third-party parts / As their notes say (pull_request) Has been cancelled
Third-party parts / The same as upstream (pull_request) Has been cancelled
159def68e2
The pool took every evicted render target's image whatever its size, and was
only drained 16 images a present (64 while VRAM was short, which was looked at
every tenth present). A sweep of several hundred targets left gigabytes of
images in VRAM that the texture budget did not count. Now an image that would
take the pool past its cap, or any image while VRAM is short, is not pooled and
is destroyed as before the pool; the pool's bytes come off the texture budget;
and VRAM pressure is looked at every present for the pool. The budget's rules,
every tenth present, look again after the dead images are freed and the pool
trimmed, so what that freed is not taken for excess. The pool's log line counts
the images it refused.

Found while reading Hookshot-old's 7bba12ad (PR 2766), which pooled evicted
targets at once with no cap and outside the budget; the cap and the accounting
are new here. Written anew for Longshot.

Tested: tests/test_target_pool_cap.py (new), tests/test_target_image_pool.py,
the renderer builds.

Assisted-by: Claude
Amateurwizrd force-pushed fix/target-pool-cap from 159def68e2
Some checks failed
PR description / Description follows the template (pull_request_target) Successful in 3s
Website / Website on Cloudflare (pull_request_target) Has been skipped
CI / Conventional Commit title (pull_request) Has been cancelled
CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Has been cancelled
CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Has been cancelled
CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Has been cancelled
CI / Workflow lint (actionlint) (pull_request) Has been cancelled
Third-party parts / As their notes say (pull_request) Has been cancelled
Third-party parts / The same as upstream (pull_request) Has been cancelled
to 1737619000
Some checks are pending
CI / Conventional Commit title (pull_request) Blocked by required conditions
CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Blocked by required conditions
CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Blocked by required conditions
CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Blocked by required conditions
CI / Workflow lint (actionlint) (pull_request) Blocked by required conditions
Guide screenshots / Guide images and local UI source review (pull_request) Blocked by required conditions
Third-party parts / As their notes say (pull_request) Blocked by required conditions
Third-party parts / The same as upstream (pull_request) Blocked by required conditions
Website / Website on Cloudflare (pull_request) Blocked by required conditions
PR description / Description follows the template (pull_request_target) Successful in 4s
2026-11-15 09:50:24 +00:00
Compare
Author
Contributor

Rebased onto main 9e6f231d (now 17376190). The vk_texture.cpp conflicts were main's new texture-budget rule (texture_rule::); kept main's rule, with this PR's second VRAM look and budget_less_pool on top. Description moved to the new template. Pool tests pass; run_all (g++) shows no failure main doesn't also have on this PC. Still WIP: needs a run on an 8 GB card and a Linux build.

Rebased onto main 9e6f231d (now 17376190). The vk_texture.cpp conflicts were main's new texture-budget rule (texture_rule::); kept main's rule, with this PR's second VRAM look and budget_less_pool on top. Description moved to the new template. Pool tests pass; run_all (g++) shows no failure main doesn't also have on this PC. Still WIP: needs a run on an 8 GB card and a Linux build.
Author
Contributor

Re-ran with the pinned llvm-mingw (the earlier run_all was g++): 541 passed, the same 18 failing as main 9e6f231d on this PC, none new; the renderer builds at 17376190.

Re-ran with the pinned llvm-mingw (the earlier run_all was g++): 541 passed, the same 18 failing as main 9e6f231d on this PC, none new; the renderer builds at 17376190.
Owner

Hi, this conflicts with main again, in gpu/vk/vk_texture.cpp (texture_frame, where the pool's offer and the driver's count are read) and docs/development.md. Main's changes there are #771 and #223. The texture_frame hunk has both sides changing the same logic (the pressure test and the budget look), so I haven't guessed at it. Please rebase onto current main.

Hi, this conflicts with `main` again, in gpu/vk/vk_texture.cpp (`texture_frame`, where the pool's offer and the driver's count are read) and docs/development.md. Main's changes there are https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/771 and https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/223. The `texture_frame` hunk has both sides changing the same logic (the pressure test and the budget look), so I haven't guessed at it. Please rebase onto current `main`.
Some checks are pending
CI / Conventional Commit title (pull_request) Blocked by required conditions
CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Blocked by required conditions
CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Blocked by required conditions
CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Blocked by required conditions
CI / Workflow lint (actionlint) (pull_request) Blocked by required conditions
Guide screenshots / Guide images and local UI source review (pull_request) Blocked by required conditions
Third-party parts / As their notes say (pull_request) Blocked by required conditions
Third-party parts / The same as upstream (pull_request) Blocked by required conditions
Website / Website on Cloudflare (pull_request) Blocked by required conditions
PR description / Description follows the template (pull_request_target) Successful in 4s
This pull request has changes conflicting with the target branch.
  • docs/development.md
Some workflows are waiting to be reviewed.
View command line instructions

Manual merge helper

Use this merge commit message when completing the merge manually.

Checkout

From your project repository, check out a new branch and test the changes.
git fetch -u 'fix/target-pool-cap:Amateurwizrd-fix/target-pool-cap'
git switch 'Amateurwizrd-fix/target-pool-cap'
Sign in to join this conversation.
No reviewers
No milestone
No project
No assignees
2 participants
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
MagicBeanDev/Longshot!518
No description provided.