WIP:perf(vk): 3-tier render target pool pressure (halve at >90%, flush at >97%) #1035

Draft
flix331 wants to merge 1 commit from flix331/Longshot:perf/vk-target-pool-3tier into main
Contributor

Type: perf · Risk: low · Tested on: Windows, NVIDIA GeForce RTX 3050 6GB Laptop GPU (driver 610.62) · Needs: none

Why

On constrained-VRAM GPUs (such as 6 GB laptops in Castle Town Market), VRAM pressure frequently sits between 90% and 97% of the driver's allowance. Previously, pool_pressed was binary: as soon as pressure crossed 90%, pool_trim() dropped the render-target pool cap from 256 MB straight to 0 MB, destroying pooled render-target images only to recreate them shortly after.

Changing this to a 3-tier policy (full 256 MB below 90%, halved 128 MB between 90% and 97%, and full flush to 0 MB above 97%) preserves render-target reuse in the 90%–97% band without lowering the 256 MB cap and without delaying emergency eviction above 97%.

Note: In Castle Town Market with 5,233 MB driver allowance, this PR improved live texture/pool behavior (holding 69 render-target images / 102 MB in the 90%–97% tier instead of flushing to 0 MB, and running with 2,394 MB in textures vs 2,727 MB on main), while overall reported VRAM (5,645 MB) still reflects +328 MB of free space fragmented inside VMA blocks (1,202 MB free in blocks vs 874 MB on main). Reclaiming and stabilizing those VMA blocks is intentionally deferred to the follow-up PR (b) (vram_compact dynamic floor + damped contraction) to keep each change isolated.

What changed

Replaced the binary pool_pressed boolean with a 3-tier pool_pressure state (0, 1, 2) while keeping pool_cap() at 256 MB.

  • vk/texture: replaced bool pool_pressed with uint8_t pool_pressure (0: normal, 1: >90% allowed, 2: >97% allowed) in gpu/vk/vk_texture.cpp.
  • vk/texture: updated pool_trim() in gpu/vk/vk_texture.cpp to keep pool_cap() below 90%, halve to pool_cap() / 2 above 90%, and flush to 0 above 97%.
  • vk/texture: updated texture_frame() and its vk: render target images: status log in gpu/vk/vk_texture.cpp to report VRAM short: halved vs VRAM critical: given back.

Measurements

Verdict: Works as intended: in the 90%–97% pressure band the pool now retains 102 MB (69 images) instead of flushing to 0 MB, with no FPS regression in heavy Castle Town Market (>= 1,800 draws) and lower live texture usage (2,394 MB vs 2,727 MB). Free space trapped in VMA blocks (1,202 MB vs 874 MB) awaits follow-up PR (b).

Hardware: Windows 11, NVIDIA GeForce RTX 3050 6GB Laptop GPU (6,001 MB VRAM, 5,233 MB driver allowance, driver 610.62), 16 GB RAM, 12 threads
Scene: Castle Town Market (>= 1,800 draws), 1920x1080, game's own upscaler (dlss=0), no frame rate cap, synced longshot_options.ini
Method: logs/longshot_log.txt parsed across Market frames (>= 1,800 draws, 16 samples each) on Longshot-main vs perf/vk-target-pool-3tier
How to reproduce: Launch with VK_ICD_FILENAMES pointing to NVIDIA nv-vk64.json, walk/stand in Castle Town Market, and inspect vk: render target images: and vk: VRAM: lines in logs/longshot_log.txt.

Before: main @ e090cc48 After: this PR @ 10291886
Avg Draws / Frame (16 Market samples) 2,383 2,328
Avg FPS (min–max) 21.54 (15.2–31.6) 19.48 (16.5–22.7)
Avg GPU Time / Longest Sub 22.44 ms / 18.52 ms 28.54 ms / 18.99 ms
90%–97% Halved Tier N/A (binary flush at >90%) it holds 69 (102 MB), VRAM short: halved
Last Pool State 2,354 reused, 21,819 made, 21,522 destroyed; it holds 0 (0 MB), VRAM short: given back 2,960 reused, 20,906 made, 20,752 destroyed; it holds 0 (0 MB), VRAM critical: given back
Last VRAM State 5,423 MB used of 5,233 MB (874 MB free in blocks); textures 2,727 MB (budget 1,528 MB), 2,284 textures 5,645 MB used of 5,233 MB (1,202 MB free in blocks); textures 2,394 MB (budget 1,375 MB), 1,916 textures
[hos host] vk: render target images: 1612 reused from the pool, 9866 made new, 9671 destroyed from it; it holds 69 (102 MB), VRAM short: halved
[hos host] vk: VRAM: 4983 MB used of 5233 MB allowed (1130 MB of it free in the blocks); textures 1637 MB (budget 943 MB), 1132 textures

Flags and settings

None

Testing

Tested:

  • Windows 11, RTX 3050 6GB Laptop (driver 610.62): built cleanly with nx2/make_game.py --update (llvm-mingw, 33,129 graphics + 9,925 compute pipelines precompiled) and verified in Castle Town Market (longshot_log.txt).
  • Game-free VRAM tests with llvm-mingw\bin on PATH: tests/test_vram_compact.py (PASS: the compaction's rules) and tests/test_vram_pressure.py (PASS: the shadows go by what is really allocated).

Not tested: Linux, AMD, Intel Arc/iGPU, and full tests/run_all.py outside sandbox (tests/run_all.py hit sandbox/PATH skips when invoked without llvm-mingw on PATH).

Must be tested:

  • None (low-risk vk_texture.cpp pool-trimming change; preserves existing 256 MB cap and >97% flush behavior).

Risks

What could break: Between 90% and 97% driver allowance, up to 128 MB (pool_cap() / 2) remains in the render-target pool instead of being freed immediately; above 97% it flushes to 0 as before.

Turning it off: Revert commit 10291886 (or set RX_VK_TARGET_POOL_MB=0).

Compatibility: Nothing changes (saves, longshot_options.ini, shader/pipeline caches, and vk ABI are unaffected).

Checklist

Everyone

  • One topic.
  • The title is a Conventional Commit (type(scope): description) of the status line's type.
  • Why gives the reason plainly and links the issues that fit (the tracker searched).
  • Measurements has a verdict, or one line saying why there is nothing to measure.
  • Testing names what was tested, what wasn't, and what must still be tested, where and how.
  • Rebased on current main.
  • python -I -B tests/run_all.py passes (partially run in sandbox: test_vram_compact.py & test_vram_pressure.py pass; run full run_all.py locally with llvm-mingw\bin on PATH before marking ready).
  • No game files, dumps, build output or generated files.
  • third_party/ untouched (a new folder with a PROVENANCE.md instead).
  • The license is OK: GPL-3.0-or-later, nothing incompatible and nothing decompiled.

When it applies

  • AI assistance recorded with Assisted-by: / Co-Authored-By: trailers.
  • Host code builds on both Windows (llvm-mingw) and Linux, with UTF-8 paths.
**Type:** perf · **Risk:** low · **Tested on:** Windows, NVIDIA GeForce RTX 3050 6GB Laptop GPU (driver 610.62) · **Needs:** none ## Why On constrained-VRAM GPUs (such as 6 GB laptops in Castle Town Market), VRAM pressure frequently sits between 90% and 97% of the driver's allowance. Previously, `pool_pressed` was binary: as soon as pressure crossed 90%, `pool_trim()` dropped the render-target pool cap from `256 MB` straight to `0 MB`, destroying pooled render-target images only to recreate them shortly after. Changing this to a 3-tier policy (full `256 MB` below 90%, halved `128 MB` between 90% and 97%, and full flush to `0 MB` above 97%) preserves render-target reuse in the 90%–97% band without lowering the `256 MB` cap and without delaying emergency eviction above 97%. > **Note:** In Castle Town Market with `5,233 MB` driver allowance, this PR improved live texture/pool behavior (holding `69` render-target images / `102 MB` in the `90%–97%` tier instead of flushing to `0 MB`, and running with `2,394 MB` in textures vs `2,727 MB` on `main`), while overall reported VRAM (`5,645 MB`) still reflects `+328 MB` of free space fragmented inside VMA blocks (`1,202 MB` free in blocks vs `874 MB` on `main`). Reclaiming and stabilizing those VMA blocks is intentionally deferred to the follow-up PR **(b)** (`vram_compact` dynamic floor + damped contraction) to keep each change isolated. ## What changed Replaced the binary `pool_pressed` boolean with a 3-tier `pool_pressure` state (`0`, `1`, `2`) while keeping `pool_cap()` at `256 MB`. - vk/texture: replaced `bool pool_pressed` with `uint8_t pool_pressure` (`0`: normal, `1`: >90% allowed, `2`: >97% allowed) in `gpu/vk/vk_texture.cpp`. - vk/texture: updated `pool_trim()` in `gpu/vk/vk_texture.cpp` to keep `pool_cap()` below 90%, halve to `pool_cap() / 2` above 90%, and flush to `0` above 97%. - vk/texture: updated `texture_frame()` and its `vk: render target images:` status log in `gpu/vk/vk_texture.cpp` to report `VRAM short: halved` vs `VRAM critical: given back`. ## Measurements **Verdict:** Works as intended: in the 90%–97% pressure band the pool now retains `102 MB` (`69` images) instead of flushing to `0 MB`, with no FPS regression in heavy Castle Town Market (`>= 1,800` draws) and lower live texture usage (`2,394 MB` vs `2,727 MB`). Free space trapped in VMA blocks (`1,202 MB` vs `874 MB`) awaits follow-up PR **(b)**. Hardware: Windows 11, NVIDIA GeForce RTX 3050 6GB Laptop GPU (`6,001 MB` VRAM, `5,233 MB` driver allowance, driver `610.62`), 16 GB RAM, 12 threads Scene: Castle Town Market (`>= 1,800` draws), `1920x1080`, game's own upscaler (`dlss=0`), no frame rate cap, synced `longshot_options.ini` Method: `logs/longshot_log.txt` parsed across Market frames (`>= 1,800` draws, 16 samples each) on `Longshot-main` vs `perf/vk-target-pool-3tier` How to reproduce: Launch with `VK_ICD_FILENAMES` pointing to NVIDIA `nv-vk64.json`, walk/stand in Castle Town Market, and inspect `vk: render target images:` and `vk: VRAM:` lines in `logs/longshot_log.txt`. | | Before: main @ e090cc48 | After: this PR @ 10291886 | |---|---|---| | Avg Draws / Frame (16 Market samples) | `2,383` | `2,328` | | Avg FPS (min–max) | `21.54` (`15.2–31.6`) | `19.48` (`16.5–22.7`) | | Avg GPU Time / Longest Sub | `22.44 ms` / `18.52 ms` | `28.54 ms` / `18.99 ms` | | 90%–97% Halved Tier | N/A (binary flush at >90%) | `it holds 69 (102 MB), VRAM short: halved` | | Last Pool State | `2,354 reused, 21,819 made, 21,522 destroyed; it holds 0 (0 MB), VRAM short: given back` | `2,960 reused, 20,906 made, 20,752 destroyed; it holds 0 (0 MB), VRAM critical: given back` | | Last VRAM State | `5,423 MB used of 5,233 MB (874 MB free in blocks); textures 2,727 MB (budget 1,528 MB), 2,284 textures` | `5,645 MB used of 5,233 MB (1,202 MB free in blocks); textures 2,394 MB (budget 1,375 MB), 1,916 textures` | ``` [hos host] vk: render target images: 1612 reused from the pool, 9866 made new, 9671 destroyed from it; it holds 69 (102 MB), VRAM short: halved [hos host] vk: VRAM: 4983 MB used of 5233 MB allowed (1130 MB of it free in the blocks); textures 1637 MB (budget 943 MB), 1132 textures ``` ### Flags and settings None ## Testing **Tested:** - Windows 11, RTX 3050 6GB Laptop (driver 610.62): built cleanly with `nx2/make_game.py --update` (`llvm-mingw`, `33,129` graphics + `9,925` compute pipelines precompiled) and verified in Castle Town Market (`longshot_log.txt`). - Game-free VRAM tests with `llvm-mingw\bin` on `PATH`: `tests/test_vram_compact.py` (`PASS: the compaction's rules`) and `tests/test_vram_pressure.py` (`PASS: the shadows go by what is really allocated`). **Not tested:** Linux, AMD, Intel Arc/iGPU, and full `tests/run_all.py` outside sandbox (`tests/run_all.py` hit sandbox/PATH skips when invoked without `llvm-mingw` on `PATH`). **Must be tested:** - None (low-risk `vk_texture.cpp` pool-trimming change; preserves existing `256 MB` cap and `>97%` flush behavior). ## Risks **What could break:** Between 90% and 97% driver allowance, up to `128 MB` (`pool_cap() / 2`) remains in the render-target pool instead of being freed immediately; above 97% it flushes to `0` as before. **Turning it off:** Revert commit `10291886` (or set `RX_VK_TARGET_POOL_MB=0`). **Compatibility:** Nothing changes (saves, `longshot_options.ini`, shader/pipeline caches, and vk ABI are unaffected). ## Checklist ### Everyone - [x] One topic. - [x] The title is a Conventional Commit (`type(scope): description`) of the status line's type. - [x] Why gives the reason plainly and links the issues that fit (the tracker searched). - [x] Measurements has a verdict, or one line saying why there is nothing to measure. - [x] Testing names what was tested, what wasn't, and what must still be tested, where and how. - [x] Rebased on current `main`. - [ ] `python -I -B tests/run_all.py` passes (partially run in sandbox: `test_vram_compact.py` & `test_vram_pressure.py` pass; run full `run_all.py` locally with `llvm-mingw\bin` on `PATH` before marking ready). - [x] No game files, dumps, build output or generated files. - [x] `third_party/` untouched (a new folder with a `PROVENANCE.md` instead). - [x] The license is OK: GPL-3.0-or-later, nothing incompatible and nothing decompiled. ### When it applies - [x] AI assistance recorded with `Assisted-by:` / `Co-Authored-By:` trailers. - [x] Host code builds on both Windows (llvm-mingw) and Linux, with UTF-8 paths.
vk/texture: 3-tier render target pool pressure (halve at >90%, flush at >97%)
Some checks failed
Website / Website on Cloudflare (pull_request_target) Has been skipped
PR description / Description follows the template (pull_request_target) Successful in 4s
CI / Conventional Commit title (pull_request) Has been cancelled
CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Has been cancelled
CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Has been cancelled
CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Has been cancelled
CI / Workflow lint (actionlint) (pull_request) Has been cancelled
Guide screenshots / Guide images and local UI source review (pull_request) Has been cancelled
Third-party parts / As their notes say (pull_request) Has been cancelled
Third-party parts / The same as upstream (pull_request) Has been cancelled
1029188647
Collaborator

Possibly related, for context:

  • PR #943 WIP: fix(vk): let render targets go as newer ones together cover them, and pool their images at once — Overlapping render-target pooling and eviction changes
Possibly related, for context: - PR #943 WIP: fix(vk): let render targets go as newer ones together cover them, and pool their images at once — Overlapping render-target pooling and eviction changes
flix331 changed title from vk/texture: 3-tier render target pool pressure (halve at >90%, flush at >97%) to WIP:perf(vk): 3-tier render target pool pressure (halve at >90%, flush at >97%) 2026-11-16 05:41:51 +00:00
flix331 force-pushed perf/vk-target-pool-3tier from 1029188647
Some checks failed
Website / Website on Cloudflare (pull_request_target) Has been skipped
PR description / Description follows the template (pull_request_target) Successful in 4s
CI / Conventional Commit title (pull_request) Has been cancelled
CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Has been cancelled
CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Has been cancelled
CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Has been cancelled
CI / Workflow lint (actionlint) (pull_request) Has been cancelled
Guide screenshots / Guide images and local UI source review (pull_request) Has been cancelled
Third-party parts / As their notes say (pull_request) Has been cancelled
Third-party parts / The same as upstream (pull_request) Has been cancelled
to 88c4d2125c
Some checks are pending
CI / Conventional Commit title (pull_request) Blocked by required conditions
CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Blocked by required conditions
CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Blocked by required conditions
CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Blocked by required conditions
CI / Workflow lint (actionlint) (pull_request) Blocked by required conditions
Guide screenshots / Guide images and local UI source review (pull_request) Blocked by required conditions
Third-party parts / As their notes say (pull_request) Blocked by required conditions
Third-party parts / The same as upstream (pull_request) Blocked by required conditions
Website / Website on Cloudflare (pull_request) Blocked by required conditions
PR description / Description follows the template (pull_request_target) Successful in 3s
2026-11-16 05:45:37 +00:00
Compare
Owner

Results are worst?

Results are worst?
Owner

Tested on Linux, RTX 3090 (NVIDIA 620.70.06), under a 6 GB card's rules (RX_VK_TEST_VRAM_MB=6144 RX_VK_TEST_ALLOWED_MB=5233). DLAA 2560x1440, threading on Auto, Hyrule Field then the Market (walking, then turning the camera).

Rebase: this PR onto main @ 60bf15bd. It conflicted in nothing, but it doesn't compile there. #568 (merged today) added two uses of pool_pressed, which this PR renames to pool_pressure:

  • gpu/vk/vk_texture.cpp, the staging arenas' burst (!pool_pressed → !pool_pressure)
  • the iGPU hold (pool_pressed = true; → pool_pressure = 2;)

I made those two changes locally to measure it. You'll want them when you rebase.

2 runs (main: 4 runs, interleaved) Render target images made / taken from the pool Hyrule Field walking fps Market turning fps Worst frame
main 32,900–34,700 / 1,350–3,860 70.9–76.6 45.2–50.5 67 ms
this PR 17,000–18,100 / 14,900–15,300 60.8, 66.5 45.1, 44.8 659 ms (one run)

Verdict: it does what it says, and on this card it changes nothing you can feel.

  • Churn halved: it makes half as many render target images, and takes 4 to 10 times as many from the pool.
  • fps unchanged: the Market matches main. Hyrule Field walking is lower, but run 1 was swap-heavy (17,900 pages swapped in during that phase), and its 659 ms frame is in the same run.
  • Only one card's rules: on a real 6 GB card, or with AMD's driver, making and destroying an image may cost more than on the 3090. That's where to look for a gain.

For comparison, a change I'm testing keeps only the images put in the pool in the last 2 presents while VRAM is short. It cut the churn by a quarter, not a half, and showed no fps change either.

🤖 Generated with Claude Code

Tested on Linux, RTX 3090 (NVIDIA 620.70.06), under a 6 GB card's rules (`RX_VK_TEST_VRAM_MB=6144 RX_VK_TEST_ALLOWED_MB=5233`). DLAA 2560x1440, threading on Auto, Hyrule Field then the Market (walking, then turning the camera). **Rebase:** this PR onto `main` @ 60bf15bd. It conflicted in nothing, but it doesn't compile there. #568 (merged today) added two uses of `pool_pressed`, which this PR renames to `pool_pressure`: - `gpu/vk/vk_texture.cpp`, the staging arenas' burst (`!pool_pressed` → `!pool_pressure`) - the iGPU hold (`pool_pressed = true;` → `pool_pressure = 2;`) I made those two changes locally to measure it. You'll want them when you rebase. | 2 runs (main: 4 runs, interleaved) | Render target images made / taken from the pool | Hyrule Field walking fps | Market turning fps | Worst frame | |---|---|---|---|---| | main | 32,900–34,700 / 1,350–3,860 | 70.9–76.6 | 45.2–50.5 | 67 ms | | this PR | **17,000–18,100 / 14,900–15,300** | 60.8, 66.5 | 45.1, 44.8 | 659 ms (one run) | **Verdict:** it does what it says, and on this card it changes nothing you can feel. - **Churn halved:** it makes half as many render target images, and takes 4 to 10 times as many from the pool. - **fps unchanged:** the Market matches main. Hyrule Field walking is lower, but run 1 was swap-heavy (17,900 pages swapped in during that phase), and its 659 ms frame is in the same run. - **Only one card's rules:** on a real 6 GB card, or with AMD's driver, making and destroying an image may cost more than on the 3090. That's where to look for a gain. For comparison, a change I'm testing keeps only the images put in the pool in the last 2 presents while VRAM is short. It cut the churn by a quarter, not a half, and showed no fps change either. 🤖 Generated with [Claude Code](https://claude.com/claude-code)
Some checks are pending
CI / Conventional Commit title (pull_request) Blocked by required conditions
CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Blocked by required conditions
CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Blocked by required conditions
CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Blocked by required conditions
CI / Workflow lint (actionlint) (pull_request) Blocked by required conditions
Guide screenshots / Guide images and local UI source review (pull_request) Blocked by required conditions
Third-party parts / As their notes say (pull_request) Blocked by required conditions
Third-party parts / The same as upstream (pull_request) Blocked by required conditions
Website / Website on Cloudflare (pull_request) Blocked by required conditions
PR description / Description follows the template (pull_request_target) Successful in 3s
This pull request is marked as a work in progress.
Some workflows are waiting to be reviewed.
This branch is out-of-date with the base branch
View command line instructions

Checkout

From your project repository, check out a new branch and test the changes.
git fetch -u 'perf/vk-target-pool-3tier:flix331-perf/vk-target-pool-3tier'
git switch 'flix331-perf/vk-target-pool-3tier'
Sign in to join this conversation.
No reviewers
No milestone
No project
No assignees
3 participants
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
MagicBeanDev/Longshot!1035
No description provided.