perf(vk): use smaller VMA block sizes on < 7 GB GPUs #1073

Open
flix331 wants to merge 1 commit from flix331/Longshot:perf/vk-vram-6gb-blocks into main
Contributor

Type: perf · Risk: low · Tested on: Windows, NVIDIA GeForce RTX 3050 6GB Laptop GPU (driver 610.62) · Needs: Linux / AMD / 8 GB+ GPU verification

Why

On < 7 GB GPUs (such as an RTX 3050 6 GB Laptop with 5,233 MB driver allowance), VMA's default 128 MB blocks for texture, image, and render-target pools (and 64 MB general heap blocks) waste substantial headroom per partially filled block and force compactions to move large amounts of texture data before a full 128 MB block can be returned to the driver. In Castle Town Market on main, 1,119 MB sat trapped free inside VMA blocks while the texture cache was squeezed to 1,553 textures (budget 1,336 MB).

Halving those pool block sizes to 64 MB (textures/images) and 32 MB (transient render-target pool and general heap) strictly when physical VRAM (DEVICE_LOCAL heap) is < 7 GB reduces wasted tail space inside blocks, keeps more textures cached, and lowers longest-submission latency without touching 8 GB+ cards.

Note on existing game-free tests: tests/test_blocks_at_start.py fails to compile its extracted pools.cpp fixture because it expects the literal identifier kImageBlock from gpu/vk/vk_vma.cpp instead of image_block_bytes(). Left unmodified pending maintainer preference (whether to update the test fixture or rename image_block_bytes() back to a helper/constant). tests/test_vram_compact.py and tests/test_vram_pressure.py both pass.

What changed

Added < 7 GB physical VRAM gating (constrained_vram()) in gpu/vk/vk_vma.cpp so only constrained cards use smaller VMA block sizes (8 GB+ cards keep main's 128 MB / 64 MB values).

  • vk/vma: added device_local_heap_bytes() and constrained_vram() (heap > 0 && heap < 7 GB) in gpu/vk/vk_vma.cpp.
  • vk/vma: set preferredLargeHeapBlockSize to 32 MB on < 7 GB cards (64 MB otherwise) in vma_init().
  • vk/vma: replaced kImageBlock with image_block_bytes() (64 MB on < 7 GB, 128 MB otherwise, still overridable via RX_VK_IMAGE_BLOCK_MB) and capped target_images_pool() at 32 MB on < 7 GB cards.

Measurements

Verdict: Across a matched 2.5-minute Castle Town Market run (p=1440..4920, 30 samples each), free space trapped in VMA blocks dropped by 485 MB (634 MB vs 1,119 MB on main), +631 more textures stayed resident (2,184 vs 1,553), longest submission dropped by 2.11 ms (16.46 ms vs 18.57 ms), and minimum FPS improved from 14.6 to 16.6 fps while eliminating 5.4–5.6 GB overshoot spikes.

Hardware: Windows 11, NVIDIA GeForce RTX 3050 6GB Laptop GPU (6,001 MB physical VRAM, 5,233 MB driver allowance, driver 610.62), 16 GB RAM, 12 threads
Scene: Castle Town Market (p=1440..4920, 30 samples each), 1920x1080, game's own upscaler (dlss=0), no frame rate cap, synced longshot_options.ini
Method: logs/longshot_log.txt parsed across the matched p=1440..4920 window (30 samples on main, 30 samples on PR (c))
How to reproduce: Launch with VK_ICD_FILENAMES pointing to NVIDIA nv-vk64.json, walk around Castle Town Market for ~2.5 minutes, and inspect vk: VRAM:, GPU time, and vk: textures from a pool in logs/longshot_log.txt.

Before: main @ e090cc48 After: this PR @ 75201832
Pool Block Sizes 128 MB tex/img/RT, 64 MB heap 64 MB tex/img, 32 MB RT & heap (< 7 GB)
Avg Draws / Frame (30 samples) 2,312 2,195
Avg FPS (min–max) 21.29 (14.6–26.9) 21.05 (16.6–27.0)
Avg GPU Time / Longest Sub 25.61 ms / 18.57 ms 24.68 ms / 16.46 ms
Completed Compactions / Overshoots 12 / 6 (5.2–5.6 GB) 12 / 9 (5.1–5.3 GB)
Last VRAM State (window) 5,098 MB used (1,119 MB free in blocks); 1,553 textures (budget 1,336 MB) 5,249 MB used (634 MB free in blocks); 2,184 textures (budget 1,605 MB)
[hos host] vk: render targets' images from blocks of their own (32 MB, memory type 1)
[hos host] vk: textures from a pool of their own, 64 MB blocks (RX_VK_COMPACT), 14 made at the start
[hos host] vk: VRAM: 5249 MB used of 5233 MB allowed (634 MB of it free in the blocks); textures 2487 MB (budget 1605 MB), 2184 textures

Flags and settings

None

Testing

Tested:

  • Windows 11, RTX 3050 6GB Laptop (driver 610.62): built cleanly with nx2/make_game.py --update (llvm-mingw) and tested in Castle Town Market (longshot_log.txt).
  • tests/test_vram_compact.py: PASS: the compaction's rules.
  • tests/test_vram_pressure.py: PASS: the shadows go by what is really allocated.

Not tested: Linux, AMD, Intel Arc/iGPU, 8 GB+ dedicated GPUs, and full tests/run_all.py.

Failed (expected pending maintainer decision on test fixture):

  • tests/test_blocks_at_start.py: fails compiling pools.cpp because its source-extraction fixture expects kImageBlock instead of image_block_bytes().

Must be tested:

  • 8 GB+ GPU (Linux/Windows): run Castle Town Market, expecting 128 MB / 64 MB block sizes and behavior identical to main, by hand: check vk: textures from a pool and vk: render targets' images lines in longshot_log.txt.

Risks

What could break: Twice as many VMA VkDeviceMemory block allocations for textures (64 MB vs 128 MB) and render targets (32 MB vs 128 MB) on < 7 GB GPUs, still well below typical driver allocation limits (~4,096). 8 GB+ GPUs keep main behavior.

Turning it off: Revert commit 75201832 (or set RX_VK_IMAGE_BLOCK_MB=128).

Compatibility: Nothing changes (saves, longshot_options.ini, shader/pipeline caches, and vk ABI are unaffected).

Issues

See #939

Checklist

Everyone

  • One topic.
  • The title is a Conventional Commit (type(scope): description) of the status line's type.
  • Why gives the reason plainly and links the issues that fit (the tracker searched).
  • Measurements has a verdict, or one line saying why there is nothing to measure.
  • Testing names what was tested, what wasn't, and what must still be tested, where and how.
  • Rebased on current main.
  • python -I -B tests/run_all.py passes (pending maintainer decision on tests/test_blocks_at_start.py fixture expecting kImageBlock).
  • No game files, dumps, build output or generated files.
  • third_party/ untouched (a new folder with a PROVENANCE.md instead).
  • The license is OK: GPL-3.0-or-later, nothing incompatible and nothing decompiled.

When it applies

  • AI assistance recorded with Assisted-by: / Co-Authored-By: trailers.
  • Host code builds on both Windows (llvm-mingw) and Linux, with UTF-8 paths.
**Type:** perf · **Risk:** low · **Tested on:** Windows, NVIDIA GeForce RTX 3050 6GB Laptop GPU (driver 610.62) · **Needs:** Linux / AMD / 8 GB+ GPU verification ## Why On `< 7 GB` GPUs (such as an RTX 3050 6 GB Laptop with `5,233 MB` driver allowance), VMA's default `128 MB` blocks for texture, image, and render-target pools (and `64 MB` general heap blocks) waste substantial headroom per partially filled block and force compactions to move large amounts of texture data before a full `128 MB` block can be returned to the driver. In Castle Town Market on `main`, `1,119 MB` sat trapped free inside VMA blocks while the texture cache was squeezed to `1,553` textures (`budget 1,336 MB`). Halving those pool block sizes to `64 MB` (textures/images) and `32 MB` (transient render-target pool and general heap) **strictly when physical VRAM (`DEVICE_LOCAL` heap) is `< 7 GB`** reduces wasted tail space inside blocks, keeps more textures cached, and lowers longest-submission latency without touching `8 GB+` cards. > **Note on existing game-free tests:** `tests/test_blocks_at_start.py` fails to compile its extracted `pools.cpp` fixture because it expects the literal identifier `kImageBlock` from `gpu/vk/vk_vma.cpp` instead of `image_block_bytes()`. Left unmodified pending maintainer preference (whether to update the test fixture or rename `image_block_bytes()` back to a helper/constant). `tests/test_vram_compact.py` and `tests/test_vram_pressure.py` both pass. ## What changed Added `< 7 GB` physical VRAM gating (`constrained_vram()`) in `gpu/vk/vk_vma.cpp` so only constrained cards use smaller VMA block sizes (`8 GB+` cards keep `main`'s `128 MB / 64 MB` values). - vk/vma: added `device_local_heap_bytes()` and `constrained_vram()` (`heap > 0 && heap < 7 GB`) in `gpu/vk/vk_vma.cpp`. - vk/vma: set `preferredLargeHeapBlockSize` to `32 MB` on `< 7 GB` cards (`64 MB` otherwise) in `vma_init()`. - vk/vma: replaced `kImageBlock` with `image_block_bytes()` (`64 MB` on `< 7 GB`, `128 MB` otherwise, still overridable via `RX_VK_IMAGE_BLOCK_MB`) and capped `target_images_pool()` at `32 MB` on `< 7 GB` cards. ## Measurements **Verdict:** Across a matched 2.5-minute Castle Town Market run (`p=1440..4920`, `30` samples each), free space trapped in VMA blocks dropped by **`485 MB`** (`634 MB` vs `1,119 MB` on `main`), **`+631` more textures** stayed resident (`2,184` vs `1,553`), longest submission dropped by **`2.11 ms`** (`16.46 ms` vs `18.57 ms`), and minimum FPS improved from `14.6` to `16.6 fps` while eliminating `5.4–5.6 GB` overshoot spikes. Hardware: Windows 11, NVIDIA GeForce RTX 3050 6GB Laptop GPU (`6,001 MB` physical VRAM, `5,233 MB` driver allowance, driver `610.62`), 16 GB RAM, 12 threads Scene: Castle Town Market (`p=1440..4920`, `30` samples each), `1920x1080`, game's own upscaler (`dlss=0`), no frame rate cap, synced `longshot_options.ini` Method: `logs/longshot_log.txt` parsed across the matched `p=1440..4920` window (`30` samples on `main`, `30` samples on `PR (c)`) How to reproduce: Launch with `VK_ICD_FILENAMES` pointing to NVIDIA `nv-vk64.json`, walk around Castle Town Market for ~2.5 minutes, and inspect `vk: VRAM:`, `GPU time`, and `vk: textures from a pool` in `logs/longshot_log.txt`. | | Before: main @ e090cc48 | After: this PR @ 75201832 | |---|---|---| | Pool Block Sizes | `128 MB` tex/img/RT, `64 MB` heap | `64 MB` tex/img, `32 MB` RT & heap (`< 7 GB`) | | Avg Draws / Frame (`30` samples) | `2,312` | `2,195` | | Avg FPS (min–max) | `21.29` (`14.6–26.9`) | `21.05` (`16.6–27.0`) | | Avg GPU Time / Longest Sub | `25.61 ms` / `18.57 ms` | `24.68 ms` / `16.46 ms` | | Completed Compactions / Overshoots | `12` / `6` (`5.2–5.6 GB`) | `12` / `9` (`5.1–5.3 GB`) | | Last VRAM State (window) | `5,098 MB used (1,119 MB free in blocks); 1,553 textures (budget 1,336 MB)` | `5,249 MB used (634 MB free in blocks); 2,184 textures (budget 1,605 MB)` | ``` [hos host] vk: render targets' images from blocks of their own (32 MB, memory type 1) [hos host] vk: textures from a pool of their own, 64 MB blocks (RX_VK_COMPACT), 14 made at the start [hos host] vk: VRAM: 5249 MB used of 5233 MB allowed (634 MB of it free in the blocks); textures 2487 MB (budget 1605 MB), 2184 textures ``` ### Flags and settings None ## Testing **Tested:** - Windows 11, RTX 3050 6GB Laptop (driver 610.62): built cleanly with `nx2/make_game.py --update` (`llvm-mingw`) and tested in Castle Town Market (`longshot_log.txt`). - `tests/test_vram_compact.py`: `PASS: the compaction's rules`. - `tests/test_vram_pressure.py`: `PASS: the shadows go by what is really allocated`. **Not tested:** Linux, AMD, Intel Arc/iGPU, `8 GB+` dedicated GPUs, and full `tests/run_all.py`. **Failed (expected pending maintainer decision on test fixture):** - `tests/test_blocks_at_start.py`: fails compiling `pools.cpp` because its source-extraction fixture expects `kImageBlock` instead of `image_block_bytes()`. **Must be tested:** - 8 GB+ GPU (Linux/Windows): run Castle Town Market, expecting `128 MB` / `64 MB` block sizes and behavior identical to `main`, by hand: check `vk: textures from a pool` and `vk: render targets' images` lines in `longshot_log.txt`. ## Risks **What could break:** Twice as many VMA `VkDeviceMemory` block allocations for textures (`64 MB` vs `128 MB`) and render targets (`32 MB` vs `128 MB`) on `< 7 GB` GPUs, still well below typical driver allocation limits (~4,096). `8 GB+` GPUs keep `main` behavior. **Turning it off:** Revert commit `75201832` (or set `RX_VK_IMAGE_BLOCK_MB=128`). **Compatibility:** Nothing changes (saves, `longshot_options.ini`, shader/pipeline caches, and vk ABI are unaffected). ## Issues See #939 ## Checklist ### Everyone - [x] One topic. - [x] The title is a Conventional Commit (`type(scope): description`) of the status line's type. - [x] Why gives the reason plainly and links the issues that fit (the tracker searched). - [x] Measurements has a verdict, or one line saying why there is nothing to measure. - [x] Testing names what was tested, what wasn't, and what must still be tested, where and how. - [x] Rebased on current `main`. - [ ] `python -I -B tests/run_all.py` passes (pending maintainer decision on `tests/test_blocks_at_start.py` fixture expecting `kImageBlock`). - [x] No game files, dumps, build output or generated files. - [x] `third_party/` untouched (a new folder with a `PROVENANCE.md` instead). - [x] The license is OK: GPL-3.0-or-later, nothing incompatible and nothing decompiled. ### When it applies - [x] AI assistance recorded with `Assisted-by:` / `Co-Authored-By:` trailers. - [x] Host code builds on both Windows (llvm-mingw) and Linux, with UTF-8 paths.
perf(vk): use smaller VMA block sizes on < 7 GB GPUs
Some checks failed
PR description / Description follows the template (pull_request_target) Successful in 4s
CI / Conventional Commit title (pull_request) Has been cancelled
CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Has been cancelled
CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Has been cancelled
CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Has been cancelled
CI / Workflow lint (actionlint) (pull_request) Has been cancelled
Guide screenshots / Guide images and local UI source review (pull_request) Has been cancelled
Third-party parts / As their notes say (pull_request) Has been cancelled
Third-party parts / The same as upstream (pull_request) Has been cancelled
7520183265
flix331 force-pushed perf/vk-vram-6gb-blocks from 7520183265
Some checks failed
PR description / Description follows the template (pull_request_target) Successful in 4s
CI / Conventional Commit title (pull_request) Has been cancelled
CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Has been cancelled
CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Has been cancelled
CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Has been cancelled
CI / Workflow lint (actionlint) (pull_request) Has been cancelled
Guide screenshots / Guide images and local UI source review (pull_request) Has been cancelled
Third-party parts / As their notes say (pull_request) Has been cancelled
Third-party parts / The same as upstream (pull_request) Has been cancelled
to 9e62352fb8
Some checks reported errors
CI / Conventional Commit title (pull_request) Blocked by required conditions
CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Blocked by required conditions
CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Blocked by required conditions
CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Blocked by required conditions
CI / Workflow lint (actionlint) (pull_request) Blocked by required conditions
Guide screenshots / Guide images and local UI source review (pull_request) Blocked by required conditions
Third-party parts / The same as upstream (pull_request) Blocked by required conditions
Third-party parts / As their notes say (pull_request) Blocked by required conditions
Website / Website on Cloudflare (pull_request) Blocked by required conditions
Website / Website on Cloudflare (pull_request_target) Has been skipped
PR description / Description follows the template (pull_request_target) Failing after 6s
beanstalk/approval approved by HumanBean (reviewed)
beanstalk/vram6gb 0/1 pass, 1 error
beanstalk/vram-ge8gb 1/1 fail
2026-11-16 06:39:16 +00:00
Compare
Owner

@Beanstalk

@Beanstalk
Member

Beanstalk test farm

Commit 9e62352 · tier reviewed · approved by HumanBean · dashboard

What will be tested · AI plan (gpt-6.1-terra)

The plan compares main and the pull request on a 6 GB GPU and an 8 GB-or-larger GPU, covering both sides of the allocator threshold. A short Market and area-change tour measures frame times, VRAM and load times, and compares screenshots for visual regressions. Crashes, failed pool creation, substantial performance or memory regressions, and screenshot differences fail the plan; each tester’s game runs take about 14 minutes.

  • Where: vram6gb (any PC with 6 GB VRAM, 1 report), vram-ge8gb (any PC with 8 GB VRAM or more, 1 report).
  • market-tour: from a new save (settings dlss=0, frame_rate_cap=6), then teleport to Market, move the stick, measure market for 60 s (after 10 s warm-up), take the picture market to compare with main, teleport to Kakariko Village, teleport to Hyrule Field, measure field-after-tour for 30 s (after 5 s warm-up), take the picture field to compare with main; 1 run on each side.
  • Measured: crashes, frame times, stutters and hitches, VRAM use, load times, the game's log.
  • The PR fails if: it crashes; or a picture differs from main by more than 5%; or peak VRAM is more than 15% above main; or longest load is more than 30% above main; or frame time (p99) is more than 20% above main; or forbidden log lines is above 0.
  • Time: about 16 min of game runs per tester (2 game starts), plus the two builds.

Done. Run 2: the farm's reports were cleared after its Windows and Linux fixes: started again. Comment @Beanstalk to run it again.

Target Reports Crash Frame p99 (ms) vs main VRAM peak (MB) vs main Result
vram6gb · 1: linux+amd+radv+igpu+handheld AMD Custom GPU 0932 (RADV VANGOGH), 26.1.99 1/1 1 none — — error
vram-ge8gb · 1: windows+nvidia+nvidia-proprietary+dgpu NVIDIA GeForce RTX 4090, 621.24.0 1/1 1 none 17.8 vs 14.3 (+25%) 13,332 vs 12,400 (+8%) fail

Measurements · vram6gb

Verdict: could not be judged on vram6gb (1/1 report).

Hardware: 1: linux+amd+radv+igpu+handheld AMD Custom GPU 0932 (RADV VANGOGH), 26.1.99 AMD Custom APU 0932, 16 GB RAM
Scene: market-tour, from a new save (dlss=0, frame_rate_cap=6); each cell's settings and switches as the table names them
Method: the Beanstalk test farm: main and this PR built and run on the same PC in one session, game starts alternating (ABBA); market-tour: 1 game start a side, market 60 s after 10 s warm-up, field-after-tour 30 s after 5 s warm-up; each number the median of the runs, then of 1 report
How to reproduce: the farm's plan above (@Beanstalk on this PR runs it again)

Before: main @ 02903b0 After: this PR @ 9e62352
market-tour · longest load (s) 26.9 27.6 (+3%)
market-tour · forbidden log lines 0.0 0.0

Flags and settings

None: the plan varied no setting, switch or build option.

Markdown for the PR description
## Measurements

**Verdict:** could not be judged on vram6gb (1/1 report).

Hardware: 1: linux+amd+radv+igpu+handheld `AMD Custom GPU 0932 (RADV VANGOGH), 26.1.99` `AMD Custom APU 0932, 16 GB RAM`  
Scene: market-tour, from a new save (dlss=0, frame_rate_cap=6); each cell's settings and switches as the table names them  
Method: the Beanstalk test farm: main and this PR built and run on the same PC in one session, game starts alternating (ABBA); market-tour: 1 game start a side, market 60 s after 10 s warm-up, field-after-tour 30 s after 5 s warm-up; each number the median of the runs, then of 1 report  
How to reproduce: the farm's plan above (`@Beanstalk` on this PR runs it again)

| | Before: main @ 02903b0 | After: this PR @ 9e62352 |
|---|---|---|
| market-tour · longest load (s) | 26.9 | 27.6 (+3%) |
| market-tour · forbidden log lines | 0.0 | 0.0 |

### Flags and settings

None: the plan varied no setting, switch or build option.

Measurements · vram-ge8gb

Verdict: fail on vram-ge8gb (1/1 report): market-tour screenshot.max_diff; market-tour market.frametime_p99_ms.max_delta_pct (+25.0%) failed.

Hardware: 1: windows+nvidia+nvidia-proprietary+dgpu NVIDIA GeForce RTX 4090, 621.24.0 13th Gen Intel(R) Core(TM) i9-13900K, 64 GB RAM
Scene: market-tour, from a new save (dlss=0, frame_rate_cap=6); each cell's settings and switches as the table names them
Method: the Beanstalk test farm: main and this PR built and run on the same PC in one session, game starts alternating (ABBA); market-tour: 1 game start a side, market 60 s after 10 s warm-up, field-after-tour 30 s after 5 s warm-up; each number the median of the runs, then of 1 report
How to reproduce: the farm's plan above (@Beanstalk on this PR runs it again)

Before: main @ 02903b0 After: this PR @ 9e62352
market-tour · market · peak VRAM (MB) 12,400 13,332 (+8%)
market-tour · field-after-tour · peak VRAM (MB) 15,522 15,428 (−1%)
market-tour · longest load (s) 19.3 8.6 (−55%)
market-tour · market · frame time (p99) (ms) 14.3 17.8 (+25%)
market-tour · field-after-tour · frame time (p99) (ms) 15.1 14.9 (−2%)
market-tour · forbidden log lines 0.0 0.0

Flags and settings

None: the plan varied no setting, switch or build option.

Markdown for the PR description
## Measurements

**Verdict:** fail on vram-ge8gb (1/1 report): `market-tour screenshot.max_diff`; `market-tour market.frametime_p99_ms.max_delta_pct (+25.0%)` failed.

Hardware: 1: windows+nvidia+nvidia-proprietary+dgpu `NVIDIA GeForce RTX 4090, 621.24.0` `13th Gen Intel(R) Core(TM) i9-13900K, 64 GB RAM`  
Scene: market-tour, from a new save (dlss=0, frame_rate_cap=6); each cell's settings and switches as the table names them  
Method: the Beanstalk test farm: main and this PR built and run on the same PC in one session, game starts alternating (ABBA); market-tour: 1 game start a side, market 60 s after 10 s warm-up, field-after-tour 30 s after 5 s warm-up; each number the median of the runs, then of 1 report  
How to reproduce: the farm's plan above (`@Beanstalk` on this PR runs it again)

| | Before: main @ 02903b0 | After: this PR @ 9e62352 |
|---|---|---|
| market-tour · market · peak VRAM (MB) | 12,400 | 13,332 (+8%) |
| market-tour · field-after-tour · peak VRAM (MB) | 15,522 | 15,428 (−1%) |
| market-tour · longest load (s) | 19.3 | 8.6 (−55%) |
| market-tour · market · frame time (p99) (ms) | 14.3 | 17.8 (+25%) |
| market-tour · field-after-tour · frame time (p99) (ms) | 15.1 | 14.9 (−2%) |
| market-tour · forbidden log lines | 0.0 | 0.0 |

### Flags and settings

None: the plan varied no setting, switch or build option.
<!-- beanstalk:sticky --> ### Beanstalk test farm Commit `9e62352` · tier **reviewed** · approved by HumanBean · [dashboard](https://beanstalk.magicbean.dev/prs/1073) **What will be tested** · AI plan (`gpt-6.1-terra`) The plan compares main and the pull request on a 6 GB GPU and an 8 GB-or-larger GPU, covering both sides of the allocator threshold. A short Market and area-change tour measures frame times, VRAM and load times, and compares screenshots for visual regressions. Crashes, failed pool creation, substantial performance or memory regressions, and screenshot differences fail the plan; each tester’s game runs take about 14 minutes. - **Where:** vram6gb (any PC with 6 GB VRAM, 1 report), vram-ge8gb (any PC with 8 GB VRAM or more, 1 report). - **market-tour:** from a new save (settings dlss=0, frame_rate_cap=6), then teleport to Market, move the stick, measure market for 60 s (after 10 s warm-up), take the picture market to compare with main, teleport to Kakariko Village, teleport to Hyrule Field, measure field-after-tour for 30 s (after 5 s warm-up), take the picture field to compare with main; 1 run on each side. - **Measured:** crashes, frame times, stutters and hitches, VRAM use, load times, the game's log. - **The PR fails if:** it crashes; or a picture differs from main by more than 5%; or peak VRAM is more than 15% above main; or longest load is more than 30% above main; or frame time (p99) is more than 20% above main; or forbidden log lines is above 0. - **Time:** about 16 min of game runs per tester (2 game starts), plus the two builds. Done. Run 2: the farm's reports were cleared after its Windows and Linux fixes: started again. Comment `@Beanstalk` to run it again. | Target | Reports | Crash | Frame p99 (ms) vs main | VRAM peak (MB) vs main | Result | | --- | --- | --- | --- | --- | --- | | vram6gb · 1: linux+amd+radv+igpu+handheld `AMD Custom GPU 0932 (RADV VANGOGH), 26.1.99` | 1/1 [1](https://beanstalk.magicbean.dev/reports/rp_4ecb9876aaeae159) | none | — | — | error | | vram-ge8gb · 1: windows+nvidia+nvidia-proprietary+dgpu `NVIDIA GeForce RTX 4090, 621.24.0` | 1/1 [1](https://beanstalk.magicbean.dev/reports/rp_b1d123012df361b1) | none | 17.8 vs 14.3 (+25%) | 13,332 vs 12,400 (+8%) | fail | #### Measurements · vram6gb **Verdict:** could not be judged on vram6gb (1/1 report). Hardware: 1: linux+amd+radv+igpu+handheld `AMD Custom GPU 0932 (RADV VANGOGH), 26.1.99` `AMD Custom APU 0932, 16 GB RAM` Scene: market-tour, from a new save (dlss=0, frame_rate_cap=6); each cell's settings and switches as the table names them Method: the Beanstalk test farm: main and this PR built and run on the same PC in one session, game starts alternating (ABBA); market-tour: 1 game start a side, market 60 s after 10 s warm-up, field-after-tour 30 s after 5 s warm-up; each number the median of the runs, then of 1 report How to reproduce: the farm's plan above (`@Beanstalk` on this PR runs it again) | | Before: main @ 02903b0 | After: this PR @ 9e62352 | |---|---|---| | market-tour · longest load (s) | 26.9 | 27.6 (+3%) | | market-tour · forbidden log lines | 0.0 | 0.0 | ### Flags and settings None: the plan varied no setting, switch or build option. <details><summary>Markdown for the PR description</summary> ```markdown ## Measurements **Verdict:** could not be judged on vram6gb (1/1 report). Hardware: 1: linux+amd+radv+igpu+handheld `AMD Custom GPU 0932 (RADV VANGOGH), 26.1.99` `AMD Custom APU 0932, 16 GB RAM` Scene: market-tour, from a new save (dlss=0, frame_rate_cap=6); each cell's settings and switches as the table names them Method: the Beanstalk test farm: main and this PR built and run on the same PC in one session, game starts alternating (ABBA); market-tour: 1 game start a side, market 60 s after 10 s warm-up, field-after-tour 30 s after 5 s warm-up; each number the median of the runs, then of 1 report How to reproduce: the farm's plan above (`@Beanstalk` on this PR runs it again) | | Before: main @ 02903b0 | After: this PR @ 9e62352 | |---|---|---| | market-tour · longest load (s) | 26.9 | 27.6 (+3%) | | market-tour · forbidden log lines | 0.0 | 0.0 | ### Flags and settings None: the plan varied no setting, switch or build option. ``` </details> #### Measurements · vram-ge8gb **Verdict:** fail on vram-ge8gb (1/1 report): `market-tour screenshot.max_diff`; `market-tour market.frametime_p99_ms.max_delta_pct (+25.0%)` failed. Hardware: 1: windows+nvidia+nvidia-proprietary+dgpu `NVIDIA GeForce RTX 4090, 621.24.0` `13th Gen Intel(R) Core(TM) i9-13900K, 64 GB RAM` Scene: market-tour, from a new save (dlss=0, frame_rate_cap=6); each cell's settings and switches as the table names them Method: the Beanstalk test farm: main and this PR built and run on the same PC in one session, game starts alternating (ABBA); market-tour: 1 game start a side, market 60 s after 10 s warm-up, field-after-tour 30 s after 5 s warm-up; each number the median of the runs, then of 1 report How to reproduce: the farm's plan above (`@Beanstalk` on this PR runs it again) | | Before: main @ 02903b0 | After: this PR @ 9e62352 | |---|---|---| | market-tour · market · peak VRAM (MB) | 12,400 | 13,332 (+8%) | | market-tour · field-after-tour · peak VRAM (MB) | 15,522 | 15,428 (−1%) | | market-tour · longest load (s) | 19.3 | 8.6 (−55%) | | market-tour · market · frame time (p99) (ms) | 14.3 | 17.8 (+25%) | | market-tour · field-after-tour · frame time (p99) (ms) | 15.1 | 14.9 (−2%) | | market-tour · forbidden log lines | 0.0 | 0.0 | ### Flags and settings None: the plan varied no setting, switch or build option. <details><summary>Markdown for the PR description</summary> ```markdown ## Measurements **Verdict:** fail on vram-ge8gb (1/1 report): `market-tour screenshot.max_diff`; `market-tour market.frametime_p99_ms.max_delta_pct (+25.0%)` failed. Hardware: 1: windows+nvidia+nvidia-proprietary+dgpu `NVIDIA GeForce RTX 4090, 621.24.0` `13th Gen Intel(R) Core(TM) i9-13900K, 64 GB RAM` Scene: market-tour, from a new save (dlss=0, frame_rate_cap=6); each cell's settings and switches as the table names them Method: the Beanstalk test farm: main and this PR built and run on the same PC in one session, game starts alternating (ABBA); market-tour: 1 game start a side, market 60 s after 10 s warm-up, field-after-tour 30 s after 5 s warm-up; each number the median of the runs, then of 1 report How to reproduce: the farm's plan above (`@Beanstalk` on this PR runs it again) | | Before: main @ 02903b0 | After: this PR @ 9e62352 | |---|---|---| | market-tour · market · peak VRAM (MB) | 12,400 | 13,332 (+8%) | | market-tour · field-after-tour · peak VRAM (MB) | 15,522 | 15,428 (−1%) | | market-tour · longest load (s) | 19.3 | 8.6 (−55%) | | market-tour · market · frame time (p99) (ms) | 14.3 | 17.8 (+25%) | | market-tour · field-after-tour · frame time (p99) (ms) | 15.1 | 14.9 (−2%) | | market-tour · forbidden log lines | 0.0 | 0.0 | ### Flags and settings None: the plan varied no setting, switch or build option. ``` </details>
HumanBean changed title from WIP:perf(vk): use smaller VMA block sizes on < 7 GB GPUs to perf(vk): use smaller VMA block sizes on < 7 GB GPUs 2026-11-16 10:12:36 +00:00
dyst left a comment

Nicely contained: the change stays in vk_vma.cpp, and 8 GB+ cards keep main's sizes. Smaller blocks waste less space at the end of each one, so the idea holds. A few things before merge:

  1. Two checks break, not one. tests/test_target_image_pool.py:56 also looks for pi.blockSize = kImageBlock;. AGENTS.md asks for checks to be updated in the same PR. Please update both, and give test_blocks_at_start's fixture a stand-in constrained_vram.
  2. Use main's device_vram_bytes() in place of the new heap loop. It goes through RX_VK_TEST_VRAM_MB (#1107, made so this PR can be measured on a big card), and it's the < 7 GB line the DLSS preset already uses.
  3. The off switch and the docs. RX_VK_IMAGE_BLOCK_MB=128 doesn't bring main's behaviour back. The RX_VK_IMAGE_PREALLOC row in docs/development.md still says 128 MiB. The block switch's default now depends on the card, so it needs a row of its own.
  4. Render targets over 32 MB (4K, or large 1440p ones) don't fit the new target blocks. Each one becomes a dedicated allocation. Only 1080p was run.
  5. Measurements: one run a side, and draws per frame differ (2,312 vs 2,195). Overshoots went from 6 to 9, which the verdict doesn't mention. Both farm runs errored, and its "6 GB" machine is a Steam Deck.

Overlap: #637 makes this same change with other numbers (≤ 8 GB, 16 MB target blocks), so the two conflict. #1057 adds its own is_constrained_vram(). One shared helper would serve both.

This is from reading the code only; nothing was built or run.

🤖 Generated with Claude Code

Nicely contained: the change stays in `vk_vma.cpp`, and 8 GB+ cards keep main's sizes. Smaller blocks waste less space at the end of each one, so the idea holds. A few things before merge: 1. **Two checks break, not one.** `tests/test_target_image_pool.py:56` also looks for `pi.blockSize = kImageBlock;`. AGENTS.md asks for checks to be updated in the same PR. Please update both, and give `test_blocks_at_start`'s fixture a stand-in `constrained_vram`. 2. **Use main's `device_vram_bytes()`** in place of the new heap loop. It goes through `RX_VK_TEST_VRAM_MB` (#1107, made so this PR can be measured on a big card), and it's the < 7 GB line the DLSS preset already uses. 3. **The off switch and the docs.** `RX_VK_IMAGE_BLOCK_MB=128` doesn't bring main's behaviour back. The `RX_VK_IMAGE_PREALLOC` row in `docs/development.md` still says 128 MiB. The block switch's default now depends on the card, so it needs a row of its own. 4. **Render targets over 32 MB** (4K, or large 1440p ones) don't fit the new target blocks. Each one becomes a dedicated allocation. Only 1080p was run. 5. **Measurements:** one run a side, and draws per frame differ (2,312 vs 2,195). Overshoots went from 6 to 9, which the verdict doesn't mention. Both farm runs errored, and its "6 GB" machine is a Steam Deck. Overlap: #637 makes this same change with other numbers (≤ 8 GB, 16 MB target blocks), so the two conflict. #1057 adds its own `is_constrained_vram()`. One shared helper would serve both. This is from reading the code only; nothing was built or run. 🤖 Generated with [Claude Code](https://claude.com/claude-code)
@ -21,6 +21,18 @@ namespace mkwvk {
VmaAllocator vma;
static uint64_t device_local_heap_bytes() {
Member

Main already has this as device_vram_bytes() (vk_memory.cpp, declared in vk_internal.h). It also applies test_vram_bytes(), so RX_VK_TEST_VRAM_MB caps it. As written, RX_VK_TEST_VRAM_MB=6144 on a 24 GB card lowers the count in blocks_at_start (which reads through test_vram_bytes on main) but keeps 128 MB blocks, a mix no real card has. Using device_vram_bytes() here also keeps the gate on the same < 7 GB line as the DLSS preset (upscaler_rows.h).

Main already has this as `device_vram_bytes()` (`vk_memory.cpp`, declared in `vk_internal.h`). It also applies `test_vram_bytes()`, so `RX_VK_TEST_VRAM_MB` caps it. As written, `RX_VK_TEST_VRAM_MB=6144` on a 24 GB card lowers the count in `blocks_at_start` (which reads through `test_vram_bytes` on main) but keeps 128 MB blocks, a mix no real card has. Using `device_vram_bytes()` here also keeps the gate on the same < 7 GB line as the DLSS preset (`upscaler_rows.h`).
@ -35,2 +47,3 @@
ci.pVulkanFunctions = &f;
ci.preferredLargeHeapBlockSize = 64ull << 20; // (64 MB blocks: Eglantine found 256 MB ones wasted VRAM)
// 64 MB blocks on >= 7 GB cards; 32 MB on < 7 GB cards to halve fragmentation in default heap blocks
ci.preferredLargeHeapBlockSize = constrained_vram() ? (32ull << 20) : (64ull << 20);
Member

Two things this changes that the Risks section doesn't cover. VMA gives an allocation over half the default block its own memory (the comment at the top of the images' pool says the same), so on < 7 GB cards that cut-off drops from 32 MB to 16 MB for everything outside the pools. This also isn't undone by RX_VK_IMAGE_BLOCK_MB=128, which the description gives as the way to turn the change off.

Two things this changes that the Risks section doesn't cover. VMA gives an allocation over half the default block its own memory (the comment at the top of the images' pool says the same), so on < 7 GB cards that cut-off drops from 32 MB to 16 MB for everything outside the pools. This also isn't undone by `RX_VK_IMAGE_BLOCK_MB=128`, which the description gives as the way to turn the change off.
@ -48,1 +61,3 @@
const uint64_t kImageBlock = (getenv("RX_VK_IMAGE_BLOCK_MB") ? (uint64_t)std::max(16, atoi(getenv("RX_VK_IMAGE_BLOCK_MB"))) : 128u) << 20;
static uint64_t image_block_bytes() {
if (const char* e = getenv("RX_VK_IMAGE_BLOCK_MB")) return (uint64_t)std::max(16, atoi(e)) << 20;
return (constrained_vram() ? 64ull : 128ull) << 20;
Member

The default of RX_VK_IMAGE_BLOCK_MB now depends on the card, but the switch has no row in docs/development.md. The RX_VK_IMAGE_PREALLOC row there still says "Initial 128 MiB image blocks". Could both go in this PR? Also, constrained_vram() scans the heaps on every call. A static const would make it read once, like the other switches in this file.

The default of `RX_VK_IMAGE_BLOCK_MB` now depends on the card, but the switch has no row in `docs/development.md`. The `RX_VK_IMAGE_PREALLOC` row there still says "Initial 128 MiB image blocks". Could both go in this PR? Also, `constrained_vram()` scans the heaps on every call. A `static const` would make it read once, like the other switches in this file.
@ -71,11 +87,12 @@ VmaPool target_images_pool() {
ai.requiredFlags = VK_MEMORY_PROPERTY_DEVICE_LOCAL_BIT;
uint32_t type = 0;
if (vmaFindMemoryTypeIndexForImageInfo(vma, &rep, &ai, &type) != VK_SUCCESS) return VK_NULL_HANDLE;
const uint64_t block_size = constrained_vram() ? std::min<uint64_t>(image_block_bytes(), 32ull << 20) : image_block_bytes();
Member

A custom VMA pool can't hold an allocation larger than its block. So a render target over 32 MB fails here and falls through to create_image's default path, where (with the 32 MB default block from vma_init) it gets a dedicated vkAllocateMemory. That covers 4K 16-bit float and depth targets (~63 MiB) and anything larger at 1440p, which is the per-target churn this pool exists to stop. It also caps the target pool at 32 MB even when RX_VK_IMAGE_BLOCK_MB is set, so the switch can't restore main. Could you check a 1440p/4K run, or keep this at image_block_bytes() (64 MB)?

A custom VMA pool can't hold an allocation larger than its block. So a render target over 32 MB fails here and falls through to `create_image`'s default path, where (with the 32 MB default block from `vma_init`) it gets a dedicated `vkAllocateMemory`. That covers 4K 16-bit float and depth targets (~63 MiB) and anything larger at 1440p, which is the per-target churn this pool exists to stop. It also caps the target pool at 32 MB even when `RX_VK_IMAGE_BLOCK_MB` is set, so the switch can't restore main. Could you check a 1440p/4K run, or keep this at `image_block_bytes()` (64 MB)?
Owner

Tested on Linux, measuring a 6 GB card's rules on an RTX 3090. This PR is merged with main c167ec67, plus #1107's RX_VK_TEST_VRAM_MB, which is merged now. That switch makes this PR's <7 GB gate take effect on a large card.

Hardware: RTX 3090, NVIDIA 620.70.06, CachyOS.
Settings: DLAA 2560x1440 with RX_VK_TEST_VRAM_MB=6144 RX_VK_TEST_ALLOWED_MB=5233, as on the RTX 3050 6 GB Laptop of the description.
Route: Hyrule Field, then the Market (teleports), walking and turning. Two sessions of this PR, interleaved with main's, five main sessions in all.

main (5 sessions) this PR (2 sessions)
VRAM peak (MB, of 5,233 allowed) 5,334 to 5,926 5,358, 5,614
Market turning: frame median (ms) 20.3 to 21.2 19.8, 21.0
Market turning: GPU ms 12.4 to 13.35 12.1, 12.2
Market turning: fps 44.1 to 46.0 43.0, 45.9
Textures retired by the budget 44,878 to 50,078 45,826, 49,268

This PR is inside main's range on every line. Its VRAM peaks sit at the lower end of main's, and GPU time is 0.2 to 1 ms lower, but two sessions on this swapping PC can't separate that from noise. So it isn't worse, and isn't measurably better here. A real 6 GB card, where going over the allowance actually pages, is the test that would show the smaller blocks paying off. On Linux NVIDIA, going over costs next to nothing.

🤖 Generated with Claude Code

Tested on Linux, measuring a 6 GB card's rules on an RTX 3090. This PR is merged with `main` c167ec67, plus #1107's `RX_VK_TEST_VRAM_MB`, which is merged now. That switch makes this PR's <7 GB gate take effect on a large card. **Hardware:** RTX 3090, NVIDIA 620.70.06, CachyOS. **Settings:** DLAA 2560x1440 with `RX_VK_TEST_VRAM_MB=6144 RX_VK_TEST_ALLOWED_MB=5233`, as on the RTX 3050 6 GB Laptop of the description. **Route:** Hyrule Field, then the Market (teleports), walking and turning. Two sessions of this PR, interleaved with main's, five main sessions in all. | | main (5 sessions) | this PR (2 sessions) | |---|---|---| | VRAM peak (MB, of 5,233 allowed) | 5,334 to 5,926 | 5,358, 5,614 | | Market turning: frame median (ms) | 20.3 to 21.2 | 19.8, 21.0 | | Market turning: GPU ms | 12.4 to 13.35 | 12.1, 12.2 | | Market turning: fps | 44.1 to 46.0 | 43.0, 45.9 | | Textures retired by the budget | 44,878 to 50,078 | 45,826, 49,268 | This PR is inside main's range on every line. Its VRAM peaks sit at the lower end of main's, and GPU time is 0.2 to 1 ms lower, but two sessions on this swapping PC can't separate that from noise. So it isn't worse, and isn't measurably better here. A real 6 GB card, where going over the allowance actually pages, is the test that would show the smaller blocks paying off. On Linux NVIDIA, going over costs next to nothing. 🤖 Generated with [Claude Code](https://claude.com/claude-code)
Owner

Can you make it clear what the gain is from this, if frame time / FPS doesn't change?

Can you make it clear what the gain is from this, if frame time / FPS doesn't change?
Some checks reported errors
CI / Conventional Commit title (pull_request) Blocked by required conditions
CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Blocked by required conditions
CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Blocked by required conditions
CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Blocked by required conditions
CI / Workflow lint (actionlint) (pull_request) Blocked by required conditions
Guide screenshots / Guide images and local UI source review (pull_request) Blocked by required conditions
Third-party parts / The same as upstream (pull_request) Blocked by required conditions
Third-party parts / As their notes say (pull_request) Blocked by required conditions
Website / Website on Cloudflare (pull_request) Blocked by required conditions
Website / Website on Cloudflare (pull_request_target) Has been skipped
PR description / Description follows the template (pull_request_target) Failing after 6s
beanstalk/approval approved by HumanBean (reviewed)
beanstalk/vram6gb 0/1 pass, 1 error
beanstalk/vram-ge8gb 1/1 fail
This pull request doesn't have enough approvals yet. 0 of 1 approvals granted.
Some workflows are waiting to be reviewed.
This branch is out-of-date with the base branch
You are not authorized to merge this pull request.
View command line instructions

Checkout

From your project repository, check out a new branch and test the changes.
git fetch -u 'perf/vk-vram-6gb-blocks:flix331-perf/vk-vram-6gb-blocks'
git switch 'flix331-perf/vk-vram-6gb-blocks'
Sign in to join this conversation.
No reviewers
No milestone
No project
No assignees
4 participants
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
MagicBeanDev/Longshot!1073
No description provided.