perf(vk): optimize < 8 GB VRAM management, dynamic floor, 3-tier target pool, and DLSS CNN presets #637

Open
flix331 wants to merge 9 commits from flix331/Longshot:perf/longshot-optimus-6gb into main
First-time contributor

Type: perf · Risk: low · Tested on: Windows, RTX 3050 6 GB Laptop · Needs: Linux, AMD, > 8 GB GPUs

## Why

On constrained GPUs (< 8 GB VRAM, e.g. RTX 3050 6 GB Laptop with ~5,233 MB driver allowance), several defaults caused VRAM overshoot into WDDM PCIe
paging, render target pool thrashing, and heavy DLSS memory overhead:
1. Adding shared VRAM headroom delayed the 90%/97% pressure rules past `allowed`, letting VRAM reach 5,637 MB / 5,233 MB.
2. A static `1,800 MB` floor and fixed `512 MB` `RX_VK_COMPACT` threshold either kept texture budgets pinned at `1,800 MB` in open areas or waited too
long to defragment blocks.
3. `pool_pressed` (`pressure > 90%`) flushed the 96 MB render target pool to `0 MB`, causing ~98k `vkCreateImage`/`vkDestroyImage` churn in dense
scenes like Castle Town Market.
4. Default DLSS/DLAA Transformer presets consumed ~231 MB of internal VRAM vs ~41–48 MB for CNN presets.

## What changed

- **Real VRAM allowance & damped contraction:** Removed `shared_vram_headroom()` in `gpu/vk/vram_compact.h` and evaluated 90%/97% rules directly
against `allowed` in `gpu/vk/vk_texture.cpp`, keeping damped contraction (`over / 4`) on `< 8 GB` GPUs.
- **Proportional compaction (`RX_VK_COMPACT`):** Added `default_min_free(allowed)` (`allowed / 16`, clamped to `64–512 MB`) in `gpu/vk/vram_compact.h`
and wired it into `compact_step()` when `RX_VK_COMPACT_MB` is unset.
- **Dynamic texture floor:** Added `update_dynamic_floor()` in `gpu/vk/vram_compact.h` targeting an 86%–92% VRAM sweet spot (`1,800 MB` to `allowed /
2`) with hysteresis, resetting to `min_floor` when `used > 97%`.
- **3-tier render target pool:** Replaced binary `pool_pressed` with `pool_pressure` (`96 MB -> 48 MB` at `> 90%` -> `0 MB` at `> 97%`) in
`gpu/vk/vk_texture.cpp`.
- **Lightweight DLSS CNN presets:** Added `MODEL_E` (preset 5) and `MODEL_F` (preset 6) to `gpu/vk/upscaler_rows.h` and `gpu/vk/vk_menu.cpp` (`DLSS
model` row).

## Measurements

**Verdict:** Eliminated `> 5,233 MB` VRAM overshoots in tested scenes, reduced DLAA internal memory from `231 MB` to `41–48 MB`, and improved Castle
Town Market from `18.2–24.9 fps` (`24.8–27.7 ms` GPU time) to `32.7–35.8 fps` (`15.0–16.7 ms` GPU time).

Hardware: Windows, NVIDIA GeForce RTX 3050 6 GB Laptop (`5,233 MB` allowed), Intel Raptor Lake, 16 GB RAM
Scene: Castle Town Market & Hyrule Field; `1536x864`, NVIDIA DLAA (`Preset F` CNN)
Method: `longshot_log.txt` periodic telemetry (`VRAM:`, `GPU time`, `status:`, `render target images:`)

| | Before (headroom + static floor + Transformer) | After (`a6c35131`, CNN F) |
|---|---|---|
| VRAM used / allowed | `5,637 / 5,233 MB` | `4,820 / 5,233 MB` |
| DLAA internal (`its own`) | `231 MB` | `41–48 MB` |
| Market FPS / GPU time | `18.2–24.9 fps` / `24.8–27.7 ms` | `32.7–35.8 fps` / `15.0–16.7 ms` |
| Slowest frame (600-frame window) | `124.4–159.7 ms` | `60.2–83.2 ms` |

### Flags and settings

None (existing `RX_VK_COMPACT_MB` and `RX_VK_DLSS_PRESET` overrides remain unchanged; `CNN (E)` and `CNN (F)` are added as optional entries in `DLSS
model`).

## Testing

**Tested:**
- Windows, RTX 3050 6 GB Laptop: compiled via `nx2/make_game.py --update` and verified live gameplay across Hyrule Field and Castle Town Market with
`DLAA` (`CNN E` and `CNN F`).

**Not tested:**
- Linux, AMD GPUs, Intel Arc, and `> 8 GB` GPUs (dynamic floor and damped contraction are gated to `allowed < 8192 MB`).

**Must be tested:**
- Linux / Windows `> 8 GB` GPU: run a quick walk in Castle Town Market, expecting unchanged default behavior and `tests/test_vram_compact.py` /
`tests/test_upscaler_rows.py` passing.
**Type:** perf · **Risk:** low · **Tested on:** Windows, RTX 3050 6 GB Laptop · **Needs:** Linux, AMD, > 8 GB GPUs ## Why On constrained GPUs (< 8 GB VRAM, e.g. RTX 3050 6 GB Laptop with ~5,233 MB driver allowance), several defaults caused VRAM overshoot into WDDM PCIe paging, render target pool thrashing, and heavy DLSS memory overhead: 1. Adding shared VRAM headroom delayed the 90%/97% pressure rules past `allowed`, letting VRAM reach 5,637 MB / 5,233 MB. 2. A static `1,800 MB` floor and fixed `512 MB` `RX_VK_COMPACT` threshold either kept texture budgets pinned at `1,800 MB` in open areas or waited too long to defragment blocks. 3. `pool_pressed` (`pressure > 90%`) flushed the 96 MB render target pool to `0 MB`, causing ~98k `vkCreateImage`/`vkDestroyImage` churn in dense scenes like Castle Town Market. 4. Default DLSS/DLAA Transformer presets consumed ~231 MB of internal VRAM vs ~41–48 MB for CNN presets. ## What changed - **Real VRAM allowance & damped contraction:** Removed `shared_vram_headroom()` in `gpu/vk/vram_compact.h` and evaluated 90%/97% rules directly against `allowed` in `gpu/vk/vk_texture.cpp`, keeping damped contraction (`over / 4`) on `< 8 GB` GPUs. - **Proportional compaction (`RX_VK_COMPACT`):** Added `default_min_free(allowed)` (`allowed / 16`, clamped to `64–512 MB`) in `gpu/vk/vram_compact.h` and wired it into `compact_step()` when `RX_VK_COMPACT_MB` is unset. - **Dynamic texture floor:** Added `update_dynamic_floor()` in `gpu/vk/vram_compact.h` targeting an 86%–92% VRAM sweet spot (`1,800 MB` to `allowed / 2`) with hysteresis, resetting to `min_floor` when `used > 97%`. - **3-tier render target pool:** Replaced binary `pool_pressed` with `pool_pressure` (`96 MB -> 48 MB` at `> 90%` -> `0 MB` at `> 97%`) in `gpu/vk/vk_texture.cpp`. - **Lightweight DLSS CNN presets:** Added `MODEL_E` (preset 5) and `MODEL_F` (preset 6) to `gpu/vk/upscaler_rows.h` and `gpu/vk/vk_menu.cpp` (`DLSS model` row). ## Measurements **Verdict:** Eliminated `> 5,233 MB` VRAM overshoots in tested scenes, reduced DLAA internal memory from `231 MB` to `41–48 MB`, and improved Castle Town Market from `18.2–24.9 fps` (`24.8–27.7 ms` GPU time) to `32.7–35.8 fps` (`15.0–16.7 ms` GPU time). Hardware: Windows, NVIDIA GeForce RTX 3050 6 GB Laptop (`5,233 MB` allowed), Intel Raptor Lake, 16 GB RAM Scene: Castle Town Market & Hyrule Field; `1536x864`, NVIDIA DLAA (`Preset F` CNN) Method: `longshot_log.txt` periodic telemetry (`VRAM:`, `GPU time`, `status:`, `render target images:`) | | Before (headroom + static floor + Transformer) | After (`a6c35131`, CNN F) | |---|---|---| | VRAM used / allowed | `5,637 / 5,233 MB` | `4,820 / 5,233 MB` | | DLAA internal (`its own`) | `231 MB` | `41–48 MB` | | Market FPS / GPU time | `18.2–24.9 fps` / `24.8–27.7 ms` | `32.7–35.8 fps` / `15.0–16.7 ms` | | Slowest frame (600-frame window) | `124.4–159.7 ms` | `60.2–83.2 ms` | ### Flags and settings None (existing `RX_VK_COMPACT_MB` and `RX_VK_DLSS_PRESET` overrides remain unchanged; `CNN (E)` and `CNN (F)` are added as optional entries in `DLSS model`). ## Testing **Tested:** - Windows, RTX 3050 6 GB Laptop: compiled via `nx2/make_game.py --update` and verified live gameplay across Hyrule Field and Castle Town Market with `DLAA` (`CNN E` and `CNN F`). **Not tested:** - Linux, AMD GPUs, Intel Arc, and `> 8 GB` GPUs (dynamic floor and damped contraction are gated to `allowed < 8192 MB`). **Must be tested:** - Linux / Windows `> 8 GB` GPU: run a quick walk in Castle Town Market, expecting unchanged default behavior and `tests/test_vram_compact.py` / `tests/test_upscaler_rows.py` passing.
Architectural notes & hardware benchmark:
- On NVIDIA Optimus (MSHybrid / no MUX), the game runs significantly smoother and more
  stable using original Nintendo settings without artificial upscaler/filter overhead.
- Memory benchmark: Total graphics memory is bounded at 7.8 GB of VRAM (5.3 GB dedicated
  physical safety limit + 2.5 GB shared VRAM) plus ~5.3 GB of system RAM for the rest of
  the host program (CPU heap, guest runtime, audio, and OS overhead).
- Parity with native Switch 2 (Tegra T239): The 7.8 GB VRAM footprint directly mirrors the
  factory game memory budget on real console hardware (~7.5-8.0 GB of accessible memory).
- UMA vs Discrete: A native Switch 2 can afford such generous memory allowances because
  CPU and GPU share Unified Memory Architecture (UMA) on a single LPDDR5X bus with zero
  transfer delay. On PC laptops with discrete GPUs, strict memory budgeting and write-back
  throttling are mandatory to prevent PCIe bus saturation and contention with the Intel
  integrated display controller.
- Display routing & memory ceiling note (Target: Acer Nitro V15 - RTX 3050 6GB, 16GB DDR5):
  Performance improves drastically when routing output directly to an external monitor via
  HDMI (dGPU bypass), representing the best possible setup without major engine-level
  refactoring. Beyond this, the only physical path for further gains is expanding system
  RAM (e.g., to 32 GB DDR5) to eliminate shared VRAM and PCIe bus contention entirely.

Key implementation details:
- Auto-detect NVIDIA nv-vk64.json ICD in DriverStore on Windows to bypass Intel iGPU
  loader hooking, layer conflicts, and PCIe display contention.
- Enable dedicated swapchain presenter thread/queue by default on NVIDIA dGPUs (0x10DE).
- Prewarm shaders before title screen (load_mode = start) when running under release
  executable (host <= 14), eliminating in-game pipeline compilation stalls and dropped draws.
- Activate idle import reclamation (high=85, low=70) on NVIDIA on Windows, freeing ~2.8 GB
  of system RAM and preventing OS page file thrashing on 16 GB machines.
- Default RX_VK_WB_MAX_KB to 1024 KB, cutting ~68 MB/frame of unread culling write-backs.
- Increase mirror budget to 1700 MB on GPUs <= 6GB, preventing mirror thrashing and
  costly PCIe fallback in open areas (Hyrule Field / Death Mountain).
- Dynamic 30% texture budget on <= 6GB cards (~1450-1550 MB) with 1 MB ASTC->BC threshold.
- Reduce VMA image chunks to 32 MB, target pool to 96 MB, dead target retention to 2 frames,
  and keep finalized normal maps to 128 MB.
- Tighten texture hold to 180 frames (3s) and reserve to 600 frames (10s) with 80% compact threshold.
- Add backward compatibility for host API version 14 in mkwvk_init.
- Support official NVIDIA NGX SDK integration (DLSS / DLAA / DLSSG).
- Update test_vram_compact.cpp assertions to match new retention thresholds (all tests green).
- Enforce strict 1,800 MB minimum texture floor in vram_compact.h and vk_texture.cpp without the previous total_bytes >= floor bypass.
- Decouple ASTC uploads in upload_defer: retain 100% synchronous uploads for multi-layer terrain arrays (layers > 1 or > 1024x1024) to prevent ground flickering while pacing single-layer NPC and prop textures to 12 ms per frame.
- Evaluate 90% and 97% memory pressure thresholds against effective budget (allowed + shared_vram_headroom ~7.79 GB), unlocking dedicated GDDR6 VRAM usage up to 5,730 MiB without triggering panic budget cuts.
- Increase idle staging arena retention from 64 MB to 128 MB on 6GB GPUs to eliminate Resizable BAR vkAllocateMemory OS kernel latency.
- Remove shared_vram_headroom() and evaluate 90%/97% pressure rules directly against driver-reported allowed to prevent WDDM PCIe paging on < 8 GB GPUs.
- Preserve damped contraction (over / 4) on constrained GPUs in held_budget() and the 90% rule to absorb transient camera/frame spikes without thrashing.
- Scale default RX_VK_COMPACT threshold proportionally (allowed / 16, clamped to 64-512 MB) via default_min_free().
- Add update_dynamic_floor() with 86%-92% hysteresis (1800 MB to allowed / 2) and immediate reset to min_floor when used > 97%.
- Replace binary pool_pressed flush with 3-tier pool_pressure (96 MB -> 48 MB at > 90% -> 0 MB at > 97%) to preserve render target reuse under pressure.
feat(vk): expose lightweight DLSS CNN presets (E and F) in the DLSS model menu
Some checks failed
PR description / Description follows the template (pull_request_target) Successful in 3s
CI / Conventional Commit title (pull_request) Has been cancelled
CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Has been cancelled
CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Has been cancelled
CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Has been cancelled
CI / Workflow lint (actionlint) (pull_request) Has been cancelled
Third-party parts / As their notes say (pull_request) Has been cancelled
Third-party parts / The same as upstream (pull_request) Has been cancelled
a6c351316e
- Add MODEL_E (preset 5) and MODEL_F (preset 6) to upscaler_rows.h while preserving existing preset indices and RX_VK_DLSS_PRESET override behavior.
- Add CNN (E) and CNN (F) entries and hints in vk_menu.cpp to reduce DLSS VRAM footprint (~48-59 MB vs ~231 MB Transformer) and improve Tensor Core throughput on constrained RTX GPUs.
flix331 changed title from perf(vk): optimize < 8 GB VRAM management, dynamic floor, 3-tier target pool, and DLSS CNN presets to WIP: perf(vk): optimize < 8 GB VRAM management, dynamic floor, 3-tier target pool, and DLSS CNN presets 2026-11-15 03:25:47 +00:00
perf(vk): reduce VMA block fragmentation and prealloc pinning on <= 8 GB GPUs
Some checks failed
PR description / Description follows the template (pull_request_target) Successful in 14s
CI / Conventional Commit title (pull_request) Has been cancelled
CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Has been cancelled
CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Has been cancelled
CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Has been cancelled
CI / Workflow lint (actionlint) (pull_request) Has been cancelled
Third-party parts / As their notes say (pull_request) Has been cancelled
Third-party parts / The same as upstream (pull_request) Has been cancelled
695e9b8193
- vram_compact.h (`blocks_at_start`): lower initial texture pool preallocation
      on < 8 GB heaps from 15% (up to 24 blocks / 768 MB) to 8% (2-12 blocks /
      up to 384 MB) so `returnable()` stays positive and `compact_step` can compact
      and return empty blocks above 384 MB instead of pinning 768 MB permanently.
    - vk_vma.cpp (`vma_init`): use 32 MB `preferredLargeHeapBlockSize` instead of
      64 MB on <= 8 GB device-local heaps (via `device_local_heap_bytes()`) to
      improve block reclamation for device buffers and sparse mirror blocks.
    - vk_vma.cpp (`target_images_pool`): use 16 MB blocks instead of 32 MB on
      <= 8 GB heaps to halve trapped free space from transient render targets.

    Observed on RTX 3050 6 GB Laptop (`5,233 MB` allowance):
    - VMA `free in the blocks` dropped from 1,448 MB to 532-649 MB.
    - `texture_pool` compacted down to 22 blocks (704 MB holding 673 MB of textures).
    - `target_image_pool` wasted space dropped from 467 MB to 242 MB, allowing
      `pool_pressure` to recover out of critical (`VRAM short: halved`).
    - Worst-case 600-frame spikes dropped from 603.9-1,168.4 ms to 75.0-99.6 ms.
flix331 force-pushed perf/longshot-optimus-6gb from 695e9b8193
Some checks failed
PR description / Description follows the template (pull_request_target) Successful in 14s
CI / Conventional Commit title (pull_request) Has been cancelled
CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Has been cancelled
CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Has been cancelled
CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Has been cancelled
CI / Workflow lint (actionlint) (pull_request) Has been cancelled
Third-party parts / As their notes say (pull_request) Has been cancelled
Third-party parts / The same as upstream (pull_request) Has been cancelled
to 70a78f9339
Some checks failed
CI / Conventional Commit title (pull_request) Blocked by required conditions
CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Blocked by required conditions
CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Blocked by required conditions
CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Blocked by required conditions
CI / Workflow lint (actionlint) (pull_request) Blocked by required conditions
Guide screenshots / Guide images and local UI source review (pull_request) Blocked by required conditions
Third-party parts / As their notes say (pull_request) Blocked by required conditions
Third-party parts / The same as upstream (pull_request) Blocked by required conditions
Website / Website on Cloudflare (pull_request) Blocked by required conditions
PR description / Description follows the template (pull_request_target) Failing after 15s
Website / Website on Cloudflare (pull_request_target) Has been skipped
beanstalk/approval approved by HumanBean (reviewed)
beanstalk/linux+vram-le6gb 0/1 pass, 0 running, 1 queued
beanstalk/linux+vram-ge9gb 0/1 pass, 0 running, 1 queued
2026-11-15 05:37:12 +00:00
Compare
flix331 changed title from WIP: perf(vk): optimize < 8 GB VRAM management, dynamic floor, 3-tier target pool, and DLSS CNN presets to perf(vk): optimize < 8 GB VRAM management, dynamic floor, 3-tier target pool, and DLSS CNN presets 2026-11-15 05:42:01 +00:00
Owner

@Beanstalk

@Beanstalk
Member

Beanstalk test farm

Commit 70a78f9 · tier reviewed · approved by HumanBean · dashboard

What will be tested · AI plan (gpt-6.1-terra)

The plan compares VRAM use, frame pacing, load times and scene images in Castle Town Market and Hyrule Field on GPUs with at most 6 GB and more than 8 GB of VRAM. Each target takes about 13 minutes, including a short walk and camera movement in both areas. Crashes, Vulkan device loss, significant performance or VRAM regressions, and substantial image differences fail the plan. Both targets require Linux gameplay testers, so they will wait until matching PCs come online. The new DLSS presets and repository unit tests are not covered by this farm plan. No tester online now matches linux+vram-le6gb, linux+vram-ge9gb: those targets wait for one.

  • Where: linux+vram-le6gb (6 GB VRAM or less, 1 report), linux+vram-ge9gb (9 GB VRAM or more, 1 report).
  • market-field-streaming: from a new save, then teleport to Market, move the stick, measure market for 30 s (after 5 s warm-up), take the picture market to compare with main, teleport to Hyrule Field, measure field for 30 s (after 5 s warm-up), take the picture field to compare with main; 1 run on each side.
  • Measured: crashes, frame times, stutters and hitches, VRAM use, load times, the game's log.
  • The PR fails if: it crashes; or a picture differs from main by more than 15%; or peak VRAM is more than 10% above main; or longest load is more than 25% above main; or frame time (p99) is more than 20% above main; or forbidden log lines is above 0.
  • Time: about 15 min of game runs per tester (2 game starts), plus the two builds.

Started.

Target Reports Crash Frame p99 (ms) vs main VRAM peak (MB) vs main Result
linux+vram-le6gb 0/1 — — — queued
linux+vram-ge9gb 0/1 — — — queued
<!-- beanstalk:sticky --> ### Beanstalk test farm Commit `70a78f9` · tier **reviewed** · approved by HumanBean · [dashboard](https://beanstalk.magicbean.dev/prs/637) **What will be tested** · AI plan (`gpt-6.1-terra`) The plan compares VRAM use, frame pacing, load times and scene images in Castle Town Market and Hyrule Field on GPUs with at most 6 GB and more than 8 GB of VRAM. Each target takes about 13 minutes, including a short walk and camera movement in both areas. Crashes, Vulkan device loss, significant performance or VRAM regressions, and substantial image differences fail the plan. Both targets require Linux gameplay testers, so they will wait until matching PCs come online. The new DLSS presets and repository unit tests are not covered by this farm plan. No tester online now matches linux+vram-le6gb, linux+vram-ge9gb: those targets wait for one. - **Where:** linux+vram-le6gb (6 GB VRAM or less, 1 report), linux+vram-ge9gb (9 GB VRAM or more, 1 report). - **market-field-streaming:** from a new save, then teleport to Market, move the stick, measure market for 30 s (after 5 s warm-up), take the picture market to compare with main, teleport to Hyrule Field, measure field for 30 s (after 5 s warm-up), take the picture field to compare with main; 1 run on each side. - **Measured:** crashes, frame times, stutters and hitches, VRAM use, load times, the game's log. - **The PR fails if:** it crashes; or a picture differs from main by more than 15%; or peak VRAM is more than 10% above main; or longest load is more than 25% above main; or frame time (p99) is more than 20% above main; or forbidden log lines is above 0. - **Time:** about 15 min of game runs per tester (2 game starts), plus the two builds. Started. | Target | Reports | Crash | Frame p99 (ms) vs main | VRAM peak (MB) vs main | Result | | --- | --- | --- | --- | --- | --- | | linux+vram-le6gb | 0/1 | — | — | — | queued | | linux+vram-ge9gb | 0/1 | — | — | — | queued |
HumanBean left a comment

Thanks for digging into the 6 GB laptop case: the numbers from Castle Town Market show there's a real problem on small cards. I can't take the PR in its current shape, though, for three reasons:

  1. It changes far more than the description says, and much of it applies to every card. The description lists five changes, but the diff also has these:

    • forcing the NVIDIA ICD on Windows
    • the presentation thread turned on for every NVIDIA GPU
    • the write-back cap turned on for everyone
    • a compatibility shim in mkwvk_init that accepts old hosts
    • a programs_load_decide override for v14 hosts
    • the target pool going from 256 to 96 MB for every card
    • image blocks going from 128 to 32 MB for every card
    • staging arenas going from 64 to 16 MB
    • the upload budget going from 40 to 12 ms, with a bypass for large textures
    • the hold and reserve times going from 600/1800 to 180/600 presents, and the hold percentage from 85 to 80
    • RX_VK_KEEP_FINALIZED_MB going from 512 to 128
    • RX_VK_DEAD_MIN_UNUSED going from 3 to 2
    • RX_VK_IMPORT_LOW going from 75 to 70
    • RX_VK_ASTC_BC_MB going from 2 to 1
    • the mirror budget on cards of 6.5 GB or less
    • a .gitignore change

    Several of these reverse values that the comments beside them say were measured. The description also mentions a shared_vram_headroom() removal that isn't in the diff (it was added and removed within the branch), and calls the 96 MB target pool the existing value, when main's is 256 MB.

  2. 8 GB cards are not untouched. The gates compare allowed, the driver's allowance, against GLOBAL_MIN_GRAPHICS_REQ = 8192 MB. An 8 GB card's driver allows about 7.3 GB, so the 42% budget, the damped contraction, the dynamic floor and the < 8 GB blocks_at_start rule all apply to 8 GB cards too. On top of that, texture_min_floor and the changed constants reach > 8 GB cards. Inline comments mark each spot.

  3. It conflicts with main, and the conflict needs a design decision, so I haven't pushed a rebase. jj rebase onto main gives two conflicts. Both start in the first commit (7173e807) and carry through every later commit:

    • .gitignore is trivial: keep main's test-results/ and _deps/ and add the .vs/, .vscode/ and CppProperties.json block (or drop that block: IDE files belong in a personal global gitignore).
    • vk_texture.cpp, texture_frame's own (the texture budget) is not trivial. Main has replaced the inline 45%/70% expression with texture_rule::dedicated_bytes() / integrated_bytes() in gpu/vk/texture_budget.h. Those size the budget from the card's VRAM (vram/12*7, capped at 3/4 of the allowance), not from a fixed share of the allowance. For a 6 GB card allowed 5,233 MB, that gives about 3,584 MB, against this PR's 42% (2,197 MB). Taking main's side drops the PR's budget change, and the PR's measurements were taken without main's rule. Porting the 42% into texture_budget.h would replace a rule main adopted after measuring it. Which one wins should be decided and measured on top of main, not guessed while resolving a conflict. So: rebase, take main's texture_rule call, and re-measure the market before adding any < 8 GB change on top of it.

What I'd ask for: split this into separate PRs, each measured on its own on top of main, with an A/B on the same build:

  • (a) the 3-tier target pool, which is small and plausible on its own, but keeps main's 256 MB cap
  • (b) the dynamic floor and the damped contraction, without the floor on the 97% rule (see the inline comment)
  • (c) the DLSS CNN presets
  • (d) anything else that turns out to matter, each value with numbers, gated by the card's real VRAM (device_vram_bytes()/heap size, not allowed)

Drop the Optimus ICD override, the default-on presentation thread for NVIDIA, the write-back cap default and the host-version shim. Those are not < 8 GB tuning, and the shim breaks the versioned interface (inline comments).

DLSS CNN presets: the presets are opt-in, so nobody's default changes, which is fine. But the PR's evidence for them is not separated from the rest. The "after" column is CNN F plus every memory change, against Transformer K plus main's memory rules, so the 18-25 to 33-36 fps gain can't be credited to either. The 231 MB against 41-48 MB of DLSS memory is a real measurement and a good start. A preset PR should show K, E and F on the same build and in the same scene (GPU time, VRAM, plus screenshots for image quality, since CNN against transformer is mostly a quality trade). It should also confirm that the DLSS runtime we ship still honours presets 5 and 6: NVIDIA has been retiring the CNN presets in the 310.x SDKs, and a value the runtime no longer supports may be silently mapped to another preset. If CNN turns out to be clearly better on small cards, the sensible-defaults rule points to picking it automatically there, not to a menu row with a performance claim in its hint.

Testing: so far it has run only on the author's Windows RTX 3050 6 GB laptop. The Beanstalk run on 70a78f9 (Linux, ≤ 6 GB and ≥ 9 GB) is queued and has no results yet. Before merging, each split PR would need a Linux run, an AMD run and a > 8 GB run, and an 8 GB NVIDIA run, since 8 GB cards are affected after all.

I haven't run any tests here; this review is from reading the code. Requesting changes.

Thanks for digging into the 6 GB laptop case: the numbers from Castle Town Market show there's a real problem on small cards. I can't take the PR in its current shape, though, for three reasons: 1. **It changes far more than the description says, and much of it applies to every card.** The description lists five changes, but the diff also has these: - forcing the NVIDIA ICD on Windows - the presentation thread turned on for every NVIDIA GPU - the write-back cap turned on for everyone - a compatibility shim in `mkwvk_init` that accepts old hosts - a `programs_load_decide` override for v14 hosts - the target pool going from 256 to 96 MB for every card - image blocks going from 128 to 32 MB for every card - staging arenas going from 64 to 16 MB - the upload budget going from 40 to 12 ms, with a bypass for large textures - the hold and reserve times going from 600/1800 to 180/600 presents, and the hold percentage from 85 to 80 - `RX_VK_KEEP_FINALIZED_MB` going from 512 to 128 - `RX_VK_DEAD_MIN_UNUSED` going from 3 to 2 - `RX_VK_IMPORT_LOW` going from 75 to 70 - `RX_VK_ASTC_BC_MB` going from 2 to 1 - the mirror budget on cards of 6.5 GB or less - a `.gitignore` change Several of these reverse values that the comments beside them say were measured. The description also mentions a `shared_vram_headroom()` removal that isn't in the diff (it was added and removed within the branch), and calls the 96 MB target pool the existing value, when main's is 256 MB. 2. **8 GB cards are not untouched.** The gates compare `allowed`, the driver's allowance, against `GLOBAL_MIN_GRAPHICS_REQ = 8192 MB`. An 8 GB card's driver allows about 7.3 GB, so the 42% budget, the damped contraction, the dynamic floor and the < 8 GB `blocks_at_start` rule all apply to 8 GB cards too. On top of that, `texture_min_floor` and the changed constants reach > 8 GB cards. Inline comments mark each spot. 3. **It conflicts with main, and the conflict needs a design decision, so I haven't pushed a rebase.** `jj rebase` onto main gives two conflicts. Both start in the first commit (7173e807) and carry through every later commit: - `.gitignore` is trivial: keep main's `test-results/` and `_deps/` and add the `.vs/`, `.vscode/` and `CppProperties.json` block (or drop that block: IDE files belong in a personal global gitignore). - `vk_texture.cpp`, `texture_frame`'s `own` (the texture budget) is not trivial. Main has replaced the inline 45%/70% expression with `texture_rule::dedicated_bytes()` / `integrated_bytes()` in `gpu/vk/texture_budget.h`. Those size the budget from the card's VRAM (`vram/12*7`, capped at 3/4 of the allowance), not from a fixed share of the allowance. For a 6 GB card allowed 5,233 MB, that gives about 3,584 MB, against this PR's 42% (2,197 MB). Taking main's side drops the PR's budget change, and the PR's measurements were taken without main's rule. Porting the 42% into `texture_budget.h` would replace a rule main adopted after measuring it. Which one wins should be decided and measured on top of main, not guessed while resolving a conflict. So: rebase, take main's `texture_rule` call, and re-measure the market before adding any < 8 GB change on top of it. **What I'd ask for:** split this into separate PRs, each measured on its own on top of main, with an A/B on the same build: - (a) the 3-tier target pool, which is small and plausible on its own, but keeps main's 256 MB cap - (b) the dynamic floor and the damped contraction, without the floor on the 97% rule (see the inline comment) - (c) the DLSS CNN presets - (d) anything else that turns out to matter, each value with numbers, gated by the card's real VRAM (`device_vram_bytes()`/heap size, not `allowed`) Drop the Optimus ICD override, the default-on presentation thread for NVIDIA, the write-back cap default and the host-version shim. Those are not < 8 GB tuning, and the shim breaks the versioned interface (inline comments). **DLSS CNN presets:** the presets are opt-in, so nobody's default changes, which is fine. But the PR's evidence for them is not separated from the rest. The "after" column is CNN F plus every memory change, against Transformer K plus main's memory rules, so the 18-25 to 33-36 fps gain can't be credited to either. The 231 MB against 41-48 MB of DLSS memory is a real measurement and a good start. A preset PR should show K, E and F on the same build and in the same scene (GPU time, VRAM, plus screenshots for image quality, since CNN against transformer is mostly a quality trade). It should also confirm that the DLSS runtime we ship still honours presets 5 and 6: NVIDIA has been retiring the CNN presets in the 310.x SDKs, and a value the runtime no longer supports may be silently mapped to another preset. If CNN turns out to be clearly better on small cards, the sensible-defaults rule points to picking it automatically there, not to a menu row with a performance claim in its hint. **Testing:** so far it has run only on the author's Windows RTX 3050 6 GB laptop. The Beanstalk run on `70a78f9` (Linux, ≤ 6 GB and ≥ 9 GB) is queued and has no results yet. Before merging, each split PR would need a Linux run, an AMD run and a > 8 GB run, and an 8 GB NVIDIA run, since 8 GB cards are affected after all. I haven't run any tests here; this review is from reading the code. Requesting changes.
@ -233,3 +233,3 @@
/// generation transformer, preset M (NVIDIA's own pick for Performance); 2 the same generation as tuned for Ultra
/// Performance, preset L; 3 NVIDIA's default, preset 0 (the runtime picks per mode, and a runtime update can change it)
enum { MODEL_K = 0, MODEL_M = 1, MODEL_L = 2, MODEL_DEFAULT = 3, kModels = 4 };
enum { MODEL_K = 0, MODEL_M = 1, MODEL_L = 2, MODEL_DEFAULT = 3, MODEL_E = 4, MODEL_F = 5, kModels = 6 };
Owner

Opt-in, so no default changes; that's fine. But please confirm that the DLSS runtime we ship honours presets 5 and 6. NVIDIA has been retiring the CNN presets in the 310.x SDKs, and a value the runtime no longer supports may be quietly mapped to another preset, so log the preset the runtime reports. Also please move this to its own PR, with K, E and F measured on the same build and scene (GPU time, VRAM, screenshots). The fps gain in this PR's table mixes CNN F with all the memory changes.

Opt-in, so no default changes; that's fine. But please confirm that the DLSS runtime we ship honours presets 5 and 6. NVIDIA has been retiring the CNN presets in the 310.x SDKs, and a value the runtime no longer supports may be quietly mapped to another preset, so log the preset the runtime reports. Also please move this to its own PR, with K, E and F measured on the same build and scene (GPU time, VRAM, screenshots). The fps gain in this PR's table mixes CNN F with all the memory changes.
@ -53,0 +53,4 @@
// NVIDIA Optimus hybrid laptop architecture (CPU-GPU PCIe bus for display): if no ICD is explicitly set,
// locate NVIDIA's nv-vk64.json in DriverStore to bind Vulkan exclusively to the dGPU, bypassing Intel iGPU loader hooking
// and preventing PCIe display contention.
if (!getenv("VK_ICD_FILENAMES")) {
Owner

This forces VK_ICD_FILENAMES for every Windows machine whose DriverStore has any nv* package with an nv-vk64.json. Setting it makes the loader load only that ICD. (1) A PC with an AMD or Intel Arc card that once had an NVIDIA driver installed (Windows keeps the old packages in DriverStore) then sees no usable GPU. (2) DriverStore usually holds several NVIDIA driver versions, and FindFirstFile order picks whichever folder comes first, not the installed one. An ICD JSON from an old package points at an old nvoglv64.dll that doesn't match the running kernel driver, so instance creation fails or the device is lost. (3) The variable is inherited by every child process, and C:\\Windows is hard-coded. The Windows loader already picks up the NVIDIA ICD from the registry. Optimus GPU selection is a matter of device choice (or the NvOptimusEnablement export), not of hiding ICDs. Please remove this block.

This forces `VK_ICD_FILENAMES` for every Windows machine whose DriverStore has any `nv*` package with an `nv-vk64.json`. Setting it makes the loader load **only** that ICD. (1) A PC with an AMD or Intel Arc card that once had an NVIDIA driver installed (Windows keeps the old packages in DriverStore) then sees no usable GPU. (2) DriverStore usually holds several NVIDIA driver versions, and `FindFirstFile` order picks whichever folder comes first, not the installed one. An ICD JSON from an old package points at an old `nvoglv64.dll` that doesn't match the running kernel driver, so instance creation fails or the device is lost. (3) The variable is inherited by every child process, and `C:\\Windows` is hard-coded. The Windows loader already picks up the NVIDIA ICD from the registry. Optimus GPU selection is a matter of device choice (or the `NvOptimusEnablement` export), not of hiding ICDs. Please remove this block.
@ -430,1 +450,3 @@
const bool presenter_wanted = pt_env && atoi(pt_env) != 0; // (off by default: no queue made for it)
// (hookshot) NVIDIA Optimus: enable dedicated presentation thread/queue by default on NVIDIA dGPUs (0x10DE)
// to decouple swapchain presentation over PCIe from the replay render loop.
const bool presenter_wanted = pt_env ? (atoi(pt_env) != 0) : (dev.props.vendorID == 0x10DE);
Owner

This turns the presentation thread and queue on by default for every NVIDIA GPU, desktop cards included. RX_VK_PRESENT_THREAD is off by default on main, and this change has no measurement and isn't in the description. It belongs in its own PR with frame-pacing numbers from more than one NVIDIA card; otherwise keep main's pt_env && atoi(pt_env) != 0.

This turns the presentation thread and queue on by default for every NVIDIA GPU, desktop cards included. `RX_VK_PRESENT_THREAD` is off by default on main, and this change has no measurement and isn't in the description. It belongs in its own PR with frame-pacing numbers from more than one NVIDIA card; otherwise keep main's `pt_env && atoi(pt_env) != 0`.
@ -1590,3 +1616,2 @@
if (!h) return nullptr;
if (h->version != MKWVK_API_VERSION) {
std::fprintf(stderr, "vk: host version mismatch: host has %u, expected %u\n", h->version, MKWVK_API_VERSION);
if (h->version < 14 || h->version > MKWVK_API_VERSION) {
Owner

This shim has to go. MKWVK_API_VERSION is 22. A v15-v21 host passes a smaller MkwVkHost than v22's, but copy_size is sizeof(MkwVkHost) for every version except 14, so the memcpy reads past the end of the host's struct. Whatever comes after it then becomes function pointers the renderer will call. For a v14 host, every field after wide_menus is zeroed, and any unchecked call through one of them is a null call. table.version = h->version (line 1667) then tells an old host that a v22 table is its own version. The interface is versioned on purpose (AGENTS.md: a change to MkwVkHost/MkwVkRenderer bumps the version, and host and renderer must match). Supporting old hosts is a design change of its own, not part of < 8 GB tuning. Please restore h->version != MKWVK_API_VERSION, host = h; and table.version = MKWVK_API_VERSION, and drop the host_v14 override in vk_programs.cpp with them.

This shim has to go. `MKWVK_API_VERSION` is 22. A v15-v21 host passes a smaller `MkwVkHost` than v22's, but `copy_size` is `sizeof(MkwVkHost)` for every version except 14, so the `memcpy` reads past the end of the host's struct. Whatever comes after it then becomes function pointers the renderer will call. For a v14 host, every field after `wide_menus` is zeroed, and any unchecked call through one of them is a null call. `table.version = h->version` (line 1667) then tells an old host that a v22 table is its own version. The interface is versioned on purpose (AGENTS.md: a change to `MkwVkHost`/`MkwVkRenderer` bumps the version, and host and renderer must match). Supporting old hosts is a design change of its own, not part of < 8 GB tuning. Please restore `h->version != MKWVK_API_VERSION`, `host = h;` and `table.version = MKWVK_API_VERSION`, and drop the `host_v14` override in `vk_programs.cpp` with them.
@ -3112,1 +3112,3 @@
return mirror_rule::auto_bytes(device_vram_bytes());
const uint64_t vram = device_vram_bytes();
// Cards of 6 GB or less: allocate up to 1700 MB to hold open-area peaks without falling back to PCIe reads.
if (vram && vram <= (6ull << 30) + (512ull << 20)) return std::min<uint64_t>(1700ull << 20, vram / 3);
Owner

On a 4 GB card this nearly triples the mirror budget: auto_bytes gives an eighth (512 MB) at 4.5 GB or less, and this gives min(1700, vram/3) = 1,365 MB. Those are the cards from issue 519, where VRAM was already oversubscribed. On 6 GB cards it goes from 1,536 to 1,700 MB, which is the opposite of what the rest of the PR is trying to do. If it's needed, please gate it to the measured range (say 5.5-6.5 GB) and show the mirror and VRAM numbers for it; it's also not in the description.

On a 4 GB card this nearly triples the mirror budget: `auto_bytes` gives an eighth (512 MB) at 4.5 GB or less, and this gives `min(1700, vram/3)` = 1,365 MB. Those are the cards from issue 519, where VRAM was already oversubscribed. On 6 GB cards it goes from 1,536 to 1,700 MB, which is the opposite of what the rest of the PR is trying to do. If it's needed, please gate it to the measured range (say 5.5-6.5 GB) and show the mirror and VRAM numbers for it; it's also not in the description.
@ -3616,2 +3619,3 @@
static const uint64_t v = [] {
const uint64_t b = wb_learn::max_bytes(getenv("RX_VK_WB_MAX_KB"));
// (hookshot) Default to 1024 KB cap to prevent ~68 MB/frame of unread culling write-backs over PCIe
const uint64_t b = getenv("RX_VK_WB_MAX_KB") ? wb_learn::max_bytes(getenv("RX_VK_WB_MAX_KB")) : (1024ull << 10);
Owner

This turns the write-back cap on for every card. wb_learn.h says it is "off by default until it shows a gain on more than one machine", and it is a correctness-sensitive switch: data the CPU later reads that wasn't written back would come back stale. It isn't in the description, and it has no measurement beyond the comment's estimate. Please keep main's wb_learn::max_bytes(getenv("RX_VK_WB_MAX_KB")), and propose enabling it in its own PR with runs on several machines.

This turns the write-back cap on for every card. `wb_learn.h` says it is "off by default until it shows a gain on more than one machine", and it is a correctness-sensitive switch: data the CPU later reads that wasn't written back would come back stale. It isn't in the description, and it has no measurement beyond the comment's estimate. Please keep main's `wb_learn::max_bytes(getenv("RX_VK_WB_MAX_KB"))`, and propose enabling it in its own PR with runs on several machines.
@ -4368,3 +4372,3 @@
#endif
static const int high = std::clamp(env_int("RX_VK_IMPORT_HIGH", high_default), 0, 100);
static const int low = std::clamp(env_int("RX_VK_IMPORT_LOW", 75), 10, 95);
static const int low = std::clamp(env_int("RX_VK_IMPORT_LOW", 70), 10, 95);
Owner

A change from 75 to 70 for every card, with no measurement, not in the description, and the comments above (lines 4197 and 4330) still say 75. Please revert it, or measure it and fix the comments.

A change from 75 to 70 for every card, with no measurement, not in the description, and the comments above (lines 4197 and 4330) still say 75. Please revert it, or measure it and fix the comments.
@ -269,3 +269,3 @@
}
const char* tx_dlss_model(int v) {
static const char* const t[] = {"Transformer (K)", "Transformer 2 (M)", "Transformer 2 (L)", "NVIDIA default"};
static const char* const t[] = {"Transformer (K)", "Transformer 2 (M)", "Transformer 2 (L)", "NVIDIA default", "CNN (E)", "CNN (F)"};
Owner

The hints ("runs faster on constrained RTX GPUs") are a performance claim, and so far no measurement separates it from the memory changes. Please keep the hint to what it is (NVIDIA's CNN model, smaller VRAM footprint) until the K/E/F comparison is in.

The hints ("runs faster on constrained RTX GPUs") are a performance claim, and so far no measurement separates it from the memory changes. Please keep the hint to what it is (NVIDIA's CNN model, smaller VRAM footprint) until the K/E/F comparison is in.
@ -350,2 +350,2 @@
env_on("RX_VK_PROGRAM_CACHE_ZSTD")); // (as vk_progcache.cpp disk_init reads it)
load_mode.store(c.create ? 1 : 0, std::memory_order_release);
env_on("RX_VK_PROGRAM_CACHE_ZSTD"));
const bool host_v14 = (!getenv("RX_VK_LOAD") && host && host->version <= 14);
Owner

This only makes sense with the host-version shim in mkwvk_init, which should go (see that comment). After that, host->version is always MKWVK_API_VERSION, so this is dead code. Its log text also says load-on-create needs host >= 21 but tests <= 14, so hosts 15-20 would still get create. Please remove it.

This only makes sense with the host-version shim in `mkwvk_init`, which should go (see that comment). After that, `host->version` is always `MKWVK_API_VERSION`, so this is dead code. Its log text also says load-on-create needs host >= 21 but tests `<= 14`, so hosts 15-20 would still get `create`. Please remove it.
@ -314,3 +314,3 @@
}
uint64_t pool_cap() {
static const uint64_t mb = getenv("RX_VK_TARGET_POOL_MB") ? (uint64_t)std::max(0, atoi(getenv("RX_VK_TARGET_POOL_MB"))) : 256;
static const uint64_t mb = getenv("RX_VK_TARGET_POOL_MB") ? (uint64_t)std::max(0, atoi(getenv("RX_VK_TARGET_POOL_MB"))) : 96;
Owner

Main's target pool is 256 MB for every card. Dropping it to 96 MB also shrinks it on > 8 GB cards, contrary to the description's claim. The 3-tier pressure logic in pool_trim is the useful part, and it works with any cap. Please keep 256, or gate a smaller cap to small cards with churn numbers (render target images: ... made new).

Main's target pool is 256 MB for every card. Dropping it to 96 MB also shrinks it on > 8 GB cards, contrary to the description's claim. The 3-tier pressure logic in `pool_trim` is the useful part, and it works with any cap. Please keep 256, or gate a smaller cap to small cards with churn numbers (`render target images: ... made new`).
@ -439,3 +440,3 @@
};
std::vector<Arena> arenas;
constexpr uint64_t kArena = 64ull << 20;
constexpr uint64_t kArena = 16ull << 20;
Owner

16 MB arenas for every card: an area load's burst now takes four times as many vkAllocateMemory calls, and the PR's own comment at 4195 says those cost 70-108 ms each. The PR doesn't measure this. Please revert to 64 MB, or show load-time numbers for small and large cards.

16 MB arenas for every card: an area load's burst now takes four times as many `vkAllocateMemory` calls, and the PR's own comment at 4195 says those cost 70-108 ms each. The PR doesn't measure this. Please revert to 64 MB, or show load-time numbers for small and large cards.
@ -2052,3 +2053,3 @@
// less: BC3's 8 keep what they have) from 2 MB as RGBA8 (RX_VK_ASTC_BC_MB), the finer ones (UI, characters)
// only from 64 MB)
static const uint64_t low = (uint64_t)(getenv("RX_VK_ASTC_BC_MB") ? atoi(getenv("RX_VK_ASTC_BC_MB")) : 2) << 20;
static const uint64_t low = (uint64_t)(getenv("RX_VK_ASTC_BC_MB") ? atoi(getenv("RX_VK_ASTC_BC_MB")) : 1) << 20;
Owner

From 2 to 1 MB for every card, with no measurement and not in the description. Please revert it, or measure it in its own change.

From 2 to 1 MB for every card, with no measurement and not in the description. Please revert it, or measure it in its own change.
@ -2509,1 +2510,3 @@
static const int64_t budget_ns = (int64_t)(getenv("RX_VK_UPLOAD_BUDGET_MS") ? atof(getenv("RX_VK_UPLOAD_BUDGET_MS")) : 40.0) * 1000000;
// Terrain materials (layer arrays > 1 or very large surfaces) must always upload synchronously
// to prevent ground flickering. Only defer smaller single-layer NPC/character/prop textures.
const bool is_terrain = (tex.layers > 1) || (tex.width > 1024 && tex.height > 1024);
Owner

This bypasses the budget for every array texture and every texture over 1024x1024, on every card. An area load's burst of large or array textures is then never spread over frames, which is the stall the deferral exists to prevent. The comment above it also records that a lower budget (8 ms) made the streamed ground flicker, and line 2516 now sets 12 ms for everyone. The ground-flicker fix on main was the 40 ms budget, not a shape test. Please revert both lines, or put them in a measured change of their own (flicker and load stalls on more than one card).

This bypasses the budget for every array texture and every texture over 1024x1024, on every card. An area load's burst of large or array textures is then never spread over frames, which is the stall the deferral exists to prevent. The comment above it also records that a lower budget (8 ms) made the streamed ground flicker, and line 2516 now sets 12 ms for everyone. The ground-flicker fix on main was the 40 ms budget, not a shape test. Please revert both lines, or put them in a measured change of their own (flicker and load stalls on more than one card).
@ -4255,2 +4271,3 @@
if (!own) // (RX_VK_VRAM_MB, else dynamic budget from GLOBAL_MIN_GRAPHICS_REQ: 42% on < 8GB, 45% on 8GB, 70% on larger cards)
own = getenv("RX_VK_VRAM_MB") ? (uint64_t)atoi(getenv("RX_VK_VRAM_MB")) << 20
: known ? std::min<uint64_t>(allowed / 100 * (allowed <= (8704ull << 20) ? 45 : 70), 6144ull << 20)
: known ? std::min<uint64_t>(allowed / 100 * (allowed < vram_compact::GLOBAL_MIN_GRAPHICS_REQ ? 42 : (allowed <= (8704ull << 20) ? 45 : 70)),
Owner

(The rebase conflict lands here.) Main has replaced this expression with texture_rule::dedicated_bytes()/integrated_bytes() (gpu/vk/texture_budget.h), which sizes the budget from the card's VRAM. A 6 GB card allowed 5,233 MB gets about 3,584 MB there, against the 2,197 MB here. Note also that allowed < 8192 MB is true for 8 GB cards, whose driver allows about 7.3 GB, so this 42% would hit them too. Please rebase, take main's texture_rule call, and re-measure before proposing a < 8 GB change on top of it, gated by device_vram_bytes().

(The rebase conflict lands here.) Main has replaced this expression with `texture_rule::dedicated_bytes()`/`integrated_bytes()` (`gpu/vk/texture_budget.h`), which sizes the budget from the card's VRAM. A 6 GB card allowed 5,233 MB gets about 3,584 MB there, against the 2,197 MB here. Note also that `allowed < 8192 MB` is true for 8 GB cards, whose driver allows about 7.3 GB, so this 42% would hit them too. Please rebase, take main's `texture_rule` call, and re-measure before proposing a < 8 GB change on top of it, gated by `device_vram_bytes()`.
@ -4376,3 +4406,2 @@
if (compact_on() && known && !by_blocks) {
static const uint64_t pct = getenv("RX_VK_COMPACT_PCT") ? (uint64_t)std::clamp(atoi(getenv("RX_VK_COMPACT_PCT")), 50, 90) : 85;
const uint64_t held = vram_compact::held_budget(budget, total_bytes, used, reclaim, allowed, pct);
static const uint64_t pct = getenv("RX_VK_COMPACT_PCT") ? (uint64_t)std::clamp(atoi(getenv("RX_VK_COMPACT_PCT")), 50, 90) : 80;
Owner

From 85 to 80 for every card. The comment just above records the measurement behind 85 (a Hyrule Field walk at DLAA 1440p: 11-12 frames over 33 ms, against 34-49). Please keep 85, or bring a new measurement on more than one card.

From 85 to 80 for every card. The comment just above records the measurement behind 85 (a Hyrule Field walk at DLAA 1440p: 11-12 frames over 33 ms, against 34-49). Please keep 85, or bring a new measurement on more than one card.
@ -4364,3 +4391,3 @@
if (known && pressure > allowed / 10 * 9) {
const uint64_t over = pressure - allowed / 10 * 9;
budget = std::min(budget, total_bytes > over ? total_bytes - over : 0);
const uint64_t damped_over = (allowed < vram_compact::GLOBAL_MIN_GRAPHICS_REQ) ? (over / 4) : over;
Owner

allowed < GLOBAL_MIN_GRAPHICS_REQ is true for 8 GB cards, so the 90% rule now gives back only a quarter of the excess per look on them too. The same applies to held_budget and update_dynamic_floor. If these are meant for cards under 8 GB, gate them by the card's size (device_vram_bytes()/heap), not by the driver's allowance, and say which cards were measured.

`allowed < GLOBAL_MIN_GRAPHICS_REQ` is true for 8 GB cards, so the 90% rule now gives back only a quarter of the excess per look on them too. The same applies to `held_budget` and `update_dynamic_floor`. If these are meant for cards under 8 GB, gate them by the card's size (`device_vram_bytes()`/heap), not by the driver's allowance, and say which cards were measured.
@ -4409,3 +4439,2 @@
const uint64_t over = used - count_line;
budget = std::min(budget, total_bytes > over ? total_bytes - over : 0);
rule_budget = std::min(rule_budget, total_bytes > over ? total_bytes - over : 0);
const uint64_t floor = vram_compact::texture_min_floor(allowed);
Owner

A floor on the 97% rule defeats it. This is the rule that keeps the count under what the driver allows: the blocks_at_start notes record a device loss at 6.6 GB of 6.5 GB allowed, and issue 519 records 4 fps from paging. With texture_min_floor, the textures can't go below 1,800 MB. On a 4 GB card the driver allows about 3.3 GB, and the game's other VRAM is the larger part of that (see the 45% comment), so the count then stays above the limit for good, which means paging or device loss. On > 8 GB cards the floor is allowed/4 (about 3.8 GB on a 16 GB card), so this also changes big cards. Please leave the 97% rule without a floor; a floor belongs, if anywhere, only on the soft lines (90% and the hold).

A floor on the 97% rule defeats it. This is the rule that keeps the count under what the driver allows: the `blocks_at_start` notes record a device loss at 6.6 GB of 6.5 GB allowed, and issue 519 records 4 fps from paging. With `texture_min_floor`, the textures can't go below 1,800 MB. On a 4 GB card the driver allows about 3.3 GB, and the game's other VRAM is the larger part of that (see the 45% comment), so the count then stays above the limit for good, which means paging or device loss. On > 8 GB cards the floor is `allowed/4` (about 3.8 GB on a 16 GB card), so this also changes big cards. Please leave the 97% rule without a floor; a floor belongs, if anywhere, only on the soft lines (90% and the hold).
@ -4609,3 +4642,3 @@
// time), VRAM at the driver's limit; the GPU's own use is checked below (last_value). RX_VK_DEAD_MIN_UNUSED=<frames>, 3)
static const uint64_t dead_min_unused =
getenv("RX_VK_DEAD_MIN_UNUSED") ? (uint64_t)std::max(1, atoi(getenv("RX_VK_DEAD_MIN_UNUSED"))) : 3;
getenv("RX_VK_DEAD_MIN_UNUSED") ? (uint64_t)std::max(1, atoi(getenv("RX_VK_DEAD_MIN_UNUSED"))) : 2;
Owner

From 3 to 2 for every card, with no measurement and not in the description. Please revert it, or measure it in its own change.

From 3 to 2 for every card, with no measurement and not in the description. Please revert it, or measure it in its own change.
@ -4619,3 +4652,3 @@
// may have put other things there since, and a write-back over scene data once crashed it)
static const uint64_t keep_finalized_max =
(uint64_t)(getenv("RX_VK_KEEP_FINALIZED_MB") ? std::max(0, atoi(getenv("RX_VK_KEEP_FINALIZED_MB"))) : 512) << 20;
(uint64_t)(getenv("RX_VK_KEEP_FINALIZED_MB") ? std::max(0, atoi(getenv("RX_VK_KEEP_FINALIZED_MB"))) : 128) << 20;
Owner

keep_finalized is a correctness keep, not a cache: the comment below explains that OoT makes its terrain tiles' normal maps again over the same storage and samples them without drawing them, so a texture evicted beyond this cap is sampled wrong. Cutting it from 512 to 128 MB for every card risks bringing back wrong or flickering terrain in large areas. Please revert, or show that 128 MB covers Hyrule Field's peak (and log how often the cap is reached).

`keep_finalized` is a correctness keep, not a cache: the comment below explains that OoT makes its terrain tiles' normal maps again over the same storage and samples them without drawing them, so a texture evicted beyond this cap is sampled wrong. Cutting it from 512 to 128 MB for every card risks bringing back wrong or flickering terrain in large areas. Please revert, or show that 128 MB covers Hyrule Field's peak (and log how often the cap is reached).
@ -35,2 +43,3 @@
ci.pVulkanFunctions = &f;
ci.preferredLargeHeapBlockSize = 64ull << 20; // (64 MB blocks: Eglantine found 256 MB ones wasted VRAM)
// (64 MB blocks on > 8 GB cards; 32 MB on <= 8 GB cards to halve fragmentation in default heap 0 blocks)
ci.preferredLargeHeapBlockSize = device_local_heap_bytes() <= vram_compact::GLOBAL_MIN_GRAPHICS_REQ ? (32ull << 20) : (64ull << 20);
Owner

<= GLOBAL_MIN_GRAPHICS_REQ includes 8 GB cards, which report an 8,192 MB heap (or just under). On those cards, VMA also gives anything over half a block (16 MB) its own dedicated allocation. The 16 MB target-pool blocks (line 84) then push render targets of 16 MB or more (2560x1440 RGBA16F is about 29 MB) out of the target pool, back to one vkAllocateMemory each. That is the churn the target pool exists to stop. Please measure it, or leave it out.

`<= GLOBAL_MIN_GRAPHICS_REQ` includes 8 GB cards, which report an 8,192 MB heap (or just under). On those cards, VMA also gives anything over half a block (16 MB) its own dedicated allocation. The 16 MB target-pool blocks (line 84) then push render targets of 16 MB or more (2560x1440 RGBA16F is about 29 MB) out of the target pool, back to one `vkAllocateMemory` each. That is the churn the target pool exists to stop. Please measure it, or leave it out.
@ -46,3 +55,3 @@
// (RX_VK_IMAGE_BLOCK_MB=<n>, 16 and up: the block's size. VMA keeps one block empty for the next image, 128 MB of VRAM that a 64 MB block
// would halve, at twice the memory objects: the driver allows ~4096, the imported pools take ~2400)
const uint64_t kImageBlock = (getenv("RX_VK_IMAGE_BLOCK_MB") ? (uint64_t)std::max(16, atoi(getenv("RX_VK_IMAGE_BLOCK_MB"))) : 128u) << 20;
const uint64_t kImageBlock = (getenv("RX_VK_IMAGE_BLOCK_MB") ? (uint64_t)std::max(16, atoi(getenv("RX_VK_IMAGE_BLOCK_MB"))) : 32u) << 20;
Owner

This changes the image and texture block size from 128 to 32 MB for every card. On a 16 GB card, blocks_at_start then preallocates 24 x 32 MB = 768 MB instead of 19 x 128 MB = 2,432 MB. That brings back the in-play vkAllocateMemory stalls (the source comment's #2438, about 170 ms on AMD) that the preallocation removed. Every texture over 32 MB now falls outside the pool, so it gets no compaction and is allocated on its own. The memory object count also quadruples, and the comment just above says the driver allows about 4,096 objects and the imported pools take about 2,400 of them. The tests don't catch this because they use their own kBlock. Please keep 128 MB; if small cards want smaller blocks, gate it by the heap size and measure it.

This changes the image and texture block size from 128 to 32 MB for **every** card. On a 16 GB card, `blocks_at_start` then preallocates 24 x 32 MB = 768 MB instead of 19 x 128 MB = 2,432 MB. That brings back the in-play `vkAllocateMemory` stalls (the source comment's `#2438`, about 170 ms on AMD) that the preallocation removed. Every texture over 32 MB now falls outside the pool, so it gets no compaction and is allocated on its own. The memory object count also quadruples, and the comment just above says the driver allows about 4,096 objects and the imported pools take about 2,400 of them. The tests don't catch this because they use their own `kBlock`. Please keep 128 MB; if small cards want smaller blocks, gate it by the heap size and measure it.
@ -40,2 +45,3 @@
inline uint64_t blocks_at_start(uint64_t heap, uint64_t block, bool holds_textures, bool integrated) {
return holds_textures && block && !integrated ? std::clamp<uint64_t>(heap / 100 * 15 / block, 2, 24) : 0;
if (!holds_textures || !block || integrated) return 0;
if (heap < GLOBAL_MIN_GRAPHICS_REQ) return std::clamp<uint64_t>(heap / 100 * 8 / block, 2, 12);
Owner

The comment above this rule says the prealloc exists because each block made during play is a ~170 ms vkAllocateMemory on the replay thread (the source comment's #2438). Cutting it to 8% and a maximum of 12 blocks puts those stalls back on small cards. The test's 8,061 MB iGPU-like case shows that 8 GB cards fall under this branch too. Please measure the stall count in play (log blocks made) before and after on a 6 GB card, or keep the 15% rule.

The comment above this rule says the prealloc exists because each block made during play is a ~170 ms `vkAllocateMemory` on the replay thread (the source comment's `#2438`). Cutting it to 8% and a maximum of 12 blocks puts those stalls back on small cards. The test's 8,061 MB iGPU-like case shows that 8 GB cards fall under this branch too. Please measure the stall count in play (log `blocks made`) before and after on a 6 GB card, or keep the 15% rule.
@ -100,3 +152,3 @@
/// camera turned: with the count held under a 7 GB allowance, a turning camera in Hyrule Field at DLAA let go of 2.3-2.6 GB of live
/// textures every 600 presents and uploaded 6.8 GB in 50 seconds, against 1.4 GB with the compaction off
inline constexpr uint64_t kHoldUnused = 600;
inline constexpr uint64_t kHoldUnused = 180;
Owner

From 600 to 180 presents (and the reserve from 1800 to 600) for every card. The comment just above records why it was raised to 600: a turning camera in Hyrule Field let go of 2.3-2.6 GB of live textures and uploaded 6.8 GB in 50 s. Three seconds is shorter than the camera turn that measurement describes. Please keep 600/1800, or bring a new Hyrule Field upload measurement (the uploads in MB per minute).

From 600 to 180 presents (and the reserve from 1800 to 600) for every card. The comment just above records why it was raised to 600: a turning camera in Hyrule Field let go of 2.3-2.6 GB of live textures and uploaded 6.8 GB in 50 s. Three seconds is shorter than the camera turn that measurement describes. Please keep 600/1800, or bring a new Hyrule Field upload measurement (the uploads in MB per minute).
@ -90,0 +98,4 @@
inline uint64_t texture_min_floor(uint64_t allowed) {
if (!allowed) return 0;
if (allowed < GLOBAL_MIN_GRAPHICS_REQ) return 1800ull << 20;
return std::max<uint64_t>(1800ull << 20, allowed / 4);
Owner

For allowed >= 8 GB this returns allowed/4, and held_budget and update_dynamic_floor apply it to > 8 GB cards: a 16 GB card's hold can no longer go below about 3.8 GB. That contradicts the claim that cards of 8 GB and more are untouched. If the floor is for small cards only, return 0 here when allowed >= GLOBAL_MIN_GRAPHICS_REQ.

For `allowed >= 8 GB` this returns `allowed/4`, and `held_budget` and `update_dynamic_floor` apply it to > 8 GB cards: a 16 GB card's hold can no longer go below about 3.8 GB. That contradicts the claim that cards of 8 GB and more are untouched. If the floor is for small cards only, return 0 here when `allowed >= GLOBAL_MIN_GRAPHICS_REQ`.
@ -95,0 +142,4 @@
target = floor;
}
uint64_t res = std::min(budget, target);
if (floor && res < floor) res = floor;
Owner

held_budget is documented as "never more than the budget was", but this line can return the floor above budget. For example, with budget at 1,000 MB and the floor at 1,800 MB it returns 1,800. Today's callers happen to ignore it (if (held < budget)), but the next caller won't know. Please return std::min(budget, std::max(target, floor)), and update the docs if the contract changes.

`held_budget` is documented as "never more than the budget was", but this line can return the floor above `budget`. For example, with `budget` at 1,000 MB and the floor at 1,800 MB it returns 1,800. Today's callers happen to ignore it (`if (held < budget)`), but the next caller won't know. Please return `std::min(budget, std::max(target, floor))`, and update the docs if the contract changes.
Some checks failed
CI / Conventional Commit title (pull_request) Blocked by required conditions
CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Blocked by required conditions
CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Blocked by required conditions
CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Blocked by required conditions
CI / Workflow lint (actionlint) (pull_request) Blocked by required conditions
Guide screenshots / Guide images and local UI source review (pull_request) Blocked by required conditions
Third-party parts / As their notes say (pull_request) Blocked by required conditions
Third-party parts / The same as upstream (pull_request) Blocked by required conditions
Website / Website on Cloudflare (pull_request) Blocked by required conditions
PR description / Description follows the template (pull_request_target) Failing after 15s
Website / Website on Cloudflare (pull_request_target) Has been skipped
beanstalk/approval approved by HumanBean (reviewed)
beanstalk/linux+vram-le6gb 0/1 pass, 0 running, 1 queued
beanstalk/linux+vram-ge9gb 0/1 pass, 0 running, 1 queued
This pull request has changes conflicting with the target branch.
  • .gitignore
  • gpu/vk/vk_texture.cpp
  • gpu/vk/vram_compact.h
Some workflows are waiting to be reviewed.
View command line instructions

Manual merge helper

Use this merge commit message when completing the merge manually.

Checkout

From your project repository, check out a new branch and test the changes.
git fetch -u 'perf/longshot-optimus-6gb:flix331-perf/longshot-optimus-6gb'
git switch 'flix331-perf/longshot-optimus-6gb'
Sign in to join this conversation.
No reviewers
No milestone
No project
No assignees
3 participants
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
MagicBeanDev/Longshot!637
No description provided.