fix(vk): let go of render targets drawn over from another start #899

Merged

ZurielAtr merged 2 commits from fix/dead-transient-targets into main

2026-11-09 08:47:09 +00:00

ZurielAtr commented

2026-11-09 00:41:59 +00:00

(Migrated from github.com)

What changed

OoT makes a frame's transient render targets (768 x 432 RGBA16F, 1536 x 864, 640 x 360) in a scratch heap whose offsets
shift with what came before them in the frame. When the camera turns, what is drawn before them changes, so almost every
frame they land ~1.2 MB away from the previous ones, overlapping them, and the renderer makes a new image for each.

drawn_over() only treats a render target as dead when another one is drawn at the same start address. These start
elsewhere, so they stayed until the texture budget or the 3600-present rule took them.

Now a render target of one level and one layer is also dead when another render target that starts elsewhere, and is not
inside it, has drawn over an eighth of it or more after its last use. The rule is in gpu/vk/target_overlap.h. Such
targets go after 3 frames unused (RX_VK_DEAD_MIN_UNUSED; 16 in the first version of this PR). Under the budget, the sweep for drawn-over targets runs every 60 presents instead of
every 600. RX_VK_DEAD_TARGETS=0 restores the old behaviour.

Why

On an 8 GB card the dead targets filled the VRAM. The driver then paged it out to system memory, and the GPU time per frame
doubled while it stayed that way.

How it was tested

RX 7700S Laptop (8 GB), Ryzen 7 7735HS, Adrenalin 26.10.1, Windows 11, v2.0.5, 1600x900, preset Custom, no frame
generation. Same route each time: out of Lon Lon Ranch into Hyrule Field, turning the camera a lot.

	Before	After
Rendered textures held	1.2-1.9 GB (up to ~300 dead 768 x 432 / 1536 x 864 targets)	0.3-0.5 GB
GPU time per frame (log)	32-37 ms once paging started	15-17 ms, steady for 10+ minutes
fps in Hyrule Field	12-18, hitches of 0.3-0.6 s every few seconds (24 in 62 s of video)	35-43
Dead targets let go	-	~14,500 in one session (~41 GB in total)

Turning the switch off (RX_VK_DEAD_TARGETS=0) in the same build brought the 1.7-2 GB of rendered textures and the low fps
back.

Game-free check: tests/test_target_overlap.py compiles tests/test_target_overlap.cpp against the rule: shifted
overlaps, views inside the target, the same start, overlaps under an eighth, and order of use. It also checks that the
eviction pass still uses the rule. It is listed in tests/run_all.py with --cxx g++.
python -I -B tests/run_all.py: everything passes here except nx2/test_nup_paths.py and
tests/test_fps_overlay_order.py, which fail the same way on a clean main on this machine.

Update: 3 frames instead of 16

With 16 frames, Hyrule Castle Town's market still held ~500 MB of dead targets at ~20 fps (59 768 x 432 RGBA16F,
135 384 x 384 and 41 1536 x 864 at a time), with the VRAM at the driver's limit. Their storage has been drawn over by
another target, so nothing the game samples is lost, and the pass still skips anything the GPU has not finished
(last_value). Played on the same RX 7700S for a long session (field, market, ranch) with 3: no missing or stale
textures. RX_VK_DEAD_MIN_UNUSED=<frames> sets it; the test checks the switch and the GPU check are still there.

🤖 Generated with Claude Code

## What changed OoT makes a frame's transient render targets (768 x 432 RGBA16F, 1536 x 864, 640 x 360) in a scratch heap whose offsets shift with what came before them in the frame. When the camera turns, what is drawn before them changes, so almost every frame they land ~1.2 MB away from the previous ones, overlapping them, and the renderer makes a new image for each. `drawn_over()` only treats a render target as dead when another one is drawn **at the same start address**. These start elsewhere, so they stayed until the texture budget or the 3600-present rule took them. Now a render target of one level and one layer is also dead when another render target that starts elsewhere, and is not inside it, has drawn over an eighth of it or more after its last use. The rule is in `gpu/vk/target_overlap.h`. Such targets go after 3 frames unused (`RX_VK_DEAD_MIN_UNUSED`; 16 in the first version of this PR). Under the budget, the sweep for drawn-over targets runs every 60 presents instead of every 600. `RX_VK_DEAD_TARGETS=0` restores the old behaviour. ## Why On an 8 GB card the dead targets filled the VRAM. The driver then paged it out to system memory, and the GPU time per frame doubled while it stayed that way. ## How it was tested RX 7700S Laptop (8 GB), Ryzen 7 7735HS, Adrenalin 26.10.1, Windows 11, v2.0.5, 1600x900, preset Custom, no frame generation. Same route each time: out of Lon Lon Ranch into Hyrule Field, turning the camera a lot. | | Before | After | |---|---|---| | Rendered textures held | 1.2-1.9 GB (up to ~300 dead 768 x 432 / 1536 x 864 targets) | 0.3-0.5 GB | | GPU time per frame (log) | 32-37 ms once paging started | 15-17 ms, steady for 10+ minutes | | fps in Hyrule Field | 12-18, hitches of 0.3-0.6 s every few seconds (24 in 62 s of video) | 35-43 | | Dead targets let go | - | ~14,500 in one session (~41 GB in total) | Turning the switch off (`RX_VK_DEAD_TARGETS=0`) in the same build brought the 1.7-2 GB of rendered textures and the low fps back. Game-free check: `tests/test_target_overlap.py` compiles `tests/test_target_overlap.cpp` against the rule: shifted overlaps, views inside the target, the same start, overlaps under an eighth, and order of use. It also checks that the eviction pass still uses the rule. It is listed in `tests/run_all.py` with `--cxx g++`. `python -I -B tests/run_all.py`: everything passes here except `nx2/test_nup_paths.py` and `tests/test_fps_overlay_order.py`, which fail the same way on a clean `main` on this machine. ### Update: 3 frames instead of 16 With 16 frames, Hyrule Castle Town's market still held ~500 MB of dead targets at ~20 fps (59 768 x 432 RGBA16F, 135 384 x 384 and 41 1536 x 864 at a time), with the VRAM at the driver's limit. Their storage has been drawn over by another target, so nothing the game samples is lost, and the pass still skips anything the GPU has not finished (`last_value`). Played on the same RX 7700S for a long session (field, market, ranch) with 3: no missing or stale textures. `RX_VK_DEAD_MIN_UNUSED=<frames>` sets it; the test checks the switch and the GPU check are still there. 🤖 Generated with [Claude Code](https://claude.com/claude-code)