feat(vk): add opt-in VRAM compaction and hold the driver's count below its limit #1821

2026-11-10T17:37:06Z

SadTransGirl commented

2026-11-10 17:37:06 +00:00

(Migrated from github.com)

At DLAA on a 12 GB card the game alone fills VRAM within a few minutes of walking, and from then on the GPU falls behind and frames wait for it. With RX_VK_COMPACT=1 the textures' unused space goes back to the driver and the texture budget keeps the total clear of the limit. Off by default.

Tracking: #995 (FPS drops in Hyrule Field); related to #993 (VRAM over budget).

What happens now

The driver counts VMA's blocks whole. An evicted texture's memory stays inside its 128 MB block, so the count keeps what the cache gave up. In a five-minute walk across Hyrule Field at DLAA 2560x1440 (RTX 4070 SUPER, 10.2-10.5 GB allowed) the count sits at 9.5-10 GB, 1.4-1.7 GB of it free inside the blocks, and:

the budget's 90% rule judges what is allocated (the count less that free space), so nothing acts while the driver's own count is at its limit;
the write-back shadows are given back at 95% and are gone for two thirds of the run (the GPU's frame is ~10.3 ms with them and 13.9-14.5 ms without, at the same draw counts);
within about 0.8 GB of the limit the GPU's frame goes from ~10 ms to 13-20 ms even with the shadows in place: the driver itself short.

The change (all behind `RX_VK_COMPACT=1`)

The textures that are not render targets come from a pool of their own (the render targets already have theirs, #1436), so every allocation in it is a texture the cache owns.
When 512 MB of that pool is free space (RX_VK_COMPACT_MB; 192 MB while the count is over 90%), VMA's defragmentation moves textures out of sparsely used blocks, at most 12 textures and 64 MB a present: a new image at the new place, the texels copied on the GPU in the present's upload commands, the views made again when next asked for, the old image destroyed once the GPU is past the copy. VMA then frees the blocks that emptied. While a pass is open nothing of the pool is freed.
The texture budget takes only that pool's free space off the count (what the other blocks hold free no texture can use) and holds the count at RX_VK_COMPACT_PCT percent of the allowance (85), letting go of the excess alone rather than cutting the cache to 7/8.
The rules are in gpu/vk/vram_compact.h; the switches are in docs/development.md; the runs are in docs/tested-configurations.md.

Numbers

Built from this branch alone at caaee0b. Unattended runs with scripted input, a five-minute walk across Hyrule Field from a save in the field, DLAA 2560x1440, RTX 4070 SUPER (driver 617.42), Ryzen 9 9950X3D, Windows 11, other programs open (10.2-10.5 GB allowed). Frames after the load, from RX_VK_FRAME_TRACE. The walk's path drifts between runs, so two runs a row. A window is 120 presents.

	fps (mean)	count, MB	windows without the shadows	windows with GPU time over 12.5 ms	frames over 25 / 33 / 50 ms	1% low, fps
as now	53.2, 54.4	9,542, 9,590	73 of 111, 80 of 114	69 of 111, 55 of 114	468/107/38, 476/112/30	19.4, 20.6
`RX_VK_COMPACT=1`	58.2, 58.6	8,253, 7,881	1 of 122, 3 of 123	6 of 122, 10 of 123	265/88/42, 188/62/12	21.1, 26.5
and `RX_VK_ASTC_BC_ALL=1` (one run)	58.9	8,031	1 of 124	2 of 124	132/47/22	26.1

No validation, device-lost or "not made" line in any run.

The branch was moved onto today's main (484cfd1) before its first push. #1506's squeeze rule, merged since, stays as it is beside the hold. The checks were run again there; the game was not measured again there.

Limits

Frames over 50 ms are not clearly fewer (30-38 as now, 12-42 with it): the walk drifts, and single runs differ a lot.
Tested on one Tier 1 configuration only (NVIDIA RTX 40, Windows), in Hyrule Field, in unattended runs without a window, plus one played session in a real window (DLSS then DLAA, a recorder beside the game): no error there, and a two-second hitch that led to the "excess alone" rule.
Not tested: AMD and Intel cards, Linux, cards with less VRAM, area changes, cutscenes, the menus beyond a pause, frame generation, RX_VK_PARALLEL.
A run costs GPU time while it lasts: one to three of each compacted run's windows over 12.5 ms hold one.
The shadows still went one to three times a run, each time the count passed 95% for a moment (a burst of new textures), and were made again. #1637 is about that rule.
Letting go of the excess 64 MB a pass instead of at once was tried (two runs): 244/100/28 and 246/78/26 frames over 25/33/50 ms, the GPU over 12.5 ms in 8 and 14 windows. Not better, so it is not in this PR.
Render targets are not compacted; their blocks held 0.5-0.8 GB of free space in these runs.

Test

python -I -B tests/test_vram_compact.py (the rules' arithmetic, and source checks that the compaction stays opt-in, that the dead list waits while a pass is open, and that create()'s own lines are as they were). Of the 26 game-free checks that read the changed files, 19 pass on my PC; the other 7 fail the same way on plain main there (no g++, no POSIX signals), so CI is the judge of those.

🤖 Generated with Claude Code

At DLAA on a 12 GB card the game alone fills VRAM within a few minutes of walking, and from then on the GPU falls behind and frames wait for it. With `RX_VK_COMPACT=1` the textures' unused space goes back to the driver and the texture budget keeps the total clear of the limit. Off by default. Tracking: #995 (FPS drops in Hyrule Field); related to #993 (VRAM over budget). ### What happens now The driver counts VMA's blocks whole. An evicted texture's memory stays inside its 128 MB block, so the count keeps what the cache gave up. In a five-minute walk across Hyrule Field at DLAA 2560x1440 (RTX 4070 SUPER, 10.2-10.5 GB allowed) the count sits at 9.5-10 GB, 1.4-1.7 GB of it free inside the blocks, and: - the budget's 90% rule judges what is allocated (the count less that free space), so nothing acts while the driver's own count is at its limit; - the write-back shadows are given back at 95% and are gone for two thirds of the run (the GPU's frame is ~10.3 ms with them and 13.9-14.5 ms without, at the same draw counts); - within about 0.8 GB of the limit the GPU's frame goes from ~10 ms to 13-20 ms even with the shadows in place: the driver itself short. ### The change (all behind `RX_VK_COMPACT=1`) - The textures that are not render targets come from a pool of their own (the render targets already have theirs, #1436), so every allocation in it is a texture the cache owns. - When 512 MB of that pool is free space (`RX_VK_COMPACT_MB`; 192 MB while the count is over 90%), VMA's defragmentation moves textures out of sparsely used blocks, at most 12 textures and 64 MB a present: a new image at the new place, the texels copied on the GPU in the present's upload commands, the views made again when next asked for, the old image destroyed once the GPU is past the copy. VMA then frees the blocks that emptied. While a pass is open nothing of the pool is freed. - The texture budget takes only that pool's free space off the count (what the other blocks hold free no texture can use) and holds the count at `RX_VK_COMPACT_PCT` percent of the allowance (85), letting go of the excess alone rather than cutting the cache to 7/8. - The rules are in `gpu/vk/vram_compact.h`; the switches are in `docs/development.md`; the runs are in `docs/tested-configurations.md`. ### Numbers Built from this branch alone at caaee0b. Unattended runs with scripted input, a five-minute walk across Hyrule Field from a save in the field, DLAA 2560x1440, RTX 4070 SUPER (driver 617.42), Ryzen 9 9950X3D, Windows 11, other programs open (10.2-10.5 GB allowed). Frames after the load, from `RX_VK_FRAME_TRACE`. The walk's path drifts between runs, so two runs a row. A window is 120 presents. | | fps (mean) | count, MB | windows without the shadows | windows with GPU time over 12.5 ms | frames over 25 / 33 / 50 ms | 1% low, fps | |---|---|---|---|---|---|---| | as now | 53.2, 54.4 | 9,542, 9,590 | 73 of 111, 80 of 114 | 69 of 111, 55 of 114 | 468/107/38, 476/112/30 | 19.4, 20.6 | | `RX_VK_COMPACT=1` | 58.2, 58.6 | 8,253, 7,881 | 1 of 122, 3 of 123 | 6 of 122, 10 of 123 | 265/88/42, 188/62/12 | 21.1, 26.5 | | and `RX_VK_ASTC_BC_ALL=1` (one run) | 58.9 | 8,031 | 1 of 124 | 2 of 124 | 132/47/22 | 26.1 | No validation, device-lost or "not made" line in any run. The branch was moved onto today's main (484cfd1) before its first push. #1506's squeeze rule, merged since, stays as it is beside the hold. The checks were run again there; the game was not measured again there. ### Limits - Frames over 50 ms are not clearly fewer (30-38 as now, 12-42 with it): the walk drifts, and single runs differ a lot. - Tested on one Tier 1 configuration only (NVIDIA RTX 40, Windows), in Hyrule Field, in unattended runs without a window, plus one played session in a real window (DLSS then DLAA, a recorder beside the game): no error there, and a two-second hitch that led to the "excess alone" rule. - Not tested: AMD and Intel cards, Linux, cards with less VRAM, area changes, cutscenes, the menus beyond a pause, frame generation, `RX_VK_PARALLEL`. - A run costs GPU time while it lasts: one to three of each compacted run's windows over 12.5 ms hold one. - The shadows still went one to three times a run, each time the count passed 95% for a moment (a burst of new textures), and were made again. #1637 is about that rule. - Letting go of the excess 64 MB a pass instead of at once was tried (two runs): 244/100/28 and 246/78/26 frames over 25/33/50 ms, the GPU over 12.5 ms in 8 and 14 windows. Not better, so it is not in this PR. - Render targets are not compacted; their blocks held 0.5-0.8 GB of free space in these runs. ### Test `python -I -B tests/test_vram_compact.py` (the rules' arithmetic, and source checks that the compaction stays opt-in, that the dead list waits while a pass is open, and that `create()`'s own lines are as they were). Of the 26 game-free checks that read the changed files, 19 pass on my PC; the other 7 fail the same way on plain main there (no g++, no POSIX signals), so CI is the judge of those. 🤖 Generated with [Claude Code](https://claude.com/claude-code)

hckrman101 commented

2026-11-10 17:51:00 +00:00

(Migrated from github.com)

Heads-up: this conflicts with main now (a lot landed today). Could you merge main into the branch (or rebase) and resolve the conflicts? Thanks!

hckrman101 commented

2026-11-10 19:05:48 +00:00

(Migrated from github.com)

I tried this branch (74d0936, built as-is) on Linux with an RTX 3090 (24 GB), DLAA at 2560x1440, in Kokiri Forest. RX_VK_TEST_ALLOWED_MB=6000 simulates a smaller card. Results so far:

Correctness: works on Linux. With RX_VK_COMPACT=1 and a 6 GB allowance, compaction ran (e.g. "159 textures moved (160 MB) in 15 passes, 0 left where they were; 2 blocks (256 MB) given back"). There were no validation, device-lost or "not made" lines, and screenshots taken after the passes show no wrong or garbled textures.

A/B, same build, only RX_VK_COMPACT changed. Last 3000 frames of each run (frame CSV):

run	fps	1% low	frames >33 ms	>50 ms	worst	VRAM used / allowed at the end
6 GB allowance, off	38.5, 38.6	25.1, 25.1	59, 61	2, 1	97, 159 ms	5630 / 6000 MB (656 MB free in blocks)
6 GB allowance, on	35.8, 36.5	5.6, 23.5	270, 286	9, 1	1684, 151 ms	5342 / 6000 MB (289 MB free in blocks)
24 GB (real), off	38.2, 47.2	3.4, 16.9	197, 17	172, 9	1287, 220 ms	~6.6 / 21.7 GB
24 GB (real), on	48.2, 39.7	11.0, 3.6	32, 171	20, 161	500, 933 ms	~6.8 / 21.7 GB

At 24 GB, compaction never starts (the game uses ~6.7 GB here). The run-to-run swings in those rows are noise: they show up the same with it off, likely from shader compiles.
At 6 GB it saves ~290 MB (94% → 89% of the allowance), but it costs ~2.5 fps and gives 4-5x as many frames over 33 ms. One run also had a single 1.7 s frame.
The extra slow frames don't come from the copy passes. Compaction ran only once per run, early on, and the slow frames are spread over the whole run. In those frames the CPU-side timings (memsync, draw, uploads, texture lookup, slot wait) are no higher than with it off, and the eviction counts are about the same. So it looks like a steady-state cost that lasts as long as the switch is on. The candidates I'm looking at:
- the separate texture pool's allocation/memory settings
- the 85% hold keeping the texture budget at ~2.4-2.5 GB while the textures need ~2.7 GB (eviction/re-creation churn)
- views or descriptor sets being rebuilt after moves

Caveats: two runs per case, one scene (not Hyrule Field), and a simulated allowance on a big card, not a real small one. A 4 GB-allowance pair is running now. I'll follow up with those numbers and, if I can pin down the cause, a suggested fix. As it stands, this looks fine as opt-in but I wouldn't make it the default yet.

I tried this branch (74d0936, built as-is) on Linux with an RTX 3090 (24 GB), DLAA at 2560x1440, in Kokiri Forest. `RX_VK_TEST_ALLOWED_MB=6000` simulates a smaller card. Results so far: **Correctness:** works on Linux. With `RX_VK_COMPACT=1` and a 6 GB allowance, compaction ran (e.g. "159 textures moved (160 MB) in 15 passes, 0 left where they were; 2 blocks (256 MB) given back"). There were no validation, device-lost or "not made" lines, and screenshots taken after the passes show no wrong or garbled textures. **A/B, same build, only `RX_VK_COMPACT` changed.** Last 3000 frames of each run (frame CSV): | run | fps | 1% low | frames >33 ms | >50 ms | worst | VRAM used / allowed at the end | |---|---|---|---|---|---|---| | 6 GB allowance, off | 38.5, 38.6 | 25.1, 25.1 | 59, 61 | 2, 1 | 97, 159 ms | 5630 / 6000 MB (656 MB free in blocks) | | 6 GB allowance, on | 35.8, 36.5 | 5.6, 23.5 | 270, 286 | 9, 1 | 1684, 151 ms | 5342 / 6000 MB (289 MB free in blocks) | | 24 GB (real), off | 38.2, 47.2 | 3.4, 16.9 | 197, 17 | 172, 9 | 1287, 220 ms | ~6.6 / 21.7 GB | | 24 GB (real), on | 48.2, 39.7 | 11.0, 3.6 | 32, 171 | 20, 161 | 500, 933 ms | ~6.8 / 21.7 GB | - At 24 GB, compaction never starts (the game uses ~6.7 GB here). The run-to-run swings in those rows are noise: they show up the same with it off, likely from shader compiles. - At 6 GB it saves ~290 MB (94% → 89% of the allowance), but it costs ~2.5 fps and gives 4-5x as many frames over 33 ms. One run also had a single 1.7 s frame. - The extra slow frames don't come from the copy passes. Compaction ran only once per run, early on, and the slow frames are spread over the whole run. In those frames the CPU-side timings (memsync, draw, uploads, texture lookup, slot wait) are no higher than with it off, and the eviction counts are about the same. So it looks like a steady-state cost that lasts as long as the switch is on. The candidates I'm looking at: - the separate texture pool's allocation/memory settings - the 85% hold keeping the texture budget at ~2.4-2.5 GB while the textures need ~2.7 GB (eviction/re-creation churn) - views or descriptor sets being rebuilt after moves Caveats: two runs per case, one scene (not Hyrule Field), and a simulated allowance on a big card, not a real small one. A 4 GB-allowance pair is running now. I'll follow up with those numbers and, if I can pin down the cause, a suggested fix. As it stands, this looks fine as opt-in but I wouldn't make it the default yet.

hckrman101 commented

2026-11-10 20:25:56 +00:00

(Migrated from github.com)

A follow-up and a correction to my earlier comment. With more runs, the stutter cost I reported doesn't hold up: it was run-to-run noise.

Same setup as before: Linux, RTX 3090, DLAA 2560x1440, Kokiri Forest, RX_VK_TEST_ALLOWED_MB to simulate a smaller card, last 3000 frames of each run.

6 GB allowance, 4 runs each way (two batches):

	fps	1% low	frames >33 ms	worst frame
off	38.5, 38.6, 32.7, 35.2	25.1, 25.1, 2.3, 7.5	59, 61, 180, 242	97, 159, 3260, 2113 ms
`RX_VK_COMPACT=1`	35.8, 36.5, 38.7, 33.2	5.6, 23.5, 26.3, 2.9	270, 286, 61, 240	1684, 151, 85, 5598 ms

The same setting swings ~33-39 fps and ~60-280 slow frames between runs, more than any off/on difference. The multi-second freezes happen with compaction off too, so they aren't from this PR. A variant with the copy passes disabled was no better. I can't see an fps or stutter cost from compaction here. The VRAM saving is real: ~100-300 MB less used at the end of each run (e.g. 5630 → 5342 MB of 6000).

4 GB allowance (soft, 2 runs off / 2 on): off 31.4 / 32.3 fps, on 30.6 / 29.7 fps, ~300 MB less VRAM with it on. Same picture: within the noise. Note the soft allowance doesn't stop the game going over it (4.4-4.75 GB used), so this isn't real pressure. A run with ~4 GB of real free VRAM (another process holding the rest) is queued, and I'll add it when it's done.

Correctness: no validation, device-lost or "not made" lines in any run, and no wrong textures in the screenshots after passes.

On whether to default it on: I'd keep it opt-in for now. It costs nothing on big cards because it never triggers, but it moves live textures, and it has only been tried on NVIDIA (your 4070 SUPER on Windows and this 3090 on Linux). The cards that would benefit most are often AMD cards and integrated GPUs. A few runs there and on a real 4-8 GB card would be the thing to see before turning it on automatically, e.g. only below ~12 GB of VRAM.

A follow-up and a correction to my earlier comment. With more runs, the stutter cost I reported doesn't hold up: it was run-to-run noise. Same setup as before: Linux, RTX 3090, DLAA 2560x1440, Kokiri Forest, `RX_VK_TEST_ALLOWED_MB` to simulate a smaller card, last 3000 frames of each run. **6 GB allowance, 4 runs each way (two batches):** | | fps | 1% low | frames >33 ms | worst frame | |---|---|---|---|---| | off | 38.5, 38.6, 32.7, 35.2 | 25.1, 25.1, 2.3, 7.5 | 59, 61, 180, 242 | 97, 159, 3260, 2113 ms | | `RX_VK_COMPACT=1` | 35.8, 36.5, 38.7, 33.2 | 5.6, 23.5, 26.3, 2.9 | 270, 286, 61, 240 | 1684, 151, 85, 5598 ms | The same setting swings ~33-39 fps and ~60-280 slow frames between runs, more than any off/on difference. The multi-second freezes happen with compaction off too, so they aren't from this PR. A variant with the copy passes disabled was no better. I can't see an fps or stutter cost from compaction here. The VRAM saving is real: ~100-300 MB less used at the end of each run (e.g. 5630 → 5342 MB of 6000). **4 GB allowance (soft, 2 runs off / 2 on):** off 31.4 / 32.3 fps, on 30.6 / 29.7 fps, ~300 MB less VRAM with it on. Same picture: within the noise. Note the soft allowance doesn't stop the game going over it (4.4-4.75 GB used), so this isn't real pressure. A run with ~4 GB of real free VRAM (another process holding the rest) is queued, and I'll add it when it's done. **Correctness:** no validation, device-lost or "not made" lines in any run, and no wrong textures in the screenshots after passes. On whether to default it on: I'd keep it opt-in for now. It costs nothing on big cards because it never triggers, but it moves live textures, and it has only been tried on NVIDIA (your 4070 SUPER on Windows and this 3090 on Linux). The cards that would benefit most are often AMD cards and integrated GPUs. A few runs there and on a real 4-8 GB card would be the thing to see before turning it on automatically, e.g. only below ~12 GB of VRAM.

hckrman101 commented

2026-11-10 20:32:50 +00:00

(Migrated from github.com)

The last runs, as promised: a real 4 GB limit. Another process held 17.6 GB of the 3090's VRAM, leaving ~4.1 GB free, so the driver's own allowance was 4.0-4.2 GB (no RX_VK_TEST_ALLOWED_MB). DLAA 2560x1440, Kokiri Forest, two pairs:

	fps	frames >33 ms
off	15.1, 27.1	1363, 1983
`RX_VK_COMPACT=1`	25.1, 22.2	2020, 1982

In both modes the game ran over the limit. Large images and mirror pools ended up in system memory, GPU time was 28-53 ms, and there were 1-1.3 s hitches, but no device lost and no crash. Compaction never started in these runs, so on vs off here is just noise. At this resolution a hard 4 GB limit is past what compaction can help with. A real 4 GB card would also differ: less bandwidth, and Windows pages VRAM out rather than falling back the way Linux NVIDIA does.

One more note on my earlier runs. Part of the first on/off gap at 6 GB came from the two modes ending up in different scenes: Saria's dialogue (~1800 draws a frame) vs after it (~2400). With every run reaching the same scene, compaction on was no worse than off, and GPU time was the same (~15-16 ms). The occasional 0.3-2.5 s freezes were CPU contention on my PC (other jobs running), with compaction on and off.

Summary for my setup: correct on Linux/NVIDIA. It does nothing at 24 GB because it never triggers. At smaller allowances it saves 100-250 MB with no fps change in Kokiri Forest. Your Hyrule Field gains are the case I couldn't reproduce here, so runs from someone on a real 8-12 GB card in the field, or on AMD/Intel, would be the useful next data.

The last runs, as promised: a real 4 GB limit. Another process held 17.6 GB of the 3090's VRAM, leaving ~4.1 GB free, so the driver's own allowance was 4.0-4.2 GB (no `RX_VK_TEST_ALLOWED_MB`). DLAA 2560x1440, Kokiri Forest, two pairs: | | fps | frames >33 ms | |---|---|---| | off | 15.1, 27.1 | 1363, 1983 | | `RX_VK_COMPACT=1` | 25.1, 22.2 | 2020, 1982 | In both modes the game ran over the limit. Large images and mirror pools ended up in system memory, GPU time was 28-53 ms, and there were 1-1.3 s hitches, but no device lost and no crash. Compaction never started in these runs, so on vs off here is just noise. At this resolution a hard 4 GB limit is past what compaction can help with. A real 4 GB card would also differ: less bandwidth, and Windows pages VRAM out rather than falling back the way Linux NVIDIA does. One more note on my earlier runs. Part of the first on/off gap at 6 GB came from the two modes ending up in different scenes: Saria's dialogue (~1800 draws a frame) vs after it (~2400). With every run reaching the same scene, compaction on was no worse than off, and GPU time was the same (~15-16 ms). The occasional 0.3-2.5 s freezes were CPU contention on my PC (other jobs running), with compaction on and off. Summary for my setup: correct on Linux/NVIDIA. It does nothing at 24 GB because it never triggers. At smaller allowances it saves 100-250 MB with no fps change in Kokiri Forest. Your Hyrule Field gains are the case I couldn't reproduce here, so runs from someone on a real 8-12 GB card in the field, or on AMD/Intel, would be the useful next data.

hckrman101 (Migrated from github.com) approved these changes

2026-11-10 20:35:58 +00:00

hckrman101 (Migrated from github.com) left a comment

Code review. My two earlier comments cover the test runs (Linux, RTX 3090). I'd keep this opt-in until it has run on AMD, an iGPU and a real 4-8 GB card.

I read the diff against b2f1f5f and checked these:

Off means off. With RX_VK_COMPACT unset, compact_step returns at once. The dead-list break never fires. reclaim stays block_free, so pressure, the 90% rule and the 97% squeeze are the same as before. evict_to stays 0, so the eviction loop still stops at budget / 8 * 7. create_image gets textures=false, so the pool choice and the allocation flags don't change. texture_pool is null, so the block log skips it. The only always-on lines are x.create_flags, present_source and compact.pressed, and they're just bookkeeping.
The move. The new image is made from the same create info create() used: flags (cube, 2D_ARRAY_COMPATIBLE, MUTABLE / BLOCK_TEXEL_VIEW, EXTENDED_USAGE), type, format, extent after skip, levels, layers, samples, usage, OPTIMAL tiling, exclusive sharing. create() adds no pNext, and the BC/ASTC usage fallbacks never land in this pool, so x.usage is the image's real usage. It's bound with vmaBindImageMemory(dstTmpAllocation). Every image here stays in GENERAL, so UNDEFINED->GENERAL on the new image and GENERAL->GENERAL copies of every level are right. The full barriers before and after are both kept (vk_work.h counts the copies, so the second barrier isn't deduped). Everything runs on one queue, so there's no ownership transfer. The copy goes into the present's r.upload, which is submitted before r.cmd. The only image r.cmd touches is the presented one, and present_source excludes it, including the direct_view and fg_after views. VMA marks the block of an ignored move immovable, so a skipped texture can't make passes loop.
Lifetime. Old views and images are destroyed only once completed >= compact.value, before vmaEndDefragmentationPass, which matches VMA's rule. destroy_gen++ makes the draw's handle cache, the descriptor slots and the temporal/FSR depth views re-resolve. A moved texture gets last_value = rec_value, so eviction and write-back skip it until the pass has ended (compact_step runs before the eviction loop in the same texture_frame). Holding the dead list while a pass is open covers destroy_later of a moved texture. The other frees in this file (target pool, BC scratch) are in other pools. A move whose allocation isn't in textures (one on the dead list) is ignored, which fails safe.
Threads. Everything runs in do_present under replay_mu. A parallel replay's segment workers have finished by then, so nothing else touches textures or the defrag context.
Budget. Units are bytes throughout and nothing can wrap (used - min(used, reclaim), total > over ? ... : 0). Without VK_EXT_memory_budget, known is false, so the hold is off and only the free-space trigger runs. On an iGPU the existing 2.5 GB texture cap applies first.
Merge and CI. Merging main resolved the conflict. The diff no longer touches tests/run_all.py, and of the 20 commits main has gained since, only docs/development.md overlaps (GitHub says mergeable). The CI failures (test_antilag, test_nup_out_of_space, test_wayland_libdecor_hint, test_draw_parameters) are in tests main has changed since; test_vram_compact.py passes on both OSes.

Nits, none blocking:

#1506's room_in_blocks still counts free space in every pool. With compaction on, the PR's own argument says only the textures' pool counts. The 85% hold mostly acts first, so it's moot in practice.
The "VRAM compacted ..." end-of-run log block appears twice in compact_step; a small helper would remove the copy.
The test checks the arithmetic for real, but checks the move/copy path only by matching source text. That's fine without a GPU in CI.

Code review. My two earlier comments cover the test runs (Linux, RTX 3090). I'd keep this opt-in until it has run on AMD, an iGPU and a real 4-8 GB card. I read the diff against b2f1f5f and checked these: - **Off means off.** With `RX_VK_COMPACT` unset, `compact_step` returns at once. The dead-list `break` never fires. `reclaim` stays `block_free`, so `pressure`, the 90% rule and the 97% squeeze are the same as before. `evict_to` stays 0, so the eviction loop still stops at `budget / 8 * 7`. `create_image` gets `textures=false`, so the pool choice and the allocation flags don't change. `texture_pool` is null, so the block log skips it. The only always-on lines are `x.create_flags`, `present_source` and `compact.pressed`, and they're just bookkeeping. - **The move.** The new image is made from the same create info `create()` used: flags (cube, 2D_ARRAY_COMPATIBLE, MUTABLE / BLOCK_TEXEL_VIEW, EXTENDED_USAGE), type, format, extent after skip, levels, layers, samples, usage, OPTIMAL tiling, exclusive sharing. `create()` adds no pNext, and the BC/ASTC usage fallbacks never land in this pool, so `x.usage` is the image's real usage. It's bound with `vmaBindImageMemory(dstTmpAllocation)`. Every image here stays in GENERAL, so UNDEFINED->GENERAL on the new image and GENERAL->GENERAL copies of every level are right. The full barriers before and after are both kept (vk_work.h counts the copies, so the second barrier isn't deduped). Everything runs on one queue, so there's no ownership transfer. The copy goes into the present's `r.upload`, which is submitted before `r.cmd`. The only image `r.cmd` touches is the presented one, and `present_source` excludes it, including the `direct_view` and fg_after views. VMA marks the block of an ignored move immovable, so a skipped texture can't make passes loop. - **Lifetime.** Old views and images are destroyed only once `completed >= compact.value`, before `vmaEndDefragmentationPass`, which matches VMA's rule. `destroy_gen++` makes the draw's handle cache, the descriptor slots and the temporal/FSR depth views re-resolve. A moved texture gets `last_value = rec_value`, so eviction and write-back skip it until the pass has ended (`compact_step` runs before the eviction loop in the same `texture_frame`). Holding the dead list while a pass is open covers `destroy_later` of a moved texture. The other frees in this file (target pool, BC scratch) are in other pools. A move whose allocation isn't in `textures` (one on the dead list) is ignored, which fails safe. - **Threads.** Everything runs in `do_present` under `replay_mu`. A parallel replay's segment workers have finished by then, so nothing else touches `textures` or the defrag context. - **Budget.** Units are bytes throughout and nothing can wrap (`used - min(used, reclaim)`, `total > over ? ... : 0`). Without `VK_EXT_memory_budget`, `known` is false, so the hold is off and only the free-space trigger runs. On an iGPU the existing 2.5 GB texture cap applies first. - **Merge and CI.** Merging main resolved the conflict. The diff no longer touches `tests/run_all.py`, and of the 20 commits main has gained since, only `docs/development.md` overlaps (GitHub says mergeable). The CI failures (test_antilag, test_nup_out_of_space, test_wayland_libdecor_hint, test_draw_parameters) are in tests main has changed since; `test_vram_compact.py` passes on both OSes. Nits, none blocking: - #1506's `room_in_blocks` still counts free space in every pool. With compaction on, the PR's own argument says only the textures' pool counts. The 85% hold mostly acts first, so it's moot in practice. - The "VRAM compacted ..." end-of-run log block appears twice in `compact_step`; a small helper would remove the copy. - The test checks the arithmetic for real, but checks the move/copy path only by matching source text. That's fine without a GPU in CI.

tests/test_vram_compact.py

					
				@ -0,0 +9,4 @@

				import tempfile

				from pathlib import Path

				RUN_ALL_ARGS = ["--cxx", "g++"]

Nit: "{cxx}" here (and no hard default below), as the #1822 convention does: run_all fills it from CXX / clang++ / g++ / c++. With "g++" hard-coded, a machine that only has clang++ (llvm-mingw on Windows, or your PC as you noted) skips the arithmetic check instead of running it.

Nit: `"{cxx}"` here (and no hard default below), as the #1822 convention does: run_all fills it from `CXX` / clang++ / g++ / c++. With `"g++"` hard-coded, a machine that only has clang++ (llvm-mingw on Windows, or your PC as you noted) skips the arithmetic check instead of running it.

SnusBlaze commented

2026-11-11 20:58:04 +00:00

(Migrated from github.com)

Tested on a real 6 GB card.

Setup: RTX 4050 Laptop (5152 MB allowed), Core 5 210H, 16 GB RAM (single-channel), Windows 11, Hookshot v2.1.0, 1080p, DLSS, Low (6GB) preset, frame generation off, RX_VK_WB_MAX_KB=1024. Same area each run.

Results:

Compaction off, RX_VK_MIRROR_MB=1300: 35-42 fps. VRAM 5.2-5.7 of 5.0 GB, textures 500+ MB over budget ("not squeezed"), 8 pools in system memory. Most slow frames were memory sync.
RX_VK_COMPACT=1 only: 19-23 fps. VRAM stayed under the limit, but the mirrors dropped to 512 MB, so GPU time went from 15-18 ms to 36-45 ms.
RX_VK_COMPACT=1 + RX_VK_COMPACT_PCT=90 + RX_VK_MIRROR_MB=1300: 33-46 fps. Best result. Memory-sync slow frames went from 45 to 7, and camera turning is much smoother. VRAM still slightly over (5.1-5.5 GB).
Same as 3 + RX_VK_ASYNC=1: 26-29 fps. Steadier frame times, but a translucent white flicker appears over ground textures.

Summary: On a 6 GB card, compaction frees space by shrinking the mirrors instead of the textures, which makes it slower. Keeping the mirror budget at 1300 MB fixes that. Logs for 3 and 4 are attached.

hookshot_log_async_flicker.txt
hookshot_log_compact_mirror1300.txt

Tested on a real 6 GB card. **Setup:** RTX 4050 Laptop (5152 MB allowed), Core 5 210H, 16 GB RAM (single-channel), Windows 11, Hookshot v2.1.0, 1080p, DLSS, Low (6GB) preset, frame generation off, `RX_VK_WB_MAX_KB=1024`. Same area each run. **Results:** 1. **Compaction off, `RX_VK_MIRROR_MB=1300`:** 35-42 fps. VRAM 5.2-5.7 of 5.0 GB, textures 500+ MB over budget ("not squeezed"), 8 pools in system memory. Most slow frames were memory sync. 2. **`RX_VK_COMPACT=1` only:** 19-23 fps. VRAM stayed under the limit, but the mirrors dropped to 512 MB, so GPU time went from 15-18 ms to 36-45 ms. 3. **`RX_VK_COMPACT=1` + `RX_VK_COMPACT_PCT=90` + `RX_VK_MIRROR_MB=1300`:** 33-46 fps. Best result. Memory-sync slow frames went from 45 to 7, and camera turning is much smoother. VRAM still slightly over (5.1-5.5 GB). 4. **Same as 3 + `RX_VK_ASYNC=1`:** 26-29 fps. Steadier frame times, but a translucent white flicker appears over ground textures. **Summary:** On a 6 GB card, compaction frees space by shrinking the mirrors instead of the textures, which makes it slower. Keeping the mirror budget at 1300 MB fixes that. Logs for 3 and 4 are attached. [hookshot_log_async_flicker.txt](https://github.com/user-attachments/files/34706776/hookshot_log_async_flicker.txt) [hookshot_log_compact_mirror1300.txt](https://github.com/user-attachments/files/34706777/hookshot_log_compact_mirror1300.txt)

Commenting is not possible because the repository is archived.

No reviewers

hckrman101