perf(vk): keep the write-back shadows while VRAM is full of free space inside VMA's blocks #1637

github-sync · 2026-11-10T11:48:17Z

SadTransGirl commented

2026-11-10 11:48:17 +00:00

(Migrated from github.com)

After some minutes of play on a 12 GB card with other programs open, the frame rate drops and stays down for the rest of the session. The renderer gives up a speed-up (the write-back shadows) when VRAM looks full, although much of that is free space it is holding itself.

What happens now

The write-back diff's shadows are given back when the driver's count of the process's VRAM passes 95% of what it allows (shadows_check_room), and made again only under 85% (shadow_room).

On a 12 GB card the texture cache fills VRAM to 93-98% of the allowance within minutes of play. Much of that is free space inside VMA's blocks, which evicting textures does not give back to the driver. So the shadows (272-448 MB) go and never come back, and every write-back is a whole copy again for the rest of the session.

In my own logs (RTX 4070 SUPER, 2560x1440, other programs open, 10.4-10.7 GB allowed) the line the shadows given back is in every slow stretch. In one of them: 58 fps at the present before it, then 26, 28, 23 fps (10,462 of 10,732 MB used, 3.5 GB of it free inside the blocks).

The change

Both rules read what is really allocated: the driver's count less the free space inside the blocks, as the texture budget's 90% rule already does (gpu/vk/vram_pressure.h).

The shadows go when really allocated passes 95%, or the driver's own count passes 99%.
A shadow is made while really allocated stays under 93% with it, and the count under 98%.
RX_VK_SHADOW_RAW=1: by the count alone, as before.

Numbers

Unattended runs (HOS_HEADLESS, HOS_INPUT_SCRIPT with the frame clock): a five-minute walk across Hyrule Field from a save in the field, DLAA 2560x1440, RTX 4070 SUPER, Ryzen 9 9950X3D. Built from 4a7ae1b. The allowance was held to 9.8 GB to stand for the other programs I normally have open (a local test cap on what vram_usage reports; it is not part of this PR). One build, RX_VK_SHADOW_RAW=1 against the default, two runs each, 129 windows a run.

	fps	windows under 55 fps	replay ms a frame	the capture's wait, ms a frame	GPU ms a frame	shadows at the end
by the count (as now)	59.5, 59.3	3, 7	8.20, 8.61	1.94, 2.15	11.24, 11.47	0 MB
with this	59.8, 59.9	1, 2	7.44, 7.41	1.10, 0.77	10.14, 10.06	384 MB

By the count, the shadows went once or twice a run and were not made again. With this change they still went 5 and 7 times a run, each time the driver's own count passed 99% under the cap (VRAM 9944 of 9800 MB), and were made again each time.

On v2.0.11's code, before the reuse pool and the sweep of dead targets, the same comparison was wider: 56.6, 58.8 fps with 32, 8 windows under 55 by the count, against 59.8, 59.6 fps with 1, 2. At 10.4 GB allowed: 58.5, 58.6 fps with 16, 13 slow windows, against 59.3 with 6.

What losing the shadows costs in a scene that repeats exactly (standing in the field with the camera turning, 2,100-3,000 draws, the same draws in both runs; measured on v2.0.11's code), RX_VK_WB_SHADOW_MB=0 against the default:

	replay ms a frame	capture's wait	GPU ms a frame
shadows	8.15	0.34	9.99
no shadows	10.64	2.07	12.49

Limits

The walk's path drifts a little from run to run, so the walk rows are not the same frames; the second table is.
These runs are on an otherwise idle PC with the allowance capped, so the driver is not really short of memory. My real sessions (26-45 fps after the line) had other programs holding VRAM as well.
In all four runs the 97% rule squeezed the texture budget. That is #1506's subject and is not touched here.
With the shadows kept, the driver's count sits where it sat before (93-98%); nothing here lowers it.
Tested on one card (NVIDIA, 12 GB). An 8 GB card reaches these lines sooner; I have none to try.

Test

python -I -B tests/test_vram_pressure.py (the rules' arithmetic with the numbers of the log above, and a source check). run_all mem_trim, texture_handle, texture_init pass.

🤖 Generated with Claude Code

After some minutes of play on a 12 GB card with other programs open, the frame rate drops and stays down for the rest of the session. The renderer gives up a speed-up (the write-back shadows) when VRAM looks full, although much of that is free space it is holding itself. ### What happens now The write-back diff's shadows are given back when the driver's count of the process's VRAM passes 95% of what it allows (`shadows_check_room`), and made again only under 85% (`shadow_room`). On a 12 GB card the texture cache fills VRAM to 93-98% of the allowance within minutes of play. Much of that is free space inside VMA's blocks, which evicting textures does not give back to the driver. So the shadows (272-448 MB) go and never come back, and every write-back is a whole copy again for the rest of the session. In my own logs (RTX 4070 SUPER, 2560x1440, other programs open, 10.4-10.7 GB allowed) the line `the shadows given back` is in every slow stretch. In one of them: 58 fps at the present before it, then 26, 28, 23 fps (10,462 of 10,732 MB used, 3.5 GB of it free inside the blocks). ### The change Both rules read what is really allocated: the driver's count less the free space inside the blocks, as the texture budget's 90% rule already does (`gpu/vk/vram_pressure.h`). - The shadows go when really allocated passes 95%, or the driver's own count passes 99%. - A shadow is made while really allocated stays under 93% with it, and the count under 98%. - `RX_VK_SHADOW_RAW=1`: by the count alone, as before. ### Numbers Unattended runs (`HOS_HEADLESS`, `HOS_INPUT_SCRIPT` with the frame clock): a five-minute walk across Hyrule Field from a save in the field, DLAA 2560x1440, RTX 4070 SUPER, Ryzen 9 9950X3D. Built from 4a7ae1b. The allowance was held to 9.8 GB to stand for the other programs I normally have open (a local test cap on what `vram_usage` reports; it is not part of this PR). One build, `RX_VK_SHADOW_RAW=1` against the default, two runs each, 129 windows a run. | | fps | windows under 55 fps | replay ms a frame | the capture's wait, ms a frame | GPU ms a frame | shadows at the end | |---|---|---|---|---|---|---| | by the count (as now) | 59.5, 59.3 | 3, 7 | 8.20, 8.61 | 1.94, 2.15 | 11.24, 11.47 | 0 MB | | with this | 59.8, 59.9 | 1, 2 | 7.44, 7.41 | 1.10, 0.77 | 10.14, 10.06 | 384 MB | By the count, the shadows went once or twice a run and were not made again. With this change they still went 5 and 7 times a run, each time the driver's own count passed 99% under the cap (`VRAM 9944 of 9800 MB`), and were made again each time. On v2.0.11's code, before the reuse pool and the sweep of dead targets, the same comparison was wider: 56.6, 58.8 fps with 32, 8 windows under 55 by the count, against 59.8, 59.6 fps with 1, 2. At 10.4 GB allowed: 58.5, 58.6 fps with 16, 13 slow windows, against 59.3 with 6. What losing the shadows costs in a scene that repeats exactly (standing in the field with the camera turning, 2,100-3,000 draws, the same draws in both runs; measured on v2.0.11's code), `RX_VK_WB_SHADOW_MB=0` against the default: | | replay ms a frame | capture's wait | GPU ms a frame | |---|---|---|---| | shadows | 8.15 | 0.34 | 9.99 | | no shadows | 10.64 | 2.07 | 12.49 | ### Limits - The walk's path drifts a little from run to run, so the walk rows are not the same frames; the second table is. - These runs are on an otherwise idle PC with the allowance capped, so the driver is not really short of memory. My real sessions (26-45 fps after the line) had other programs holding VRAM as well. - In all four runs the 97% rule squeezed the texture budget. That is #1506's subject and is not touched here. - With the shadows kept, the driver's count sits where it sat before (93-98%); nothing here lowers it. - Tested on one card (NVIDIA, 12 GB). An 8 GB card reaches these lines sooner; I have none to try. ### Test `python -I -B tests/test_vram_pressure.py` (the rules' arithmetic with the numbers of the log above, and a source check). `run_all` mem_trim, texture_handle, texture_init pass. 🤖 Generated with [Claude Code](https://claude.com/claude-code)

hckrman101 (Migrated from github.com) reviewed

2026-11-10 16:08:39 +00:00

hckrman101 (Migrated from github.com) left a comment

Thanks, the numbers make a good case, and the change itself is small and safe: vram_usage() is just vmaGetHeapBudgets (atomic counters plus VMA's cached budget, no vmaCalculateStatistics), it runs under pools_mu as before, a failed create_device_buffer still just means no shadow (whole copies), and RX_VK_SHADOW_RAW=1 keeps the old rules. The test is registered in run_all.py and CI is green. Without VK_EXT_memory_budget, used/allowed are VMA's own block bytes and 80% of the heap, so these lines are rarely reached there. Same as before.

One thing I'd like you to look at before merging. The premise is that the free space inside VMA's blocks is usable, but it isn't usable for the shadows themselves. With preferredLargeHeapBlockSize = 64 MB (vk_vma.cpp), VMA 3.3 makes any allocation over 32 MB a dedicated allocation (size > GetPreferredBlockSize() / 2). At 272-448 MB total, most shadows are pool-sized and well over that. So each shadow raises the driver's count by its full size, and freeing it lowers the count by its full size. That has two effects:

Shadows now come before textures. shadow_fits lets the count go up to 98% and the really-allocated figure up to 93%. The texture budget squeezes at 97% count and at 90% really-allocated (vk_texture.cpp ~2911-2921). So making a shadow can now be what starts texture eviction. Evicting textures doesn't lower the count (that space stays in the blocks), so the squeeze stays on while the shadows stay. The comment on shadow_room still says "the textures and the mirrors come first". That may well be the right trade (your tables say the shadows are worth more than a few hundred MB of textures), but then the comment should say so. Otherwise keep shadow_fits below both texture lines, e.g. used + n <= 96% and allocated + n <= 89%. This may also be part of why the texture budget was squeezed in all four runs. #1506 (squeeze waits while blocks have room) changes the 97% side of this, so the two PRs interact.
The count path has 1% hysteresis. Shadows go when the count passes 99%. Freeing them (dedicated) drops it by about S, and shadow_fits lets them back up to 98%. That's the 5-7 go/remake cycles per run you saw. Each remake reallocates and starts with shadow_known all zero (full fill reads). The "do not chase each other" check in the test only covers the 95/93 really-allocated pair, not 99/98. Widening the gap a bit, or adding a test case for it, would help.

Smaller: block_free and used/allowed are summed over all device-local heaps. On AMD with ReBAR, the host-visible device-local heap's block free space counts toward allocated even though AUTO_PREFER_DEVICE shadows won't land there. The texture budget's rule already works this way, so it's a nit, not new.

Nit: the long comment above shadow_rules_raw quotes the v2.0.11 numbers (56.6/58.8 -> 59.8/59.6, 32/8 windows), not the current-base ones in the PR's first table. Either is fine, but it'd be good to say which base they're from.

Thanks, the numbers make a good case, and the change itself is small and safe: `vram_usage()` is just `vmaGetHeapBudgets` (atomic counters plus VMA's cached budget, no `vmaCalculateStatistics`), it runs under `pools_mu` as before, a failed `create_device_buffer` still just means no shadow (whole copies), and `RX_VK_SHADOW_RAW=1` keeps the old rules. The test is registered in run_all.py and CI is green. Without VK_EXT_memory_budget, `used`/`allowed` are VMA's own block bytes and 80% of the heap, so these lines are rarely reached there. Same as before. One thing I'd like you to look at before merging. The premise is that the free space inside VMA's blocks is usable, but it isn't usable for the shadows themselves. With `preferredLargeHeapBlockSize = 64 MB` (vk_vma.cpp), VMA 3.3 makes any allocation over 32 MB a dedicated allocation (`size > GetPreferredBlockSize() / 2`). At 272-448 MB total, most shadows are pool-sized and well over that. So each shadow raises the driver's count by its full size, and freeing it lowers the count by its full size. That has two effects: 1. **Shadows now come before textures.** `shadow_fits` lets the count go up to 98% and the really-allocated figure up to 93%. The texture budget squeezes at 97% count and at 90% really-allocated (vk_texture.cpp ~2911-2921). So making a shadow can now be what starts texture eviction. Evicting textures doesn't lower the count (that space stays in the blocks), so the squeeze stays on while the shadows stay. The comment on `shadow_room` still says "the textures and the mirrors come first". That may well be the right trade (your tables say the shadows are worth more than a few hundred MB of textures), but then the comment should say so. Otherwise keep `shadow_fits` below both texture lines, e.g. `used + n <= 96%` and allocated `+ n <= 89%`. This may also be part of why the texture budget was squeezed in all four runs. #1506 (squeeze waits while blocks have room) changes the 97% side of this, so the two PRs interact. 2. **The count path has 1% hysteresis.** Shadows go when the count passes 99%. Freeing them (dedicated) drops it by about S, and `shadow_fits` lets them back up to 98%. That's the 5-7 go/remake cycles per run you saw. Each remake reallocates and starts with `shadow_known` all zero (full fill reads). The "do not chase each other" check in the test only covers the 95/93 really-allocated pair, not 99/98. Widening the gap a bit, or adding a test case for it, would help. Smaller: `block_free` and `used`/`allowed` are summed over all device-local heaps. On AMD with ReBAR, the host-visible device-local heap's block free space counts toward `allocated` even though `AUTO_PREFER_DEVICE` shadows won't land there. The texture budget's rule already works this way, so it's a nit, not new. Nit: the long comment above `shadow_rules_raw` quotes the v2.0.11 numbers (56.6/58.8 -> 59.8/59.6, 32/8 windows), not the current-base ones in the PR's first table. Either is fine, but it'd be good to say which base they're from.

gpu/vk/vk_memory.cpp

Nit: with the 93%/98% lines, "the textures and the mirrors come first" isn't quite true anymore (texture squeeze is at 90% allocated / 97% count). Update the comment or the lines.

gpu/vk/vram_pressure.h

					
				@ -0,0 +1,40 @@

				/* How full VRAM really is, for the rules that give memory back near the driver's limit (vk_memory.cpp's write-back shadows).

Shadows over 32 MB are dedicated allocations in VMA (64 MB preferred block size, size > blockSize/2), so they never go into the block free space that allocated() subtracts. Each one adds n to used directly. With used + n <= 98% here and the texture budget squeezing at used > 97% (and at allocated > 90%, below the 93% here), making a shadow can now push the texture cache into eviction. Either keep these two lines under the texture ones (e.g. 96% / 89%), or make it explicit that shadows take priority. See the review body.

Shadows over 32 MB are dedicated allocations in VMA (64 MB preferred block size, `size > blockSize/2`), so they never go into the block free space that `allocated()` subtracts. Each one adds `n` to `used` directly. With `used + n <= 98%` here and the texture budget squeezing at `used > 97%` (and at allocated > 90%, below the 93% here), making a shadow can now push the texture cache into eviction. Either keep these two lines under the texture ones (e.g. 96% / 89%), or make it explicit that shadows take priority. See the review body.

gpu/vk/vram_pressure.h

					
				@ -0,0 +22,4 @@

				/// at the next 1%, five to seven times in a five-minute run, each time read whole again)

				inline bool shadows_go(uint64_t used, uint64_t allowed, uint64_t block_free) {

				    if (!allowed) return false;

				    return allocated(used, block_free) > allowed / 100 * 95 || used > allowed / 100 * 99;

Shadows go at a 99% count, and shadow_fits lets them come back up to a 98% count. Since freeing dedicated shadows lowers used by their full size, this 1% gap is what lets them cycle (the 5-7 go/remake per run in the description). Each remake starts with shadow_known cleared. Maybe widen it, and add a 99/98 case next to the 95/93 "do not chase" check in the test.

Shadows go at a 99% count, and `shadow_fits` lets them come back up to a 98% count. Since freeing dedicated shadows lowers `used` by their full size, this 1% gap is what lets them cycle (the 5-7 go/remake per run in the description). Each remake starts with `shadow_known` cleared. Maybe widen it, and add a 99/98 case next to the 95/93 "do not chase" check in the test.

hckrman101 commented

2026-11-10 17:49:57 +00:00

(Migrated from github.com)

Heads-up: this conflicts with main now (a lot landed today). Could you merge main into the branch (or rebase) and resolve the conflicts? Thanks!

hckrman101 (Migrated from github.com) approved these changes

2026-11-10 20:53:16 +00:00

hckrman101 (Migrated from github.com) left a comment

Thanks, all three of my points are dealt with, and the merge conflict is gone (it merges cleanly into main now).

Fixed:

shadow_fits is now 90% really allocated / 97% count, so a shadow is only made where the textures' own lines (90% allocated, 97% count in vk_texture.cpp) are not passed. The comment on shadow_room is true again, and the runs you added to vram_pressure.h (96/89 never got the shadows back, 90/97 did) are a good reason for not going lower.
The go/remake cycle: 99% go against 97% fit leaves a 2% gap, and the test now has the 99/97 case next to the 95/90 one, so they cannot chase each other.
The comment above shadow_rules_raw now says it is v2.0.11's code.

#1821 (compaction, merged): I read it against this. It does not touch vk_memory.cpp, and the two compose. Compact is opt-in; without it nothing changes here. With it the texture budget holds the count near 85%, so your 97% count line does not bind and the shadows are only judged by the 90% allocated line. One small mismatch, a nit and not a blocker: with compaction the texture budget takes only the textures' pool's free space off the count (vram_compact::reclaimable), while allocated() here subtracts the free space of all blocks. So a shadow can be made where the texture side, which sees less free space, is a little more pressed. With the hold at 85% there are roughly 5 points of room before the textures' 90% line, so I would not expect it to matter for 272-448 MB; just worth knowing if someone sees shadows and texture eviction trading off with RX_VK_COMPACT=1.

CI: the shadows test passes on both Linux and Windows. The two failing checks (nx2/test_draw_parameters.py, tests/test_antilag.py; also test_wayland_libdecor_hint on Windows) are in code this PR does not touch, from a base that was moving that day. Please re-run after the next merge of main to be sure. Not tested on my side: I did not build or run it.

Thanks, all three of my points are dealt with, and the merge conflict is gone (it merges cleanly into main now). Fixed: - `shadow_fits` is now 90% really allocated / 97% count, so a shadow is only made where the textures' own lines (90% allocated, 97% count in `vk_texture.cpp`) are not passed. The comment on `shadow_room` is true again, and the runs you added to `vram_pressure.h` (96/89 never got the shadows back, 90/97 did) are a good reason for not going lower. - The go/remake cycle: 99% go against 97% fit leaves a 2% gap, and the test now has the 99/97 case next to the 95/90 one, so they cannot chase each other. - The comment above `shadow_rules_raw` now says it is v2.0.11's code. #1821 (compaction, merged): I read it against this. It does not touch `vk_memory.cpp`, and the two compose. Compact is opt-in; without it nothing changes here. With it the texture budget holds the count near 85%, so your 97% count line does not bind and the shadows are only judged by the 90% allocated line. One small mismatch, a nit and not a blocker: with compaction the texture budget takes only the textures' pool's free space off the count (`vram_compact::reclaimable`), while `allocated()` here subtracts the free space of all blocks. So a shadow can be made where the texture side, which sees less free space, is a little more pressed. With the hold at 85% there are roughly 5 points of room before the textures' 90% line, so I would not expect it to matter for 272-448 MB; just worth knowing if someone sees shadows and texture eviction trading off with `RX_VK_COMPACT=1`. CI: the shadows test passes on both Linux and Windows. The two failing checks (`nx2/test_draw_parameters.py`, `tests/test_antilag.py`; also `test_wayland_libdecor_hint` on Windows) are in code this PR does not touch, from a base that was moving that day. Please re-run after the next merge of main to be sure. Not tested on my side: I did not build or run it.

github-sync referenced this pull request

2026-11-12 06:26:38 +00:00

chore(main): release 2.1.0-rc.1 #2026

Commenting is not possible because the repository is archived.

No reviewers

hckrman101