WIP: fix(vk): the tree-shadow mask gives back the history images and views a new size or a gone texture replaces #1534

Draft
mooki wants to merge 2 commits from mooki/Longshot:fix/tree-shadow-mask-history into main
Member

Type: fix · Risk: low · Tested on: Linux, RTX 4090: one game run with the switch on, and game-free checks · Needs: Windows, any GPU; Linux, AMD GPU; the maintainer's Hyrule Castle save

TL;DR

An optional tree-shadow smoothing pass, off by default, leaked two video-memory images every time the game changed its picture size, and never gave back what it made for textures that had gone. This PR gives them back about two seconds later. In the game with the pass on, 33 size changes in 220 seconds kept the renderer's image count between 7 and 9, where main without the fix grew from 2 to 27 in 16 changes, and nothing hung or faulted. The hang that made the pass opt-in was not reproduced here and is not claimed fixed.

Breakdown

  • Who it helps: anyone who switches the pass on (RX_VK_MASK_BLEND), for example to hunt the hang that made it off by default; with the default settings it never runs.
  • What was wrong: at each new picture size the pass made its two history images again without freeing the old ones, and it kept a view for every texture it had touched after the texture was gone.
  • What changes: the replaced images and the views of gone textures are held 120 presents (about two seconds), as the TAAU's replaced histories are, and then destroyed once.
  • What does not change: the pass stays off by default and its blend is untouched; with the switch off nothing here runs.
  • How it was checked: a new game-free test (red on main, green here); the whole suite by name (the same 38 failures as main); the Windows-target syntax check; and one game run with the switch on next to a run of main without the fix (no fault, image count 7 to 9 against 2 to 27).
  • What is still open: Hyrule Castle and the recorded 8 minutes (this run reached 12,656 presents, the report about 30,000), Windows, AMD, and a counter for the views.
  • Risk: low: only the pass with its switch on is touched; an image destroyed 120 presents after its last use could still be read if the GPU were that far behind (the TAAU history has the same bound).

Why

The tree-shadow mask blend (RX_VK_MASK_BLEND, commit 746fa1158) evens OoT's tree-shadow mask out over frames where the TAAU network is left out. It has been off by default since commit 7bba1a151, because with it on the GPU hung on the maintainer's save near Hyrule Castle (device lost after about 8 minutes, about 30,000 presents; see the comment above mask_weight()). Anyone who turns it on to look for that hang meets a leak first: each time the mask's size changes it makes its two history images again over the same variables without destroying the old ones, and the game's dynamic resolution changes the size every few frames. It also keeps a view for every texture image it was blended into and never lets one go, so a view outlives its image and can be found again under the same handle for another image. Players with the default settings are not affected.

Found while reading every place a render target's image can lose its owner (the search for #1443); this is not the cause of #1443 itself.

Refs #1443.

How to reproduce

Seen on: Linux (Arch, kernel 7.2.15), NVIDIA RTX 4090, driver 615.78.08
Settings and switches: RX_VK_MASK_BLEND=1 (clamped to 0.95; any weight above 0 runs the same code), the Render resolution row changed every few seconds
Steps: start the game on main with the switch, stand in Hyrule Field by day, set the Render resolution row to 50, 100, 75 and 67 % in turn every few seconds (the menu's row, or the control bridge's option_set command), and read the vk: image blocks: line the renderer logs every 600 presents
Expected: the image count stays flat while the size changes
Actual: on main it grows with every change: 2 images at the start, 9 after 2 changes, 15 after 4, 23 after 8, 27 after 12 (the last 5 changes were in Kokiri Forest and added none); the shared image blocks grow from 1 block (32 MB) to 8 blocks (256 MB)

What changed

The replaced history pair and the views of gone textures are put on a list and destroyed 120 presents later, once, as the TAAU's replaced histories already are (old_hist in the same file). Nothing else about the pass changes.

  • gpu/vk/vk_draw.cpp: mask_retire() and mask_reap() (new, just above mask_stabilize): the first moves an image, its allocation and its view to a list and clears the caller's handles, the second destroys the entries older than 120 presents. mask_stabilize retires the old pair before it makes a new one at a new size, reaps the list at each call, and when texture_destroy_generation() changes retires every cached view and empties the map (the ones still wanted are made again at the next call).
  • tests/test_mask_history_retire.py and tests/test_mask_history_retire.cpp (new): see Testing.
  • Unchanged: the pass stays off by default, mask_weight(), the blend and the shader, the guard in draw() that arms the pass, every other file.

Measurements

Verdict: with the pass on and the same schedule, this PR holds the renderer's shared image pool at 7 to 9 images (the last line: 2 blocks 64 MB, 7 images 23 MB) across 33 resolution changes, where main without the fix grows it from 2 to 27 images (8 blocks 256 MB, 27 images 219 MB) in 16 changes. Neither run hung or faulted: no device loss, no fatal or crash line, no NVRM, Xid or kmemsysDoCacheOp line in the kernel journal. These are counts from one run each, not frame rates, and the mask pass seems to have run in Hyrule Field only (see below).

Hardware: Linux (Arch, kernel 7.2.15), RTX 4090, driver 615.78.08, i9-13900K, 96 GB; the PC had other agents' CPU jobs beside the runs (load 8 to 23), the card did not (see Method)
Scene: Hyrule Field by day, child Link on the market side (a generated save), then a debug teleport to Kokiri Forest; native mesh shaders, 2560x1440 at the start, RX_VK_MASK_BLEND=1 (the renderer logs weight 0.95), defaults otherwise
Method: windowless runs through the workbench's run wrapper (https://github.com/mooki000/ls-workbench), one after the other inside one hold of the card (the quiet-window and measuring locks held, the wrapper's whole-card gate, nvidia-smi read before each start: only the desktop and two resident helpers were on the card, and in every sample that listed the game (62 of 63, and 30 of 30) the other compute processes held the same 1,587 MiB). A small script drove the game over the control bridge (docs/control.md): walking with camera swings, and every 4 to 9 seconds option_set resolution stepping through 50, 100, 75, 67, 100, 50 ... % (1280x720, 2560x1440, 1920x1080, 1707x960), so each change gives the mask a size different from the one before; every 4 seconds it read the renderer's latest vk: image blocks: line from the game's log. The head ran 220 s of play (125 s in the field, 89 s in the forest, 33 changes, 12,656 presents); main without the fix ran 93 s (field 50 s, forest 40 s, 16 changes, 6,931 presents), started right after the head's run. The kernel journal was read by the script every 2 s (it would have stopped the game at the first NVRM, Xid or kmemsysDoCacheOp line) and again by the wrapper after each run; "changes so far" counts the script's accepted option_set replies up to each sample.
How to reproduce: as in How to reproduce; main is the parent of this PR's first commit, built the same way (a native build of each commit over the same donor build, 418 new and 100 cached objects)

Resolution changes so far Before: main @ 645d58e68 After: this PR @ 618091d77
0 (start) 2 images 2 images
2 9 7
4 15 7
8 23 7
12 27 (the run was in Kokiri Forest from change 11) 7
16 27 (end of this run) 9
33 (end of the head's run) not run 7
range after the first change 2 to 27 7 to 9
last vk: image blocks: line 8 blocks 256 MB, 27 images 219 MB 2 blocks 64 MB, 7 images 23 MB
exit, fatal, crash, device lost 0, 0, 0, 0 0, 0, 0, 0
kernel journal: NVRM, Xid, kmemsysDoCacheOp lines 0 0
presents, resolution changes 6,931, 16 12,656, 33
this PR, the mask pass's own log lines (one run each; they appear at its first call):
  vk: tree-shadow mask blend ready
  vk: tree-shadow mask blend on #267 2560x1440, weight 0.95
this PR, shared image pool, first and last lines:
  vk: image blocks: 1 blocks 32 MB, 2 images 12 MB; 1 free ranges, the largest 19 MB
  vk: image blocks: 2 blocks 64 MB, 7 images 23 MB; 3 free ranges, the largest 24 MB
main without the fix, first and last lines:
  vk: image blocks: 1 blocks 32 MB, 2 images 12 MB; 1 free ranges, the largest 19 MB
  vk: image blocks: 8 blocks 256 MB, 27 images 219 MB; 8 free ranges, the largest 25 MB
run summaries: this PR: exit 0, 12657 frames, fatal=0 crash=0 devlost=0 kernel=0 nvrm=0, journal complete
               main:    exit 0, 6932 frames, fatal=0 crash=0 devlost=0 kernel=0 nvrm=0, journal complete
GPU after both runs: 2137 MiB used (the desktop and two helpers), 2 % utilisation, 38 C, no game process left

How to read it: the fix has no counters or log lines of its own, and a view is not a memory allocation, so this shows the images (the history pair) and not the views. Only the first dispatch of the pass is logged, so I cannot count its dispatches; its activity after that is read from main's growth. That growth stopped when the run moved to Kokiri Forest (the last 5 changes added no image), and this PR's count was flat there as well, so the pass probably does not run in Kokiri Forest and only Hyrule Field (about 8,000 presents, 19 changes of this PR's run) exercised it. Counters that moved with the sizes and say nothing about the mask: render-target images 7 to 130 (this PR) and 11 to 207 (main), device heap allocations about 1.4 thousand to 20 thousand at the first change (new render-target sets), the renderer's VRAM line 1.4 to 10.9 GB, the process 11.4 GiB at the end of both runs.

Pictures of the runs, looked at before they were cited (frames taken during play, 1280x720; no streak judgement was made, which needs a switch-off against switch-on pair at full size):

four frames with the switch on, after control: 1 this PR in Hyrule Field (Link fighting an enemy in the grass), 2 main without the fix in the same field, 3 this PR in Kokiri Forest on the path by the signpost, 4 main without the fix in a Kokiri Forest passage. All four are normal frames: no tearing, garbage or missing terrain.

six frames of this PR's run in order: 1 the start (black: a load fade), 2 Hyrule Field after the Render resolution went to 50 %, 3 the field by a fence, 4 the arrival in Kokiri Forest (black: a load fade), 5 and 6 the forest. The two black frames are load fades taken before control, not a fault; the other four are normal.

Flags and settings

None.

Testing

Tested: in the game, on my own PC (Linux, RTX 4090, driver 615.78.08): one run of this PR's head, 618091d77 (version v3.0.0-rc.7-10-g618091d77), with RX_VK_MASK_BLEND=1, 220 s of play in Hyrule Field and Kokiri Forest with 33 resolution changes, native mesh shaders, alone on the card: it quit cleanly (exit 0), fatal=0 crash=0 devlost=0, no NVRM, Xid or kmemsysDoCacheOp line in the journal (checked every 2 s during the run and again after), and the card was idle and normal afterwards. The same schedule on main without the fix (645d58e68, 93 s, 16 changes) was clean too and grew the image pool as shown above, which is How to reproduce no longer showing here. Also: tests/test_mask_history_retire.py (new; red on main, green here); the whole suite by name against main (python -I -B tests/run_all.py --jobs 6 --strict: this PR @ 618091d77 669 passed, 38 failed, 17 skipped; main @ 645d58e68 668 passed, 38 failed, 17 skipped; the same 38 names fail on both, none touches the mask); the Windows-target syntax check of the new fixture (the workbench's Windows fixture check, llvm-mingw 20261104: PASS, 1 translation unit, 0 failures, 0 unjudged; Windows execution NOT TESTED); a trial merge of this head with current main (430eef4f2 on 18 Nov, 73 commits later; first checked against 4b68c42be, 57 commits later) is clean, only gpu/vk/vk_draw.cpp is merged automatically, and the three mask checks (test_mask_history_retire.py, test_shadow_mask_guard.py, test_tree_shadow_mask.py) pass on the merged tree. The merged tree was not built or run.

Not testable by hand: the recorded device loss near Hyrule Castle. It was seen on the maintainer's PC and save, twice, after about 8 minutes with the pass on; it was not seen on this PC in the days the pass was on by default, so it cannot be reproduced here, and the stand-in is the clean run above. The head already contains the fix commit b5d679321 for its suspected cause (a temporary guard that ran the pass before its draw, inside an open render pass), so a clean run of this head says nothing about whether that fix cured it.

Not tested: the pass's own time cost (a paired exclusive timing is queued for the night of 18 Nov, see Risks); the owner's Hyrule Castle save and the recorded 8 minutes or about 30,000 presents (this run reached 12,656, about 8,000 of them with the pass probably active); a second run of either build (the numbers are one run each); RX_VK_MASK_BLEND=0.85, the weight at which the hang was recorded (the run used 1, which the renderer clamps to 0.95); that the views are destroyed (no counter); tree-shadow visual quality; Windows, AMD, Intel; a rebuilt game on current main; that destroying the history images 120 presents after a size change is late enough for every submission in flight (the TAAU's identical delay is the precedent). The head's run logged seven ran without a core after 2000 ms lines from the game's save thread (a CPU contention notice, not a driver or GPU line, with the PC loaded by other jobs); the main run had none and I did not attribute them to this change.

Must be tested:

  • Windows, any GPU: start the game with RX_VK_MASK_BLEND=0.85, walk in Hyrule Field by day for five minutes while the Render resolution row is changed every few seconds, expecting the vk: image blocks: image count to stay flat instead of climbing with each change, and no device loss or validation message about a destroyed image in use (by hand: read longshot_log.txt).
  • Linux, AMD GPU: the same five minutes in Hyrule Field with RX_VK_MASK_BLEND=0.85, expecting the same flat image count and no device loss (by hand: read longshot_log.txt).
  • The maintainer's Hyrule Castle save, any NVIDIA GPU: play ten minutes or more near the castle with RX_VK_MASK_BLEND=0.85, expecting no device loss, to learn whether the recorded hang still happens now that its suspected cause is fixed (by hand: the maintainer; this PR does not claim to cure it).

Risks

What could break: only the pass with its switch on: a view or image destroyed 120 presents after it was last used could still be read by a submission in flight if the GPU were more than 120 presents behind (the TAAU history has the same bound). With the switch off nothing here runs. The pass's own time cost is not measured yet: it now also walks a short list at each call (empty unless a size changed in the last 120 presents). A paired exclusive timing, main against this PR with the switch on at a fixed picture size, is queued for the night of 18 Nov (no earlier than 22:00); its numbers go into a dated update, or a note that the hold was not quiet and no number. The runs above were ordinary runs, not timing runs.

Overlaps: none found. Of the 299 open pull requests (read on 2026-11-17), 45 change gpu/vk/vk_draw.cpp and none adds or removes a line that names mask_stabilize, mask_weight, RX_VK_MASK_BLEND or MaskGuard; a trial merge of each of the 55 that touch vk_draw.cpp, switches.def or the mask checks, against current main and against current main plus this change, shows no conflict that only this change causes. Two WIP ray-tracing pull requests (727, 1151) add code immediately after the end of mask_stabilize, which this change does not touch.

Turning it off: it is off by default; revert the commit.

Compatibility: Nothing changes.

Issues

Refs #1443.

Checklist

Everyone

  • One topic.
  • The title is a Conventional Commit (type(scope): description) of the status line's type.
  • Why gives the reason plainly and links the issues that fit (the tracker searched).
  • Tested on my own PC where it runs (in the game, for a change the game runs), as Testing's Tested says; what
    can't be run by hand is in Testing's Not testable by hand, with why.
  • Measurements has a verdict, or one line saying why there is nothing to measure.
  • Nothing gets worse that I know of: any cost (speed, latency, memory, a setup that worked before) is measured and
    in Risks. Not ticked yet: the pass's own time cost is not measured; the paired timing is queued for the night of 18 Nov (see Risks).
  • Testing names what was tested, what wasn't, and what must still be tested, where and how.
  • Rebased on current main. Not ticked yet: the head is on 645d58e68 (current main 430eef4f2 is 73 commits later) because the game run is of this exact head; the trial merge is clean.
  • python -I -B tests/run_all.py passes (any failure that also fails on main is named): 38 tests fail on both main and this PR (the same names, this head against main @ 645d58e68): test_bug_report_defaults.py, test_cpu_report.py, test_crash_record.py, test_crt_options.py, test_damage_debug_menu.py, test_display_startup_failure.py, test_dlss_feature_size.py, test_failed_compile_keeps_objects.py, test_framegen_preset.py, test_fsr_uniform_ownership.py, test_guest_imported.py, test_hud_layout_row.py, test_import_reach.py, test_invalidate_vcore.py, test_linux_preflight.py, test_linux_pressure.py, test_linux_update.py, test_mirror_budget.py, test_missing_nup.py, test_mods_size_table.py, test_nonuniform_image_arrays.py, test_parallel_init_order.py, test_pending_guest_import.py, test_pipeline_play_cache.py, test_present_thread.py, test_profiling_mode.py, test_readback_timeout_source.py, test_replay_write_order.py, test_rumble_menu.py, test_run_all_changed.py, test_slow_frames.py, test_staging_burst.py, test_target_image_pool.py, test_upscaler_round_trip.py, test_upscaler_rows.py, test_vcore_escape.py, test_vk_occlusion_queries.py, test_walk_key.py.
  • No game files, dumps, build output or generated files.
  • third_party/ untouched (a new folder with a PROVENANCE.md instead).
  • The license is OK: GPL-3.0-or-later, nothing incompatible and nothing decompiled.

When it applies

  • AI assistance recorded with Assisted-by: / Co-Authored-By: trailers.
  • A game-free test added for the fix, failing on main and passing here.
  • A check under tests/ that reads changed source updated in this PR, not weakened (the new test reads mask_stabilize; no existing check changed, and the two existing mask checks pass unchanged on this head and on the trial merge).
  • Host code builds on both Windows (llvm-mingw) and Linux, with UTF-8 paths (the Windows-target syntax check of the new fixture passed; vk_draw.cpp built natively on Linux for the game run).

New on this forge; not opened on GitHub.

**Type:** fix · **Risk:** low · **Tested on:** Linux, RTX 4090: one game run with the switch on, and game-free checks · **Needs:** Windows, any GPU; Linux, AMD GPU; the maintainer's Hyrule Castle save ## TL;DR An optional tree-shadow smoothing pass, off by default, leaked two video-memory images every time the game changed its picture size, and never gave back what it made for textures that had gone. This PR gives them back about two seconds later. In the game with the pass on, 33 size changes in 220 seconds kept the renderer's image count between 7 and 9, where main without the fix grew from 2 to 27 in 16 changes, and nothing hung or faulted. The hang that made the pass opt-in was not reproduced here and is not claimed fixed. ## Breakdown - **Who it helps:** anyone who switches the pass on (`RX_VK_MASK_BLEND`), for example to hunt the hang that made it off by default; with the default settings it never runs. - **What was wrong:** at each new picture size the pass made its two history images again without freeing the old ones, and it kept a view for every texture it had touched after the texture was gone. - **What changes:** the replaced images and the views of gone textures are held 120 presents (about two seconds), as the TAAU's replaced histories are, and then destroyed once. - **What does not change:** the pass stays off by default and its blend is untouched; with the switch off nothing here runs. - **How it was checked:** a new game-free test (red on main, green here); the whole suite by name (the same 38 failures as main); the Windows-target syntax check; and one game run with the switch on next to a run of main without the fix (no fault, image count 7 to 9 against 2 to 27). - **What is still open:** Hyrule Castle and the recorded 8 minutes (this run reached 12,656 presents, the report about 30,000), Windows, AMD, and a counter for the views. - **Risk:** low: only the pass with its switch on is touched; an image destroyed 120 presents after its last use could still be read if the GPU were that far behind (the TAAU history has the same bound). ## Why The tree-shadow mask blend (`RX_VK_MASK_BLEND`, commit `746fa1158`) evens OoT's tree-shadow mask out over frames where the TAAU network is left out. It has been off by default since commit `7bba1a151`, because with it on the GPU hung on the maintainer's save near Hyrule Castle (device lost after about 8 minutes, about 30,000 presents; see the comment above `mask_weight()`). Anyone who turns it on to look for that hang meets a leak first: each time the mask's size changes it makes its two history images again over the same variables without destroying the old ones, and the game's dynamic resolution changes the size every few frames. It also keeps a view for every texture image it was blended into and never lets one go, so a view outlives its image and can be found again under the same handle for another image. Players with the default settings are not affected. Found while reading every place a render target's image can lose its owner (the search for #1443); this is not the cause of #1443 itself. Refs #1443. ## How to reproduce **Seen on:** Linux (Arch, kernel 7.2.15), NVIDIA RTX 4090, driver 615.78.08 **Settings and switches:** `RX_VK_MASK_BLEND=1` (clamped to 0.95; any weight above 0 runs the same code), the Render resolution row changed every few seconds **Steps:** start the game on main with the switch, stand in Hyrule Field by day, set the Render resolution row to 50, 100, 75 and 67 % in turn every few seconds (the menu's row, or the control bridge's `option_set` command), and read the `vk: image blocks:` line the renderer logs every 600 presents **Expected:** the image count stays flat while the size changes **Actual:** on main it grows with every change: 2 images at the start, 9 after 2 changes, 15 after 4, 23 after 8, 27 after 12 (the last 5 changes were in Kokiri Forest and added none); the shared image blocks grow from 1 block (32 MB) to 8 blocks (256 MB) ## What changed The replaced history pair and the views of gone textures are put on a list and destroyed 120 presents later, once, as the TAAU's replaced histories already are (`old_hist` in the same file). Nothing else about the pass changes. - `gpu/vk/vk_draw.cpp`: `mask_retire()` and `mask_reap()` (new, just above `mask_stabilize`): the first moves an image, its allocation and its view to a list and clears the caller's handles, the second destroys the entries older than 120 presents. `mask_stabilize` retires the old pair before it makes a new one at a new size, reaps the list at each call, and when `texture_destroy_generation()` changes retires every cached view and empties the map (the ones still wanted are made again at the next call). - `tests/test_mask_history_retire.py` and `tests/test_mask_history_retire.cpp` (new): see Testing. - Unchanged: the pass stays off by default, `mask_weight()`, the blend and the shader, the guard in `draw()` that arms the pass, every other file. ## Measurements **Verdict:** with the pass on and the same schedule, this PR holds the renderer's shared image pool at 7 to 9 images (the last line: 2 blocks 64 MB, 7 images 23 MB) across 33 resolution changes, where main without the fix grows it from 2 to 27 images (8 blocks 256 MB, 27 images 219 MB) in 16 changes. Neither run hung or faulted: no device loss, no fatal or crash line, no NVRM, Xid or kmemsysDoCacheOp line in the kernel journal. These are counts from one run each, not frame rates, and the mask pass seems to have run in Hyrule Field only (see below). Hardware: Linux (Arch, kernel 7.2.15), RTX 4090, driver 615.78.08, i9-13900K, 96 GB; the PC had other agents' CPU jobs beside the runs (load 8 to 23), the card did not (see Method) Scene: Hyrule Field by day, child Link on the market side (a generated save), then a debug teleport to Kokiri Forest; native mesh shaders, 2560x1440 at the start, `RX_VK_MASK_BLEND=1` (the renderer logs `weight 0.95`), defaults otherwise Method: windowless runs through the workbench's run wrapper (https://github.com/mooki000/ls-workbench), one after the other inside one hold of the card (the quiet-window and measuring locks held, the wrapper's whole-card gate, `nvidia-smi` read before each start: only the desktop and two resident helpers were on the card, and in every sample that listed the game (62 of 63, and 30 of 30) the other compute processes held the same 1,587 MiB). A small script drove the game over the control bridge (docs/control.md): walking with camera swings, and every 4 to 9 seconds `option_set resolution` stepping through 50, 100, 75, 67, 100, 50 ... % (1280x720, 2560x1440, 1920x1080, 1707x960), so each change gives the mask a size different from the one before; every 4 seconds it read the renderer's latest `vk: image blocks:` line from the game's log. The head ran 220 s of play (125 s in the field, 89 s in the forest, 33 changes, 12,656 presents); main without the fix ran 93 s (field 50 s, forest 40 s, 16 changes, 6,931 presents), started right after the head's run. The kernel journal was read by the script every 2 s (it would have stopped the game at the first NVRM, Xid or kmemsysDoCacheOp line) and again by the wrapper after each run; "changes so far" counts the script's accepted `option_set` replies up to each sample. How to reproduce: as in How to reproduce; main is the parent of this PR's first commit, built the same way (a native build of each commit over the same donor build, 418 new and 100 cached objects) | Resolution changes so far | Before: main @ 645d58e68 | After: this PR @ 618091d77 | |---|---|---| | 0 (start) | 2 images | 2 images | | 2 | 9 | 7 | | 4 | 15 | 7 | | 8 | 23 | 7 | | 12 | 27 (the run was in Kokiri Forest from change 11) | 7 | | 16 | 27 (end of this run) | 9 | | 33 (end of the head's run) | not run | 7 | | range after the first change | 2 to 27 | 7 to 9 | | last `vk: image blocks:` line | 8 blocks 256 MB, 27 images 219 MB | 2 blocks 64 MB, 7 images 23 MB | | exit, fatal, crash, device lost | 0, 0, 0, 0 | 0, 0, 0, 0 | | kernel journal: NVRM, Xid, kmemsysDoCacheOp lines | 0 | 0 | | presents, resolution changes | 6,931, 16 | 12,656, 33 | ``` this PR, the mask pass's own log lines (one run each; they appear at its first call): vk: tree-shadow mask blend ready vk: tree-shadow mask blend on #267 2560x1440, weight 0.95 this PR, shared image pool, first and last lines: vk: image blocks: 1 blocks 32 MB, 2 images 12 MB; 1 free ranges, the largest 19 MB vk: image blocks: 2 blocks 64 MB, 7 images 23 MB; 3 free ranges, the largest 24 MB main without the fix, first and last lines: vk: image blocks: 1 blocks 32 MB, 2 images 12 MB; 1 free ranges, the largest 19 MB vk: image blocks: 8 blocks 256 MB, 27 images 219 MB; 8 free ranges, the largest 25 MB run summaries: this PR: exit 0, 12657 frames, fatal=0 crash=0 devlost=0 kernel=0 nvrm=0, journal complete main: exit 0, 6932 frames, fatal=0 crash=0 devlost=0 kernel=0 nvrm=0, journal complete GPU after both runs: 2137 MiB used (the desktop and two helpers), 2 % utilisation, 38 C, no game process left ``` How to read it: the fix has no counters or log lines of its own, and a view is not a memory allocation, so this shows the images (the history pair) and not the views. Only the first dispatch of the pass is logged, so I cannot count its dispatches; its activity after that is read from main's growth. That growth stopped when the run moved to Kokiri Forest (the last 5 changes added no image), and this PR's count was flat there as well, so the pass probably does not run in Kokiri Forest and only Hyrule Field (about 8,000 presents, 19 changes of this PR's run) exercised it. Counters that moved with the sizes and say nothing about the mask: render-target images 7 to 130 (this PR) and 11 to 207 (main), device heap allocations about 1.4 thousand to 20 thousand at the first change (new render-target sets), the renderer's VRAM line 1.4 to 10.9 GB, the process 11.4 GiB at the end of both runs. Pictures of the runs, looked at before they were cited (frames taken during play, 1280x720; no streak judgement was made, which needs a switch-off against switch-on pair at full size): ![four frames with the switch on, after control: 1 this PR in Hyrule Field (Link fighting an enemy in the grass), 2 main without the fix in the same field, 3 this PR in Kokiri Forest on the path by the signpost, 4 main without the fix in a Kokiri Forest passage. All four are normal frames: no tearing, garbage or missing terrain.](https://git.magicbean.dev/attachments/9d5b4a9f-b833-4f14-844f-b379f0d02512) ![six frames of this PR's run in order: 1 the start (black: a load fade), 2 Hyrule Field after the Render resolution went to 50 %, 3 the field by a fence, 4 the arrival in Kokiri Forest (black: a load fade), 5 and 6 the forest. The two black frames are load fades taken before control, not a fault; the other four are normal.](https://git.magicbean.dev/attachments/80feeb92-46e4-4240-8251-68e953279079) ### Flags and settings None. ## Testing **Tested:** in the game, on my own PC (Linux, RTX 4090, driver 615.78.08): one run of this PR's head, `618091d77` (version `v3.0.0-rc.7-10-g618091d77`), with `RX_VK_MASK_BLEND=1`, 220 s of play in Hyrule Field and Kokiri Forest with 33 resolution changes, native mesh shaders, alone on the card: it quit cleanly (exit 0), `fatal=0 crash=0 devlost=0`, no NVRM, Xid or kmemsysDoCacheOp line in the journal (checked every 2 s during the run and again after), and the card was idle and normal afterwards. The same schedule on main without the fix (`645d58e68`, 93 s, 16 changes) was clean too and grew the image pool as shown above, which is How to reproduce no longer showing here. Also: `tests/test_mask_history_retire.py` (new; red on main, green here); the whole suite by name against main (`python -I -B tests/run_all.py --jobs 6 --strict`: this PR @ 618091d77 669 passed, 38 failed, 17 skipped; main @ 645d58e68 668 passed, 38 failed, 17 skipped; the same 38 names fail on both, none touches the mask); the Windows-target syntax check of the new fixture (the workbench's Windows fixture check, llvm-mingw 20261104: PASS, 1 translation unit, 0 failures, 0 unjudged; Windows execution NOT TESTED); a trial merge of this head with current main (430eef4f2 on 18 Nov, 73 commits later; first checked against 4b68c42be, 57 commits later) is clean, only `gpu/vk/vk_draw.cpp` is merged automatically, and the three mask checks (`test_mask_history_retire.py`, `test_shadow_mask_guard.py`, `test_tree_shadow_mask.py`) pass on the merged tree. The merged tree was not built or run. **Not testable by hand:** the recorded device loss near Hyrule Castle. It was seen on the maintainer's PC and save, twice, after about 8 minutes with the pass on; it was not seen on this PC in the days the pass was on by default, so it cannot be reproduced here, and the stand-in is the clean run above. The head already contains the fix commit `b5d679321` for its suspected cause (a temporary guard that ran the pass before its draw, inside an open render pass), so a clean run of this head says nothing about whether that fix cured it. **Not tested:** the pass's own time cost (a paired exclusive timing is queued for the night of 18 Nov, see Risks); the owner's Hyrule Castle save and the recorded 8 minutes or about 30,000 presents (this run reached 12,656, about 8,000 of them with the pass probably active); a second run of either build (the numbers are one run each); `RX_VK_MASK_BLEND=0.85`, the weight at which the hang was recorded (the run used 1, which the renderer clamps to 0.95); that the views are destroyed (no counter); tree-shadow visual quality; Windows, AMD, Intel; a rebuilt game on current main; that destroying the history images 120 presents after a size change is late enough for every submission in flight (the TAAU's identical delay is the precedent). The head's run logged seven `ran without a core after 2000 ms` lines from the game's save thread (a CPU contention notice, not a driver or GPU line, with the PC loaded by other jobs); the main run had none and I did not attribute them to this change. **Must be tested:** - Windows, any GPU: start the game with `RX_VK_MASK_BLEND=0.85`, walk in Hyrule Field by day for five minutes while the Render resolution row is changed every few seconds, expecting the `vk: image blocks:` image count to stay flat instead of climbing with each change, and no device loss or validation message about a destroyed image in use (by hand: read `longshot_log.txt`). - Linux, AMD GPU: the same five minutes in Hyrule Field with `RX_VK_MASK_BLEND=0.85`, expecting the same flat image count and no device loss (by hand: read `longshot_log.txt`). - The maintainer's Hyrule Castle save, any NVIDIA GPU: play ten minutes or more near the castle with `RX_VK_MASK_BLEND=0.85`, expecting no device loss, to learn whether the recorded hang still happens now that its suspected cause is fixed (by hand: the maintainer; this PR does not claim to cure it). ## Risks **What could break:** only the pass with its switch on: a view or image destroyed 120 presents after it was last used could still be read by a submission in flight if the GPU were more than 120 presents behind (the TAAU history has the same bound). With the switch off nothing here runs. The pass's own time cost is not measured yet: it now also walks a short list at each call (empty unless a size changed in the last 120 presents). A paired exclusive timing, main against this PR with the switch on at a fixed picture size, is queued for the night of 18 Nov (no earlier than 22:00); its numbers go into a dated update, or a note that the hold was not quiet and no number. The runs above were ordinary runs, not timing runs. **Overlaps:** none found. Of the 299 open pull requests (read on 2026-11-17), 45 change `gpu/vk/vk_draw.cpp` and none adds or removes a line that names `mask_stabilize`, `mask_weight`, `RX_VK_MASK_BLEND` or `MaskGuard`; a trial merge of each of the 55 that touch `vk_draw.cpp`, `switches.def` or the mask checks, against current main and against current main plus this change, shows no conflict that only this change causes. Two WIP ray-tracing pull requests (727, 1151) add code immediately after the end of `mask_stabilize`, which this change does not touch. **Turning it off:** it is off by default; revert the commit. **Compatibility:** Nothing changes. ## Issues Refs #1443. ## Checklist ### Everyone - [x] One topic. - [x] The title is a Conventional Commit (`type(scope): description`) of the status line's type. - [x] Why gives the reason plainly and links the issues that fit (the tracker searched). - [x] Tested on my own PC where it runs (in the game, for a change the game runs), as Testing's Tested says; what can't be run by hand is in Testing's Not testable by hand, with why. - [x] Measurements has a verdict, or one line saying why there is nothing to measure. - [ ] Nothing gets worse that I know of: any cost (speed, latency, memory, a setup that worked before) is measured and in Risks. Not ticked yet: the pass's own time cost is not measured; the paired timing is queued for the night of 18 Nov (see Risks). - [x] Testing names what was tested, what wasn't, and what must still be tested, where and how. - [ ] Rebased on current `main`. Not ticked yet: the head is on `645d58e68` (current main `430eef4f2` is 73 commits later) because the game run is of this exact head; the trial merge is clean. - [x] `python -I -B tests/run_all.py` passes (any failure that also fails on `main` is named): 38 tests fail on both main and this PR (the same names, this head against main @ 645d58e68): test_bug_report_defaults.py, test_cpu_report.py, test_crash_record.py, test_crt_options.py, test_damage_debug_menu.py, test_display_startup_failure.py, test_dlss_feature_size.py, test_failed_compile_keeps_objects.py, test_framegen_preset.py, test_fsr_uniform_ownership.py, test_guest_imported.py, test_hud_layout_row.py, test_import_reach.py, test_invalidate_vcore.py, test_linux_preflight.py, test_linux_pressure.py, test_linux_update.py, test_mirror_budget.py, test_missing_nup.py, test_mods_size_table.py, test_nonuniform_image_arrays.py, test_parallel_init_order.py, test_pending_guest_import.py, test_pipeline_play_cache.py, test_present_thread.py, test_profiling_mode.py, test_readback_timeout_source.py, test_replay_write_order.py, test_rumble_menu.py, test_run_all_changed.py, test_slow_frames.py, test_staging_burst.py, test_target_image_pool.py, test_upscaler_round_trip.py, test_upscaler_rows.py, test_vcore_escape.py, test_vk_occlusion_queries.py, test_walk_key.py. - [x] No game files, dumps, build output or generated files. - [x] `third_party/` untouched (a new folder with a `PROVENANCE.md` instead). - [x] The license is OK: GPL-3.0-or-later, nothing incompatible and nothing decompiled. ### When it applies - [x] AI assistance recorded with `Assisted-by:` / `Co-Authored-By:` trailers. - [x] A game-free test added for the fix, failing on `main` and passing here. - [x] A check under `tests/` that reads changed source updated in this PR, not weakened (the new test reads `mask_stabilize`; no existing check changed, and the two existing mask checks pass unchanged on this head and on the trial merge). - [x] Host code builds on both Windows (llvm-mingw) and Linux, with UTF-8 paths (the Windows-target syntax check of the new fixture passed; `vk_draw.cpp` built natively on Linux for the game run). --- New on this forge; not opened on GitHub.
Cuts the retiring helpers out of vk_draw.cpp and runs them over 40 size changes, and reads mask_stabilize for the wiring: the old pair
retired before a new one is made over its handles, every view retired when a texture went.

Assisted-by: Claude Sonnet 5.5
Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
fix(vk): the tree-shadow mask gives back the history images and views a new size or a gone texture replaces
Some checks failed
CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Has been skipped
CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Has been skipped
CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Has been skipped
Third-party parts / The same as upstream (pull_request) Has been skipped
CI / Conventional Commit title (pull_request) Successful in 10s
Guide screenshots / Guide images and local UI source review (pull_request) Failing after 16s
Website / Website on Cloudflare (pull_request_target) Has been skipped
CI / Workflow lint (actionlint) (pull_request) Successful in 17s
Third-party parts / As their notes say (pull_request) Successful in 19s
PR description / Description follows the template (pull_request_target) Successful in 6s
618091d777
mask_stabilize (RX_VK_MASK_BLEND, off by default) made its two history images again at every new mask size over the same static handles:
the old images, their memory and their views were never destroyed, and the dynamic resolution makes a new size every few frames. It also
kept a view for every texture image it was blended into in a map nothing emptied, so a view outlived its image and was found again
under the same handle for another image. The replaced pair and the views of gone textures now wait 120 presents (as the TAAU's replaced
histories do) and are destroyed once.

Assisted-by: Claude Sonnet 5.5
Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
Some checks failed
CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Has been skipped
CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Has been skipped
CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Has been skipped
Third-party parts / The same as upstream (pull_request) Has been skipped
CI / Conventional Commit title (pull_request) Successful in 10s
Guide screenshots / Guide images and local UI source review (pull_request) Failing after 16s
Website / Website on Cloudflare (pull_request_target) Has been skipped
CI / Workflow lint (actionlint) (pull_request) Successful in 17s
Third-party parts / As their notes say (pull_request) Successful in 19s
PR description / Description follows the template (pull_request_target) Successful in 6s
This pull request is marked as a work in progress.
This branch is out-of-date with the base branch
View command line instructions

Checkout

From your project repository, check out a new branch and test the changes.
git fetch -u 'fix/tree-shadow-mask-history:mooki-fix/tree-shadow-mask-history'
git switch 'mooki-fix/tree-shadow-mask-history'
Sign in to join this conversation.
No reviewers
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
MagicBeanDev/Longshot!1534
No description provided.