WIP: fix(vk): the tree-shadow mask gives back the history images and views a new size or a gone texture replaces #1534
No reviewers
Labels
No labels
area/accessibility
area/audio
area/input
area/install
area/menu
area/renderer
area/saves
area/tooling
area/upscaling
beanstalk: approved
beanstalk: failed
beanstalk: needs approval
beanstalk: running
beanstalk: tested
ci: approved
good first issue
gpu/amd
gpu/igpu
gpu/intel
gpu/nvidia
help wanted
kind
bug
kind
chore
kind
docs
kind
feature
kind
performance
kind
question
needs measurements
platform/linux
platform/macos
platform/steam-deck
platform/windows
preview
priority
critical
priority
high
priority
low
regression
risk
high
risk
low
risk
medium
status
blocked
status
confirmed
status
duplicate
status
invalid
status
needs-info
status
wontfix
ci: approved
No milestone
No project
No assignees
1 participant
Notifications
Due date
No due date set.
Dependencies
No dependencies set
Reference
MagicBeanDev/Longshot!1534
Loading…
Reference in a new issue
No description provided.
Delete branch "mooki/Longshot:fix/tree-shadow-mask-history"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Type: fix · Risk: low · Tested on: Linux, RTX 4090: one game run with the switch on, and game-free checks · Needs: Windows, any GPU; Linux, AMD GPU; the maintainer's Hyrule Castle save
TL;DR
An optional tree-shadow smoothing pass, off by default, leaked two video-memory images every time the game changed its picture size, and never gave back what it made for textures that had gone. This PR gives them back about two seconds later. In the game with the pass on, 33 size changes in 220 seconds kept the renderer's image count between 7 and 9, where main without the fix grew from 2 to 27 in 16 changes, and nothing hung or faulted. The hang that made the pass opt-in was not reproduced here and is not claimed fixed.
Breakdown
RX_VK_MASK_BLEND), for example to hunt the hang that made it off by default; with the default settings it never runs.Why
The tree-shadow mask blend (
RX_VK_MASK_BLEND, commit746fa1158) evens OoT's tree-shadow mask out over frames where the TAAU network is left out. It has been off by default since commit7bba1a151, because with it on the GPU hung on the maintainer's save near Hyrule Castle (device lost after about 8 minutes, about 30,000 presents; see the comment abovemask_weight()). Anyone who turns it on to look for that hang meets a leak first: each time the mask's size changes it makes its two history images again over the same variables without destroying the old ones, and the game's dynamic resolution changes the size every few frames. It also keeps a view for every texture image it was blended into and never lets one go, so a view outlives its image and can be found again under the same handle for another image. Players with the default settings are not affected.Found while reading every place a render target's image can lose its owner (the search for #1443); this is not the cause of #1443 itself.
Refs #1443.
How to reproduce
Seen on: Linux (Arch, kernel 7.2.15), NVIDIA RTX 4090, driver 615.78.08
Settings and switches:
RX_VK_MASK_BLEND=1(clamped to 0.95; any weight above 0 runs the same code), the Render resolution row changed every few secondsSteps: start the game on main with the switch, stand in Hyrule Field by day, set the Render resolution row to 50, 100, 75 and 67 % in turn every few seconds (the menu's row, or the control bridge's
option_setcommand), and read thevk: image blocks:line the renderer logs every 600 presentsExpected: the image count stays flat while the size changes
Actual: on main it grows with every change: 2 images at the start, 9 after 2 changes, 15 after 4, 23 after 8, 27 after 12 (the last 5 changes were in Kokiri Forest and added none); the shared image blocks grow from 1 block (32 MB) to 8 blocks (256 MB)
What changed
The replaced history pair and the views of gone textures are put on a list and destroyed 120 presents later, once, as the TAAU's replaced histories already are (
old_histin the same file). Nothing else about the pass changes.gpu/vk/vk_draw.cpp:mask_retire()andmask_reap()(new, just abovemask_stabilize): the first moves an image, its allocation and its view to a list and clears the caller's handles, the second destroys the entries older than 120 presents.mask_stabilizeretires the old pair before it makes a new one at a new size, reaps the list at each call, and whentexture_destroy_generation()changes retires every cached view and empties the map (the ones still wanted are made again at the next call).tests/test_mask_history_retire.pyandtests/test_mask_history_retire.cpp(new): see Testing.mask_weight(), the blend and the shader, the guard indraw()that arms the pass, every other file.Measurements
Verdict: with the pass on and the same schedule, this PR holds the renderer's shared image pool at 7 to 9 images (the last line: 2 blocks 64 MB, 7 images 23 MB) across 33 resolution changes, where main without the fix grows it from 2 to 27 images (8 blocks 256 MB, 27 images 219 MB) in 16 changes. Neither run hung or faulted: no device loss, no fatal or crash line, no NVRM, Xid or kmemsysDoCacheOp line in the kernel journal. These are counts from one run each, not frame rates, and the mask pass seems to have run in Hyrule Field only (see below).
Hardware: Linux (Arch, kernel 7.2.15), RTX 4090, driver 615.78.08, i9-13900K, 96 GB; the PC had other agents' CPU jobs beside the runs (load 8 to 23), the card did not (see Method)
Scene: Hyrule Field by day, child Link on the market side (a generated save), then a debug teleport to Kokiri Forest; native mesh shaders, 2560x1440 at the start,
RX_VK_MASK_BLEND=1(the renderer logsweight 0.95), defaults otherwiseMethod: windowless runs through the workbench's run wrapper (https://github.com/mooki000/ls-workbench), one after the other inside one hold of the card (the quiet-window and measuring locks held, the wrapper's whole-card gate,
nvidia-smiread before each start: only the desktop and two resident helpers were on the card, and in every sample that listed the game (62 of 63, and 30 of 30) the other compute processes held the same 1,587 MiB). A small script drove the game over the control bridge (docs/control.md): walking with camera swings, and every 4 to 9 secondsoption_set resolutionstepping through 50, 100, 75, 67, 100, 50 ... % (1280x720, 2560x1440, 1920x1080, 1707x960), so each change gives the mask a size different from the one before; every 4 seconds it read the renderer's latestvk: image blocks:line from the game's log. The head ran 220 s of play (125 s in the field, 89 s in the forest, 33 changes, 12,656 presents); main without the fix ran 93 s (field 50 s, forest 40 s, 16 changes, 6,931 presents), started right after the head's run. The kernel journal was read by the script every 2 s (it would have stopped the game at the first NVRM, Xid or kmemsysDoCacheOp line) and again by the wrapper after each run; "changes so far" counts the script's acceptedoption_setreplies up to each sample.How to reproduce: as in How to reproduce; main is the parent of this PR's first commit, built the same way (a native build of each commit over the same donor build, 418 new and 100 cached objects)
645d58e68618091d77vk: image blocks:lineHow to read it: the fix has no counters or log lines of its own, and a view is not a memory allocation, so this shows the images (the history pair) and not the views. Only the first dispatch of the pass is logged, so I cannot count its dispatches; its activity after that is read from main's growth. That growth stopped when the run moved to Kokiri Forest (the last 5 changes added no image), and this PR's count was flat there as well, so the pass probably does not run in Kokiri Forest and only Hyrule Field (about 8,000 presents, 19 changes of this PR's run) exercised it. Counters that moved with the sizes and say nothing about the mask: render-target images 7 to 130 (this PR) and 11 to 207 (main), device heap allocations about 1.4 thousand to 20 thousand at the first change (new render-target sets), the renderer's VRAM line 1.4 to 10.9 GB, the process 11.4 GiB at the end of both runs.
Pictures of the runs, looked at before they were cited (frames taken during play, 1280x720; no streak judgement was made, which needs a switch-off against switch-on pair at full size):
Flags and settings
None.
Testing
Tested: in the game, on my own PC (Linux, RTX 4090, driver 615.78.08): one run of this PR's head,
618091d77(versionv3.0.0-rc.7-10-g618091d77), withRX_VK_MASK_BLEND=1, 220 s of play in Hyrule Field and Kokiri Forest with 33 resolution changes, native mesh shaders, alone on the card: it quit cleanly (exit 0),fatal=0 crash=0 devlost=0, no NVRM, Xid or kmemsysDoCacheOp line in the journal (checked every 2 s during the run and again after), and the card was idle and normal afterwards. The same schedule on main without the fix (645d58e68, 93 s, 16 changes) was clean too and grew the image pool as shown above, which is How to reproduce no longer showing here. Also:tests/test_mask_history_retire.py(new; red on main, green here); the whole suite by name against main (python -I -B tests/run_all.py --jobs 6 --strict: this PR @618091d77669 passed, 38 failed, 17 skipped; main @645d58e68668 passed, 38 failed, 17 skipped; the same 38 names fail on both, none touches the mask); the Windows-target syntax check of the new fixture (the workbench's Windows fixture check, llvm-mingw 20261104: PASS, 1 translation unit, 0 failures, 0 unjudged; Windows execution NOT TESTED); a trial merge of this head with current main (430eef4f2on 18 Nov, 73 commits later; first checked against4b68c42be, 57 commits later) is clean, onlygpu/vk/vk_draw.cppis merged automatically, and the three mask checks (test_mask_history_retire.py,test_shadow_mask_guard.py,test_tree_shadow_mask.py) pass on the merged tree. The merged tree was not built or run.Not testable by hand: the recorded device loss near Hyrule Castle. It was seen on the maintainer's PC and save, twice, after about 8 minutes with the pass on; it was not seen on this PC in the days the pass was on by default, so it cannot be reproduced here, and the stand-in is the clean run above. The head already contains the fix commit
b5d679321for its suspected cause (a temporary guard that ran the pass before its draw, inside an open render pass), so a clean run of this head says nothing about whether that fix cured it.Not tested: the pass's own time cost (a paired exclusive timing is queued for the night of 18 Nov, see Risks); the owner's Hyrule Castle save and the recorded 8 minutes or about 30,000 presents (this run reached 12,656, about 8,000 of them with the pass probably active); a second run of either build (the numbers are one run each);
RX_VK_MASK_BLEND=0.85, the weight at which the hang was recorded (the run used 1, which the renderer clamps to 0.95); that the views are destroyed (no counter); tree-shadow visual quality; Windows, AMD, Intel; a rebuilt game on current main; that destroying the history images 120 presents after a size change is late enough for every submission in flight (the TAAU's identical delay is the precedent). The head's run logged sevenran without a core after 2000 mslines from the game's save thread (a CPU contention notice, not a driver or GPU line, with the PC loaded by other jobs); the main run had none and I did not attribute them to this change.Must be tested:
RX_VK_MASK_BLEND=0.85, walk in Hyrule Field by day for five minutes while the Render resolution row is changed every few seconds, expecting thevk: image blocks:image count to stay flat instead of climbing with each change, and no device loss or validation message about a destroyed image in use (by hand: readlongshot_log.txt).RX_VK_MASK_BLEND=0.85, expecting the same flat image count and no device loss (by hand: readlongshot_log.txt).RX_VK_MASK_BLEND=0.85, expecting no device loss, to learn whether the recorded hang still happens now that its suspected cause is fixed (by hand: the maintainer; this PR does not claim to cure it).Risks
What could break: only the pass with its switch on: a view or image destroyed 120 presents after it was last used could still be read by a submission in flight if the GPU were more than 120 presents behind (the TAAU history has the same bound). With the switch off nothing here runs. The pass's own time cost is not measured yet: it now also walks a short list at each call (empty unless a size changed in the last 120 presents). A paired exclusive timing, main against this PR with the switch on at a fixed picture size, is queued for the night of 18 Nov (no earlier than 22:00); its numbers go into a dated update, or a note that the hold was not quiet and no number. The runs above were ordinary runs, not timing runs.
Overlaps: none found. Of the 299 open pull requests (read on 2026-11-17), 45 change
gpu/vk/vk_draw.cppand none adds or removes a line that namesmask_stabilize,mask_weight,RX_VK_MASK_BLENDorMaskGuard; a trial merge of each of the 55 that touchvk_draw.cpp,switches.defor the mask checks, against current main and against current main plus this change, shows no conflict that only this change causes. Two WIP ray-tracing pull requests (727, 1151) add code immediately after the end ofmask_stabilize, which this change does not touch.Turning it off: it is off by default; revert the commit.
Compatibility: Nothing changes.
Issues
Refs #1443.
Checklist
Everyone
type(scope): description) of the status line's type.can't be run by hand is in Testing's Not testable by hand, with why.
in Risks. Not ticked yet: the pass's own time cost is not measured; the paired timing is queued for the night of 18 Nov (see Risks).
main. Not ticked yet: the head is on645d58e68(current main430eef4f2is 73 commits later) because the game run is of this exact head; the trial merge is clean.python -I -B tests/run_all.pypasses (any failure that also fails onmainis named): 38 tests fail on both main and this PR (the same names, this head against main @645d58e68): test_bug_report_defaults.py, test_cpu_report.py, test_crash_record.py, test_crt_options.py, test_damage_debug_menu.py, test_display_startup_failure.py, test_dlss_feature_size.py, test_failed_compile_keeps_objects.py, test_framegen_preset.py, test_fsr_uniform_ownership.py, test_guest_imported.py, test_hud_layout_row.py, test_import_reach.py, test_invalidate_vcore.py, test_linux_preflight.py, test_linux_pressure.py, test_linux_update.py, test_mirror_budget.py, test_missing_nup.py, test_mods_size_table.py, test_nonuniform_image_arrays.py, test_parallel_init_order.py, test_pending_guest_import.py, test_pipeline_play_cache.py, test_present_thread.py, test_profiling_mode.py, test_readback_timeout_source.py, test_replay_write_order.py, test_rumble_menu.py, test_run_all_changed.py, test_slow_frames.py, test_staging_burst.py, test_target_image_pool.py, test_upscaler_round_trip.py, test_upscaler_rows.py, test_vcore_escape.py, test_vk_occlusion_queries.py, test_walk_key.py.third_party/untouched (a new folder with aPROVENANCE.mdinstead).When it applies
Assisted-by:/Co-Authored-By:trailers.mainand passing here.tests/that reads changed source updated in this PR, not weakened (the new test readsmask_stabilize; no existing check changed, and the two existing mask checks pass unchanged on this head and on the trial merge).vk_draw.cppbuilt natively on Linux for the game run).New on this forge; not opened on GitHub.
View command line instructions
Checkout
From your project repository, check out a new branch and test the changes.