fix(vk): a finalized render target goes once the game has written over its storage #1462
No reviewers
Labels
No labels
area/accessibility
area/audio
area/input
area/install
area/menu
area/renderer
area/saves
area/tooling
area/upscaling
beanstalk: approved
beanstalk: failed
beanstalk: needs approval
beanstalk: running
beanstalk: tested
ci: approved
good first issue
gpu/amd
gpu/igpu
gpu/intel
gpu/nvidia
help wanted
kind
bug
kind
chore
kind
docs
kind
feature
kind
performance
kind
question
needs measurements
platform/linux
platform/macos
platform/steam-deck
platform/windows
preview
priority
critical
priority
high
priority
low
regression
risk
high
risk
low
risk
medium
status
blocked
status
confirmed
status
duplicate
status
invalid
status
needs-info
status
wontfix
ci: approved
No milestone
No project
No assignees
4 participants
Notifications
Due date
No due date set.
Dependencies
No dependencies set
Reference
MagicBeanDev/Longshot!1462
Loading…
Reference in a new issue
No description provided.
Delete branch "mooki/Longshot:fix/finalized-target-obsolete"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Type: fix · Risk: medium · Tested on: Linux, RTX 4090 (a 6 GB card's allowance by switch); game-free checks · Needs: Windows, any GPU; an 8 GB card; a long walk over Hyrule Field
TL;DR
In long sessions the game's video memory slowly fills up until it stutters or closes (#1443). The cause: small terrain tiles the renderer keeps after the game has already reused their memory, so they are never let go. This PR lets those stale tiles go. Walking the same route on a simulated 6 GB card, main climbs from 129 to 166 to 221 render images; with this PR it stays at 128, 131, 129. One rule in the texture eviction pass and one new test; nothing that is still in use is touched.
Breakdown
Why
In a long session the game's video memory keeps growing until it slows down, runs out, or closes (#1443 collects the reports). Players' logs show the render-target images climbing by about 200 an hour while walking, and about 84 % of them (median over 151 logs) are small terrain tiles the renderer keeps after the game has finished with them. The renderer does not lose them; it never lets them go: inside the texture budget it retires a render target only when another target was drawn over it, and a tile whose slot the game later reuses with CPU writes is never drawn over.
Refs #1443.
How to reproduce
Seen on: Linux, Arch, RTX 4090, NVIDIA 615.78.08 (a 6 GB card's allowance set by
RX_VK_TEST_ALLOWED_MB=5200)Settings and switches: defaults, plus
RX_VK_TEST_ALLOWED_MB=5200Steps: load a save in Kokiri Forest, then three or more laps of: teleport to Hyrule Field, walk about 200 m over open ground for 6000 presents along a planned route, teleport back; read the "image blocks (render targets)" line every 600 presents
Expected: the render-target images stay flat from lap to lap
Actual: on main they climb every lap: 129, 166, 221 images at the ends of the first three marks (arrival, lap 1, lap 2); other runs on main with a diagnostic build: 171, 237, 209, 297 at the default allowance
What changed
A finalized render target whose storage the game has written over since it finalized it goes, within the budget as over it. The terrain tiles (260 x 260 RG8, nine levels) are drawn once, written back into memory, finalized, and the game later reuses their slots; the image of each stayed for good.
finalized_obsolete()ingpu/vk/vk_texture.cpp: a texture the GPU drew and the game finalized, in a pool's own memory, of 32 KB or more, with three quarters of its storage written by the CPU since it was finalized (memory_written_runsfrom itsrelease_gen).texture_frame's pass lets such a texture go whatever the budget says: the sweep (if (sweep && !overwritten) continue;) no longer keeps it,keep_finalizedno longer counts it, and the compaction's hold treats it as it treats a drawn-over target. Nothing is written back (its texels are gone from that memory on the console too).tests/test_target_retention.pyand.cppcut that pass andfinalized_obsoleteout ofvk_texture.cppand run 30 laps of 60 tile-sized targets;tests/test_vram_compact.pyfollows the changedhold_liveline.Unchanged: the cap on kept finalized textures (
RX_VK_KEEP_FINALIZED_MB, 128 MB: it is still applied over the budget only and still counts texels, not the allocator's padding), the dead-target sweep (#1255), the target pool. #943's summed overlap is a different class (single-level targets) and is not touched.Measurements
Verdict: on this head the render-target images stay flat across the two walking laps that ended (128, 131, 129) while main's climb (129, 166, 221); the finalized tiles the renderer keeps are 60 and 41 against 93 and 131. Only lap 1 is a clean comparison (131 against 166 images, 60 against 93 kept): in lap 2 of both runs a Deku Baba killed Link in Hyrule Field (the pictures show Game Over; the game restarts him at the arrival point). It is two laps of one route, not hours of walking, so it shows the direction, not the size, of the field's growth.
Hardware: Linux, RTX 4090, NVIDIA 615.78.08, i9-13900K, 96 GB; other games on the card (ordinary run, not a timing measurement)
Scene: Kokiri Forest and Hyrule Field (child), DLAA 2048x1152, 6 GB allowance by switch
Method: the lap script (
LapleakDriver), 3 laps,longshot_log.txtlines every 600 presents; the diagnostic that counted the kinds and owners (RX_VK_LOG_TARGETS) is not in this PRHow to reproduce: as above
645d58e681c09f9d12Supporting runs with the diagnostic (not this exact head: the diagnostic build plus the same single rule,
50eaceee8, against the diagnostic build on main; 24 GB card, no allowance switch, a longer route in open Hyrule Field): main's images 171, 237, 209, 297 and finalized tiles unused 600+ presents 1, 50, 79, 105; with the rule 101, 120, 131 and 0, 19, 14, of which written over by the CPU 1, 1, 1. At the 6 GB allowance on main with the diagnostic: 144, 152, 198, 240 images, tiles 1, 22, 77, 93, of which written over 1, 18, 44, 70.On main, with the diagnostic: 0 orphans in 95 samples (every image in the render-target blocks owned by a live texture, the dead list or the pool); at lap ends the finalized 260 x 260 tiles unused 600+ presents were 1, 22, 77, 93 of which 1, 18, 44, 70 had their storage written over by the CPU.
Fixture:
tests/test_target_retention.py, main @645d58e68: storage written over two laps on, within the budget 1800 of 1800 tiles alive after 30 laps, over the budget 744 of 1800 (the keep cap), and the three-quarters case fails; this PR: 120 of 1800 in both, the quarter, half and 16 KB cases keep all.Pictures of the walk
Flags and settings
None.
How it was tested
Everything here was run by an AI agent (Claude Sonnet 5.5) working for me; commands are given so anyone can rerun them. The game run described in section 3 is one run of this head and one of main, the same afternoon; its files are named below.
1. What was looked for, and how the cause was found
gpu/vkwhere a render target's image can be made, replaced, destroyed, pooled or erased (create(),destroy_later, the dead list drain intexture_frame,pool_put/pool_take/pool_trim, the compaction's image swap, the upscalers' and frame generation's images) was read at main50170f819(later645d58e68, which did not touch these files). No path loses an image.longshot_log.txtandhookshot_log.txtattached to the issues of this tracker that had 20 or more samples of "image blocks (render targets)" was downloaded (151 logs from 393 files, v3.0.0-rc.3 to nightlies) and read by a small Python script (regular expressions over four log lines every 600 presents: "image blocks (render targets): N blocks .. MB, M images .. MB", "render target images: .. made new, .. destroyed from it; it holds ..", "VRAM: .. textures .. MB (budget .. MB)", "finalized textures the GPU drew once, kept for being made again: N (KB)"). Check 1: images == made new - destroyed in every sample (92 % of 14,447 samples exact). Check 2: kept count against image count at the end of each log (median 0.84).RX_VK_LOG_TARGETS=<presents>) was added on a separate branch. Every N presents it matches every image in the render-target blocks (VMA's pool statistics) with its owner (a live Texture, the dead list, the target pool), counts the live render-target textures by kind (never written, live, finalized, one level or many) and by presents unused, and counts how many finalized ones have had their storage written by the CPU since the game finalized them (memory_written_runsfrom the texture'srelease_gen). It was run on main in the game (section 3) and showed: 0 orphans in 95 samples, and the unused finalized tiles' storage written over by the CPU (17/17, 36/37, 38/38 in the older buckets; in the walking run 70 of 93).2. Game-free checks (anyone can rerun)
python -I -B tests/test_target_retention.py(the pinned build Python;--cxx g++is the default). It cuts, by their text,texture_frame's choice of which textures go (fromorderto the end of its loop) and the newfinalized_obsolete()out ofgpu/vk/vk_texture.cpp(tests/srccut.py), compiles them intotests/test_target_retention.cppwith stand-ins for the Texture fields they read, and runs 30 laps of 60 render targets the size of the terrain tiles (260 x 260 RG8, nine levels, 176 KB of texels, 325 KB as the allocator gives it: the numbers come from the diagnostic's lines). Each lap's targets are drawn once, finalized, never used again, and no later target is drawn over them; the game writes over a lap's storage two laps on (all of it, a quarter, half, three quarters; a 16 KB texture as well). Pass: the tiles whose storage was written over (three quarters or more, 32 KB or more) are gone and the others stay.645d58e68with the test copied in: within the budget 1800 of 1800 tiles alive after 30 laps, over the budget 744 of 1800 (the keep cap), the three-quarters case fails. On this PR's head: 120 of 1800 in both cases; a quarter, half and the 16 KB texture keep all.python -I -B tests/run_all.py --jobs 6 --strictthrough the workbench's idle-class gate (https://github.com/mooki000/ls-workbench: the idle wrapper, three slots, core dumps off, a fresh HOME), once on this head and once on main645d58e68, outputs compared by test name. 669 passed / 38 failed / 17 skipped here, 668 / 38 / 17 on main; the same 38 names fail on both (listed in the checklist).ucrt-x86_64) compiles the new C++ fixture for the Windows target: PASS, 2 translation units, 0 failures, 0 unjudged. Windows execution was not run.3. In the game (Linux, headless)
645d58e68and this head1c09f9d12built the same way for the game: a native Linux build with the workbench's build wrapper through the build gate, release profile (RX_RELEASE=1, the shipped profile data), no diagnostic code, over a finished donor build of main36de9b0cewhose 42 lifted-code inputs and 517 HLE names were checked identical (the changes are renderer-only). This head: 123 s, 418 objects new and 100 from the cache, source stamp d08d307d; main645d58e68: 87 s, source stamp 35eea065; both exit 0. (The build receipts stay on my PC: they hold local paths.)RX_VK_TEST_ALLOWED_MB=5200, which makes the renderer see a driver allowance of 5200 MB, so the 24 GB card plays the budget and pressure paths of a 6 GB card (seegpu/vk/vram_compact.h,gpu/vk/texture_budget.h).docs/control.mdin this repository: https://git.magicbean.dev/MagicBeanDev/Longshot/src/branch/main/docs/control.md) on a private socket, by a lap script (the workbench's scripted-play drivers,hsap, https://github.com/mooki000/ls-workbench). A lap: debug teleport to Hyrule Field (child), a planned route of open-ground walking (a stick held toward the next waypoint, the pad's analog axes through the bridge, 6000 presents a lap), debug teleport back to Kokiri Forest, wait. The script marks each lap's start and end in a CSV with the present count. The plan asked for 3 laps of the walking leg (6000 presents, about 100 s) and a 300-present leg back in Kokiri Forest; the run's time budget stopped it after two completed laps, so each run has the arrival mark and two laps. One run per build, the head first and main 7 minutes later, same save, same plan, same settings. A planned route over open ground (waypoints chosen from the workbench's local world notes, not shipped) is followed by the script holding the stick toward the next waypoint and checking the position the bridge reports./proc, the card's memory from nvidia-smi, and the game's frame log. A picture of the game's frame every 20 s of walking through the bridge'sshot(1280 x 720). At each lap end the table takes the last value of each log line seen before the lap's end mark.kmemsysDoCacheOp, any NVRM line): 0 for both runs, journal coverage complete; a game exit code 0, no crash, no device lost, no core dump.leak-fix-pr-lapleak.py.txt, 776 lines, standard library only: it starts the game through the workbench's run wrapper with a save and switches, drives it through the bridge, writesmarks.csvwith the present count at each lap mark andsamples.csvevery 5 s, and takes the bridge's pictures). Its way of use: the workbench's scripted-play skill. Each run's own files (laps.md,samples.csv,memlines.logwith every game log line of the run, the pictures) are named in the comment.4. What was not tested
See "Not tested" under Testing: hours of walking, Windows, AMD, 8 GB cards, any speed claim, and the process-memory (RAM) creep of the 2-hour co-op soak.
Testing
Tested:
tests/test_target_retention.py(red on main, green here); the whole suite by name against main (python -I -B tests/run_all.py --jobs 6 --strict: this PR @1c09f9d12669 passed, 38 failed, 17 skipped; main @645d58e68668 passed, 38 failed, 17 skipped; the failing set is the same 38 names on both, none of them touches the render targets' eviction); the Windows-target syntax check of the new fixture (the workbench's Windows fixture check, llvm-mingw 20261104: PASS, 2 translation units, 0 failures, 0 unjudged; Windows execution NOT TESTED); one game run of this head through the lane (one run each of this head and main through the workbench's lane, 13:01 to 13:15 on 2026-11-17, exit 0, no fatal, crash, device lost or kernel lines; only two laps completed, and Link died in lap 2 of both); pictures of a long Hyrule Field walk (pictures of the walk: two contact sheets, 13 frames each, taken every 20 s).Not tested: that the field's growth (about 200 images an hour, walking through terrain) is all this kind (the laps here are 6000-present walks, about 100 s each, not hours); whether three quarters is the right line for a tile whose slot the game only partly reused; a long walk for ground streaks (issues 107 and 386) beyond the 13 pictures a build here (I looked at them at contact-sheet size: a subtle streak would not show); the third lap of the plan (the time budget ended the run); Windows, AMD, an 8 GB card; the RAM growth in the 2-hour soak (it follows time, not area, and is not render targets).
Must be tested:
Risks
What could break: a tile the renderer lets go of is made again when the game next uses its storage and is then filled from memory; if the game still relied on the image's texels for a slot it only partly rewrote, the ground could show streaks (issues 107 and 386). The rule requires three quarters of the storage written after finalization and 32 KB or more, to avoid that.
Cost: the rule reads the written ranges of a finalized texture's storage (
memory_written_runs) when the eviction pass looks at that texture, and only for textures the GPU drew, finalized and of 32 KB or more (the first checks return before the read). The texture pass's own time (the frame trace'stex_frame_ms) was not measured on either build; the runs above were ordinary runs on a shared machine, no speed claim.Turning it off: revert the commit.
Compatibility: Nothing changes.
Issues
Refs #1443.
Checklist
Everyone
type(scope): description) of the status line's type.can't be run by hand is in Testing's Not testable by hand, with why.
in Risks. Not ticked yet: Risks says the texture pass's own time cost was not measured on either build.
main.python -I -B tests/run_all.pypasses (any failure that also fails onmainis named): 38 tests fail on both main and this PR: test_bug_report_defaults.py, test_cpu_report.py, test_crash_record.py, test_crt_options.py, test_damage_debug_menu.py, test_display_startup_failure.py, test_dlss_feature_size.py, test_failed_compile_keeps_objects.py, test_framegen_preset.py, test_fsr_uniform_ownership.py, test_guest_imported.py, test_hud_layout_row.py, test_import_reach.py, test_invalidate_vcore.py, test_linux_preflight.py, test_linux_pressure.py, test_linux_update.py, test_mirror_budget.py, test_missing_nup.py, test_mods_size_table.py, test_nonuniform_image_arrays.py, test_parallel_init_order.py, test_pending_guest_import.py, test_pipeline_play_cache.py, test_present_thread.py, test_profiling_mode.py, test_readback_timeout_source.py, test_replay_write_order.py, test_rumble_menu.py, test_run_all_changed.py, test_slow_frames.py, test_staging_burst.py, test_target_image_pool.py, test_upscaler_round_trip.py, test_upscaler_rows.py, test_vcore_escape.py, test_vk_occlusion_queries.py, test_walk_key.py.third_party/untouched (a new folder with aPROVENANCE.mdinstead).When it applies
Assisted-by:/Co-Authored-By:trailers.tests/that reads changed source updated in this PR, not weakened (tests/test_vram_compact.pyfollows the changedhold_liveline; nothing removed).New on this forge; not opened on GitHub.
WIP: fix(vk): a finalized render target goes once the game has written over its storageto fix(vk): a finalized render target goes once the game has written over its storageThe lap scripts used for the game runs in the description, attached as text (rename to .py): lapleak.py drives the game over the control bridge (
docs/control.md) and samples every 4 s; lapsumm.py turns a run'ssamples.csv,memlines.logandmarks.csvinto the per-lap tables. Both are standard library only. The planned walking route is a file of waypoints I made from my own game files, so it is not attached; without it the walk turns at obstacles and stays near the arrival point. The 24 GB laps and the first form of the fix are in #1443 (comment).@Beanstalk
Beanstalk test farm
Commit
1c09f9d· tier reviewed · approved by llun · dashboardWhat will be tested · AI plan (
gpt-6.1-terra)On Windows and on an 8 GB GPU, the plan repeats short Hyrule Field walks and returns to Kokiri Forest under a simulated 6 GB allowance, then checks the Deku Tree interior. Each target takes roughly 15 minutes, comparing VRAM, frame times and pictures against main; crashes, device loss, substantial performance or VRAM regressions, and visual differences fail. The VRAM probe is a proxy for retention: this plan cannot count render-target images or establish long-session growth.
RX_VK_TEST_ALLOWED_MB=5200), then 3 times: teleport to Hyrule Field, move the stick, teleport to Kokiri Forest, teleport to Hyrule Field, take the picture field-after-laps to compare with main, measure field for 20 s (after 5 s warm-up), teleport to Inside the Deku Tree, take the picture deku-interior to compare with main, measure deku for 20 s (after 5 s warm-up); 1 run on each side.Done (started 20:01 UTC, last report 20:56 UTC). Comment
@Beanstalkto run it again.AMD Radeon RX 9070 XT, 2.0.416; 2: windows+nvidia+nvidia-proprietary+dgpuNVIDIA GeForce RTX 4070, 621.1.0NVIDIA GeForce RTX 3070, 621.24.0Measurements · windows
Verdict: fail on windows (2/1 report):
field-retention field.frametime_p99_ms.max_delta_pct (+26.9%)failed.Hardware: 1: windows+amd+amd-proprietary+dgpu
AMD Radeon RX 9070 XT, 2.0.416AMD Ryzen 7 5800X3D 8-Core Processor, 32 GB RAM; 2: windows+nvidia+nvidia-proprietary+dgpuNVIDIA GeForce RTX 4070, 621.1.013th Gen Intel(R) Core(TM) i7-13700F, 32 GB RAMScene: field-retention, from a new save (RX_VK_TEST_ALLOWED_MB=5200); each cell's settings and switches as the table names them
Method: the Beanstalk test farm: main and this PR built and run on the same PC in one session, game starts alternating (ABBA); field-retention: 1 game start a side, field 20 s after 5 s warm-up, deku 20 s after 5 s warm-up; each number the median of the runs, then of 2 reports
How to reproduce: the farm's plan above (
@Beanstalkon this PR runs it again)645d58e1c09f9dFlags and settings
None: the plan varied no setting, switch or build option.
Markdown for the PR description
Measurements · vram8gb
Verdict: pass on vram8gb (1/1 report): no rule failed and no PR run crashed.
Hardware: 1: windows+nvidia+nvidia-proprietary+dgpu
NVIDIA GeForce RTX 3070, 621.24.0AMD Ryzen 5 5600X3D 6-Core Processor, 32 GB RAMScene: field-retention, from a new save (RX_VK_TEST_ALLOWED_MB=5200); each cell's settings and switches as the table names them
Method: the Beanstalk test farm: main and this PR built and run on the same PC in one session, game starts alternating (ABBA); field-retention: 1 game start a side, field 20 s after 5 s warm-up, deku 20 s after 5 s warm-up; each number the median of the runs, then of 1 report
How to reproduce: the farm's plan above (
@Beanstalkon this PR runs it again)645d58e1c09f9dFlags and settings
None: the plan varied no setting, switch or build option.
Markdown for the PR description
Code review (read only: nothing was built or run here). Verdict: comment. The code is correct as far as I can read it and I would approve it, but its own Must be tested asks for an 8 GB card and a 20-minute Hyrule Field walk, and nobody has posted those yet.
What it does for players: in long sessions, mostly while walking through terrain, the renderer stops keeping the finalized 260 x 260 terrain normal tiles after the game has reused their memory with CPU writes. Before this, within the texture budget, nothing ever let them go. It is one rule in
texture_frame's eviction pass, plus a game-free test.Checked, no problem found
frame_no - last >= need_unused(120 presents in a sweep) andtex->last_value > completedbefore it is chosen. After that it goes through the usualdestroy_later/forget_gpupath, so its images, views and descriptors have the same lifetime as any other eviction. Nothing new here.flushis!released && ..., so an obsolete texture is never written back. That is right: writing old texels over memory the game has reused is the crash thatwrite_back_staleand the CPU-runs clip exist to prevent.texture_touch:finalized_obsoletereadsmemory_written_runs(t.guest, t.guest_size, t.release_gen)for a pool texture (gpu < 1<<40). That is exactly whattexture_dirty(*tex, max(upload_gen, release_gen))would re-upload if the old Texture were sampled again. So evicting it and making it again over that storage (gpu_owned_sinceplus dirty runs: a full upload from memory) gives the same texels for the rewritten part. A different format or view at the same start is stilldrawn_over's case, andobsoleteis only computed when!overwritten.drawn_writebackwrites back throughnvn_gpu_write, which callsHOS_WRITTEN, so a tile written back after it was finalized also counts as CPU-written. But in that case memory holds the tile's own texels, so the texture made again from it is correct. No picture risk, at most an earlier eviction.continue,hold_live,keep_finalizedand its pre-count, the sweep'scontinue). The over-budget pass'sbreakstill applies.tests/test_vram_compact.pyfollows its line rather than being weakened.finalized_obsoletewithsrccutand runs them. It fails on main for the bug's reason: 1800 of 1800 within the budget, the cap's 744 over it. Undoing either!obsoletein the sweep line or the one inkeep_finalizedmakes a case fail. The 1/4, 1/2 and 16 KB cases pin the thresholds. What it doesn't model: page granularity (its stub hands back one byte run) and the generation itself (release_genis always 1, and the stub ignoresgen). That is fine for a pass-level test, sincememory_written_runsis main's code.third_party/change. No new switch.Evidence
The numbers are from the PR's own head
1c09f9d12against its base645d58e68. Same save, route, settings andRX_VK_TEST_ALLOWED_MB=5200, the same afternoon, and the log lines are pasted. One run per side, with the clean comparison in lap 1 only (131 vs 166 images, 60 vs 93 kept). The 24 GB laps in #1443 (update 3) are the first form of the fix plus the logging commit, not this head. The description says so. "Refs #1443", not "Fixes", is right. At lap 2 the game's VRAM is about the same on both sides (4762 vs 4752 MB) and the render-target blocks are 704 vs 800 MB, so this is one contributor to the issue's growth, and the issue stays open.Test Farm so far: Windows, AMD RX 9070 XT: pass (pictures within 3%; field VRAM peak 4917 vs 4918 MB; Deku interior peak +6% and p99 +5%, one run each, which I read as noise). vram8gb: still running. Neither is the 20-minute walk the description asks for. Everything else is NVIDIA Linux only. No Intel.
Suggestions (not blocking)
vk_texture.cpp:4580, a texture that was never written back. For the terrain tiles the untouched quarter is in memory already (drawn_writebackcovers them: up to 1 MB, levels > 1).finalized_obsoletehas no size cap and no write-back condition, though. So a finalized single-level target, or one over 1 MB, whose storage the game rewrote by 3/4 now goes within the budget, and if it is made again the remaining quarter comes from memory that never held its texels. Before this PR, within the budget, that quarter stayed in the kept image andtexture_touchpatched only the dirty runs. Over the budget the old pass let it go anyway, so the exposure is small. Still,|| (t.written_back_for == t.last_written)as a condition for the 3/4 line (and requiring all of it otherwise), or the 4 MB capkeep_finalizeduses, would keep the rule to what was measured.vk_texture.cpp:5273, the retired line's reason: passingoverwritten || obsoletemakesevict_reason::classifycount these asDRAWN_OVER("a render target another image has drawn over"). A reader of the "textures retired" line will look for a drawing that never happened. A reason of their own, or at least the comment atevict_reason.h:15, would help the next leak hunt.vk_texture.cpp:5225, cost: whenobsoleteis false,keep_finalized(tex)runsfinalized_obsoletea second time, and the pre-count loop runs it for every kept finalized texture on every pass (about 50 page lookups per tile). That is cheap, but caching it per pass (or readingobsoleteinside the lambda) would avoid walking the page generations three times.tex_frame_mswasn't measured.Overlaps and merge order
evict.push_backline and theRX_VK_EVICT_NO_FLUSHswitch. It also widens memory generations touint64_t(memory_written_runs,upload_gen,gpu_gen). Textual conflict on that line.finalized_obsoleteand the fixture'smemory_written_runsstub (uint32_t gen) need to follow the type there. Merge this first; 1339 rebases over it.vk_texture.cpp, but not this pass. No conflict, and no change of meaning: they change what is uploaded, not what is kept.destroy_laterinto that pool like any other eviction.Before approval: the Test Farm's vram8gb report, and someone's 20-minute walk over Hyrule Field on an 8 GB or smaller card (any OS; Windows preferred). Look at the ground for streaks and check that the "image blocks (render targets)" images stay flat. With that posted and clean, I'd approve as it is.
@ -4569,0 +4577,4 @@memory_written_runs(t.guest, t.guest_size, t.release_gen, written);uint64_t sum = 0;for (const auto& r : written) sum += r.second - r.first;return sum * 4 >= t.guest_size * 3;Suggestion: for a texture
drawn_writebacknever covered (single level, or over 1 MB), the quarter the game didn't rewrite comes back from memory that never held its texels when the texture is made again. Before this PR, within the budget, the kept image held it. Requiringt.written_back_for == t.last_writtenfor the 3/4 line (all of it otherwise), orkeep_finalized's 4 MB cap, would keep the rule to the tiles that were measured.@ -5209,0 +5222,4 @@// held is gone from that memory on the console too, and a draw that sampled it would take the memory's contents over its// texels (texture_touch). Within the budget the pass is a sweep that lets go of nothing the GPU did not draw over, and these// were never drawn over by a target: the game's own writes to the memory they stood in kept every one for good (issue 1443)const bool obsolete = !overwritten && finalized_obsolete(*tex);Nit: when this is false,
keep_finalized(tex)below computesfinalized_obsoleteagain, and the pre-count loop computes it once more for each kept finalized texture. That is cheap (about 50 page lookups per tile), but it could be computed once per texture per pass.@ -5253,3 +5271,3 @@// tiled pattern over the Deku Tree's planks and Hyrule Field's path, with a 16 GB card's textures at the 6 GB budget// (issue 386))evict.push_back({tex, !overwritten && tex->gpu_written && !tex->released && flushable(*tex), shifted, overwritten});evict.push_back({tex, !overwritten && tex->gpu_written && !tex->released && flushable(*tex), shifted, overwritten || obsolete});Nit:
overwritten || obsoletemakesevict_reason::classifycount these as DRAWN_OVER in the "textures retired" line, though nothing was drawn over them. A reason of their own would make the next leak hunt easier.Brought up to the ready-for-review standard (#1454): the checklist's Everyone list has two new boxes, added here from what this PR's description and comments show (please check they match what you know).
fix(vk): a finalized render target goes once the game has written over its storageto WIP: fix(vk): a finalized render target goes once the game has written over its storageThank you for adding the two boxes; they match what I know. Under the ready-for-review list (#1454) I have moved this back to WIP until one cost is measured and named in Risks: the texture pass's own time (
tex_frame_ms), main against this head; I am answering the review separately. I will tick the box and mark it ready again when that number is in.Verdict: Comment: would approve once out of WIP, because the code still reads correct and the Test Farm's one failure looks like noise from a single run, not this change. Two runs are still missing before I'd approve: the
tex_frame_msnumber you are measuring now, and the 20-minute Hyrule Field walk on an 8 GB or smaller card that this PR's Must be tested asks for.For players: in long sessions, mostly while walking through terrain, the renderer stops keeping finalized 260 x 260 terrain tiles once the game has reused their memory, so render-target VRAM stops climbing lap after lap. The code is better for it: one rule in
texture_frame's pass and a game-free test. No regression is shown.Reviewed:
1c09f9d1against645d58e6(its merge base). The head is unchanged since my last review, sojj interdiffis empty. I readgpu/vk/vk_texture.cpp's changed pass in full again, along with the code it relies on: the pre-count loop,keep_finalized, the sweep cadence (dead_every,sweep = total_bytes <= budget) andmemory_written_runs. I skimmedtests/test_target_retention.*andtests/test_vram_compact.py. I read the three Test Farm reports in full. Nothing was run (a code review).Fixed in this review: nothing. Both commits rebase onto current main (
e8e92f97) with no conflict at all, and the interdiff against this head is empty. I didn't push that rebase, for three reasons: it resolves nothing, maintainer edits are off on this branch, and you are measuring on this head. Rebase it when you push thetex_frame_msnumbers (CONTRIBUTING's point 7).Checked and fine: everything in my earlier review still holds (in-flight lifetime, no write-back of an obsolete texture, "written over" meaning what
texture_touchre-uploads, the hold, keep cap and sweep handled consistently, the test failing on main for the bug's reason). The added CPU work is bounded.finalized_obsoletereturns before reading memory for anything that isn't a released, GPU-drawn pool texture of 32 KB or more. For a tile it walks about 80 page generations, up to three times a pass. With the 20 to 130 kept tiles the logs show, that comes to tens of microseconds a pass, not milliseconds. Main's newer commits don't touch this pass (texture_aliveat :2711 is the only change tovk_texture.cpp), and the description has the TL;DR and Breakdown that #1470 now asks for.The Test Farm's failure: noise, not this PR (my reading)
The Windows target failed on one rule in one report: RTX 4070 (futondestroyer), Hyrule Field frame time p99 18.27 to 23.19 ms (+26.9%). I don't read it as a regression:
A rerun with more runs a side on the 4070 (
@Beanstalk,runs: 3for field-retention) would clear thebeanstalk: failedlabel. I don't hold approval on it. Yourtex_frame_msmeasurement is the direct check of this PR's own cost, so it matters more than the farm's p99 here.Suggestions (unchanged from my last review, the author may take or leave)
vk_texture.cpp:4580: limit the 3/4 rule to texturesdrawn_writebackcovered (t.written_back_for == t.last_written), or applykeep_finalized's 4 MB cap, so the quarter the game didn't rewrite still has its texels in memory when the texture is made again.vk_texture.cpp:5273: give these their ownevict_reasoninstead of counting them as DRAWN_OVER.vk_texture.cpp:5225: computefinalized_obsoleteonce per texture per pass.Testing: what exists: the author's two-lap Linux run (RTX 4090 at a 6 GB allowance, 131 against 166 images in the clean lap), the game-free test, the suite by name against main, and the Test Farm. The farm passed on the RTX 3070 8 GB, passed on the RX 9070 XT (pictures 0 diff, VRAM flat) and failed on the RTX 4070 on field p99 (noise, above). The farm's plan can't count render-target images or show long-session growth, as it says itself. Still missing, and holding approval:
tex_frame_msmain against this head (the unticked Nothing gets worse box), and a 20-minute Hyrule Field walk on an 8 GB or smaller card (Windows preferred) with the "image blocks (render targets)" line staying flat and no streaked ground. No Intel run either. That isn't required, since nothing here is vendor-specific.Overlaps and merge order: #1339 (WIP) touches the same
evict.push_backline and widens memory generations touint64_t. Merge this first, and 1339 rebases over it (finalized_obsoleteand the fixture'smemory_written_runsstub follow the type). #1162 and #1223 touchvk_texture.cppoutside this pass: no conflict.WIP: fix(vk): a finalized render target goes once the game has written over its storageto fix(vk): a finalized render target goes once the game has written over its storage