WIP: perf(vk): a new ASTC texture's blocks are hashed at upload only when a finalized one has its shape #1281
No reviewers
Labels
No labels
area/accessibility
area/audio
area/input
area/install
area/menu
area/renderer
area/saves
area/tooling
area/upscaling
beanstalk: approved
beanstalk: failed
beanstalk: needs approval
beanstalk: running
beanstalk: tested
ci: approved
good first issue
gpu/amd
gpu/igpu
gpu/intel
gpu/nvidia
help wanted
kind
bug
kind
chore
kind
docs
kind
feature
kind
performance
kind
question
needs measurements
platform/linux
platform/macos
platform/steam-deck
platform/windows
preview
priority
critical
priority
high
priority
low
regression
risk
high
risk
low
risk
medium
status
blocked
status
confirmed
status
duplicate
status
invalid
status
needs-info
status
wontfix
ci: approved
No milestone
No project
No assignees
4 participants
Notifications
Due date
No due date set.
Dependencies
No dependencies set
Reference
MagicBeanDev/Longshot!1281
Loading…
Reference in a new issue
No description provided.
Delete branch "TrashPanda/Longshot:perf/astc-hash-later"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Type: perf · Risk: medium · Tested on: Linux, RTX 4070 Ti · Needs: Windows build, AMD (RADV), Steam Deck
Why
When an area loads, more new textures than before show up a frame or more late. Since #1222, every new ASTC texture's
blocks are hashed on the replay thread at its upload, so a revisit can reuse an earlier decode. That hashing took about
0.55 s of the replay thread over a Kokiri tour, inside the 40 ms per-frame upload budget, so the budget runs out sooner
in a load and more textures wait for a later frame. Only some of those hashes can ever find a match. See #1069.
What changed
A new texture's blocks are hashed at its upload only when a finalized texture of the same shape is listed, since no
other can take an image there. The rest are hashed on a background thread, and the hash is kept only if the CPU did not
write the texture's storage between the upload and the hash. That way, when the game finalizes the texture it can still
be listed as a source for a later revisit.
gpu/vk/astc_reuse.h:Freedcounts its entries by shape (has_shape, withlistandunlisttaking the shape),and
hash_storage()hashes a storage in the same 64 MB pieces as the upload, so both places give the same hash.gpu/vk/vk_texture.cpp,reuse_upload(): hashes now whenfreed.has_shape()is true. Otherwise it queues thestorage (a live guest-window pointer, its size and the memory generation) for
reuse_hash_queue()'s thread.Storage that is sparse, virtual, not contiguous or shadowed by the capture is hashed at upload, as before.
reuse_hash_frame(), called fromtexture_frame(): takes the hashes that came back. It keeps a hash only iftexture_dirty()finds no writes since the upload's generation, and lists a texture the game finalized before itshash came (
list_when_hashed). A texture that goes or is uploaded again in the meantime drops its job inreuse_drop()(the result is matched by a sequence number, never by a pointer alone).main@876dc360droppedRX_VK_ASTC_HASH_LATER, since #1309 deleted the switches that only turn a change off (andRX_VK_ASTC_REUSEwith them).Measurements
Verdict: helps, less than hoped. The replay thread hashes about 100 ms less per tour and about 23% fewer new textures
wait a frame, with no loss of reuse. Most of the remaining upload-time hashing is for textures that do find a match;
the decode it replaces runs on the GPU, so that cost stays.
Hardware: Ubuntu 26.04.1, kernel 7.0.0-43, RTX 4070 Ti 12 GB, NVIDIA 595.99.02, Ryzen 7 9800X3D, 32 GB
Scene: Kakariko and the Market (child), four visits each; 2560x1440, defaults,
RX_VK_VRAM_MB=8192Method: the last
vk: decoded ASTC used againandleft for a later frame ... N so farlines and the slowestvk: statusframe in each run'slongshot_log.txt. Three runs a side, interleaved, each alone on the PC throughscripts/job_queue.pytiming mode; the median of the three.How to reproduce:
RX_VK_VRAM_MB=8192 python -I scripts/memtour.py <game> --repo-save kokiri-start --dests 2,3,2,3,2,3,2,3 --dwell 25 --settle 20, with and without--env RX_VK_ASTC_HASH_LATER=0Baseline: the same build with
RX_VK_ASTC_HASH_LATER=0, which hashed everything at upload as main does. It's anin-session A/B on one build instead of two builds. These runs are from
824fa2c4, before the rebase dropped that switch; the Steam Deck run in Must be tested comparesmainwith this PR instead.824fa2c4,RX_VK_ASTC_HASH_LATER=0824fa2c4An earlier A/B, before the fallback for storage that can't be hashed later, gave the same picture: 531 to 426 ms
hashed and 662 to 385 left for a later frame. Its reuse count was lower (6,657 to 5,814), which is why the fallback
was added.
The last such line of the second run with the change:
Flags and settings
None. (The measurements'
RX_VK_ASTC_HASH_LATERwas dropped in the rebase, as #1309 asks.)Testing
Tested:
python -I -B tests/run_all.pyon Linux at72bf99a9(rebased onmain@876dc360): 713 passed, 45 failed, 22 skipped. All 45 also fail onmain@876dc360run alone, excepttest_perf_report, which passes here run alone too (it timed out under the 16-at-once run).gpu/vk/vk_texture.cppcompiles (clang++ -fsyntax-onlywith the renderer's flags). Six tours inthe game as above, with no crash, hang or visible texture problem.
Not tested: the Windows build (llvm-mingw); only
std::threadandstd::condition_variableare new to this code.Other GPUs, and a Steam Deck, where the extra thread competes with the game for four cores.
Must be tested:
vk: decoded ASTC used againline to show textures hashed later and kept (Test Farm: @Beanstalk windows, the ASTC hash line after a few area changes)
mainagainst this PR on a Kakariko Village and Market route by the farm's teleports, expecting the replay thread's upload and hash time lower and no more slow frames thanmain(Test Farm: @Beanstalk steamdeck, Kakariko and the Market, main against this PR, thevk: decoded ASTC used againline and slow frames)Risks
What could break: a hash that no longer matches the decoded image, if a write to the storage got past the
generation check, would let a later revisit copy the wrong image. That would show as a wrong texture on a revisit.
texture_dirty()is the same check the texture cache uses to decide on re-uploads.Cost: the textures not hashed at upload are hashed on one more thread, about the same bytes as before in all (4.7 of 6.2 GB moved off the replay thread over the tour). On a 4-core Steam Deck that thread competes with the game's; not measured yet (Must be tested).
Turning it off: revert the commit.
Compatibility: nothing changes.
Issues
See #1069
Checklist
Everyone
type(scope): description) of the status line's type.can't be run by hand is in Testing's Not testable by hand, with why.
in Risks. Not ticked yet: Risks names the extra thread's cost on the Steam Deck, but it is not measured until the Deck run.
main.python -I -B tests/run_all.pypasses (any failure that also fails onmainis named).third_party/untouched (a new folder with aPROVENANCE.mdinstead).When it applies
Assisted-by:/Co-Authored-By:trailers.docs/tested/for each new measured run (format indocs/tested-configurations.md).tests/that reads changed source updated in this PR, not weakened.Tested in game: Linux (Ubuntu 26.04.1), RTX 4070 Ti, NVIDIA 595.99.02: six Kokiri tours (Kakariko and the Market, child, four visits each), three with RX_VK_ASTC_HASH_LATER=0 and three without, interleaved. No crash or wrong texture; replay-thread hashing 554 -> 452 ms, new textures left for a later frame 678 -> 519 (medians), reuse unchanged.
@Beanstalk windows and steamdeck: build this PR, then from the repository save kokiri-start teleport Kakariko, the Market, Kakariko, the Market (scripts/memtour.py --repo-save kokiri-start --dests 2,3,2,3,2,3,2,3). Expect the last 'vk: decoded ASTC used again' line in longshot_log.txt to show textures hashed later and kept with few or none written meanwhile, no wrong textures on the revisits, and on the Deck no more slow frames than the same tour with RX_VK_ASTC_HASH_LATER=0.
@TrashPanda only members of
BigBeanscan start the farm.One of them can approve this request by reacting 👍 to it or to this comment.This request was for824fa2c, and the PR has changed since (72bf99a), so a 👍 no longer approves it. Comment@Beanstalkagain to ask for the new commit.A new texture is hashed on the replay thread only when a finalized one of its shape is already listed. The rest are hashed later, and the hash is kept only when nothing wrote the storage after the upload. I read head
824fa2c4. It applies onto current main. This account opened the pull request, so this is not an approval.The later job keeps the guest pointer, the size and the memory generation.
texture_frametakes the result only when that job is still the texture's, then drops it iftexture_dirtysaw a write. A texture that goes, or is uploaded again, drops the job before the result is applied. Sparse, virtual and captured storage is still hashed at the upload. The same 64 MB pieces are hashed in both places.RX_VK_ASTC_HASH_LATER=0is the old path.The two runs in Must be tested are still open. Windows, any GPU: a few area changes, and the
vk: decoded ASTC used againline showing textures hashed later and kept. A Steam Deck: the same tour no slower thanRX_VK_ASTC_HASH_LATER=0. Only a BigBean can start those.Brought up to the ready-for-review standard (#1454): the checklist's Everyone list has two new boxes, added here from what this PR's description and comments show (please check they match what you know).
Farm readiness review for
824fa2c4: the branch currently conflicts with main and the added hash cost remains unmeasured in the checklist. Please rebase and refresh the request for the new head. Describe the Kakariko/Market route using farm teleports with the relevant switch on and off, and compare measured upload/hash cost and slow frames on the same Deck.824fa2c4bd72bf99a96eperf(vk): a new ASTC texture's blocks are hashed at upload only when a finalized one has its shapeto WIP: perf(vk): a new ASTC texture's blocks are hashed at upload only when a finalized one has its shape@futondestroyer thanks. Rebased onto
main@876dc360; head is now72bf99a9. The rebase also dropsRX_VK_ASTC_HASH_LATER: #1309 deleted the switches that only turn a change off (RX_VK_ASTC_REUSEamong them), so the A/B is nowmainagainst this PR. The extra thread's cost is named in Risks, and I put this back to WIP until the Deck run measures it. Fresh request below.@Beanstalk steamdeck: main against this PR @
72bf99a9. From the repository save kokiri-start, teleport Kakariko Village, the Market, Kakariko, the Market (the farm's teleports, 25 s in each). Measure slow frames and frame time (p99) in each area, and compare the last 'vk: decoded ASTC used again' line (MB and ms of blocks hashed at upload; on this PR also 'hashed later, kept, written meanwhile') and 'left for a later frame' between the two. Expect less hashing on the replay thread and no more slow frames than main. Also windows (any GPU), this PR only: the same route, expecting the ASTC line to show textures hashed later and kept, and no crash.@TrashPanda only members of
BigBeanscan start the farm. One of them can approve this request by reacting 👍 to it or to this comment.View command line instructions
Checkout
From your project repository, check out a new branch and test the changes.