perf(vk): a new ASTC texture's blocks are hashed at upload only when a finalized one has its shape #1281
No reviewers
Labels
No labels
area/accessibility
area/audio
area/input
area/install
area/menu
area/renderer
area/saves
area/tooling
area/upscaling
beanstalk: approved
beanstalk: failed
beanstalk: needs approval
beanstalk: running
beanstalk: tested
ci: approved
good first issue
gpu/amd
gpu/igpu
gpu/intel
gpu/nvidia
help wanted
kind
bug
kind
chore
kind
docs
kind
feature
kind
performance
kind
question
needs measurements
platform/linux
platform/macos
platform/steam-deck
platform/windows
preview
priority
critical
priority
high
priority
low
regression
risk
high
risk
low
risk
medium
status
blocked
status
confirmed
status
duplicate
status
invalid
status
needs-info
status
wontfix
ci: approved
No milestone
No project
No assignees
3 participants
Notifications
Due date
No due date set.
Dependencies
No dependencies set
Reference
MagicBeanDev/Longshot!1281
Loading…
Reference in a new issue
No description provided.
Delete branch "TrashPanda/Longshot:perf/astc-hash-later"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Type: perf · Risk: medium · Tested on: Linux, RTX 4070 Ti · Needs: Windows build, AMD (RADV), Steam Deck
Why
When an area loads, more new textures than before show up a frame or more late. Since #1222, every new ASTC texture's
blocks are hashed on the replay thread at its upload, so a revisit can reuse an earlier decode. That hashing took about
0.55 s of the replay thread over a Kokiri tour, inside the 40 ms per-frame upload budget, so the budget runs out sooner
in a load and more textures wait for a later frame. Only some of those hashes can ever find a match. See #1069.
What changed
A new texture's blocks are hashed at its upload only when a finalized texture of the same shape is listed, since no
other can take an image there. The rest are hashed on a background thread, and the hash is kept only if the CPU did not
write the texture's storage between the upload and the hash. That way, when the game finalizes the texture it can still
be listed as a source for a later revisit.
gpu/vk/astc_reuse.h:Freedcounts its entries by shape (has_shape, withlistandunlisttaking the shape),and
hash_storage()hashes a storage in the same 64 MB pieces as the upload, so both places give the same hash.gpu/vk/vk_texture.cpp,reuse_upload(): hashes now whenfreed.has_shape()is true. Otherwise it queues thestorage (a live guest-window pointer, its size and the memory generation) for
reuse_hash_queue()'s thread.Storage that is sparse, virtual, not contiguous or shadowed by the capture is hashed at upload, as before.
reuse_hash_frame(), called fromtexture_frame(): takes the hashes that came back. It keeps a hash only iftexture_dirty()finds no writes since the upload's generation, and lists a texture the game finalized before itshash came (
list_when_hashed). A texture that goes or is uploaded again in the meantime drops its job inreuse_drop()(the result is matched by a sequence number, never by a pointer alone).docs/development.md: theRX_VK_ASTC_HASH_LATERswitch.Measurements
Verdict: helps, less than hoped. The replay thread hashes about 100 ms less per tour and about 23% fewer new textures
wait a frame, with no loss of reuse. Most of the remaining upload-time hashing is for textures that do find a match;
the decode it replaces runs on the GPU, so that cost stays.
Hardware: Ubuntu 26.04.1, kernel 7.0.0-43, RTX 4070 Ti 12 GB, NVIDIA 595.99.02, Ryzen 7 9800X3D, 32 GB
Scene: Kakariko and the Market (child), four visits each; 2560x1440, defaults,
RX_VK_VRAM_MB=8192Method: the last
vk: decoded ASTC used againandleft for a later frame ... N so farlines and the slowestvk: statusframe in each run'slongshot_log.txt. Three runs a side, interleaved, each alone on the PC throughscripts/job_queue.pytiming mode; the median of the three.How to reproduce:
RX_VK_VRAM_MB=8192 python -I scripts/memtour.py <game> --repo-save kokiri-start --dests 2,3,2,3,2,3,2,3 --dwell 25 --settle 20, with and without--env RX_VK_ASTC_HASH_LATER=0Baseline: the same build with
RX_VK_ASTC_HASH_LATER=0, which hashes everything at upload as main does. It's anin-session A/B on one build instead of two builds.
824fa2c4,RX_VK_ASTC_HASH_LATER=0824fa2c4An earlier A/B, before the fallback for storage that can't be hashed later, gave the same picture: 531 to 426 ms
hashed and 662 to 385 left for a later frame. Its reuse count was lower (6,657 to 5,814), which is why the fallback
was added.
The last such line of the second run with the change:
Flags and settings
One new switch, on by default.
RX_VK_ASTC_HASH_LATER=0only to compareTesting
Tested:
python -I -B tests/run_all.pyon Linux: 652 passed, 21 failed, 21 skipped. The same 21 also fail on thisbranch's base without the change: game-file caches, Linux preflight, DLSS and others not touched here. Six tours in
the game as above, with no crash, hang or visible texture problem.
Not tested: the Windows build (llvm-mingw); only
std::threadandstd::condition_variableare new to this code.Other GPUs, and a Steam Deck, where the extra thread competes with the game for four cores.
Must be tested:
vk: decoded ASTC used againline to show textures hashed later and kept (Test Farm: @Beanstalk windows, the ASTC hash line after a few area changes)
RX_VK_ASTC_HASH_LATER=0(Test Farm: @Beanstalk steamdeck)Risks
What could break: a hash that no longer matches the decoded image, if a write to the storage got past the
generation check, would let a later revisit copy the wrong image. That would show as a wrong texture on a revisit.
texture_dirty()is the same check the texture cache uses to decide on re-uploads.Turning it off:
RX_VK_ASTC_HASH_LATER=0, orRX_VK_ASTC_REUSE=0for no reuse at all.Compatibility: nothing changes.
Issues
See #1069
Checklist
Everyone
type(scope): description) of the status line's type.can't be run by hand is in Testing's Not testable by hand, with why.
in Risks. Not ticked yet: Testing says the extra thread's CPU cost on the Steam Deck is not measured, and Risks does not name it.
main.python -I -B tests/run_all.pypasses (any failure that also fails onmainis named).third_party/untouched (a new folder with aPROVENANCE.mdinstead).When it applies
Assisted-by:/Co-Authored-By:trailers.docs/tested/for each new measured run (format indocs/tested-configurations.md).tests/that reads changed source updated in this PR, not weakened.Tested in game: Linux (Ubuntu 26.04.1), RTX 4070 Ti, NVIDIA 595.99.02: six Kokiri tours (Kakariko and the Market, child, four visits each), three with RX_VK_ASTC_HASH_LATER=0 and three without, interleaved. No crash or wrong texture; replay-thread hashing 554 -> 452 ms, new textures left for a later frame 678 -> 519 (medians), reuse unchanged.
@Beanstalk windows and steamdeck: build this PR, then from the repository save kokiri-start teleport Kakariko, the Market, Kakariko, the Market (scripts/memtour.py --repo-save kokiri-start --dests 2,3,2,3,2,3,2,3). Expect the last 'vk: decoded ASTC used again' line in longshot_log.txt to show textures hashed later and kept with few or none written meanwhile, no wrong textures on the revisits, and on the Deck no more slow frames than the same tour with RX_VK_ASTC_HASH_LATER=0.
@TrashPanda only members of
BigBeanscan start the farm. One of them can approve this request by reacting 👍 to it or to this comment.A new texture is hashed on the replay thread only when a finalized one of its shape is already listed. The rest are hashed later, and the hash is kept only when nothing wrote the storage after the upload. I read head
824fa2c4. It applies onto current main. This account opened the pull request, so this is not an approval.The later job keeps the guest pointer, the size and the memory generation.
texture_frametakes the result only when that job is still the texture's, then drops it iftexture_dirtysaw a write. A texture that goes, or is uploaded again, drops the job before the result is applied. Sparse, virtual and captured storage is still hashed at the upload. The same 64 MB pieces are hashed in both places.RX_VK_ASTC_HASH_LATER=0is the old path.The two runs in Must be tested are still open. Windows, any GPU: a few area changes, and the
vk: decoded ASTC used againline showing textures hashed later and kept. A Steam Deck: the same tour no slower thanRX_VK_ASTC_HASH_LATER=0. Only a BigBean can start those.Brought up to the ready-for-review standard (#1454): the checklist's Everyone list has two new boxes, added here from what this PR's description and comments show (please check they match what you know).
View command line instructions
Manual merge helper
Use this merge commit message when completing the merge manually.
Checkout
From your project repository, check out a new branch and test the changes.