perf(vk): cheaper replay: uniform-only descriptor rebuilds and flat program picks #2644
Open
hckrman101 wants to merge 3 commits from
hckrman101/perf/replay-cheaper into main
pull from: hckrman101/perf/replay-cheaper
merge into: MagicBeanDev:main
MagicBeanDev:main
MagicBeanDev:github/pr/2893
MagicBeanDev:github/pr/2890
MagicBeanDev:github/pr/2887
MagicBeanDev:github/pr/2878
MagicBeanDev:github/pr/2875
MagicBeanDev:github/pr/2874
MagicBeanDev:github/pr/2873
MagicBeanDev:github/pr/2871
MagicBeanDev:github/pr/2756
MagicBeanDev:github/pr/2897
MagicBeanDev:github/pr/2896
MagicBeanDev:github/pr/2718
MagicBeanDev:github/pr/2842
MagicBeanDev:github/pr/2691
MagicBeanDev:github/pr/2895
MagicBeanDev:github/pr/2743
MagicBeanDev:github/pr/2765
MagicBeanDev:github/pr/2708
MagicBeanDev:github/pr/2732
MagicBeanDev:github/pr/2755
MagicBeanDev:github/pr/2817
MagicBeanDev:github/pr/2876
MagicBeanDev:release-please--branches--main
MagicBeanDev:release-please--branches--main--release-notes
MagicBeanDev:github/pr/2778
MagicBeanDev:github/pr/2888
MagicBeanDev:github/pr/2866
MagicBeanDev:github/pr/2877
MagicBeanDev:github/pr/2800
MagicBeanDev:github/pr/2829
MagicBeanDev:github/pr/2850
MagicBeanDev:github/pr/2889
MagicBeanDev:github/pr/2882
MagicBeanDev:github/pr/2860
MagicBeanDev:github/pr/2849
MagicBeanDev:github/pr/2886
MagicBeanDev:github/pr/2794
MagicBeanDev:github/pr/2840
MagicBeanDev:github/pr/2830
MagicBeanDev:github/pr/2834
MagicBeanDev:github/pr/2719
MagicBeanDev:github/pr/2742
MagicBeanDev:github/pr/2741
MagicBeanDev:github/pr/2820
MagicBeanDev:github/pr/2771
MagicBeanDev:github/pr/2859
MagicBeanDev:github/pr/2858
MagicBeanDev:github/pr/2823
MagicBeanDev:github/pr/2757
MagicBeanDev:github/pr/2769
MagicBeanDev:github/pr/2844
MagicBeanDev:github/pr/2843
MagicBeanDev:github/pr/2673
MagicBeanDev:github/pr/2855
MagicBeanDev:github/pr/2739
MagicBeanDev:github/pr/2807
MagicBeanDev:github/pr/2851
MagicBeanDev:github/pr/2789
MagicBeanDev:github/pr/2814
MagicBeanDev:github/pr/2818
MagicBeanDev:github/pr/2774
MagicBeanDev:github/pr/2772
MagicBeanDev:github/pr/2750
MagicBeanDev:github/pr/2786
MagicBeanDev:github/pr/2785
MagicBeanDev:github/pr/2782
MagicBeanDev:github/pr/2779
MagicBeanDev:github/pr/2775
MagicBeanDev:github/pr/2706
MagicBeanDev:github/pr/2766
MagicBeanDev:github/pr/2747
MagicBeanDev:github/pr/2759
MagicBeanDev:github/pr/2745
MagicBeanDev:github/pr/2740
MagicBeanDev:github/pr/2733
MagicBeanDev:github/pr/2725
MagicBeanDev:github/pr/2727
MagicBeanDev:github/pr/2728
MagicBeanDev:github/pr/2729
MagicBeanDev:github/pr/2721
MagicBeanDev:github/pr/2722
MagicBeanDev:github/pr/2715
MagicBeanDev:github/pr/2714
MagicBeanDev:github/pr/2707
MagicBeanDev:github/pr/2703
MagicBeanDev:github/pr/2705
MagicBeanDev:github/pr/2697
MagicBeanDev:github/pr/2679
MagicBeanDev:github/pr/2676
MagicBeanDev:github/pr/2669
MagicBeanDev:docs/what-hookshot-connects-to
MagicBeanDev:fix/hdr-encode-test
MagicBeanDev:fix/main-tests
MagicBeanDev:fix/play-cache-files
MagicBeanDev:perf/game-barrier-order
MagicBeanDev:feat/heap-lazy
MagicBeanDev:feat/texture-detail
MagicBeanDev:perf/astc-opaque-bc1
MagicBeanDev:fix/fs-dir-lock
MagicBeanDev:fix/framegen-native-kept
MagicBeanDev:fix/update-no-downgrade
MagicBeanDev:port/k10-gcn-f16-mirrors
MagicBeanDev:fix/framegen-mult-cap
MagicBeanDev:chore/release-as-2.1.1
MagicBeanDev:perf/pcache-load-switch
MagicBeanDev:fix/prepare-progress-log
MagicBeanDev:fix/update-closes-stale-game
MagicBeanDev:fix/hdr-scaled-sampled
MagicBeanDev:perf/first-start-preset
MagicBeanDev:perf/zero-upload
MagicBeanDev:fix/vcore-escape-2s
MagicBeanDev:test/fake-gh-atomic
MagicBeanDev:perf/batch-idle-flush
MagicBeanDev:test/finish-stub-a
MagicBeanDev:chore/record-2.1.0
MagicBeanDev:fix/main-red-tests
MagicBeanDev:fix/test-display-startup-driver-cache
MagicBeanDev:feat/mesh-shader-mode
MagicBeanDev:fix/menu-probes-at-startup
MagicBeanDev:fix/fps-counter-classic
MagicBeanDev:fix/test-log-batch-flake
MagicBeanDev:fix/narrow-screen-hud
MagicBeanDev:ci/prereleases
MagicBeanDev:release/v2.0.15
MagicBeanDev:hotfix/v2.0.15
MagicBeanDev:fix/present-ahead-replay
MagicBeanDev:release/v2.0.13
MagicBeanDev:fix/bracket-paths-2013
MagicBeanDev:fix/hdr-overlays-2013
MagicBeanDev:fix/fps-counter-scale-2013
MagicBeanDev:fix/stick-seen-at-rest
MagicBeanDev:hotfix/menu-fits-screen
MagicBeanDev:fix/f1-always-opens-menu
MagicBeanDev:fix/ini-keep-lines
MagicBeanDev:claude/cranky-pike-7cf247
MagicBeanDev:perf/texture-answer-flat
MagicBeanDev:fix/stamp-test-modules
MagicBeanDev:fix/appimage-fonts
MagicBeanDev:fix/display-awake
MagicBeanDev:release/v2.0.12
MagicBeanDev:fix/discord-windows
MagicBeanDev:rc/v2.0.12-tests
MagicBeanDev:fix/dlss-keep-feature
MagicBeanDev:rc/v2.0.12
MagicBeanDev:button-icons
No reviewers
Labels
Clear labels
accessibility
Barrier affecting people with disabilities
autorelease: pending
autorelease: tagged
bug
Something isn't working
dependencies
Pull requests that update a dependency file
documentation
Improvements or additions to documentation
duplicate
This issue or pull request already exists
enhancement
New feature or request
github_actions
Pull requests that update GitHub Actions code
good first issue
Good for newcomers
help wanted
Extra attention is needed
invalid
This doesn't seem right
question
Further information is requested
testing wanted
wontfix
This will not be worked on
No labels
accessibility
autorelease: pending
autorelease: tagged
bug
dependencies
documentation
duplicate
enhancement
github_actions
good first issue
help wanted
invalid
question
testing wanted
wontfix
Milestone
Clear milestone
No items
No milestone
Assignees
Clear assignees
No assignees
2 participants
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".
No due date set.
Dependencies
No dependencies set
Reference
MagicBeanDev/Hookshot!2644
Loading…
Reference in a new issue
No description provided.
Delete branch "hckrman101/perf/replay-cheaper"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
After #2602 the frame is serial and the replay of the game's command stream is its longest stretch (~7 ms a frame in Hyrule Field on an RTX 3090 / 7900X), so CPU time taken off the replay thread is frame time. This takes three pieces of per-draw work off it; none changes what is drawn.
Not tested in the game yet (code reading, existing profiles, a build and the checks only): see the last section for what to measure.
Where the time goes (existing profile, Hyrule Field, replay thread)
HOS_SAMPLE_ALLprofile of the #2602 build, 951 samples underreplay(~6.4 ms a frame, one sample ~6.7 us):drawbuild_descriptorsprogram_forbind_descriptors(50 in the driver's push)dispatchStageand descriptor (answer_handle), the stages' descriptor lists inbuild_descriptors.124 draws kept their neighbour's descriptorsof 1262: OoT binds a uniform buffer for nearly every draw, which movesbind_gen, so nine draws in ten build their whole set again. Yet the picks' own log says a third of the graphics picks areby the last picks' textures, a uniform buffer bound since(2.36 of 7.2 million), and another quarterwith other textures.The changes
1. A draw after a uniform buffer's bind builds its uniform ranges alone (
RX_VK_DESC_PARTIAL=0: off)rebuild_uniform_rangeswrites the uniform ranges (throughuniform_range, as the full build does) over that build;bind_descriptorsthen pushes what differs, as it already does. The storage buffers, textures, samplers and the list of sampled textures are the last build's.desc_rest_stands):bind_descriptors(desc_built_prog; a dispatch's build over it is seen throughdw().prog);uniform_only_of, asked once a program). Those words are the stages'ssbo_genandtex_genand nothing else of the state, so no uniform buffer's contents can change them;desc_slots_kept, as for a draw that keeps its whole set), the texture epoch,gpu_bind_stamp()(pools and mirrors: what a buffer binding resolves to) and a new generation that moves when a texture becomes GPU-written (what the build kept for the pass hazards). The last three are read before the build begins, so a build that moves one itself is followed by a whole build.RX_VK_DESC_REUSE=0/never, and with the switches that log, count or widen each binding (RX_VK_DRAW_STATS,RX_VK_WATCH_PROG,RX_VK_SSBO_STRETCH, ...).descriptors(1.35-1.4 ms today). Up to a third of the graphics draws qualify by the picks' log; for those the build skipstexture_view_of(28 samples), the storage buffers'gpu_buffer_bind, the handle reads and most of the function's own 67 samples. Honest low end: an earlier per-stage texture memo measured nothing above the noise, andRX_VK_PROF_FINE's large "textures" time is mostly its own clock reads.a frame: N draws kept their neighbour's descriptors (...), M more built their uniform ranges alone.2. Program picks read their answers' handles from a flat list, once
AnswerSource: the stage and the c2 offsets, Eden's TexturePair shifts) and itsanswer_stage_mask.pick_handlesreads the handles straight from the state; anything that is not a c2 word of the answer's own stage is still asked of the program. A pick "with other textures" checked each handle twice (answer_handles, thenstill_valid);still_valid_handlesjudges the handles already read.driver_bank_word(s, 2, stage, offset)is whatstage_cbuf_wordcalls for c2), the same comparison, the same verdict.tests/test_program_answer_domains.pyholdspick_handlesandstill_valid_handlestoanswer_handlesandstill_validon the real descriptor types for every kind of answer. No switch: identical output.programs(0.7-1.1 ms).answer_handleis 21 samples, 17 of them on two loads of the stage's descriptor;still_valid17; the mask walked the answers a third time.3.
build_descriptorsasks for its writer oncedw()is a call and a thread-local read; the build asked it about ten times a descriptor (12 samples underbuild_descriptors). Expected: ~0.07 ms. Identical output.In all: an estimated 0.3-0.5 ms of the ~6.8 ms replay (5-8 %). Estimates from sample counts, not measurements.
Switches
RX_VK_DESC_PARTIAL=0: every draw builds its whole set, as before (indocs/development.md). Changes 2 and 3 have no switch: the same output by construction.Looked at and left for later
mesh_program_forevery time (two locks, three hash lookups). Only ~25-40 of the field's 1262 draws are mesh draws, so a picks cache for them is worth ~0.01 ms there; written and dropped from this PR.Shader::Info), and a flat table for the picks.memory_end_submission(0.38 ms:cpu_words_merge, the sort inshadows_forget_written): left alone, next to the capture work of #2622.Tests
tests/test_desc_uniform_only.py: which programs may, what refuses a draw, what is written, and the conditions indraw(), on the realProgram/ Eden descriptor types.tests/test_program_answer_domains.py(the flat sources against the program's own reads),tests/test_pick_tex_gen.py(the mask kept with the pick).hookshot-build --vk-only: builds, ABI check ok.python -I -B tests/run_all.py(Linux): 444-446 pass, 25 skipped; 21 fail, the same 21 on unchanged main (a016b7a6) with the same headers, none in files this PR touches (audio, control, docker packages, menu i18n,vk_memory.cppfixtures, ...).tests/test_release_scripts.pyfailed once in a full run (its two-uploads-at-once case) and passes alone.Not yet tested in the game
Nothing here was run in the game. What testers should measure, Hyrule Field and Kokiri Forest, default options, prepared shaders:
replay(N): submit, draw, descriptors, programs, commandsline ofoot-status --timingruns, orvk: a frame's replay time (ms)in the log):descriptorsshould fall by 0.2-0.3 ms andprogramsby 0.1-0.2 ms;submitby 0.3-0.5 ms. Alternate runs: two runs of one build differ by 10 %.M more built their uniform ranges alonein the log. If M is near 0, change 1 does nothing here and the eligibility (uniform_only_of) is too narrow for this game's materials.RX_VK_DESC_PARTIAL=0, andRX_VK_DESC_REUSE=neveras the reference for descriptors.RX_VK_FRAMELOG/ the flash detector in Kokiri and the field for wrong or missing textures and one-frame flashes; scene changes, the pause menu, a texture streaming burst (riding across the field), DLSS/FSR and frame generation on (their passes push descriptors of their own between draws); an AMD card (RADV) and a card whose sets go over the push limit (allocated sets instead of pushes).RX_VK_PARALLEL, off by default): the new state is per recording thread like the state beside it, but it was not run.WIP: perf(vk): cheaper replay: uniform-only descriptor rebuilds and flat program picksto perf(vk): cheaper replay: uniform-only descriptor rebuilds and flat program picksTiming result: no measurable gain, so this stays a draft.
One alternating pair, standing in Hyrule Field, RTX 3090 + Ryzen 9 7900X, Linux (NVIDIA, Wayland), uncapped, pipeline cache loading off on both sides:
Setup: RX 5600 XT 6 GB (RDNA1, no mesh shaders), Adrenalin 26.10.1, Windows 11, Ryzen 5 5600X, 24 GB, 1920x1080, frame generation off. Every run starts from the same copied save and follows a scripted route through the control bridge (Kokiri Forest and Hyrule Field: settle 15 s, then 60 s standing); fps are the medians of the log's
N presentslines. Run-to-run noise on mainde866017(4 runs): Kokiri 31.5-34.6 fps, Hyrule Field 32.1-34.8.Merged on main
de866017. Because this PR changesvk_shader.cpp(the translator stamp), both builds ran withRX_VK_PRECOMPILE=0and a cache set of their own. Each set was warmed by a first run, then the order was base, PR, base, PR. Same save and same route (Kokiri Forest, Hyrule Field, 60 s standing).The mechanism works on this card:
a frame: 127 draws kept their neighbour's descriptors ..., 381 more built their uniform ranges alone(of ~890-1,040 draws).programsis down 12-15% in both places.drawanddescriptorsare down about 0.2-0.3 ms together, a little more than the 0.1-0.15 ms you saw on the RTX 3090. The frame rate doesn't show it at this card's ~34 fps: Kokiri 35.95 / 34.4 against 35.1 / 35.2, and Hyrule Field swings 33.9-42.2 even between base runs. The replay isn't this card's limit (GPU 17 ms, submit 7-10 ms). The screenshots (one a place) look the same as main's; no frame log was taken for flashes.Tested in the game: the partial rebuild is taken, but no change in frame rate here.
Hardware: RTX 3060 12 GB, NVIDIA 615.71.09, Manjaro (kernel 6.12), Core Ultra 7 265KF (8 P-cores + 12 E-cores), Wayland. Built on main
51da0464: each PR merged onto it, onlyhookshot_vk.sobuilt (container build,RX_MARCH=x86-64-v3 RX_RELEASE=1) and swapped into the same game folder; main'shookshot_vk.sobuilt the same way as the baseline.Settings: 3440x1440 monitor, picture 2576x1080 (Aspect ratio: monitor's), frame interpolation 2x, Reflex on, V-Sync off, no frame rate cap; otherwise unchanged between runs.
Scene: Hyrule Field from the same save, standing, no input, 3.5 to 7 minutes a run; only the 5 s windows after loading (3,370-3,760 draws in every run) are counted. One run each, in the order main, #2633, #2644, #2559.
(replay columns: medians of the last 20
a frame's replay timelines;GPU-bound waitwas 0.0 ms in all runs)112 draws kept their neighbour's descriptors ..., 505 more built their uniform ranges alone(504-510 per frame in the stable part).descriptors2.9 ms against main's 3.5, but #2559, which does not touch descriptors, shows 2.7 ms: within run-to-run noise. Memory sync (~9.5 ms) is the largest part of the replay in every run.Some data from another machine, in case it helps (RTX 5070 Laptop, Core Ultra 9 275HX, v2.0.14, two 30 s HOS_SAMPLE_ALL profiles in gameplay at 20-40 fps):
submit23-28 ms a frame for 2300-3000 draws (GPU ~12 ms), so the frame is replay-bound here too;build_descriptors10.5% inclusive of the render thread,program_for5.5%,texture_view_of3.6%,gpu_buffer_bind3.4%;I also tried the same idea independently, with a wider eligibility (texture and image descriptors kept across uniform buffer binds, handles re-read and compared when they are not plain c2 words, so handle programs and c3+ handles qualify too). A differential test cut from the real code (38k draws over random streams with UBO rebinds and updates, slot re-registrations, texture destroys, handle programs, captures and recordings) matched a whole build at every draw, and dropping any of its 24 conditions made it fail. But on a frame-shaped synthetic stream it was neutral: keeping saved about 22-34 ns a draw, while recording what is kept cost every whole build 17-38 ns, because the draws that qualify reuse their neighbour's handles and their lookups are already warm. So the gain may be at the low end of your estimate, and an in-game A/B (
RX_VK_DESC_PARTIAL=0against the default) is what will tell.Two things from that attempt that could be useful here: a check switch that runs a whole build after each partial one and logs any difference (so a player can verify it in game), and the differential test. Happy to share either.