WIP: perf(vk): a descriptor build asks for its stages' descriptor lists first #2045
No reviewers
Labels
No labels
area/accessibility
area/audio
area/input
area/install
area/menu
area/renderer
area/saves
area/tooling
area/upscaling
beanstalk: approved
beanstalk: failed
beanstalk: needs approval
beanstalk: running
beanstalk: tested
ci: approved
good first issue
gpu/amd
gpu/igpu
gpu/intel
gpu/nvidia
help wanted
kind
bug
kind
chore
kind
docs
kind
feature
kind
performance
kind
question
needs measurements
platform/linux
platform/macos
platform/steam-deck
platform/windows
preview
priority
critical
priority
high
priority
low
regression
risk
high
risk
low
risk
medium
status
blocked
status
confirmed
status
duplicate
status
invalid
status
needs-info
status
wontfix
ci: approved
No milestone
No project
No assignees
3 participants
Notifications
Due date
No due date set.
Dependencies
No dependencies set
Reference
MagicBeanDev/Longshot!2045
Loading…
Reference in a new issue
No description provided.
Delete branch "siantm/Longshot:perf/vk-desc-lists-first"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Summary
The busiest instructions of a descriptor build (
build_descriptors,gpu/vk/vk_draw.cpp) were the first loads of each stage's descriptor lists, kept inside the stage'sShader::Info(about ten cache lines of it): a miss a list, one after another. The build now asks for each stage'sInfolines (from the uniform buffers' used sizes to the end of the image descriptors) before anything else, so the misses overlap. A hint only: the descriptors written are the same; the result expected is a shorter build.No switch; a revert gives back main's code.
From f61b2820 on
exp/x86-asm-sync.Read first: the prefetch at the start of
build_descriptorsingpu/vk/vk_draw.cpp.Testing
Tested: Windows 11, i7-9700, RTX 4060, the builder's llvm-mingw clang 23; head
41c1c6deon main @75317e1f. Every check ran throughtests/run_all.py --strict --exactin build jobs of the PC's job queue, in a hidden console (logt-dc3-vk-desc-lists-first):tests/test_desc_info_prefetch.py(new: the prefetch is the build's first work and every list the build reads lies inside the prefetched range ofShader::Info, inshader_info.h's order): pass.--changed 75317e1f; its own are among them): 146 passed, 15 failed. Failed:tests/test_sampler_admission.py,tests/test_fsr_uniform_ownership.py,tests/test_pipeline_play_cache.py,tests/test_draw_stream_visibility.py,tests/test_device_lost_exit.py,tests/test_draw_binds.py,tests/test_dev_packs.py,tests/test_frame_overlap.py,tests/test_linux_pressure.py,tests/test_mesh_conv.py,tests/test_prepare_cache_save.py,tests/test_recorder_own_idle.py,tests/test_upscaler_rows.py,tests/test_vk_occlusion_queries.py: each fails on main @75317e1ftoo (the same checks on main in the same queue batch, logt-dc3-base).tests/test_recording_admission.pyalso failed in that run (itsparallelcase, "worker slot ownership") and passed when run again on this head (logt-dc4-vk-desc-lists-first); main passed it in that batch and in five runs of the next (logst-dc4-base-ra1tot-dc4-base-ra5).It has NOT run in the game on this PC yet.
Still to test:
75317e1fagainst this PR, each warmed a minute in the scene first, then interleaved (main, this PR, this PR, main), two runs each, expecting the same picture and the replay's time a frame the same as or lower than main's (by hand)tests/test_desc_info_prefetch.pywith--cxx g++and--cxx clang++, expecting PASS (by hand)Measurements
Verdict: with the stages' lists out of the caches (a build's first look at a program) a build takes 8-15 % less (1.09-1.17x; 19-140 ns a build), in both builds; with them in cache and work between the lists it is the same (1.00x), with no work 1.03-1.10x. Whether the lists are out of cache in play, and how often, is not measured; the third party's in-game run below showed no difference within about 5 %. A micro-benchmark on authored input; not measured in play.
Micro-benchmark (
vk_desc_lists_first_bench.py: a script on this PC, not in this PR): the head's prefetch loop cut out ofbuild_descriptors, put in front of an authored walk of the lists in the build's order (the old side: the walk alone), each descriptor followed by an authored amount of dependent work (rounds of a multiply-xorshift, a stand-in for its write); vk_shader.h'sStagewith Eden's realShader::Info(848 bytes a stage); programs of a vertex stage (3 uniform buffers, 1 storage buffer, 2 textures) and a fragment stage (4, 2, 8 and 1 image), a third (2, 3, 1) in the 3-stage case; three rounds run alone in this PC's job queue (2026-11-22, 01:21-01:23 UTC; logbench-vk-walks.log), the builder's llvm-mingw clang 23.1.5 at -O2 for baseline x86-64 and at -O2 -march=native (this PC's i7-9700, as the game is built here), 301 repetitions a case, old and new taking turns on the same authored inputs, their results equal at every repetition. Each cell: ns per build, the median of the three rounds' medians, in parentheses the lowest and highest of them; Δ: old over new (above 1, this PR is quicker).75317e1f41c1c6dezeldaootr's result (#issuecomment-69131), as posted:
9dd4132641c1c6de(merged into main9dd41326)Hardware and settings: Steam Deck OLED, SteamOS 3.9.5, RADV VANGOGH, Mesa 26.2.0-devel, GPU clock set to 1600 MHz, headless; Minimum, render 50 % (640x400), output 1280x800, uncapped, V-Sync and frame generation off. Method:
dev.py ab(with a local change to compare two prebuilt runtimes), the Market, three interleaved pairs of 60 s after the warm-up, the median; no FATAL, device-lost,VK_ERRORor validation line in the six logs. Its own stated limits: three runs a side on one Deck in one standing scene resolve about 5 %, so a saving of a few tenths of a millisecond of the replay's CPU time a frame would not show.Risks
None for the picture: a prefetch changes no value. It could cost a little where the lines were already in cache (more loads issued); that is what the in-game A/B is for. A revert is enough (no switch). The renderer is compiled again after updating.
Checklist
Ready for review
main; the title is a Conventional Commit.mise run testpasses (any failure that also fails onmainis named). Not yet: only the checks in Testing ran.main, or Measurements says why there is nothing to measure. Not yet (Measurements).third_party/untouched; nothing GPL-incompatible.When it applies
Assisted-by:/Co-Authored-By:trailers.@Beanstalk This changes how the renderer builds descriptors, lists first (gpu/vk), for every draw. No switch: a revert gives back main's code. It has not run in the game on these PCs yet. Please play Kokiri Forest and Hyrule Field, two minutes each, the PR against main on the same PC, with a picture in each compared with main's, on Linux with NVIDIA, Linux with AMD (RADV), Windows with Intel and the Steam Deck. The same picture as main, no crash, device lost, validation error or FATAL line; frame time p99 equal to main's or better, within the run-to-run spread.
@siantm only members of
BigBeanscan start the farm. One of them can approve this request by reacting 👍 to it or to this comment.Steam Deck result: I ran this PR against main on a Steam Deck OLED, standing in the Market, three pairs. It showed no difference the Deck can resolve (-0.6 % on the median, the two sides' runs overlapping), and no errors in any log.
9dd4132641c1c6dedev.py ab's verdict: no difference shown: -0.6 % fps (median 26.5 against 26.7), within the runs' spread (A 26.2 to 26.7, B 26.4 to 27.0). NoFATAL, device-lost,VK_ERRORor validation line in the 6 logs; every run exited normally.Hardware: Steam Deck OLED, SteamOS 3.9.5, RADV VANGOGH, Mesa 26.2.0-devel, GPU clock manual 1600 MHz, headless.
Settings: Minimum, Render 50 % (640x400), output 1280x800, uncapped, V-Sync and frame generation off.
Game data: complete (the dump, with the three damaged textures replaced).
Built: main @
9dd41326and main9dd41326with this PR's head41c1c6demerged in (1 commit, no conflicts), x86-64-v3, shaders prepared on the Deck; only the renderer differs.Measured:
dev.py ab, Market, 3 interleaved pairs of 60 s after the warm-up, the median (the project's tool, with a local change to compare two prebuilt runtimes, since the Deck does not build).Limits: three runs a side on one Deck, headless, one standing scene; on this Deck that resolves about 5 %, so a saving of a few tenths of a millisecond of the replay's CPU a frame would not show.
@zeldaootr thank you for running this on a Steam Deck OLED and for writing down the setup and its limits. It is now cited in Measurements as your run, with the about 5 % it resolves, beside the A/B still to do on this PC;
41c1c6deis still this pull request's head.@Beanstalk This replaces my request 65289 on this head, so that it follows the farm's picture and hardware rules. This changes how the renderer builds descriptors for every draw, asking for its stages' descriptor lists first (gpu/vk). No switch: a revert gives back main's code. Please start from the kokiri-start save and take the picture right after it loads and settles, before any input, with a main-against-itself picture on the same PC at that point; then, for frame times only, play Kokiri Forest and Hyrule Field, two minutes each; at least two starts a side, interleaved, the PR against main on the same PC, on Windows with an RTX 4070, Windows with an Arc B580, Linux with an RX 6600 (RADV) and the Steam Deck. The same picture as main within main's own difference from itself; frame time p99 equal to main's or better, within the run-to-run spread. The log must have no FATAL, device lost or validation error line.
@siantm only members of
BigBeanscan start the farm. One of them can approve this request by reacting 👍 to it or to this comment.Measurements: a micro-benchmark on authored input (old against new, three rounds, x86-64 baseline and -march=native; not measured in play) and the Verdict rewritten to match; the third party's in-game run kept.
@Beanstalk One correction to my request 69679 above, for its picture point: please take each picture 5 s (300 frames at 60 fps) after the first frame whose picture is not a single uniform colour, with no input in between, and take the main-against-itself picture from the same point. A picture taken during the load's fade-in from black is not comparable: run 70269 on #2156 caught the fade in 3 of 4 starts.
@siantm only members of
BigBeanscan start the farm. One of them can approve this request by reacting 👍 to it or to this comment.View command line instructions
Checkout
From your project repository, check out a new branch and test the changes.