perf(vk): opt-in pipeline binary cache, lighter precompile and a wait budget for AMD Windows #2814
Open
github-sync
wants to merge 3 commits from
github/pr/2814 into main
pull from: github/pr/2814
merge into: MagicBeanDev:main
MagicBeanDev:main
MagicBeanDev:github/pr/2893
MagicBeanDev:github/pr/2890
MagicBeanDev:github/pr/2887
MagicBeanDev:github/pr/2878
MagicBeanDev:github/pr/2875
MagicBeanDev:github/pr/2874
MagicBeanDev:github/pr/2873
MagicBeanDev:github/pr/2871
MagicBeanDev:github/pr/2756
MagicBeanDev:github/pr/2897
MagicBeanDev:github/pr/2896
MagicBeanDev:github/pr/2718
MagicBeanDev:github/pr/2842
MagicBeanDev:github/pr/2691
MagicBeanDev:github/pr/2895
MagicBeanDev:github/pr/2743
MagicBeanDev:github/pr/2765
MagicBeanDev:github/pr/2708
MagicBeanDev:github/pr/2732
MagicBeanDev:github/pr/2755
MagicBeanDev:github/pr/2817
MagicBeanDev:github/pr/2876
MagicBeanDev:release-please--branches--main
MagicBeanDev:release-please--branches--main--release-notes
MagicBeanDev:github/pr/2778
MagicBeanDev:github/pr/2888
MagicBeanDev:github/pr/2866
MagicBeanDev:github/pr/2877
MagicBeanDev:github/pr/2800
MagicBeanDev:github/pr/2829
MagicBeanDev:github/pr/2850
MagicBeanDev:github/pr/2889
MagicBeanDev:github/pr/2882
MagicBeanDev:github/pr/2860
MagicBeanDev:github/pr/2849
MagicBeanDev:github/pr/2886
MagicBeanDev:github/pr/2794
MagicBeanDev:github/pr/2840
MagicBeanDev:github/pr/2830
MagicBeanDev:github/pr/2834
MagicBeanDev:github/pr/2719
MagicBeanDev:github/pr/2742
MagicBeanDev:github/pr/2741
MagicBeanDev:github/pr/2820
MagicBeanDev:github/pr/2771
MagicBeanDev:github/pr/2859
MagicBeanDev:github/pr/2858
MagicBeanDev:github/pr/2823
MagicBeanDev:github/pr/2757
MagicBeanDev:github/pr/2769
MagicBeanDev:github/pr/2844
MagicBeanDev:github/pr/2843
MagicBeanDev:github/pr/2673
MagicBeanDev:github/pr/2855
MagicBeanDev:github/pr/2739
MagicBeanDev:github/pr/2807
MagicBeanDev:github/pr/2851
MagicBeanDev:github/pr/2789
MagicBeanDev:github/pr/2818
MagicBeanDev:github/pr/2774
MagicBeanDev:github/pr/2772
MagicBeanDev:github/pr/2750
MagicBeanDev:github/pr/2786
MagicBeanDev:github/pr/2785
MagicBeanDev:github/pr/2782
MagicBeanDev:github/pr/2779
MagicBeanDev:github/pr/2775
MagicBeanDev:github/pr/2706
MagicBeanDev:github/pr/2766
MagicBeanDev:github/pr/2747
MagicBeanDev:github/pr/2759
MagicBeanDev:github/pr/2745
MagicBeanDev:github/pr/2740
MagicBeanDev:github/pr/2733
MagicBeanDev:github/pr/2725
MagicBeanDev:github/pr/2727
MagicBeanDev:github/pr/2728
MagicBeanDev:github/pr/2729
MagicBeanDev:github/pr/2721
MagicBeanDev:github/pr/2722
MagicBeanDev:github/pr/2715
MagicBeanDev:github/pr/2714
MagicBeanDev:github/pr/2707
MagicBeanDev:github/pr/2703
MagicBeanDev:github/pr/2705
MagicBeanDev:github/pr/2697
MagicBeanDev:github/pr/2679
MagicBeanDev:github/pr/2676
MagicBeanDev:github/pr/2669
MagicBeanDev:docs/what-hookshot-connects-to
MagicBeanDev:fix/hdr-encode-test
MagicBeanDev:fix/main-tests
MagicBeanDev:fix/play-cache-files
MagicBeanDev:perf/game-barrier-order
MagicBeanDev:feat/heap-lazy
MagicBeanDev:feat/texture-detail
MagicBeanDev:perf/astc-opaque-bc1
MagicBeanDev:fix/fs-dir-lock
MagicBeanDev:fix/framegen-native-kept
MagicBeanDev:fix/update-no-downgrade
MagicBeanDev:port/k10-gcn-f16-mirrors
MagicBeanDev:fix/framegen-mult-cap
MagicBeanDev:chore/release-as-2.1.1
MagicBeanDev:perf/pcache-load-switch
MagicBeanDev:fix/prepare-progress-log
MagicBeanDev:fix/update-closes-stale-game
MagicBeanDev:fix/hdr-scaled-sampled
MagicBeanDev:perf/first-start-preset
MagicBeanDev:perf/zero-upload
MagicBeanDev:fix/vcore-escape-2s
MagicBeanDev:test/fake-gh-atomic
MagicBeanDev:perf/batch-idle-flush
MagicBeanDev:test/finish-stub-a
MagicBeanDev:chore/record-2.1.0
MagicBeanDev:fix/main-red-tests
MagicBeanDev:fix/test-display-startup-driver-cache
MagicBeanDev:feat/mesh-shader-mode
MagicBeanDev:fix/menu-probes-at-startup
MagicBeanDev:fix/fps-counter-classic
MagicBeanDev:fix/test-log-batch-flake
MagicBeanDev:fix/narrow-screen-hud
MagicBeanDev:ci/prereleases
MagicBeanDev:release/v2.0.15
MagicBeanDev:hotfix/v2.0.15
MagicBeanDev:fix/present-ahead-replay
MagicBeanDev:release/v2.0.13
MagicBeanDev:fix/bracket-paths-2013
MagicBeanDev:fix/hdr-overlays-2013
MagicBeanDev:fix/fps-counter-scale-2013
MagicBeanDev:fix/stick-seen-at-rest
MagicBeanDev:hotfix/menu-fits-screen
MagicBeanDev:fix/f1-always-opens-menu
MagicBeanDev:fix/ini-keep-lines
MagicBeanDev:claude/cranky-pike-7cf247
MagicBeanDev:perf/texture-answer-flat
MagicBeanDev:fix/stamp-test-modules
MagicBeanDev:fix/appimage-fonts
MagicBeanDev:fix/display-awake
MagicBeanDev:release/v2.0.12
MagicBeanDev:fix/discord-windows
MagicBeanDev:rc/v2.0.12-tests
MagicBeanDev:fix/dlss-keep-feature
MagicBeanDev:rc/v2.0.12
MagicBeanDev:button-icons
No reviewers
Labels
Clear labels
accessibility
Barrier affecting people with disabilities
autorelease: pending
autorelease: tagged
bug
Something isn't working
dependencies
Pull requests that update a dependency file
documentation
Improvements or additions to documentation
duplicate
This issue or pull request already exists
enhancement
New feature or request
github_actions
Pull requests that update GitHub Actions code
good first issue
Good for newcomers
help wanted
Extra attention is needed
invalid
This doesn't seem right
question
Further information is requested
testing wanted
wontfix
This will not be worked on
No labels
accessibility
autorelease: pending
autorelease: tagged
bug
dependencies
documentation
duplicate
enhancement
github_actions
good first issue
help wanted
invalid
question
testing wanted
wontfix
Milestone
Clear milestone
No items
No milestone
Assignees
Clear assignees
No assignees
1 participant
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".
No due date set.
Dependencies
No dependencies set
Reference
MagicBeanDev/Hookshot!2814
Loading…
Reference in a new issue
No description provided.
Delete branch "github/pr/2814"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
On an 8 GB AMD laptop with 32 GB of RAM, v2.1.0 grows past the PC's RAM after a few area changes (33-37 GB private), so Windows pages the game out and single frames take 5-10 s. First-time cutscenes and boss fights also stop the picture for minutes while AMD's compiler builds new pipelines (10-57 s each).
This draft adds opt-in switches against both, plus a pipeline cache built on
VK_KHR_pipeline_binarythat turned out much faster than the pipeline cache on this AMD driver. Everything is off by default: with no new variable set, behaviour is main's. It is a draft because the rebased build has not been run in the game yet (see Status). It groups several pieces under one topic (RAM and long waits on AMD Windows); I am happy to split it into separate PRs if you prefer.Status, and what was tested where
51da0464). It builds with llvm-mingw, and the game-free suite result is under Tests. The rebased build has not been run in the game.What is in it
1. A pipeline cache made of pipeline binaries:
RX_VK_BINARY_CACHE=1,RX_VK_PCACHE_SKIP=1mkw_vk_pipelines.binis read whole at every start and the driver holds it for the whole session (2.4-2.5 GB here). WithRX_VK_BINARY_CACHE=1, each pipeline's binaries are kept on their own inhookshot_pipeline_binaries.bin, with an index of keys in memory, and only the binaries of the pipeline being made are read from the file.How it works:
cached_graphics/cached_compute(every pipeline goes through them), the pipeline's key is computed withvkGetPipelineKeyKHR, using the create info without the ask-only flag.vkCreatePipelineBinariesKHRis called from that data, and the pipeline is created with the binaries and a null cache.create_cached). Its binaries are then taken from the driver's own cache by probing it with the create info (pipelineBinaryInternalCacheis true on this driver), so nothing is compiled twice, and they are appended to the file..oldand a new one is started.VK_KHR_maintenance5andVK_KHR_pipeline_binaryare enabled on the device only when one of these switches is set.RX_VK_PCACHE_SKIP=1needs the binary cache. It does what main'sRX_VK_PCACHE_LOAD=0does (the file is not read and not saved), but meant for play:mkw_vk_pipelines.binstays as it was, and the play files are not merged into the empty cache, only numbered on.RX_VK_BINARY_TEST=<n>is a measurement only. On the first n pipelines made in play, it logs the key time, whether the driver's cache has the pipeline, a capturing build, a build from the captured data, and a build from the previous run's data (kept inhookshot_binary_test.bin). Everything it makes is destroyed.Measured on the RX 7700S:
RX_VK_PRECOMPILE_GRAPHICS=release, see 2)mkw_vk_pipelines.binThis is the opposite of what #2760 found on NVIDIA (1.2-1.7 ms median against 0.4-0.6 with the pipeline cache, 2.2 MB a pipeline, 159 compute pipelines), so it looks driver-dependent. Numbers from NVIDIA and Intel with this branch would tell.
A finding that may help others: AMD's own Vulkan cache is one file of a fixed 896 MiB (
%LOCALAPPDATA%\AMD\VkCache\*.parc, exactly 939,524,096 bytes) and evicts. Pipelines that came from the driver's cache fell from 3,245 in one session to 11 two sessions later.2. How much the precompile keeps:
RX_VK_PRECOMPILE_GRAPHICS=keep|release|noneThe start builds every noted graphics pipeline (15,000-17,000 here) and keeps all of them, and their programs, for the whole session.
keep(default): as main.release: each pipeline is built (so it is in the driver's cache) and destroyed.none: the notes are not built and their programs not read. Play makes them on first use.A controlled test at the title screen, same install,
RX_VK_PRECOMPILE=0against the default:With
release, the start was at 22.7 GB. So the programs are ~8 GB of that, and the pipelines themselves ~1.2 GB.Caveat: with
noneon v2.1.0, programs were then read fromhookshot_programs.binon the replay thread at their first draw. That gave 100-1,700 ms frames: micro-stutter that the player noticed. Locally I had a switch that read them on a worker (left out here, see below). On main,nonewill most likely stutter the same way unless programs are loaded ahead of their first draw, which is what #2760's load-at-create does better.noneprobably belongs with #2760, andreleaseis the safe one.3. A per-frame wait budget:
RX_VK_PIPE_WAIT_MS=<ms>A frame waits at most this long, in all, for the pipelines of draws that main already allows to be left out (none of their stages writes memory). Past the budget they are left out until a worker has them.
jobs_wait_formoves the waited job to the front of the queue and waits with a timeout; it does not run the job inline, where an AMD compile can take seconds.jobs_wait's steal. With the budget (50 ms) the game kept presenting while they compiled, at a low frame rate with missing parts.This overlaps #1946, which gives a waited draw an unoptimized stand-in. On this AMD card, an earlier benchmark found
VK_PIPELINE_CREATE_DISABLE_OPTIMIZATION_BITdid not make compiles faster (47 compute shaders, 182 s against 181 s), so stand-ins may help less on AMD. The two could live together.4. Cache hits made at the draw:
RX_VK_PIPE_TRY_CACHED=<ms>A pipeline the driver's cache or ours already has is made at the draw, asked without compiling (
FAIL_ON_PIPELINE_COMPILE_REQUIRED), up to that many ms a frame. Only pipelines that must be compiled go to the workers.In a boss area on v2.1.0 the queue reached 286 jobs for 4 workers. Pipelines the caches had sat behind compiles of 10-44 s each, and the boss model was not drawn for minutes. In a later 30-minute session with the switch at 8 ms, 4,000 pipelines were made at the draw and 824 went to the workers.
5. A log line for slow pipelines in play
Each graphics pipeline built in play that takes 50 ms or more is logged (the first 60, then every 100th), with the program key and the time.
Left out of this rebase (tested on v2.1.0 only)
RX_VK_PROGRAM_ASYNC: programs read on a worker, with a stand-in for the draw, and translations moved ahead of pipeline builds. Main now waits for translations whole, so the draw would wait for the read anyway. It also touchedvk_shader.cpp, which changes the translator stamp, and without it this PR does not change the stamp: no new prepare for anyone. #2760 covers this better.RX_VK_WORKERSand 6 workers with 16+ threads now.Sessions, for context (v2.1.0, local builds; real fps = presents / time)
Different routes, so read this as context, not an A/B. In every session I also had
RX_VK_KEEP_RETIRED=0in the environment (retired imports given back at once: 0.9-3.7 GB freed in a session, no device loss in about ten sessions). Main has #1906's graceful version of this now, so it is not part of this PR.Private memory still grows during long, varied sessions. The game window's committed memory went from 7.8 to 15.2 GB in one session, more than the game's 9.8 GB, and I have not found why.
The frame rate in busy places
Fps drop in busy places such as Kakariko Village and Castle Town. In one 30-minute session it went from 62 fps (first two thirds) to 46 fps (last third): draws from ~930 to ~1,600 a frame, replay from 8.8 to 13.8 ms, GPU from 9.7 to 12.9 ms. I don't think these switches cause it: the cost rises with what the scene draws, and the replay thread (CPU) is the limit here. It has not been A/B tested in those places.
Other findings on this machine
RX_VK_GPL=1): 937 of 42,549 program variants in about 4 minutes, so about 3 hours for the whole pass. Keeping every library inmkw_vk_pipelines.binwould also grow it to an estimated ~8 GB, read whole into RAM. I did not use it.RX_VK_KEEP_FINALIZED_POOLS=1the same area was fine. This was on a backport, so it may not apply to #2759 as it stands on main, but it is AMD evidence for it.Not tested
RX_VK_GPL=1; pipelines that come back with more than one binary (all of mine had one); a binary file damaged by a crash mid-write (records cut short are written over, but this was not forced).Tests
RX_VK_ONLY=1 RX_RELEASE=1 nx2/build.py --standalone).python -I -B tests/run_all.pyon Windows: 449 passed, 11 failed, 35 skipped, the same as clean main (51da0464) on the same PC, with the same 11 failures (builder, portable archive, updater digest,test_input_background,test_gpu_hang_watch,test_spawn_emit,test_fpx_sticky,nx2/test_nup_paths.py: environment ones, nothing these changes touch).tests/test_draw_side_effects.py(its fixture gets stubs fortexture_frame_no,jobs_wait_forandlogf, which the budgeted wait uses; with the budget unset the decisions it checks are unchanged, underRX_VK_PIPE_SKIPunset,0and2), andtests/test_precompile_pipeline_cleanup.py(no change to the test: thereleasebranch was rewritten so the block it extracts still compiles).See #993, #995. Related: #2760, #1946, #2649, #2759, #1906.
This was developed with AI assistance (Claude Code). The measurements come from my own runs of the game.
🤖 Generated with Claude Code