WIP: perf(vk): programs with the same bindings share one set layout and pipeline layout, so a program change pushes only what changed #800
No reviewers
Labels
No labels
area/accessibility
area/audio
area/input
area/install
area/menu
area/renderer
area/saves
area/tooling
area/upscaling
beanstalk: approved
beanstalk: failed
beanstalk: needs approval
beanstalk: running
beanstalk: tested
ci: approved
good first issue
gpu/amd
gpu/igpu
gpu/intel
gpu/nvidia
help wanted
kind
bug
kind
chore
kind
docs
kind
feature
kind
performance
kind
question
needs measurements
platform/linux
platform/macos
platform/steam-deck
platform/windows
preview
priority
critical
priority
high
priority
low
regression
risk
high
risk
low
risk
medium
status
blocked
status
confirmed
status
duplicate
status
invalid
status
needs-info
status
wontfix
ci: approved
No milestone
No project
No assignees
4 participants
Notifications
Due date
No due date set.
Dependencies
No dependencies set
Reference
MagicBeanDev/Longshot!800
Loading…
Reference in a new issue
No description provided.
Delete branch "dyst/Longshot:perf/vk-shared-pipeline-layouts"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Type: perf · Risk: low · Tested on: Windows 11 (the game-free checks; the game A/B is next) · Needs: a game run on NVIDIA, AMD and Intel
Why
Each shader program had its own descriptor set layout and pipeline layout, and the replay pushes only the descriptors that changed only while the layout is the one it pushed with: every program change pushed every descriptor again, even between programs whose bindings are laid out the same way. Programs change often within a pass. Found in an audit of the renderer's per-draw path; no issue on the tracker.
What changed
Programs whose layouts' create infos are the same now share one set layout and pipeline layout, kept for the device's life, so a program change between them pushes only the writes that differ. Pipelines are made from the same create infos as before and no pipeline key holds a layout handle.
layout_signature,layouts_for,SharedLayoutsandensure_layout; the preparation's compute warm leaves shared layouts alone (gpu/vk/vk_draw.cpp)program_cache_borrowleaves shared layouts alone (gpu/vk/vk_progcache.cpp);Program::layout_shared(gpu/vk/vk_shader.h, not a stamped translator file)RX_VK_LAYOUT_EACH=1row in docs/development.mdtests/test_shared_layouts.pyMeasurements
Verdict: not measured yet: the measurements are pending from the RX_VK_BENCH A/B of this PR @
e2a24c1aagainst main @b9e95173(under way), which will replace this section.Flags and settings
One switch for an A/B.
RX_VK_LAYOUT_EACH=1=1for an A/BTesting
Tested: Windows 11 (Ryzen 9 3900X), llvm-mingw and the Vulkan headers of the builder's v2.0.14 tools: rebased onto main @
b9e95173(this PR @e2a24c1a; it stands alone on main, on no other PR). The 135 checks that read this diff (listed below: those under tests/ and nx2/ that name a changed file, and the ones this PR adds or changes), not the whole suite:python -I -B tests/run_all.py --jobs 6 <those checks>withRX_CPMat the builder's CPM cache: 127 passed, 7 failed; each failure (tests/test_crash_dialog.py,tests/test_linux_driver_fault_guard.py,tests/test_oot_state_tap.py,tests/test_pipeline_tour.py,tests/test_vk_occlusion_queries.py,tests/test_vk_pgo.py,tests/test_walk_key.py) also fails on main @b9e95173in the same checks, on the same PC and the same day.The 135 checks
nx2/test_cbuf_table_extent.pynx2/test_dump_ssbo_read.pynx2/test_mesh_dynamic_state.pytests/test_amd_fragment_warp.pytests/test_bake_ds.pytests/test_bank_gen.pytests/test_bc_raw_targets.pytests/test_check_pr_description.pytests/test_commit_room.pytests/test_crash_dialog.pytests/test_desc_clock.pytests/test_desc_counts.pytests/test_dev_packs.pytests/test_device_lost_exit.pytests/test_dispatch_clock.pytests/test_dlss_checks.pytests/test_draw_binds.pytests/test_draw_side_effects.pytests/test_draw_stream_visibility.pytests/test_drawn_page_map.pytests/test_driver_cache_room.pytests/test_driver_faults.pytests/test_dummy_views.pytests/test_equal_depth_margin.pytests/test_fallback_buffer_cleanup.pytests/test_fallback_depth_hold.pytests/test_fallback_loaded.pytests/test_fallback_pipeline_state.pytests/test_fallback_size_policy.pytests/test_flat_map.pytests/test_framegen_preset.pytests/test_fsr_uniform_ownership.pytests/test_global_table_loop.pytests/test_gpl_cold_library.pytests/test_gpl_library_cache.pytests/test_handle_lookup.pytests/test_handle_table_arrays.pytests/test_hold_gpu.pytests/test_intel_compiler_gate.pytests/test_intel_quick_compute.pytests/test_jobs_failure.pytests/test_jobs_wait_runs_queued.pytests/test_linux_driver_fault_guard.pytests/test_linux_pressure.pytests/test_load_default.pytests/test_masked_clear_failure.pytests/test_mem_trim.pytests/test_menu_env_on.pytests/test_mesh_args_cleanup.pytests/test_mesh_fallback.pytests/test_mesh_fallback_failed_once.pytests/test_mesh_query_feature.pytests/test_met_wait_capture.pytests/test_module_ids.pytests/test_native_float_atomic_add.pytests/test_oot_state_tap.pytests/test_parallel_dry_marks.pytests/test_parallel_rb_refresh.pytests/test_pass_hazard.pytests/test_pcache_fetch_lock.pytests/test_pcache_header_read.pytests/test_pcache_stamp.pytests/test_pipeline_cache_save.pytests/test_pipeline_notes_open.pytests/test_pipeline_play_cache.pytests/test_pipeline_share.pytests/test_pipeline_state_computes.pytests/test_pipeline_state_counts.pytests/test_pipeline_state_portable.pytests/test_pipeline_states.pytests/test_pipeline_tour.pytests/test_precompile_compute_guard.pytests/test_precompile_pipeline_cleanup.pytests/test_predict_resume.pytests/test_predict_sees_loaded.pytests/test_predict_types.pytests/test_predicted_check.pytests/test_prepare_cache_save.pytests/test_prepare_cpus.pytests/test_prepare_seed_stamp.pytests/test_prepare_stall.pytests/test_prepare_workers.pytests/test_prepared_stamps.pytests/test_prof_sampling.pytests/test_program_answer_domains.pytests/test_program_cache_cleanup.pytests/test_program_cache_format.pytests/test_program_cache_index.pytests/test_program_cache_one_variant.pytests/test_program_cache_reader.pytests/test_program_load_wait.pytests/test_program_record_head.pytests/test_program_stamp_knobs.pytests/test_psychov35_dlss.pytests/test_psychov35_taau.pytests/test_push_state.pytests/test_quick_pipelines.pytests/test_radv_mesh_optimizer.pytests/test_reactive_mask.pytests/test_recomplauncher_json.pytests/test_recorder_own_idle.pytests/test_recorder_sync.pytests/test_recording_admission.pytests/test_replay_copies.pytests/test_rumble_menu.pytests/test_same_draw.pytests/test_same_dyn.pytests/test_sampler_admission.pytests/test_seed_variant_cleanup.pytests/test_shader_module_guard.pytests/test_shader_stamp_cache.pytests/test_shader_stamp_inputs.pytests/test_shadow_mask_guard.pytests/test_shared_layouts.pytests/test_split_before_draw.pytests/test_startup_progress.pytests/test_state_list_plain.pytests/test_taau_env_lookup.pytests/test_taau_fxaa.pytests/test_taau_typed_dispatch.pytests/test_table_track.pytests/test_texture_handle_banks.pytests/test_texture_squeeze.pytests/test_third_party_objects.pytests/test_threaded_rendering.pytests/test_tls_objects.pytests/test_tree_shadow_mask.pytests/test_ub_threads.pytests/test_upscale_fallback.pytests/test_upscaler_rows.pytests/test_vertex_bind.pytests/test_vk_occlusion_queries.pytests/test_vk_pgo.pytests/test_walk_key.pytests/test_warm_bounded.pyNot tested: the game yet. This PR is WIP until its A/B is in: the RX_VK_BENCH A/B against main is running on an RTX 3080 with a Ryzen 9 3900X, Windows 11; AMD and Intel need other testers.
Must be tested:
RX_VK_LAYOUT_EACH=1: Kokiri Forest and Castle Town a few minutes, expecting the same picture and, withRX_VK_PROF_FINE=1, fewerfull pushesa draw (Test Farm: @Beanstalk any, those scenes with and without RX_VK_LAYOUT_EACH=1)Risks
What could break: A push elsewhere in the same command buffer at the same bind point that does not mark the game's pushed descriptors as disturbed (the temporal actor pass, frame generation, FSR, ASTC decode): the next game draw's delta push would leave the other bindings as that push set them. Two draws of the same program already relied on this; shared layouts make it more frequent.
Turning it off:
RX_VK_LAYOUT_EACH=1Compatibility: Nothing changes: no caches, settings or vk ABI involved.
Checklist
Everyone
type(scope): description) of the status line's type.can't be run by hand is in Testing's Not testable by hand, with why. Not ticked yet: Testing says the game is not tested yet; only game-free checks are named, and game files changed.
in Risks. Not ticked yet: Measurements only says not measured yet, and Risks names a possible missed descriptor disturbance that is unmeasured.
main.python -I -B tests/run_all.pypasses (any failure that also fails onmainis named).third_party/untouched (a new folder with aPROVENANCE.mdinstead).When it applies
Assisted-by:/Co-Authored-By:trailers.mainand passing here.🤖 Generated with Claude Code
The checks on this PC (Windows 11, RTX 3080, Ryzen 9 3900X, llvm-mingw from the builder's v2.0.14 tools):
python -I -B tests/run_all.pyon main @aa95d35ffails these 19 here, and so does this branch:test_appimage_player_build.py,test_bake_portability.py,test_cpm_discovery.py,test_cpp_preflight.py,test_crash_dialog.py,test_env_on.py,test_fpx_sticky.py,test_lift_cleanup_error.py,test_linux_preflight.py,test_make_game_cache.py,test_make_game_prepare.py,test_missing_nup.py,test_oot_state_tap.py,test_options_budget_once.py,test_readback_late.py,test_spawn_emit.py,test_tested_configurations.py,test_vk_pgo.py,test_walk_key.py. Any other failure in a full run on this branch was rerun on its own and passed (the full runs shared the PC with dev_packs builds: timing checks).Next: an interleaved A/B in the game against main on one dev_packs base of
aa95d35f; the numbers will replace the Measurements section.67156a1f65e2a24c1a90Rebased onto main @
b9e95173(nowe2a24c1a; stands alone on main, not stacked); the 135 checks that read this diff, not the whole suite: 127 passed, 7 failed, each failure also failing on main @b9e95173in the same checks. Stays WIP: measurements pending from the RX_VK_BENCH A/B.Brought up to the ready-for-review standard (#1454): the checklist's Everyone list has two new boxes, added here from what this PR's description and comments show (please check they match what you know).
**Not testable by hand:**line saying why and what stands in for it. Then tick it.#800 on today's main: the standard test, +1.6 % fps (diagnostic); it also invalidates the shader caches
I measured this PR on a second PC, on today's main.
Verdict: a small gain that holds in each pair, but it is under the 5 % that three pairs can call. Medians: 66.99
against 68.09 fps (+1.6 %), with the PR ahead in every pair (+3.7 %, +1.6 %, +2.6 %). CPU time a frame is 70.6
against 71.3 ms, unchanged within noise. The render thread's own line agrees with a small cut: its jobs a frame go from
13.79 to 13.54 ms, mostly in
others(7.22 to 6.88), anddescriptorsgo from 2.53 to 2.38 ms. One caution: the PRarm's GPU clock read higher (2,579 against 2,495 MHz median), with the same
gpu_ms(7.30 against 7.35). Some of thefps difference may be the GPU's state rather than the change.
One thing to fix before merge: this PR invalidates every player's shader caches. The description says "no caches
... involved". But
Programgainsbool layout_shared(gpu/vk/vk_shader.h), andtranslator_stamp()(
gpu/vk/vk_shader.cpp) foldssizeof(Program)into the stamp, so a translator stamp change follows. On its firststart the PR's runtime logged
vk: pipeline cache: 1 files made for another translator dropped (1435 MB); it starts emptyandprogram cache ...: 0 program keys kept. It then began translating all 38,809 programs at start-up, andthe harness stopped that first start at its 240 s limit. Preparing the runtime again with the game's own
--prepare-shaderstook 28 min here. The player's cost is the same as for a change to a stamped translator file. Ifthat isn't wanted, the flag could live outside
Program(for example, a set of shared layout handles kept besidelayouts_for's cache), so thatsizeof(Program)stays the same.What was run
Base: Longshot main @
ef6038ab(2026-11-18 03:06Z), built as one development base with llvm-mingw 20261104, asthe official builds are. Control and PR share the same
longshot.exe(0a53b387). The renderer DLL is the base's(ed1771bb) for the control and the PR's (afa43e5d) for the PR arm, built over the same base (
scripts/dev_packs.py).How the PR was applied: your commit
e2a24c1a(the PR's only commit) was cherry-picked ontoef6038ab. There weretwo conflicts, both text only:
docs/development.md: the PR's row sits next to rows main has changed since. I kept main's side, including its newRX_VK_STAGING_BURST_Srow and without theRX_VK_DESC_COUNTS=0row, which main has deleted. I added the PR's onenew row,
RX_VK_LAYOUT_EACH=1, right after it.gpu/vk/vk_progcache.cpp: main addedprogram_cache_borrow_all(the preparation's compute warm, run on all CPUs)directly above
program_cache_borrow, whose doc comment the PR rewrites. I kept main's new function unchanged andtook the PR's two-line comment for
program_cache_borrow.The result has exactly the PR's added and removed lines (320+/7-). I wrote no code of my own.
A merge point for you: main's new
program_cache_borrow_alldestroys each variant'slayoutandset_layoutwithout your
layout_sharedguard. Your owntests/test_shared_layouts.pycatches it on the merged tree:"vk_progcache.cpp:857 destroys a program's layout without leaving a shared one". It looks harmless as written, because
its only caller passes
warm_compute, which destroys under your guard and then nulls both handles beforeborrow_all's loop runs. Still, the guard belongs there when you rebase (if (p->layout_shared) continue;, as inprogram_cache_borrow). It is in the preparation only, not in the frames measured here.PC: Ryzen 7 5700X3D (8 cores, 16 threads), Radeon RX 9070 XT, AMD's Windows driver 26.11.1, Windows 11.
Harness: our standard test (
std_test.py604af9c9,run_profile.py6d998641). It runs Hyrule Field from a fixedsave, with the camera turning at full speed (right stick held from the load). The game runs in a 1280x720 window
(16:9), with the frame rate unlocked (
HOS_FPS_MAX=240) and the game clock checked (HOS_FPS_LOG=1, 1.000 ±0.01). The render thread is on (
RX_VK_ASYNC=1, as Threaded rendering Auto gives a 16-thread CPU), and the fixedtest settings file is used. The window is 20 s after the load, with 3 alternating pairs, no warm-up and no
replacements.
The PR arm sets no switch: the change is on by default, and
RX_VK_LAYOUT_EACH=1is its off switch. The controlis main without the PR.
Background load: a small fixed GPU load runs beside the game in every run of both arms: 500 blended quads a
frame at 100 fps, in its own window. On this PC, AMD's driver otherwise keeps the RX 9070 XT at ~1,550-1,850 MHz
because the game's bursty load never reaches its boost threshold. Runs then measure a slower GPU and compare
poorly. With the load, both arms run at ~2,440-2,580 MHz.
Order: each runtime had one unmeasured first start before the A/B; the PR runtime's came after its re-prepare.
The control runtime is the one the #795 A/B used just before. No runtime copy, pack or preparation ran within 15 min
before the runs, and a disk-write probe was clean.
Numbers
Per run, in run order.
RT jobsandRT idleare the render thread's own line (vk: the render thread (600 presents ...): its jobs a frame, split into submits, presents and others).draw,descriptorsandprogramscome fromvk: a frame's replay time (ms). Each is the mean of the last two lines inside the window.error linescounts lineswith error, fail, VUID, lost or validation. In every run of both arms it is one line, the crash watcher's start line.
No run was rejected, and no frame went over 100 ms in any run. The PR's own line in every PR run:
vk: 4096 programs have 548 pipeline layouts between them(1,024 have 171; 2,048 have 280-300).Checked, not checked
tests/run_all.py --changed ef6038ab, 171) on the cherry-pickedtree, and the same checks on
ef6038ab. The only failure the base doesn't have istests/test_shared_layouts.py, forthe
program_cache_borrow_allpoint above. The other 12 failures fail onef6038abtoo, on this PC.test_switch_table.py(failing on main here already) gains one line here, "RX_VK_LAYOUT_EACH is read(gpu/vk/vk_draw.cpp) and has no row in gpu/vk/switches.def". Main's switch table (#1300) wants a row for it.
programs and 9,689 compute pipelines of 4,059 compute programs, none failed to make.
with
RX_VK_SHOTS(a picture every 600 presents) and looked at the pictures. They showed the menu, the load and theturning field with the HUD, and nothing was wrong: no missing or wrong textures, no stale descriptors showing as
another object's texture. This is a glance, not an image comparison: the camera is at a different angle in each run.
disturbed (FSR, frame generation, the temporal actor pass, ASTC decode). None of those were on in this test. Also not
checked:
RX_VK_LAYOUT_EACH=1,RX_VK_PROF_FINE=1's push counts, other scenes, NVIDIA, Intel, Linux and the SteamDeck. No validation layers, no image comparison, no code review beyond the merge.
How to reproduce
(
devpacks.shis this PC's front forscripts/dev_packs.py;std_test.pyis our standard-test harness.)Steam Deck OLED results for this PR.
Setup: Steam Deck OLED (SteamOS, RADV Mesa 26.2.0-devel (git-5c65d4fc31)). Main
544229a2vs that main plus this PR (e2a24c1a, rebased asd8c55aa). Same game build, renderer-only difference, the same pinned Minimum-derived settings with overrides in both arms: fixed 50% render resolution, 1280×800 output on the 90 Hz panel, Handheld Profile off, plain window scaling and sharpness off, frame generation off, 15 W, main's clock policy, the FPS counter at its Dev level.Method: four distinct-launch round pairs, M P / P M / M P / P M: Kokiri standing (45 s) and Hyrule Field turning (30 s). The pairs are time-ordered and descriptive (n = 4); this doesn't claim statistical independence. Arrows are M → P; percentage changes are 100*(P/M−1). Renderer frame tracing was enabled in both arms.
Kokiri standing: no consistent throughput benefit resolved.
Hyrule Field turning: effect unresolved. Only one pair was admitted (round 3: 27.88 → 30.97 fps), so no general Field benefit is claimed. Main's Field window was refused in 3 of its 4 measured runs by our "1 jobs queued" admission rule, each time when its window crossed present 7200; the PR's windows started later and crossed no 1800-present boundary. The admission imbalance coincided with different present phases; it does not establish a PR effect or a fix for queued jobs.
Safety: No GPU fault, reset or timeout was recorded in this block. The covered Kokiri and Field safety windows had zero flash flags, and the views we inspected showed no obvious new artifact. Zero detector flags and sampled views are not universal correctness proof.
Limits:
View command line instructions
Manual merge helper
Use this merge commit message when completing the merge manually.
Checkout
From your project repository, check out a new branch and test the changes.