WIP: perf(vk): a draw keeps its pipeline and dynamic state while the NVN state they are made from stands #2059
No reviewers
Labels
No labels
area/accessibility
area/audio
area/input
area/install
area/menu
area/renderer
area/saves
area/tooling
area/upscaling
beanstalk: approved
beanstalk: failed
beanstalk: needs approval
beanstalk: running
beanstalk: tested
ci: approved
good first issue
gpu/amd
gpu/igpu
gpu/intel
gpu/nvidia
help wanted
kind
bug
kind
chore
kind
docs
kind
feature
kind
performance
kind
question
needs measurements
platform/linux
platform/macos
platform/steam-deck
platform/windows
preview
priority
critical
priority
high
priority
low
regression
risk
high
risk
low
risk
medium
status
blocked
status
confirmed
status
duplicate
status
invalid
status
needs-info
status
wontfix
ci: approved
No milestone
No project
No assignees
2 participants
Notifications
Due date
No due date set.
Dependencies
No dependencies set
Reference
MagicBeanDev/Longshot!2059
Loading…
Reference in a new issue
No description provided.
Delete branch "siantm/Longshot:perf/vk-draw-state-gen"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Summary
Each draw past
pipe_fastput its pipeline key together and looked it up, and packed its dynamic state's inputs to compare them with the command buffer's. The replay now counts, on each thread, the changes to what those are made from (draw_state_gen,gpu/vk/vk_draw.h):gpu/vk/vk_replay.cppmoves it when a state record (render targets, viewports, scissor, depth range, the bound states, vertex attributes and streams, stencil, polygon offset, blend colour, primitive restart, depth clamp, rasterizer discard) writes another value, at any record not known to leave that state alone (op_moves_draw_state) and at each replay's start. While it stands, with the same state object, the same resolved targets (targets_gen) and the same program (object, key and variant) and topology, a draw takes the pipeline entry of the draw before (pipe_kept) and, with the command buffer's dynamic state valid, leaves its dynamic state as it is (dyn_kept), without packing or comparing. What is kept goes at each recording's begin. The result expected is less work for draws of the same state in a row.Depends on #2049 (and through it #2048 and #2047: the switch,
targets_gen, the worddynamic_statereads once). Their commits show in this diff until they merge. Open #1297 (a pipeline only with the program object it was built from) does not conflict:pipe_keptkeys on the program object too.Switch:
RX_VK_BIND_CHANGED(kill, on).=0: every draw as on main.From bd3ea307 on
exp/x86-asm-sync.Read first:
draw_state_gen,kOpsDrawStateandop_moves_draw_stateingpu/vk/vk_draw.handgpu/vk/vk_replay.cpp;pipe_keptanddyn_keptingpu/vk/vk_draw.cpp.Testing
Tested: Windows 11, i7-9700, RTX 4060, the builder's llvm-mingw clang 23; head
c27bd1f3on #2049's head3562fe6b(on main @75317e1f). Every check ran throughtests/run_all.py --strict --exactin build jobs of the PC's job queue, in a hidden console (logt-dc4-vk-draw-state-gen):tests/test_draw_state_gen.py(new: every case ofreplay()that writes a fielddyn_packor the key reads moves the generation; the records listed as leaving the state alone write none; the state records are exactlykOpsDrawState; any other record and each replay's start move it;pipe_keptanddyn_keptcompare everything they were made from): pass.--changed 3562fe6b; its own are among them): 246 passed, 32 failed. Failed:tests/test_fsr_image_retirement.py,tests/test_fsr_uniform_ownership.py,tests/test_pipeline_play_cache.py,tests/test_sampler_admission.py,tests/test_draw_stream_visibility.py,tests/test_capture_readback_visibility.py,tests/test_crash_record.py,tests/test_device_lost_exit.py,tests/test_dev_packs.py,tests/test_draw_binds.py,tests/test_dev_ab.py,tests/test_crash_dialog.py,tests/test_flush_nothing_held.py,tests/test_frame_overlap.py,tests/test_gpu_wait_stats.py,tests/test_linux_pressure.py,tests/test_mesh_conv.py,tests/test_prepare_cache_save.py,tests/test_perf_report.py,tests/test_present_queue_default.py,tests/test_profiling_mode.py,tests/test_present_source_crop.py,tests/test_renderer_new_handler.py,tests/test_recorder_own_idle.py,tests/test_replay_write_order.py,tests/test_slow_frames.py,tests/test_sparse_mirror.py,tests/test_target_reuse_memory.py,tests/test_ub_update_batch.py,tests/test_upscaler_rows.py,tests/test_vk_occlusion_queries.py: each fails on main @75317e1ftoo (the same checks on main in the batch before, logt-dc3-base).tests/test_recording_admission.pyfails on main @75317e1ftoo, now and then: its "worker slot ownership" case failed on main in 7 of 10 runs made alternately with this branch's (logst-dc5-base-ra-*), with the same message.It has NOT run in the game on this PC yet.
Still to test:
RX_VK_BIND_CHANGED=0and this PR with it unset (on), each warmed a minute in the scene first, then interleaved (base, =0, unset, unset, =0, base), two runs each, expecting the same picture all three ways (no wrong blend, depth or stencil, no wrong-sized viewport) and unset the replay's time a frame the same as or lower than the base's (by hand)RX_VK_BIND_CHANGED=0and unset, warmed, in ABBA order, expecting the same picture every way and no crash or validation error (@Beanstalk linux+nvidia, linux+amd, windows+amd, windows+intel, steamdeck: Kokiri Forest a minute, its base PR alone against this PR with RX_VK_BIND_CHANGED=0 and unset, ABBA, the same picture)tests/test_draw_state_gen.pywith--cxx g++and--cxx clang++, expecting PASS (by hand)Measurements
Verdict: not measured: no micro-benchmark of this port and no run in the game on this PC yet; the in-game A/B is in Still to test.
RX_VK_BIND_CHANGEDships on (kind kill), the branch's default; it is not measured on main.Risks
A state record that changes what a pipeline or the dynamic state is made from but is missing from
kOpsDrawStateand listed as leaving it alone would let a draw keep the state of the draw before: a wrong blend, depth, stencil or viewport for that draw. The list is checked againstreplay()'s cases by the test; any record not listed moves the generation. Thevk_replay.cpphooks are fitted onto main's rewrittenreplay(). Turned off byRX_VK_BIND_CHANGED=0. The renderer is compiled again after updating.Checklist
Ready for review
main; the title is a Conventional Commit.mise run testpasses (any failure that also fails onmainis named). Not yet: only the checks in Testing ran.main, or Measurements says why there is nothing to measure. Not yet (Measurements).third_party/untouched; nothing GPL-incompatible.When it applies
gpu/vk/switches.def. Its row is in; not measured yet.tests/that reads changed source is updated, not weakened.Assisted-by:/Co-Authored-By:trailers.push_constants pushed the program's whole block (page table, task first, render area, fallback addresses) at every draw whose program reads it, though the draws of a pass push the same block one after another. It now leaves the push out when the last block this thread recorded was the same bytes, for the same stages, into the same command buffer (push_standing). vk_work.h counts every vkCmdPushConstants on the thread (push_epoch), so a push of anyone's in between makes the next draw push again. The block is forgotten where the command buffer's may be something else: at each recording's begin (binds_begin_recording), a masked clear, the temporal upscalers' stand-ins, each dispatch's end (its stand-ins call NGX / FidelityFX, which push past vk_work.h), and (new here, beside the branch's sites) in descriptors_disturbed, which every upscaler and library pass on main calls after recording its own pipelines. A direct mesh draw no longer pushes entry 0's first a second time: it is in the block just sent or left standing. The first of RX_VK_BIND_CHANGED's parts on the branch (bd3ea307, "draw-binds"); its other parts follow as their own commits. Its index-buffer part (bind at the pool buffer's start, draw from the first index) is main's RX_VK_INDEX_REBASE already and is not ported. Switch: RX_VK_BIND_CHANGED, kill, on by default (the branch's default); =0 (or false, off, no) pushes at every draw as before. Read once, at its first use. Row in gpu/vk/switches.def. Not measured on its own on the branch ("the saving is not measured yet"); not measured in play. Test: tests/test_push_changed.py runs push_standing (cut from the source) and reads the wiring: vk_work.h's count, the check before the push, the block kept after it, every place it is forgotten, the mesh first; and the switch's reader runs in fresh processes (unset, 0, off, false, 1, on and another value, twice each, set again after its first read). The three fixtures that compile draw_begin_recording stand in binds_begin_recording. Assisted-by: Claude Code@Beanstalk This makes the renderer track every draw's pipeline and dynamic state with a generation count (gpu/vk), behind RX_VK_BIND_CHANGED, a kill switch on by default; RX_VK_BIND_CHANGED=0 is main's way. It is on #2049, #2048 and #2047, whose commits are in its diff. It has not run in the game on these PCs yet. Please play Kokiri Forest and Hyrule Field, two minutes each, the PR against main on the same PC, on the PR's side with the switch unset and with RX_VK_BIND_CHANGED=0, with a picture in each compared with main's, on Linux with NVIDIA, Linux with AMD (RADV), Windows with Intel and the Steam Deck. The same picture as main, no crash, device lost, validation error or FATAL line; unset, frame time p99 equal to main's or better and to the switch off.
@siantm only members of
BigBeanscan start the farm. One of them can approve this request by reacting 👍 to it or to this comment.View command line instructions
Checkout
From your project repository, check out a new branch and test the changes.