WIP: perf(vk): a mirror sync's pages from the captures go in one compute dispatch, not one copy region each #326
No reviewers
Labels
No labels
area/accessibility
area/audio
area/input
area/install
area/menu
area/renderer
area/saves
area/upscaling
good first issue
gpu/amd
gpu/igpu
gpu/intel
gpu/nvidia
help wanted
kind
bug
kind
chore
kind
docs
kind
feature
kind
performance
kind
question
needs measurements
platform/linux
platform/steam-deck
platform/windows
preview
priority
critical
priority
high
priority
low
status
blocked
status
confirmed
status
duplicate
status
invalid
status
needs-info
status
wontfix
testing wanted
No milestone
No project
No assignees
2 participants
Notifications
Due date
No due date set.
Dependencies
No dependencies set
Reference
MagicBeanDev/Longshot!326
Loading…
Reference in a new issue
No description provided.
Delete branch "proci/Longshot:perf/mirror-sync-pass"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Why
A mirror sync copies the pages a capture holds into the mirror as copy regions. In Hyrule Field that is about 1,300 a frame, a page or a few each, and the GPU reads them from host memory over PCIe one after another. At the spot below they took 1.12 ms of GPU time a frame on
main. This sends them through the write-back diff's compute pass instead, one dispatch a sync, and there they took 0.33 ms.This is the change merged in Hookshot as NotRexed/Hookshot#3023 (2026-11-13), ported to Longshot and measured again here. None of the restore PRs carries it.
What changed
mirror_sync(gpu/vk/vk_memory.cpp) no longer records a capture buffer's runs of pages asvkCmdCopyBufferregions.msync_pass_addturns them into the write-back diff's chunks, andmsync_pass_flushrecords them as one dispatch in the upload command buffer, where the copies were.shaders/wb_diff.compis a copy when its shadow is its destination: it reads the source, compares it with the destination and writes it there where they differ. So each chunk is{mirror = the capture's page, shadow = guest = the mirror's}.wbd_split): 16-byte vectors of at most 64 KB where source and destination are alike within 16 bytes, words otherwise.memory_sync_mirrors_before_segments(the parallel replay's presync) holds every pool's chunks until its loop is done, and dispatches them once.texture_uploads_recorded) waits for the dispatch as it did for the copies.wbd_table_get), whose tables are 128 KB or more a dispatch.vk: mirror syncs' pass: no table room). It never showed in the runs below.STORAGE_BUFFERandSHADER_DEVICE_ADDRESStoo. That iscap_usage(), which the late pages' fix (#248) added for the same bits: it now gives them when either is on (used bycap_spare_makeandcap_buf_get).shadow_sync_copy, from #248), as the copies' do. The pass writes the mirror only.vecsis 0 or 1, and the push constant the write-back uses for the stamp's value is 0.wbd_pipe_ready(): no subgroup votes in compute);vk: mirror syncs' pass (600 submits): N dispatches, N chunks; N syncs as copies (no table room).tests/test_msync_pass.py(new, game-free).RX_VK_MSYNC_PASS=0in docs/development.md, afterRX_VK_WB_DIFF=0and #248's two switches; one file in docs/tested/ for each of the ten runs.Cost:
RX_VK_PARALLEL=4(the presync's, 1,290-1,350 chunks), 29 with the serial replay.memory synctime is the same within the runs' spread (table below).Flags and settings
RX_VK_MSYNC_PASS=0Read once by the switches' rule (
env_value_off) and logged once at the start:vk: mirror syncs' pages from the captures by a compute pass (RX_VK_MSYNC_PASS), or... as copy regions, as before. The flag starts with none oftranslator_stamp's prefixes, so setting it keeps the program cache. It is in docs/development.md's switch table, afterRX_VK_WB_DIFF=0and #248's two switches. The README lists the switches players use, so it is not there.Measurements
main:
e51e28e9(2026-11-13)this PR: perf/mirror-sync-pass @ d49ffb35 (this change on
e51e28e9), rebased since onto main76699aadase089b486Hardware: RTX 5080 16 GB, driver 610.88; Ryzen 9 7900X3D (SMT on), 64 GB; Windows 11 Pro 26200, fullscreen
Scene: Hyrule Field, the same save every run, standing still with the camera untouched. 2560x1440, DLAA, frame generation off, frame rate cap Uncapped, Reflex off, VRAM budget 11 GB;
RX_VK_PARALLEL=4,HOS_PIN=*=FFF.Method: each run loads the save, stands still for 2 minutes or more, then opens the pause menu and closes the game from it.
e51e28e9(longshot e51e28e9 (source 8dd02aab)). Each side'slongshot_vk.dllwas built alone from its commit (RX_VK_ONLY, release, llvm-mingw). The main one is the same file as the full build's.longshot_log.txt's frame lines (vk: N presents, X fps, every 120 presents), about 53 s.RX_VK_FRAME_LOG, its presents matched to the frame lines over the same window. The 1% low is 1000 over the mean of the slowest 1% of frame times.RX_VK_PASS_TIMING=1), main then this PR. These are the medians of the profile's reports (one every 600 presents) in the same window.gprof_on()in vk_replay.cpp's segment split), so this pair ran the serial replay. Its timestamps also order every span. Those runs' fps (64 instead of 68) are not used.How to reproduce: the settings above, a save standing in Hyrule Field, 2 minutes still. Read the frame lines and the
replay timelines. Then a pair withRX_VK_PASS_TIMING=1, reading itsmsync copies (CPU pages from a capture -> mirror)(on main) ormsync pass (CPU pages from a capture -> mirror, one dispatch)span.e51e28e9(run 1 / run 2)memory sync, ms a frameRX_VK_PARALLEL=4(frame-time runs, from its(600 submits)lines)mainruns differ more (2.7 fps) than the two sides' medians (0.3).submitabout 6.3 ms of a 14.5 ms frame, the GPU's recorded work 1.8 ms), not by the GPU, so the GPU time the pass saves does not show as frame rate here.GPU timeline. It counts the recordings' command buffers only, not the upload ones where the syncs are. So it reads 1.8 ms a frame on both sides.main @
e51e28e9, frame-time run 1 (a frame line in the window and the lines before it):this PR @ d49ffb35, frame-time run 1:
The GPU profile, one report each (main, then this PR):
On Hookshot the same change measured the same way gave 1.13 -> 0.34 ms with the serial replay (NotRexed/Hookshot#3023). With
RX_VK_PARALLEL=4the pass is one dispatch a frame instead of 29. The unprofiled runs can't time it, so its GPU time there is not measured.After the rebase. The runs are on
e51e28e9. The branch is now on main76699aad, which has the late pages' fix (#248); the conflict with it is resolved (Reviewer notes).RX_VK_PARALLELis above 1, and all ten runs set it to 4 (the profile pair too, though the profile ran the serial replay). So with these settings it stays off. Without the switch, the default, it is on with this pass, and that has not run in the game.Testing
Tested:
Game-free:
tests/test_msync_pass.py(new) cutsmsync_pass_on,wbd_split,WbdChunk,MsyncCopyandmsync_pass_addout of vk_memory.cpp, as they are. It compiles them with-std=c++20 -Wall -Wextra -Werroragainst a fake pool, fake buffers and their addresses. It checks:shadow == guest, and a vector chunk 16-byte aligned at both ends;0,false,OFF,Noare off;cap_usage(), which gives them storage and an address when the pass is on;shadow_sync_copy);msync_pass_flush's push constants are the write-back's, and that the shader'sChunkfields areWbdChunk's, in order.Each of the three new checks for #248 (the usage function, its storage and address bits, the shadow copy) fails when its rule is undone in a copy of vk_memory.cpp.
With
main's vk_memory.cpp it fails at the first cut (the pass is not there); with this change it passes (llvm-mingw clang 23.1.5, Windows 11).run_all:
python -I -B tests/run_all.py --jobs 8on Windows 11 (llvm-mingw clang 23.1.5).e089b486: 517 passed, 4 failed, 34 skipped.main76699aad: 516 passed, the same 4 failed (test_finish_output,test_oot_state_tap,test_pad_layout,test_tested_configurations), 34 skipped.test_tested_configurationson both: only its check of the CLI through an ASCII pipe fails, by\r\nagainst\non Windows. The check that validates the repository's run files passes, this PR's ten included.tests/andnx2/read it besides this PR's, #248's three among them, and all pass unchanged on the branch. Three C++ fixtures read it too, and the checks that compile them pass (test_mirror_inherit.py,test_recorder_sync.py,test_retired_give_back.py). None needed changing.Renderer:
longshot_vk.dllbuilt frome089b486(RX_VK_ONLY, release, llvm-mingw): 0 failed, no warnings.In the game: the ten runs in Measurements, on
e51e28e9, withRX_VK_PARALLEL=4(the profile pair serial). I played them and closed each from the pause menu.VK_ERRORline.Not tested:
mirror_sync(the fix's dispatch, then the runs taken) and in the shadow copy; the test checks those from the text only.RX_VK_PARALLEL=4. The profile turns that replay off.Needs other testers:
RX_VK_MSYNC_PASS=0. Expect the log'smirror syncs' passlines with0 syncs as copies, nothing wrong on screen, and withRX_VK_PASS_TIMING=1themsync passspan below themsync copiesone (request to open).Screenshots
Nothing on screen: the change moves where the GPU copies the same bytes.
Risk and rollback
RX_VK_MSYNC_PASS=0turns it off, or the commit can be reverted alone.Reviewer notes
msync_pass_add/msync_pass_flushand their call inmirror_sync(gpu/vk/vk_memory.cpp), and the presync inmemory_sync_mirrors_before_segments.N from guest memory as it was at the replayin themirror syncs (600 submits)lines, is 0 in every run's window on both sides. Non-zero ones show only at the save's load and at the close, on both sides alike.cap_usage(), this PR withcap_buf_usage(). This PR's is gone:cap_usage()gives the two bits when the fix or the pass is on.shadow_sync_copy), after the mirror's copy. The runs the pass takes get that copy as well, recorded where the mirror's copy was. Merged as git left it, they skipped it.mirror_syncfirst runslate_fix_emit, whosefull_barrierafter the fix's dispatch covers all reads, shader reads included. The pass's dispatch comes after that, at the end ofmirror_syncor after the presync's loop, so it reads the fixed words. The fix's own barrier before its dispatch, which orders the shadow copies before it reads them, is unchanged.wb_diff.compstamps: bit 1 of a chunk'svecs, and the push constant that was padding. The pass sets neither.Issues
No Longshot issue found for the mirror syncs' copies (searched 2026-11-13).
Checklist
For everyone
type(scope): description).main.python -I -B tests/run_all.pyrun and passing (any failure that also fails onmainis named above).third_party/untouched (a new folder with aPROVENANCE.mdinstead).If applicable
Assisted-by:/Co-Authored-By:trailers.docs/tested/for each new measured run (format indocs/tested-configurations.md; no appended row or assembled view).Written with Claude Code (Claude Opus 5.5). I played the in-game runs. The commit carries a
Co-Authored-Bytrailer.🤖 Generated with Claude Code
08c81e4c58e089b48642Read head
e089b486against maincf605c85: the complete diff, description, empty comments/reviews, and the capture, upload, shader, recorder and allocation paths. Eight compiled fixtures pass with GCC and Clang; the shader and tested-run checks pass too. The new test against main stops at the missing pass before compiling. No game or renderer build was run.What's good
mirror_sync: each capture buffer's regions become a copy command. Reusing the existing chunk format and shader keeps the change small, and the shader consumes the new chunks as intended.docs/tested/, not additions to the frozen table.Must change
nonCoherentAtomSize. Ingpu/vk/vk_memory.cpp, lines 5278-5279, the new table is written and flushed atb->fill, which advances only by its byte count at line 5288.msync_block_get, lines 5194-5198, permits appending while earlier readers of the block are still pending.create_bufferdoes not require coherent memory. With a 128-byte atom, two one-chunk tables occupy bytes[0,32)and[32,64); flushing the second also covers the first table while its submitted dispatch may still read it. Vulkan counts that whole atom as a host write even when the neighboring bytes are unchanged, so this introduces a host/GPU data race in the chunk table. Align each independently published table, including the capacity calculation, asstagingalready does ingpu/vk/vk_texture.cpp, lines 456-463. Add a test of the actual block allocator and flush path with an earlier table pending and a non-coherent atom larger than one chunk. The rule is explicit at https://docs.vulkan.org/refpages/latest/refpages/source/vkFlushMappedMemoryRanges.html.Measurement and smaller points
memcpyand never executesmsync_block_getormsync_pass_flush. Its passing result does not cover the lifetime and flush issue above.cf605c85. The source-check jobs are blocked, not passing: https://git.magicbean.dev/MagicBeanDev/Longshot/actions/runs/3782.Not checked
No GPU execution, Windows execution, full build, full suite or new performance measurement here. The non-coherent race is established from the allocator and Vulkan's synchronization rule, not a reproduced visual fault. The default serial replay with the late-page fix needs current-head gameplay and synchronization-validation evidence; the published runs predate that combination. AMD, Intel, Steam Deck, long-session behavior and the underlying timing captures remain unjudged.
View command line instructions
Checkout
From your project repository, check out a new branch and test the changes.