perf(runtime): keep the virtual core for an invalidate with nothing to wait for #1150
No reviewers
Labels
No labels
area/accessibility
area/audio
area/input
area/install
area/menu
area/renderer
area/saves
area/tooling
area/upscaling
beanstalk: approved
beanstalk: failed
beanstalk: needs approval
beanstalk: running
beanstalk: tested
ci: approved
good first issue
gpu/amd
gpu/igpu
gpu/intel
gpu/nvidia
help wanted
kind
bug
kind
chore
kind
docs
kind
feature
kind
performance
kind
question
needs measurements
platform/linux
platform/macos
platform/steam-deck
platform/windows
preview
priority
critical
priority
high
priority
low
regression
risk
high
risk
low
risk
medium
status
blocked
status
confirmed
status
duplicate
status
invalid
status
needs-info
status
wontfix
ci: approved
No milestone
No project
No assignees
1 participant
Notifications
Due date
No due date set.
Dependencies
No dependencies set
Reference
MagicBeanDev/Longshot!1150
Loading…
Reference in a new issue
No description provided.
Delete branch "dyst/Longshot:perf/invalidate-fast-path"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Type: perf · Risk: medium · Tested on: Windows 11, RTX 3080, in the game (Kokiri Forest, Hyrule Field) and game-free; Linux game-free · Needs: a Linux run in the game
Why
In CPU-bound places such as Castle Town and Hyrule Field, the game's model work checks each object's occlusion result once for every view, and each check gave up the thread's virtual core and then queued to get one back, even though there was nothing to wait for. On the Switch that call is a cheap cache operation and the thread just keeps running. See https://github.com/NotRexed/Hookshot/issues/995 (low fps with low GPU use in Castle Town and Hyrule Field).
What changed
Before it gives up the virtual core,
rx_nvn_invalidatenow asks the renderer whether anything in the range could make the call wait. When nothing could (no GPU write-back ever landed there, no texture the GPU wrote was ever listed there, and no live texture's storage lies there), the renderer does the one thinginvalidatewould have done on that path (it notes the CPU's read) and the call returns with the core still held. Otherwise the call runs as before, with the core given up. The answer comes from marks that are read without a lock: they are set under the existing tables' locks and never cleared.rx_nvn_invalidateinhost/rx_backend.ccallsnvn_invalidate_quickfirst and returns when it answers 1.MkwVkRenderergainsnvn_invalidate_quick(gpu/vk/vk_api.h,MKWVK_API_VERSION25 to 26).invalidate_quickingpu/vk/vk_replay.cpp, right afterinvalidate. It keeps the core only whentex_pages_any,texture_gpu_written_everandmemory_writeback_everare all false for the range. It always gives the core up when one ofinvalidate's diagnostics is on (RX_VK_LOG_INVALIDATE,RX_VK_LOG_READBACK), withRX_VK_INVALIDATE_WAIT=1, for ranges at or past 512 GB (virtual pools, which are not tracked), and withRX_VK_INVALIDATE_QUICK=0.gpu/vk/page_marks.h(new) holdsPageMarks: one bit per 4 KB page, in chunks made on first use. A writer marks with a release while it holds the lock of what the mark stands for. A reader loads with an acquire.wb_pages_noteingpu/vk/vk_memory.cppmarkswb_everunderwb_pages_mu, before it sets the pages' values. Every write-back value goes through it, and the values are never dropped, somemory_writeback_evergives exactly theeverthatmemory_writeback_valuegives, without the lock.vw_noteingpu/vk/vk_texture.cppmarksvw_everundervw_muwhen it lists a texture.texture_gpu_written_everis true wherevertexture_gpu_written_inis (it can also be true after a texture has left the list, which only sends that call the slow way).vk: invalidates since the last readbacks line: N kept their virtual core ..., M gave it up, follows eachvk: readbacks of GPU resultsline (every 3000 readbacks of written-back ranges). Each thread counts in thread-local counters and adds them to the shared ones every 256 calls, so the kept path writes no shared cache line per call. The readbacks line itself is unchanged.docs/development.md, next to the readback paragraph.tests/test_invalidate_quick.pywithtests/invalidate_quick_test.cpp(new).tests/test_invalidate_vcore.pygains thequickandquick-declinedcases.test_readback_part.py,test_late_fix_build.pyandtest_direct_buffer_writes.pygive their cut-out pieces the marks.test_audio_output_devices.pyandtest_microphone_test_gain.pypin version 26.Why it is exact. With the switches above off,
invalidate(vk_replay.cpp) does these things for a range with no page ever written back, no listed texture and no live texture storage. It notes the CPU's read (memory_note_cpu_read). It skips the write-back branch, which is behindever. It reaches the texture branch withtexture_gpu_written_infalse, then returns at!replay_behind()or at!tex_pages_any(...)whether or not the replay is behind. So it never waits, never takesreplay_muand never writes texels back. The look tests a superset of each of those conditions, so it keeps the core only whereinvalidatewould have done nothing but the note.test_invalidate_quick.pyreadsinvalidatefor these gates and fails if it gains a switch the look does not know.Threading. While it holds the core, the kept path takes no lock except
memory_note_cpu_read'swb_learn_mu. That is a leaf lock, taken only withRX_VK_WB_MAX_KBset (off by default), and nobody holds it across a wait. So the kept path cannot block on a thread that needs this core. Take a write-back noted just after the look said "never". Any reader the game orders after that write-back (a fence, an event or a lock, all of which come after the note's unlock) acquires past the release mark and goes the slow way. A reader the game does not order after it was already racing the note ininvalidate's locked look on main, and that look can come first too. So the guest sees what it saw before: the same values, the same waits. The occlusion-query slots that the hot calls read are written on the CPU (REPORT_COUNTERandocc_collect, viaguest_cpu_write, kept out of the mirrors' write-backs), never by a write-back, so the expected answer for them is "keep".Other PRs. #303 (closed, not merged) and the merged #700 change only the waits inside the written-back branch (
evertrue), which this PR leaves as it is and sends down the slow way. #700's count sits on the readbacks line, whose format is kept, and its check (test_readback_part.py) passes. #977 (open) changes the same waits and adds a line next towb_pages_note. Whichever lands second fixes a small textual conflict there and near the readbacks line, andtest_invalidate_quick.pythen asks for its new switch (RX_VK_READBACK_EARLY) to be listed. #799 (open) changesnvn_pool_createdinMkwVkRenderer, so whichever lands second bumpsMKWVK_API_VERSIONagain.Measurements
Verdict: a small, steady gain where the frame is CPU-bound, and no cost elsewhere. Hyrule Field: 51.4 fps against 50.5 (+1.7%; each build's two runs within 0.1 fps of each other, both of this PR's above both of main's), 1% low 30.0 against 27.3 (+10%). Kokiri Forest with the camera turning: 37.7 against 38.3, within that scene's own spread (main 38.37 and 38.32, this PR 38.78 and 36.62). The fast path is on and taken: on the busiest worker thread 31,444 invalidates kept their virtual core against 2,604 that gave it up.
Hardware: Windows 11 Pro N (22631), Ryzen 9 3900X (12 cores, 24 threads), 64 GB, RTX 3080, a hard drive
Scene: Kokiri Forest from the repository's
kokiri-startsave, the camera turning; Hyrule Field (the debug teleport toHyruleField_Child), the camera still; uncapped (HOS_FPS_MAX=240), headlessMethod: one game folder updated in place (
make_game.py --update, warm stores) between main and this PR, the order main, PR, PR, main; after each install one run of each scene withab2.py(25 s settle, a 45 s window): fps over the window, 1% low and frame times from the present trace; through the PC's job queue (--mode timing, nothing else running); with the session warm (an unmeasured run of each scene first); medians of 2. Castle Town Market was dropped: after its teleport the measured window was a blank screen (5 draws a frame) on main tooHow to reproduce: the two scenes above with each build; the counts are the
vk: invalidates since the last readbacks linelines in the log71b1edd8b2e0ebdcA first, colder set (right after the folder was updated to this main) is not used: main's own runs rose from 37-41 to 51 fps in Hyrule Field over that job as the session warmed, larger than any difference between the builds; its numbers are in the PR's comments.
Flags and settings
One switch added, on by default. Three existing switches of
invalidatealso turn the look off now, so they keep doing what they did (they read every call).RX_VK_INVALIDATE_QUICK0gives the core up for every call, as before0is for the A/B and for diagnosisRX_VK_LOG_INVALIDATERX_VK_LOG_READBACKRX_VK_INVALIDATE_WAITTesting
Tested:
python -I -B tests/run_all.py --jobs 8 --changed longshot/main. This is the 217 checks whose text names a changed file (the new one among them), on this head (b2e0ebdc, rebased on main71b1edd8), run withRX_CPMpointing at the builder's CPM cache so the Vulkan-header checks run too. Result: 205 passed, 11 failed, 6 skipped. All 11 failures also fail on a clean main worktree (6c8d6e6a, the base before the rebase; main's one new commit since touches the lift only):test_audio_output_devices,test_crash_dialog,test_fsr_uniform_ownership,test_guest_imported,test_import_reach,test_pads_after_menu,test_parallel_init_order,test_pending_guest_import,test_replay_write_order,test_vk_occlusion_queries,test_walk_key. The 6 skips are checks that need a Vulkan device, spirv-val or Linux.python:3.12, gcc 14.2), before the rebase (f3e13988 on6c8d6e6a): the same names with--exact. Result: 181 passed, 8 failed. The same 8 fail on main (byte_compile_tracked,test_bake_ds,test_bug_report*andtest_dev_packs, which need git in the container, plustest_guest_imported,test_pending_guest_import,test_vk_occlusion_queries,test_walk_key).vk_memory.cpp,vk_texture.cpp,vk_replay.cppandvk_main.cppcompiled to objects with the renderer's flags fromnx2/build.py, on Windows (llvm-mingw, x86_64-w64-mingw32) and Linux (clang 19).host/rx_backend.ccompiled with the host's flags on both.tests/test_invalidate_quick.pyand thequickcase oftests/test_invalidate_vcore.py, run against main's sources: both fail there.Tested: (head
5ea93a58:b2e0ebdcrebased onto main @55e8fbd5with no conflict, the same patches; on it the 226 checks that read the diff, not the whole suite,run_all.py --changed longshot/main: 186 passed, 32 skipped, 8 failed, and the same 8 fail on a clean main @55e8fbd5: test_audio_output_devices, test_bug_report, test_pads_after_menu, test_walk_key, test_guest_imported, test_pending_guest_import, test_crash_dialog, test_vk_occlusion_queries; test_invalidate_quick and test_invalidate_vcore pass) in the game, Windows 11, RTX 3080: Kokiri Forest and Hyrule Field, main71b1edd8against this PRb2e0ebdc, interleaved with the session warm (Measurements); the fast path's log lines.Not tested: Linux in the game; Castle Town Market (its teleport's window was a blank screen); a pool where most invalidates hit pages once written back (the slow path's share would grow: Question 2).
Must be tested:
vk: invalidates since the last readbacks linecounts with most kept (thehost/posixbuild ofrx_backend.c).Risks
What could break: guest threads' scheduling. The core is no longer handed to another guest thread at these calls. On the Switch there was no hand-off there either, but a guest loop that polled such a range with no other call into the OS would now keep its core where it used to let another thread on. The lifted code's interrupt polls (
hos_vcore_checkpoint) and the virtual cores' escape time still apply. Another risk is a write-back path added later that bypasseswb_pages_note, or a texture list that bypassesvw_note: the look would then say "keep" whereinvalidatemust wait.test_invalidate_quick.pychecks thatwb_pages.setandvw[t.gpu]are reached only through the marking paths.Turning it off:
RX_VK_INVALIDATE_QUICK=0(every invalidate gives up the core, as before), or revert the commit.Compatibility: the vk ABI changes (
MKWVK_API_VERSION26): the game andlongshot_vkmust come from the same build, which a normal build guarantees. The shader stamp is not touched, so the shader and pipeline caches stay valid. Nothing changes for saves orlongshot_options.ini.Issues
See https://github.com/NotRexed/Hookshot/issues/995
Questions for reviewers
686ae93eadded the release for calls that wait. It did not mention calls that don't wait.Checklist
Everyone
type(scope): description) of the status line's type.main.python -I -B tests/run_all.pypasses (any failure that also fails onmainis named).third_party/untouched (a new folder with aPROVENANCE.mdinstead).When it applies
Assisted-by:/Co-Authored-By:trailers.MKWVK_API_VERSIONbumped (gpu/vk/vk_api.h'sMkwVkHostorMkwVkRendererchanged).mainand passing here.tests/that reads changed source updated in this PR, not weakened.🤖 Generated with Claude Code
Tested in game: Windows 11, RTX 3080: Kokiri Forest (camera turning) and Hyrule Field, main
71b1edd8against this PR interleaved; Field 44.6 vs 39.1 fps, Kokiri unsettled; the fast path's log lines show it on and takenCorrection to the numbers in the description, before anyone reads too much into them: main's own runs in the same job kept getting faster as the session warmed up (Hyrule Field 40.85, 37.38, then 43.66 and 51.12 fps in the next PR's set; Kokiri 31.25 to 38.48), so this PR's Field gain (44.6 against 39.1) is within that drift and not shown yet. I'm re-running main against this PR with the game warm (the same order, main, PR, PR, main, both scenes) and will replace the Measurements with those; it stays WIP until then.
b2e0ebdc9f5ea93a5849The warm re-run is in, and it replaces the Measurements (the first set was taken while the session was still warming up, as the correction above said).
With the session warm, main @
71b1edd8against this PR @b2e0ebdc, main, PR, PR, main, Windows 11, RTX 3080:Also rebased onto main @
55e8fbd5(head5ea93a58, no conflict, the same patches;MKWVK_API_VERSIONstill 25 -> 26 there). The checks that read the diff run again on it next, then this leaves WIP.The checks that read the diff, on the rebased head
5ea93a58: 186 passed, 32 skipped, 8 failed, each of the 8 failing on a clean main @55e8fbd5too (named in Testing). Out of WIP.WIP: perf(runtime): keep the virtual core for an invalidate with nothing to wait forto perf(runtime): keep the virtual core for an invalidate with nothing to wait forView command line instructions
Manual merge helper
Use this merge commit message when completing the merge manually.
Checkout
From your project repository, check out a new branch and test the changes.