perf(nvn): recording threads wait less on the recordings' copy lock #766
No reviewers
Labels
No labels
area/accessibility
area/audio
area/input
area/install
area/menu
area/renderer
area/saves
area/upscaling
beanstalk: approved @ 1697603
beanstalk: approved @ 2c5a62e
beanstalk: approved @ 4fd82b7
beanstalk: approved @ 64d6d09
beanstalk: approved @ 6f91fae
beanstalk: approved @ 70a78f9
beanstalk: approved @ 8d361dc
beanstalk: approved @ 9d8356e
beanstalk: approved @ abf2227
beanstalk: approved @ d82f202
beanstalk: approved @ e5f1e40
beanstalk: approved @ ee8ab5f
ci: approved
good first issue
gpu/amd
gpu/igpu
gpu/intel
gpu/nvidia
help wanted
kind
bug
kind
chore
kind
docs
kind
feature
kind
performance
kind
question
needs measurements
platform/linux
platform/macos
platform/steam-deck
platform/windows
preview
priority
critical
priority
high
priority
low
regression
status
blocked
status
confirmed
status
duplicate
status
invalid
status
needs-info
status
wontfix
testing wanted
ci: approved
No milestone
No project
No assignees
2 participants
Notifications
Due date
No due date set.
Dependencies
No dependencies set
Reference
MagicBeanDev/Longshot!766
Loading…
Reference in a new issue
No description provided.
Delete branch "HumanBean/Longshot:perf-copy-lock"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Type: perf · Risk: medium · Tested on: Linux, RTX 3090 · Needs: Windows, any GPU
Why
Where the game is limited by the CPU (Kokiri Forest, the Market, Hyrule Field on a fast GPU), part of every frame is spent by the game's job workers waiting in line for one lock, not doing work. The workers record the game's command buffers, and each recording made, each control memory block added and each recording called took the same lock (
copy_lockinhost/nvn_native.c). In the Market on an RTX 3090 and a Ryzen 9 7900X, the ModuleSystemWorkers, which sit on the frame's critical path, spent 13% of their running time asleep on it. Every 25,000 recordings (about every 2 s there) one of them also held it for a sweep over all ~68,000 live recordings while the others waited.See #410 (CPU-side bottleneck in Kokiri Forest, low GPU use) and the work list's "Low fps with low GPU use in Castle Town and Hyrule Field" row.
What changed
Calling a recording no longer takes the lock in the normal case, the copies let go are only gone through once the replay can free some, and the sweep is done in small slices. What the game sees is the same: the same handles are accepted and refused, and the same copies are kept and freed.
check_handleinhost/nvn_native.cfirst looks the handle up in the live-handle set withoutcopy_lock(live_handle_seen). The set's table and size are published together at each growth (live_pub; old tables are never freed, so a reader can still be in one), and its slots are written with atomic stores, a moved entry into its new slot before it leaves the old one (live_handle_remove). A miss is asked again under the shared lock exactly as before, so a handle is only refused as before. A hit can only be a handleEndRecordingmade, which the game got from that call.free_dead_copiesreturns at once until the replay has passed the oldest copy waiting (dead_first, kept bydead_copy_add). Before, everyAddControlMemorywent through all of them with the exclusive lock held.sweep_overwritten_copiesgoes over 2688 of the 65536 buckets every 1024 recordings (SWEEP_SLICE,SWEEP_EVERY), every bucket once in ~25,000 recordings as before, instead of all of them at once every 25,000.tests/test_nvn_host_tables.pycompileslive_handle_seenand checks that it agrees with the locked lookup after adds and removals, and through a growth of the table.Measurements
Verdict: faster in all six phases measured, by 1 to 4% (median frame time); the most while turning the camera in the Market (11.33 to 10.89 ms, +4% fps). In three of the six phases the slowest session with the change beat the fastest without it.
Hardware: CachyOS (Linux 7.3.3), RTX 3090 24 GB, NVIDIA 620.70.06, Ryzen 9 7900X (the game kept to 12 of its 24 threads, CPUs 0-5 and 12-17, with nothing else timed or building beside it), 32 GB
Scene: a fresh copy of one save in Kokiri Forest each session, then Hyrule Field and the Market by the debug teleport; child Link, day. Headless sway, fullscreen; the game chose a 1280x720 output there (the frame is CPU-bound: GPU 4.5 to 8.7 ms of a 9 to 12 ms frame). Defaults: Threaded rendering Auto (on with 24 threads), the game's own upscaler, no frame generation;
HOS_FPS_MAX=240(uncapped).Method: in each area 15 s standing, then 20 s turning the camera at full speed (right stick held); frame times from
RX_VK_FRAME_LOG, fps from the log's 120-present lines. Six sessions interleaved (this PR, main, main, this PR, this PR, main), the same binaries otherwise: main is adev_packs.pybase ata00c485c, this PR is that base with this change as a pack. The median of each session's median frame, then the median of the three sessions.How to reproduce: any save;
RX_VK_FRAME_LOG=<csv>andHOS_FPS_MAX=240, standing and turning in the same spot each session.a00c485c64d6d098(ona00c485c)Where the time went, from one sampled Market session each (
HOS_SAMPLE_ALL, every game thread every 2 ms): the share of the ModuleSystemWorkers' running samples in libc, by the host call above it. The libc time undercheck_handle,EndRecordingandAddControlMemoryispthread_rwlock_rdlock/wrlocksleeping on the futex:a00c485ccheck_handle(CallCommands), the lockEndRecording, the lockAddControlMemory, the lockThe PR is rebased on main
aa95d35fsince: that commit changes only how Arc cards read occlusion counts (gpu/vk, Intel Arc alone), which does not run on this NVIDIA card, and nothing this change touches, so the numbers hold for this head.Each session's numbers are in
docs/tested/2026-11-15-recording-copy-lock-humanbean-run01.mdtorun06.md.Flags and settings
None. The sweep's
HOS_NVN_NO_SWEEP=1(off) andHOS_NVN_REC_SHIFTare unchanged.Testing
Tested: in the game on Linux (RTX 3090, NVIDIA 620.70.06, CachyOS, headless sway): the six timed sessions above plus one sampled one, each Kokiri Forest, Hyrule Field and the Market by the debug teleport, about 3 minutes of play each. No crash, no device lost, no
command handle ... is not a live recording, and the sweep still let copies go (nvn: N copies of recordings whose control memory was reused let goin the debug log, as on main). The game-free checks (tests/run_all.py) ran on Linux: nothing fails that doesn't fail on main too (the checklist names them), andtests/test_nvn_host_tables.py, which covers the new read, passes.Not tested: Windows, where the SRW lock spins before it sleeps, so the lock's waits there may have cost less and the gain may be smaller; AMD and Intel GPUs; the Steam Deck and CPUs with 8 threads or fewer; sessions longer than a few minutes (the live copies' count after the slices).
Must be tested:
Risks
What could break: the host's recordings of command buffers. A recording refused that should be accepted would end the game with
command handle ... is not a live recording; the lock-free read can only miss, and a miss is checked again under the lock as before. A recording accepted that has been let go would need the game to call it while adding its control memory again, the same race main has; its copy's magic is cleared before it leaves the set and its memory is freed only after the replay passes it. Memory: the live-handle set's old tables are kept (at most as much again as the current one: ~2 MB with ~68,000 live recordings).Turning it off: revert the commit.
Compatibility: Nothing changes: no save, option, cache or ABI change.
Checklist
Everyone
type(scope): description) of the status line's type.main.python -I -B tests/run_all.pypasses (any failure that also fails onmainis named). (Linux, at64d6d098: 509 passed, 14 failed, 70 skipped. The 14 fail the same way on mainaa95d35fon this PC, none near this change:test_options_budget_once,test_readback_late,test_vk_pgo,test_bake_portability,test_cpm_discovery,test_crash_dialog,test_walk_key,test_make_game_cache,test_lift_cleanup_error,test_linux_preflight,test_pipeline_tour,test_env_on,test_mods_size_table,test_appimage_player_build.test_nvn_host_tables.pypasses.)third_party/untouched (a new folder with aPROVENANCE.mdinstead).When it applies
Assisted-by:/Co-Authored-By:trailers.docs/tested/for each new measured run (format indocs/tested-configurations.md).tests/that reads changed source updated in this PR, not weakened.🤖 Generated with Claude Code
Description filled in: the measurements (six interleaved sessions on an RTX 3090 + 7900X, Linux: 1-4% faster in all six phases) and the sampled lock waits before and after. Windows and the Steam Deck still to be run.
Why the numbers measured on
a00c485chold on the rebased head (aa95d35fonly changes Arc's occlusion reads); the Windows and Steam Deck runs asked of the Test Farm below.@Beanstalk windows: on main and on this PR, the same settings each, play a save in Kokiri Forest, then go to Hyrule Field and the Castle Town market (Debug tab, Teleport): in each area stand still for 15 s, then turn the camera at full speed for 20 s. Expecting the same or a better frame rate with this PR (median frame time per area from RX_VK_FRAME_LOG, or the FPS counter), and no crash or 'command handle ... is not a live recording' in longshot_log.txt.
@Beanstalk steamdeck: the same route at the Deck's default settings, main and this PR: Kokiri Forest, then Hyrule Field and the Castle Town market by the Debug tab's Teleport, 15 s still and 20 s turning the camera in each. Expecting the same or better fps with this PR and no crash.
Beanstalk test farm
Commit
64d6d09· tier reviewed · approved by HumanBean · dashboardWhat will be tested · AI plan (
gpt-6.1-terra)On Steam Deck at default settings, main and the PR will follow the same child-Link route through Kokiri Forest, Hyrule Field and the Market, taking about 16 minutes. Each area has a 15-second still measurement and a 20-second measurement after a full-stick camera turn; the plan cannot measure while the stick step is running. It compares FPS, frame times and pictures, and fails for lower FPS, a crash, a device-lost or invalid-recording log message, or a substantial picture difference. No Steam Deck is listed online, so this target will wait for one. No tester online now matches steamdeck: that target waits for one.
Done. Run 2: re-planned by HumanBean. Comment
@Beanstalkto run it again.Ran
tests/run_all.pyon Linux at64d6d098: 509 passed, 14 failed, 70 skipped. The 14 also fail on mainaa95d35fon the same PC (listed in the checklist), andtest_nvn_host_tables.pypasses, so the run_all box is ticked. The Windows llvm-mingw build is still open.WIP: perf(nvn): recording threads wait less on the recordings' copy lockto perf(nvn): recording threads wait less on the recordings' copy lock