Perf: possible performance improvement leads #1020
Labels
No labels
area/accessibility
area/audio
area/input
area/install
area/menu
area/renderer
area/saves
area/tooling
area/upscaling
beanstalk: approved
beanstalk: failed
beanstalk: needs approval
beanstalk: running
beanstalk: tested
ci: approved
good first issue
gpu/amd
gpu/igpu
gpu/intel
gpu/nvidia
help wanted
kind
bug
kind
chore
kind
docs
kind
feature
kind
performance
kind
question
needs measurements
platform/linux
platform/macos
platform/steam-deck
platform/windows
preview
priority
critical
priority
high
priority
low
regression
risk
high
risk
low
risk
medium
status
blocked
status
confirmed
status
duplicate
status
invalid
status
needs-info
status
wontfix
ci: approved
No milestone
No project
No assignees
2 participants
Notifications
Due date
No due date set.
Dependencies
No dependencies set
Reference
MagicBeanDev/Longshot#1020
Loading…
Reference in a new issue
No description provided.
Delete branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
A place to collect leads for making frames faster: where profiling shows frame time going, what has been tried, and what has been ruled out. Comment with a lead (the area, the hardware, what the profile showed, and how much of the frame it is), or with a result when you try one. When someone takes a lead, they say so here and link the pull request.
The first leads, from profiling Kokiri Forest, Hyrule Field and the Market on Linux (RTX 3090, Ryzen 9 7900X), are in the first comment. Related: #410 (CPU-side bottleneck in Kokiri Forest), #766 (the first fix from it, merged).
Leads from profiling on Linux (RTX 3090, NVIDIA 620.70.06, Ryzen 9 7900X with the game on 12 of its 24 threads, CachyOS; main
a00c485c, Threaded rendering Auto (on), uncapped, 1280x720 output). Kokiri Forest, Hyrule Field and the Market are all limited by the CPU there: GPU time is 4.5 to 8.7 ms of a 9 to 12.5 ms frame. The render thread is not the limit either. The frame's critical path is the game's main thread waiting on its job workers (ModuleSystemWorkers, BaseProcExecutors).Ranked by what I'd expect each to gain:
copy_lock,host/nvn_native.c). After #766, the job workers still spend ~4% of their running samples waiting on it: about half undernvnCommandBufferEndRecording, about half underAddControlMemory→release_copies, both exclusive. On Linux the SRW shim (host/posix/windows.h) maps SRW locks topthread_rwlock, which sleeps on the futex at once, where Windows' SRW lock spins first. A short spin in the shim would help every SRW lock on Linux. Holding the lock for less time in EndRecording would cut it further. Guess: 1-2% in CPU-bound areas.capture_make,copy_part,collect_in; ~0.2 ms per submit, ~6 submits a frame). The default 4 capture helper threads are already best:RX_VK_CAPTURE_THREADS=0was 0.254 vs 0.195 ms per submit with lower fps, and 2 was slower too. The gain would have to come from copying less per submit or merging submits. It is the biggest item, but it is in the renderer and riskier.vkQueuePresentKHRblocks the render thread for 3-4 ms a frame.RX_VK_PRESENT_THREAD=1freed ~3 ms of it (idle 1.3 → 4.2 ms a frame) but moved fps by under 2% here, because this PC is limited by the game threads. With Threaded rendering off, on 8 threads or fewer, and maybe on the Steam Deck, it could matter more. It is off for swapchain external-sync and a 5 s drain; one idea is to drain the presenter beforevkAcquireNextImageKHRinpresent_image. First step: measure with Threaded rendering off to see if it is worth it.obj_lookup: ~4.4% of the critical path in the Market. Not looked into yet: what it looks up, and whether it can be cached.Ruled out:
rx_nvn_invalidate→wait_rec_seq(~2.6% of the workers' time): a real wait for a GPU readback.How it was measured:
HOS_SAMPLE_ALL(every game thread every 2 ms) andHOS_CHAINfor the critical path, frame times fromRX_VK_FRAME_LOGwithHOS_FPS_MAX=240; in each area 15 s standing and 20 s turning the camera at full speed. One gotcha: on Linux,prof.waitingdoesn't countpthread_rwlock_*lockfutex sleeps, so they show up as "running" libc time. That is how the copy lock waits were found.Possibly related, for context:
More leads, and results since rc.4 (Linux, RTX 3090, NVIDIA 620.70.06, Ryzen 9 7900X on all 24 threads, CachyOS). This time at 2560x1440: the game's own upscaler at 100%, uncapped, V-Sync off, frame generation off. In the Market: 5 s walking forward, then 20 s walking with the camera turning. Builds are player builds from the Builder AppImage, with runs interleaved between versions.
Where main stands against rc.4 in the Market (medians of 4 clean runs; walking / walking + turning):
53f96d78)1764fed26fb13fe5That's about +5% walking and +12% turning. The float helpers (
a64_fpx_*) went from ~5.5% of the game threads' running samples to ~2.5%. #838 (6fb13fe5) moved nothing measurable on this PC. Turning numbers carry some noise, because the circle sees 1,000-1,400 draws from run to run.New leads:
nvnSyncWait→mkwvk::wait) more than in rc.4. In the Market it's in 1.8% of the main thread's samples on rc.4, 2.8% ondbad7d12and 3.9% on6fb13fe5: it grows step by step. In a 6-minute play session one4d035e9(DLSS on, 55-70 fps) it was 6.7-8.8%. The main thread is the critical path, so this is frame time. First step: which syncs the game waits on, and which commit since rc.4 made it grow. Then whether they can be signalled at the CPU-side point where the work is known done, rather than at GPU completion.capture_make→copy_split→copy_partis in 7.3-8.1% of the main thread's samples in the Market (rc.4 8.1%, main 7.3%), and 7-7.5% in the play session (lead 2 above, the same order of size at 1440p).e4d035e9, the render thread's summary line shows presents at 3-8 ms a frame. The game waited for room at 528-598 of every 600 presents, 0.7-1.0 ms each, and 4.8 ms each in one heavy stretch. So the render thread does sit on the critical path some of the time on this PC, through the blocking present.Ruled out:
RX_VK_PRESENT_THREAD=1in the Market, on 24 threads at 1440p: no gain. 3 runs each way, interleaved on6fb13fe5. Off: 61.2 / 67.7 fps; on: 60.0 / 70.4 fps. The higher turning median comes from runs that saw fewer draws (1,038-1,094 against ~1,300); the one pair with matching draws was 64.6 on against 67.7 off. The presenter thread starts on its own queue (family 0, queue 1, withVK_KHR_present_wait). So freeing the render thread from the present (lead 3 above) doesn't shorten the frame on its own here; the queue waits in lead 3 of this comment need another approach.Side finding: main
dbad7d12(the main of earlier today) peaked at ~18.2 GB of the game process's RAM in the Market, against ~12.5 GB for rc.4,1764fed2and6fb13fe5. On a 30 GB PC that pushed the rest of the PC into swap, with hitches up to 650 ms. One of the 31 commits up to1764fed2fixed it. The commit that caused it and the one that fixed it aren't pinned down yet.How it was measured:
HOS_SAMPLE_ALLwhile walking and turning,RX_VK_PASS_TIMING=1,RX_VK_FRAME_LOG, and the game's RSS and the PC's swap-ins every 2 s. Runs with heavy swapping from other programs (several GB of swap-ins during a run) are left out of the medians. The game code isn't named in any of this: the profile names only Longshot's own functions.