Game: Periodic ~4.6 s stalls of 100-200 ms (wall-clock cadence, camera-movement dependent) in Castle Town market and Kokiri Forest: host blocked in present or texture lookups #1988
Labels
No labels
area/accessibility
area/audio
area/input
area/install
area/menu
area/renderer
area/saves
area/tooling
area/upscaling
beanstalk: approved
beanstalk: failed
beanstalk: needs approval
beanstalk: running
beanstalk: tested
ci: approved
good first issue
gpu/amd
gpu/igpu
gpu/intel
gpu/nvidia
help wanted
kind
bug
kind
chore
kind
docs
kind
feature
kind
performance
kind
question
needs measurements
platform/linux
platform/macos
platform/steam-deck
platform/windows
preview
priority
critical
priority
high
priority
low
regression
risk
high
risk
low
risk
medium
status
blocked
status
confirmed
status
duplicate
status
invalid
status
needs-info
status
wontfix
ci: approved
No milestone
No project
No assignees
2 participants
Notifications
Due date
No due date set.
Dependencies
No dependencies set
Reference
MagicBeanDev/Longshot#1988
Loading…
Reference in a new issue
No description provided.
Delete branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Longshot version
v3.0.0 stable (source bd6a0cdd); Nightly 649 (source
846e5399)Channel
Release
What kind of problem
Low fps or stutter
What happened
Disclaimer
I’m not a developer, and I’ve been looking into my problem with Claude AI, who has also written the following.
Summary
With the camera moving, the game stalls for 100-200 ms (up to ~240 ms) about every 4.6 s in busy scenes. The cadence is wall-clock: the period is the same at a 30 fps cap and at ~52-60 fps, and with different graphics settings. The GPU is at ~35 % (~8 ms per frame), no shader is built, there are 0-1 uploads per frame, and the system measures clean.
RX_VK_SLOW_LOG=50andHOS_PROFILE=1place the lost time either intexture lookups(a single frame) or inpresentfollowed bytexture lookupson the next frame. In both cases the game's main thread is blocked and no game thread is on the CPU. Nothing I changed (system, kernel, driver, desktop, V-Sync, FPS cap, ~12 Longshot switches) removes it. From outside I cannot tell what the game waits for, so I am looking for pointers: which call blocks, and what runs on a ~4.6 s cadence?Environment
846e5399) also tested, same resultTwo configurations show the same stalls:
Another recomp (HeyListen) shows the same symptom on this machine. I have not checked whether it shares this renderer.
What I see
Evidence
1. Cadence
RX_VK_COMPACT=0, 3 without, same result).RX_VK_COMPACT=0Of 52 intervals between stalls, 44 are 3.6-5.8 s (median 4.6 s, SD 0.46 s). The other 8 are 8.6-10.7 s, i.e. two beats with one below 50 ms. So it is not an exact timer.
RX_VK_SLOW_LOGrun (61 s): 11 stalls >= 95 ms (100-195 ms), period fit 4.58 s (R = 0.91).2. Where the time goes
All stalls have 0-1 uploads, 0 new pipelines / programs, and 1600-2300 draws.
(a) One frame, submit dominated by
texture lookups(normally ~1-2 ms per frame):(b) A slow
present, thentexture lookupson the next frame (4 of the 11 largest stalls):The same pattern appears at presents 1172 (frame 241.9 ms, submit 46.9, present 169.1), 1588 (104.6 / 15.5 / 83.8, next frame lookups 78.5) and 1833 (135.2 / 24.1 / 103.9, next frame lookups 37.5). My reading: one blockage of ~150-230 ms that lands in whichever host call is running.
HOS_PROFILE=1(88 s, Castle Town market;report.mdand the profile folder attached):tex_lookup_ms(80-166 ms) ortex_frame_ms(168 ms, withwait_render_ms164 ms). They have 0 or 1 uploads, so this is not upload volume.main_blocked_msis 79 ms on average (12.6 ms in steady frames) whilemain_vcore_msstays at 0.06 ms. The main thread is not waiting for a virtual core.sead::MainThreadwaiting inlibc.so.6100 % of the time, and in most sweeps no game thread is on the CPU. The wait is in a lock or the driver.longshot_vk.sooffsets are unresolved on my side, so I cannot name the call.slow upload 59.2 ms: ... 512x512 ... ASTC, 236 KB.perf(other session):mkw_vk displayblocked 122.7 ms at 159.14 s and 169.4 ms at 163.66 s (4.5 s apart).3. Not the system
cyclictest(30 s): max latency 6-157 µs.perf sched: no game thread waits more than ~5 ms for a CPU.turbostat: SMI = 0, no thermal or power limit.vkcuberun fine on the same machine.4. Tried without effect
nvidia-smi,pidstat, MangoHud)RX_VK_PRESENT_THREAD=1, threaded rendering offRX_VK_COMPACT=0,HOS_TEX_COMPACT_MS=off/10000RX_VK_UPLOAD_BUDGET_MS=10,RX_VK_ASTC_RGBA8=1HOS_CORES8 / 10 / 12HOS_RESERVE_CORE=0The sparse-mirror destruction fix described in your docs is active here (
0 destroyed).5. Thread-level observations
NavMeshCAStepDepWorker1→tera::FileResourceMgr→PreActorModelCompose→Porteron core 1. Only 7 of the 94 overlap a stall, and none of the 11 largest stalls has one within ±4 log lines (14 % expected by chance). The waiting threads are loaders (tera::FileResourceMgr,PreActorModelCompose,GeneralFormatAsync_), blocked whilesead::MainThreadholds the core. I read this as a consequence of the main thread being blocked rather than the cause.mkw_vk replay): scheduled ~3000 times per second (~100 per frame, asched_yieldspin aroundRecorder::wait_idle). In a 45 sbpftracecapture with stalls it showed no off-CPU wait >= 40 ms. A 40 sperf recordshows 60 % on CPU, no continuous on-CPU stretch >= 80 ms, longest gap 85 ms. It appears to wait for work rather than cause the stall; the in-game overlay shows "Waiting on: Game's main thread" rising during the holes. This is hard to reconcile withtex_lookup_msof 165 ms.bpftraceon all threads, 58 s, no controller connected: of allioctl/mmap/mprotect/munmap/mremap/madvisecalls, only one lasted >= 40 ms (a 42 msioctlinmkw_vk replay). So there is no long NVIDIA ioctl or memory syscall during the stalls.futexwaits >= 60 ms inPresentation Th(60-101 ms) occur at seconds 3, 13, 18, 22, 27, 32 and 46 of the capture (period fit 4.76 s, R = 0.94, 1 s clock resolution). At 3, 22 and 46 s,sead::MainThread(52-63 ms) andmkw_vk recorder(41 ms) also wait at the same moment, so the whole pipeline stalls at once. The larger events fall on the same ~4.76 s grid every 4-5 cycles. The main replay thread never waits >= 40 ms infutex; the presentation thread is starved while the replay does not sleep.OptimizeChangeBanc: in a separate 37 sperfcapture it was the only thread with CPU bursts >= 80 ms: 94 ms at t = 22.05 s and 203 ms at t = 26.51 s, 4.46 s apart (hot functionf_805aeea0, 18 % of its samples). In the profile above it is active in 2 of the 8 largest stalls. Two events are too few to conclude.6. Side note (not the same problem)
A performance power profile (
powerprofilesctl set performance,min_perf_pct=100) cuts replay CPU per frame from 15.8 to 10.5 ms and 50-65 ms frames from ~85/min to ~35/min (30 fps cap). It does not remove the periodic 100-200 ms stalls.Questions
libc.so.6. Is there a switch, a log or a Tracy build that names the object it waits on? Could the log break downslow present(acquire vs queue present) andtexture lookups(per-call time)?tex_lookup_msreach 165 ms with no upload? Couldpresentandtexture lookupsbe waiting on the same lock shared with a game thread, or on the same driver operation?perfstacks the replay thread is often inRecorder::spin<…wait_idle…>/sched_yieldbefore long gaps. What can keep it waiting 100+ ms there, and what is the recorder doing at that time?846e5399) and the stalls are unchanged. Are #994 (render-target reuse), #838 (render thread leaving its core) and the occlusion virtual-core fix included in that nightly? If not, is there a branch or build with them that I can test? I can also run a debug build or any switch you suggest, and report.Where in the game
Kokiri Forest, Town Market, while moving the camera
How often
Every time
Graphics card and driver version
RTX 3060 12 GB, NVIDIA 595.99.02 and 615.71.09 (same result)
CPU, RAM and operating system
i5-12400F (6 cores, 12 threads), 32 GB DDR4, Linux Mint 22.3, X11 (Cinnamon; Openbox without compositor also tried)
Changes from the defaults
Frame generation: Auto (Built-in FG) (default: Off)
Frame generation by: Built-in FG (default: FSR 3.1 FG)
Fullscreen: On (default: Off)
GPU device: 0: NVIDIA GeForce RTX 3060 (default: Auto)
Volume: 50% (default: 100%)
FPS counter: Dev (default: Off)
Menu size: Small (default: Medium)
Depends on this PC (the values now, not counted as changes):
Quality preset: Auto (this PC) (default depends on this PC)
Output resolution: 1920x1200 (native) (default depends on this PC)
Profile: Docked (default depends on this PC)
Upscaler: NVIDIA DLSS (default depends on this PC)
Render resolution: Quality 67% (1280x800) (default depends on this PC)
Sharpness: 40% (default depends on this PC)
Window scaling filter: Not needed (window = output) (default depends on this PC)
Game speed: Real time (default depends on this PC)
Frame pacing: On (default depends on this PC)
Switches set:
LONGSHOT_NO_UPDATE_CHECK=1
SDL_VIDEO_WAYLAND_WMCLASS=(value left out)
SDL_VIDEO_X11_WMCLASS=(value left out)
longshot_log.txt
No response
Screenshots or video
No response
Before you send it
Possibly related, for context: