diag(host): HOS_AWAY_LOG_MS logs a serial thread's released-core stretches #1666
No reviewers
Labels
No labels
area/accessibility
area/audio
area/input
area/install
area/menu
area/renderer
area/saves
area/tooling
area/upscaling
beanstalk: approved
beanstalk: failed
beanstalk: needs approval
beanstalk: running
beanstalk: tested
ci: approved
good first issue
gpu/amd
gpu/igpu
gpu/intel
gpu/nvidia
help wanted
kind
bug
kind
chore
kind
docs
kind
feature
kind
performance
kind
question
needs measurements
platform/linux
platform/macos
platform/steam-deck
platform/windows
preview
priority
critical
priority
high
priority
low
regression
risk
high
risk
low
risk
medium
status
blocked
status
confirmed
status
duplicate
status
invalid
status
needs-info
status
wontfix
ci: approved
No milestone
No project
No assignees
2 participants
Notifications
Due date
No due date set.
Dependencies
No dependencies set
Reference
MagicBeanDev/Longshot!1666
Loading…
Reference in a new issue
No description provided.
Delete branch "Molier/Longshot:diag/away-log-ms"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Summary
A diagnostic:
HOS_AWAY_LOG_MS=<ms>logs every released-core stretch (hos_vcore_away..hos_vcore_back) of aserial thread that lasts at least that many ms, with the stretch's CPU time and the host call site that started it
(
away:lines, symbolised module+offset) — the same quantity the frame trace'smain_blocked_mscolumnalready totals, but per occurrence instead of per frame. Off (unset or negative): one cached load and a branch, no
behaviour change. Read first:
host/hos_thread.c(away_log_ms,away_cpu_ns,away_log, and the two call sitesin
hos_vcore_away/hos_vcore_back).Testing
Tested: Linux (Ubuntu), host compile clean (
-fsyntax-only,host/hos_thread.c; one pre-existing unrelatedwarning,
unused function 'pick_cache_cores', not from this change).tests/run_all.py away_log:tests/test_away_log.pypasses on the rebased head.Still to test:
HOS_AWAY_LOG_MS=1during a known stall (an area change, a long load) and confirmthe
away:line's call site matches a thread actually blocked there in a Tracy capture (by hand).Measurements
Nothing to measure: a diagnostic switch, off by default, with no effect on the game's own behaviour when unset.
Risks
None when unset. When set, extra CPU time (one
clock_gettime/GetThreadTimescall and a log line) only on thestretches it reports;
HOS_AWAY_LOG_MSis meant for short diagnostic sessions, not left on.Checklist
Ready for review
main; the title is a Conventional Commit.tests/run_all.py away_logpasses.third_party/untouched.When it applies
HOS_AWAY_LOG_MSrow added todocs/dev/reference/switches.md, #1802's location).#if defined(_WIN32)/__linux__CPU-time split mirrors existing patterns elsewhere in the file).Assisted-by:trailers.Ready to merge
5f478884b14ec23e9629Tested in game on Linux: Fedora 45 (kernel 7.3.4), AMD Radeon RX 9070 XT (RADV GFX1201, Mesa 26.2.7), Ryzen 5 5600X (12 logical processors), 32 GB RAM. This PR's head
4ec23e96and #1667'sa871f3ff, both cherry-picked onto their common base2f1c0c9c, in one test build (longshot abc-probes (source 166c4da4)), withHOS_AWAY_LOG_MS=50andHOS_PROFILE=1(which setsRX_VK_FRAME_TRACE=1). No errors, hangs or device losses in any run.The probe (this PR's Needs item): it logs as described on a real GPU: 72 and 81
away:lines, each with its thread, wall and CPU time and call site, and the call sites resolve against the build's symbols withnm. Three sites cover sead::MainThread:n_nvnSyncWaitn_nvnQueueSubmitCommandshos_cv_sleepn_nvnQueueSubmitCommandsthe stretch is the submit-time capture: its wall time equals the slow-capture line logged just before it (#1667's probe), for example 773.0 ms againstvk: slow capture 773.0 ms.n_nvnSyncWaitthe game waits on a GPU sync with almost no CPU, and each long one is most of a slow frame: 751 ms in a 764 ms frame, 408 in 454, 303 in 314, 284 in 323. The Presentation Thread waits through the same stretch inn_nvnQueuePresentTexture(754 ms against the main thread's 752). It isn't GPU execution: the longest submission in the log's GPU time lines is 20.6 and 18.9 ms. Each long wait comes right after a frame of 100 to 300 MB of texture uploads (95 to 379 uploads, many of them 57 MB ASTC textures re-encoded as BC3). In the 752 ms one the replay thread was 73% inprograms_load_wait(program_cache_loadunderprogram_pick); in most of the others (407, 409, 175 ms) it was idle. So the work the fence waits for is held somewhere before the replay, and where is still open.HOS_NVN_SYNC_LOGshould show which fence each wait wants against how far the replay is; that's our next run.main_blocked_ms, because a stretch can span a present: the 148 ms frame showsmain_blocked_ms69 against a 122 ms stretch ending in it. Splitting a stretch at the present would make the two comparable.hos_cv_sleeplines (20 and 29, up to 9.9 s at start) are that thread waiting for work, with almost no CPU. A cutoff per thread, or leaving out a known idle wait, would cut that noise.(In this build
+0x1f73a4b8is inn_nvnSyncWait,+0x1f741e17inn_nvnQueuePresentTexture,+0x1f74147dinn_nvnQueueSubmitCommands.)Not compared against main: the probe changes nothing while it's off, so these are its readings, not a measurement of this PR.
Settings: the player's own (Custom preset), 2560x1440 fullscreen, HDR10, AMD FSR 4 (1920x1080 to 2560x1440), V-Sync Auto (mailbox), no frame cap, frame generation off, FPS counter off; one kept options file for every run. Threaded rendering Auto (on).
RX_VK_CAPTURE_SLOW_MS=50(#1667) in every run.Method: a scripted route through the control bridge from the repository save
kokiri-start: the debug teleport to the Market, the clock at 12:00, out of the gate into Hyrule Field, then 120 s of laps along Link's facing between 20 m and 320 m out with a 180° camera turn every 10 s (the trigger in #1720). A warm-up run first, then 2 runs; Link's hearts protected (HOS_OOT_PROTECT_HEARTS=1).4ec23e9629338d0ab083WIP: diag(host): HOS_AWAY_LOG_MS logs a serial thread's released-core stretchesto diag(host): HOS_AWAY_LOG_MS logs a serial thread's released-core stretchesRebased onto
main@89c3d21e. One real conflict inhost/hos_thread.c(main reworded the neighbouring comment): kept main's wording, this PR's functions unchanged after it. Docs row moved todocs/dev/reference/switches.md(#1802).tests/test_away_log.pypasses; host compile clean. Off by default, a diagnostic only, so out of WIP; body rewritten in the new template.View command line instructions
Checkout
From your project repository, check out a new branch and test the changes.