fix(host): the samplers' stack walks stay on the walked thread's stack, so HOS_PROFILE=1 no longer crashes the game on Windows #1343
No reviewers
Labels
No labels
area/accessibility
area/audio
area/input
area/install
area/menu
area/renderer
area/saves
area/tooling
area/upscaling
beanstalk: approved
beanstalk: failed
beanstalk: needs approval
beanstalk: running
beanstalk: tested
ci: approved
good first issue
gpu/amd
gpu/igpu
gpu/intel
gpu/nvidia
help wanted
kind
bug
kind
chore
kind
docs
kind
feature
kind
performance
kind
question
needs measurements
platform/linux
platform/macos
platform/steam-deck
platform/windows
preview
priority
critical
priority
high
priority
low
regression
risk
high
risk
low
risk
medium
status
blocked
status
confirmed
status
duplicate
status
invalid
status
needs-info
status
wontfix
ci: approved
No milestone
No project
No assignees
2 participants
Notifications
Due date
No due date set.
Dependencies
No dependencies set
Reference
MagicBeanDev/Longshot!1343
Loading…
Reference in a new issue
No description provided.
Delete branch "dyst/Longshot:fix/sampler-stack-walk"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Type: fix · Risk: low · Tested on: Windows 11, RTX 3080 (NVIDIA 621.14) · Needs: none
Why
On Windows the profiling mode (
HOS_PROFILE=1, #1279) can crash the game: its all-thread sampler walks the stack of each thread it stops, and when a thread is stopped where its stack can't be walked, the walk reads memory that isn't there and takes the game down. With the mode sampling from the boot and listing the threads again every 5 s, that happened about 2 minutes into the first Windows run. Fixes #383.How to reproduce
Seen on: Windows 11, RTX 3080, NVIDIA 621.14 (main @
8e9cfc3a); #383 saw it on an RX 6700 XT before #1279Settings and switches:
HOS_PROFILE=1(orHOS_SAMPLE_ALL=<file>with<file>.go), headless, thekokiri-startsaveSteps: play; on main @
8e9cfc3athe game closed after 135 s of play, in Kokiri ForestExpected: the run goes on, and the profile folder holds the whole session
Actual:
CRASH: exception 0xc0000005 (read 0x438, ...) at ntdll.dll+0x21580, fromlongshot.exe+0x1c6fc27c, which issample_all_thread+0xb2cWhat changed
Every stack walk in
host/mkw_glue.cgoes one frame at a time through one helper that only reads the walked thread's own stack. A frame it can't walk ends that sample's stack there, and the game goes on.host:
unwind_step()inhost/mkw_glue.cis the one call ofRtlVirtualUnwind. Before it reads anything, it checks that the stack pointer is on the walked thread's committed stack, and the frame register too when the function's unwind data names one. After the step it checks the new stack pointer. A step outside ends the walk. ARUNTIME_FUNCTIONentry chained to another is followed.host:
thread_stack()gets a suspended thread's stack limits (its TEB'sStackLimittoStackBase) withNtQueryInformationThread, looked up from ntdll at run time, so the link line is unchanged.own_stack()gets the current thread's limits, for the crash report.host: the three walks use it:
report_crash's host backtrace,sample_thread(HOS_SAMPLE) andsample_all_thread(HOS_SAMPLE_ALL, the profiling mode). Their output format is unchanged.Why checks and not a guard: the host is C, and this clang takes
__try/__exceptin C only with-fms-extensions, which the host isn't built with (the renderer's__tryis in C++). So the walk can't fault, rather than catching the fault.tests:
tests/test_sampler_stack_walk.py(new):RtlVirtualUnwindandRtlLookupFunctionEntryare called only inunwind_step;It fails against main's
mkw_glue.c.Measurements
Verdict: pass. With the fix, the same profiled session ran 454 s of play to a clean stop, where main crashed at 135 s. The stacks the sampler writes are as deep as before.
Hardware: Windows 11 Pro 22631, RTX 3080 (NVIDIA 621.14), Ryzen 9 3900X (12 cores, 24 threads), 64 GB
Scene:
kokiri-start, Kokiri Forest; 1920x1080, no upscaler, no frame generation; headlessMethod: one
HOS_PROFILE=1session each, started throughscripts/longshot_ctl.py launch --headlesswith heart protection, then teleports with walks and camera swings. On main the session reached Kokiri and died. With the fix, a Kokiri cutscene kept refusing the teleports, so the session sampled Kokiri until stopped by hand. Stack depth is frames per sample over each run's first 100 s of play, from itsall.txt.How to reproduce: a
scripts/dev_packs.pybase of8e9cfc3a, a pack of this branch over it, then the same launch withHOS_PROFILE=1on each runtime8e9cfc3a41e601e0(on main544229a2)HOS_PROFILE=1CRASH:linesWhy main @
8e9cfc3a, not the current544229a2: it is the base the pack was built over, and the numbers still hold. Of the 6 commits since, only #1291 toucheshost/mkw_glue.c, inhos_alloc_failed, not a sampler or the crash report's walk, and none toucheshost/profiling.corhost/posix/prof.c.The crash depends on where a thread is stopped (#383 saw it in 1 of 8 runs before the profiling mode). One run each shows that the walk survives what main's didn't, not a rate. The deep-stack share fell 3 points. That could be the frame-register check ending some walks, or simply the different scene (a cutscene against main's walk). The 1-frame share is unchanged, so no walk is cut at its first frame, and the stack limits are read right.
Flags and settings
None
Testing
Tested:
dev_packs.pypack of this branch (longshot.exeandlongshot_vk.dll) over a base of main @8e9cfc3a.tests/test_sampler_stack_walk.py --cc <llvm-mingw clang>passes, and fails against main'smkw_glue.c.host/mkw_glue.c, throughrun_all.py:test_host_alloc_wait,test_input_background,test_nvn_host_tablesandtest_commit_limitpass.test_crash_record,test_slow_frames,test_profiling_modeandtest_renderer_new_handlerfail with the same message on main @544229a2:void hos_prof_start(void)is inhost/posix/prof.ctwice, and TDM gcc failstest_crash_record's stubs. That's 8 checks, not the whole suite.host/mkw_glue.ccompiled with gcc 14 (Docker,-fsyntax-only -D_GNU_SOURCE). Every changed line is inside#ifdef _WIN32. The only error was line 967'spollfield, which is newer than the lift headers I compiled against and outside this change.Not tested: the
HOS_SAMPLEsampler and the crash report's backtrace in the game (the same helper, and their checks only), and AMD and Intel GPUs (the walk doesn't depend on the GPU).Must be tested: None
Risks
What could break: a sampled stack, or a crash report's host backtrace, ending earlier than before where a frame would have read outside its thread's stack. That only affects the profiling tools and the report's backtrace, and players never run the profilers. #1044 (WIP) changes the same
sample_all_thread, for its samples' timestamps: whichever lands second resolves a small textual overlap.Turning it off: revert the commit
Compatibility: Nothing changes
Issues
Fixes #383
Checklist
Everyone
type(scope): description) of the status line's type.main.python -I -B tests/run_all.pypasses (any failure that also fails onmainis named).third_party/untouched (a new folder with aPROVENANCE.mdinstead).When it applies
Assisted-by:/Co-Authored-By:trailers.mainand passing here.🤖 Generated with Claude Code
Tested in game: Windows 11, RTX 3080 (NVIDIA 621.14): HOS_PROFILE=1 from kokiri-start, headless, 454 s of play with no crash and a clean stop (main @
8e9cfc3acrashed at 135 s in sample_all_thread); the profile folder complete, stacks as deep as main'sWIP: fix(host): the samplers' stack walks stay on the walked thread's stack, so HOS_PROFILE=1 no longer crashes the game on Windowsto fix(host): the samplers' stack walks stay on the walked thread's stack, so HOS_PROFILE=1 no longer crashes the game on WindowsProfiling on Windows no longer takes the game down when a stopped thread's stack cannot be walked. I read head
41e601e0. It merges, and I approve it.unwind_stepis the only call ofRtlVirtualUnwindandRtlLookupFunctionEntry. It checks the stack pointer against the walked thread's committed stack before it reads unwind data, the frame register when the function has one, and the stack pointer again after the step. A step outside that range ends the sample there. The crash report,HOS_SAMPLEandHOS_SAMPLE_ALLall go through it, and both samplers suspend the thread before they ask for its limits. A thread whose limits cannot be read keeps one frame and goes on. The lines the profile writes are unchanged.The stack limits come from the thread's TEB through
NtQueryInformationThread, looked up at run time, so the link line stays as it is. The frame register is the same numbering asCONTEXT'sRaxthroughR15. A chained unwind entry is followed once so that check sees the real frame register, andRtlVirtualUnwindstill gets the entry the lookup returned.Windows, RTX 3080: one profiled session ran 454 seconds and stopped cleanly, where the same session on main closed at 135 seconds inside
sample_all_thread. The stacks stayed about as deep. Must be tested is None. The check reads the source and, with a Windows compiler, compiles the helpers on their own.Work-in-progress #1044 also edits
sample_all_thread. Take this one; that branch rebases onto it.View command line instructions
Checkout
From your project repository, check out a new branch and test the changes.