fix(scripts): the profile readers count a Windows wait as a wait and name the renderer's functions #1347
No reviewers
Labels
No labels
area/accessibility
area/audio
area/input
area/install
area/menu
area/renderer
area/saves
area/tooling
area/upscaling
beanstalk: approved
beanstalk: failed
beanstalk: needs approval
beanstalk: running
beanstalk: tested
ci: approved
good first issue
gpu/amd
gpu/igpu
gpu/intel
gpu/nvidia
help wanted
kind
bug
kind
chore
kind
docs
kind
feature
kind
performance
kind
question
needs measurements
platform/linux
platform/macos
platform/steam-deck
platform/windows
preview
priority
critical
priority
high
priority
low
regression
risk
high
risk
low
risk
medium
status
blocked
status
confirmed
status
duplicate
status
invalid
status
needs-info
status
wontfix
ci: approved
No milestone
No project
No assignees
3 participants
Notifications
Due date
No due date set.
Dependencies
No dependencies set
Reference
MagicBeanDev/Longshot!1347
Loading…
Reference in a new issue
No description provided.
Delete branch "dyst/Longshot:fix/profile-report-windows"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Type: fix · Risk: low · Tested on: Windows 11, RTX 3080 (a real HOS_PROFILE=1 profile; the game-free checks) · Needs: none
Why
On Windows,
scripts/profile_report.py(#1279's report) points at the wrong code. Its "hottest" on each thread, and the code it says runs more in stutters, are mostly threads sleeping. The renderer's functions also show as mangled names. So a Windows profile sends you after idle loops instead of what the frame spends its time on. Fixes #1337.How to reproduce
Seen on: Windows 11, RTX 3080, NVIDIA 621.14, main @
8e9cfc3aSettings and switches:
HOS_PROFILE=1, thenprofile_report.py <profile folder> --maps <game folder>Steps: read the report's At a glance
Expected: each thread's hottest functions are work, under readable names
Actual:
sead::MainThread's hottest are_ZN5mkwvk12_GLOBAL__N_14waitEyy(6.7 %) andsleep_away(5.7 %), and the Presentation Thread's is a condition variable wait (7.8 %). "Running far more in stutters" names those waits.What changed
prof.waitingnow recognises Longshot's waits whatever their parameter list, and knows the ones it was missing.prof.load_symbolsdemangles a Windows map's names. Every reader built onprof.pygets both:profile_report.py,crit.py,hotlist.py,chain.py,vkprof.py.scripts:
prof.base_name()drops a name's closing parameter list, and a trailingconst. It keeps(anonymous namespace)andoperator().waiting()matchesWAIT_FRAMESagainst it. Before,endswithfailed on any namenm -Corllvm-cxxfiltwrites with parameters, on Linux too: even the listedcondition_variable::waitnever matched there.scripts:
WAIT_FRAMESgains the waits a Windows run showed counted as running. Of its 1.44 M samples with the leaf in a system module, these were counted as running:sleep_away: 132,579 samples;__libcpp_condvar_waitand__libcpp_condvar_timedwait: 137,869 together;gpu_hang_watch_start's sleep: 14,415;::wait;__libcpp_mutex_lock.Work inside the system under Longshot's frames stays running:
operator new,steady_clock::now,event_signal,discard_pages. That's why the rule is a list, not "a leaf in ntdll is a wait", as #1337 first suggested.scripts:
prof.demangler()findsllvm-cxxfilt(the toolchain's underLONGSHOT_MINGW, else onPATH, elsec++filt).demangle_all()runs it once over a Windows map's_Znames inload_symbols. Without one, the names stay as the map has them.tests:
tests/test_prof_symbols.pychecks:nm -Cform, and work in the system staying running;base_name's three shapes;It fails against main's
prof.py.Measurements
Verdict: pass. On a real Windows profile, the report's hottest functions and its stutter attribution go from waits to work, under readable names.
Hardware: Windows 11 Pro 22631, RTX 3080 (NVIDIA 621.14), Ryzen 9 3900X, 64 GB
Scene: a
HOS_PROFILE=1run of main @8e9cfc3afromkokiri-start(135 s of play, 1,902,180 samples, 76 stutters)Method:
profile_report.py <that folder> --maps <its game folder>, once with main'sscripts/prof.pyand once with this branch's, everything else the same (LONGSHOT_MINGWset to the Builder's llvm-mingw). The two reports' At a glance compared.How to reproduce: any Windows
HOS_PROFILE=1folder, the report run with eachprof.py544229a20934572fsead::MainThreadrunning_ZN5mkwvk12_GLOBAL__N_14waitEyy6.7 %,sleep_away5.7 %,_ZN5mkwvk12guarded_copyEPvPKvy4.2 %mkwvk::guarded_copy(void*, void const*, unsigned long long)4.2 %,collect_in2.3 %,spin_until_set2.2 %__libcpp_condvar_timedwait7.8 %collect_in2.4 %wait(39 %)f_805aeea0(12 %, 0 % otherwise), then BaseProcCreateAndDeleteThreadFlags and settings
None (
LONGSHOT_MINGW, which the build already reads, is also where the demangler is looked for).Testing
Tested:
run_all.py test_link_skip test_prof_symbols test_profile_stream test_profiling_mode): three pass.test_profiling_modefails on its first line with the same message on main @544229a2:void hos_prof_start(void)is inhost/posix/prof.ctwice. Itspacked_formatandreportparts, run on their own, fail with the sameTypeErroron main and here. That's 4 checks, not the whole suite.tests/test_prof_symbols.pyfails against main'sprof.pyand passes here.profile_report.pywith eachprof.py.Not tested: a Linux profile. The same
base_namematch applies there, and the check coversnm -C's form, but no Linux profile was read here. The game is untouched.Must be tested: None
Risks
What could break: a sample counted as a wait that runs, where its innermost Longshot frame's name ends with a listed name and its leaf is outside Longshot (a
::waitthat spins inside the driver, say). It only affects the profile readers' numbers; the game is untouched.Turning it off: revert the commit
Compatibility: Nothing changes
Issues
Fixes #1337
Checklist
Everyone
type(scope): description) of the status line's type.main.python -I -B tests/run_all.pypasses (any failure that also fails onmainis named).third_party/untouched (a new folder with aPROVENANCE.mdinstead).When it applies
Assisted-by:/Co-Authored-By:trailers.mainand passing here.tests/that reads changed source updated in this PR, not weakened.🤖 Generated with Claude Code
WIP: fix(scripts): the profile readers count a Windows wait as a wait and name the renderer's functionsto fix(scripts): the profile readers count a Windows wait as a wait and name the renderer's functionsPossibly related, for context:
On the Windows HOS_PROFILE=1 run the fix was checked against (profile-20261116-211517), the hang watch's thread still counted as running: 60,340 samples, 19.4% of the running samples, all in vk_replay.cpp. Its frame is the thread's lambda, "mkwvk::gpu_hang_watch_start()::$_0::operator()() const" as llvm-cxxfilt writes it, and base_name took only the closing parameter list off, so "...::$_0::operator()" never ended with the listed gpu_hang_watch_start. The test named the frame "mkwvk::gpu_hang_watch_start()", which the run never has. - base_name names a lambda's body (clang's ::$_<n>::operator(), GCC's ::{lambda(...)#<n>}::operator()) as the function it is written in. - tests/test_prof_symbols.py: the hang watch as the run names it, both compilers' lambdas, and a lambda that does not wait staying running. On that run, the samples in a system module counted as running go from 91,913 to 31,573; what is left is the work in the system under Longshot's frames that stays running by design (event_signal, read_at, operator new, steady_clock::now). Co-Authored-By: Claude Opus 5.5 <[email protected]>Follow-up pushed:
3af0c93f.After
0934572f, the hang watch's sleep still counted as running. On a later WindowsHOS_PROFILE=1run (base8e9cfc3a,profile-20261116-211517) it was 60,340 samples, 19.4 % of the running ones, all charged tovk_replay.cpp. The frame the run has is the thread's lambda,mkwvk::gpu_hang_watch_start()::$_0::operator()() const.base_nametook off only the closing(), which left...::$_0::operator(), and that doesn't end withgpu_hang_watch_start. The check passed because it named the framemkwvk::gpu_hang_watch_start(), which no run has.base_namenames a lambda's body as the function it's written in: clang's::$_<n>::operator()and GCC's::{lambda(...)#<n>}::operator().tests/test_prof_symbols.pychecks the hang watch under the name the run gives it, both compilers' lambda forms, and a lambda that doesn't wait staying running. It fails against 0934572f'sprof.py.On that profile, the running samples with their leaf in a system module go from 91,913 to 31,573. What's left is work in the system under Longshot's frames, which stays running as this PR intends:
event_signal,read_at,operator new,steady_clock::now.Checks, Windows 11:
test_prof_symbols,test_profile_streamandtest_profile_path_keypass.test_profiling_modefails as on main (the doubledhos_prof_startmarker).test_prof_clockandtest_prof_samplingneed ag++and a binary this machine doesn't have, and fail the same with and without this commit.A Windows profile counts Longshot's waits as waits, and names the renderer's functions. I read head
3af0c93f. It applies onto current main.base_namedrops a parameter list and a trailingconst, and names a lambda as the function it is written in, so the hang watch's sleep matchesgpu_hang_watch_start. The wait list also gainssleep_away, libc++'s condition waits and mutex lock, and a frame whose name ends in::wait. Work under a Longshot frame, such asoperator neworsteady_clock::now, stays running. A Windows map's_Znames are demangled once, throughLONGSHOT_MINGWorllvm-cxxfilton the path, and the old MSVC reader still handles a?name@symbol. On an RTX 3080 profile, the main thread's hottest line goes from the mangled fence wait tomkwvk::guarded_copy, and the stutter list names game code instead of those waits.This is the prof.py change to take. #1321 does the same demangle and the same waits, and also gives each thread one row in the Threads table. Do not land both copies of
prof.py. Rebase #1321 and keep that row change.Nothing further has to be run before this merges.
View command line instructions
Checkout
From your project repository, check out a new branch and test the changes.