WIP: perf(scripts): hotlist.py ranks a hot list by the FP helper calls it inlines #1124
No reviewers
Labels
No labels
area/accessibility
area/audio
area/input
area/install
area/menu
area/renderer
area/saves
area/tooling
area/upscaling
beanstalk: approved
beanstalk: failed
beanstalk: needs approval
beanstalk: running
beanstalk: tested
ci: approved
good first issue
gpu/amd
gpu/igpu
gpu/intel
gpu/nvidia
help wanted
kind
bug
kind
chore
kind
docs
kind
feature
kind
performance
kind
question
needs measurements
platform/linux
platform/macos
platform/steam-deck
platform/windows
preview
priority
critical
priority
high
priority
low
regression
risk
high
risk
low
risk
medium
status
blocked
status
confirmed
status
duplicate
status
invalid
status
needs-info
status
wontfix
ci: approved
No milestone
No project
No assignees
2 participants
Notifications
Due date
No due date set.
Dependencies
No dependencies set
Reference
MagicBeanDev/Longshot!1124
Loading…
Reference in a new issue
No description provided.
Delete branch "Marcuss2/Longshot:perf/hotlist-fp-ranking"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Type: perf · Risk: low · Tested on: Linux, RX 9070 XT (RADV) · Needs: none
Why
The lift can compile a chosen list of lifted functions together at
-O2with the FP helpers' fast paths inlined (A64_HOT_FUNCS+A64_HOT_CHUNK=1,code_hot.c).scripts/hotlist.pychose that list by sample time alone, which is the wrong measure for what the list is for: the inlining pays per *call toa64_fpx_*that goes away, so a function worth listing is one that runs often and holds many of those calls. The time-only cut also keeps functions with no FP call at all — the hot chunk compiles them for-O2alone.On a 12 s Market profile the time-only cut at 0.85 takes 441 functions and 80 MB of hot-chunk C; ranking by samples × FP calls reaches most of the same FP work with 47 functions and 25 MB.
What changed
scripts/hotlist.pygains--fp-calls <gen>: with the lift's own C folder it counts each lifted function's scalara64_fpx_*call sites (one definition per function, its body ending where the next begins) and ranks by samples × calls, cutting at<share>of that total. The summary line names the weighted share. Without the flag nothing changes; the Zstd decoder is appended as before.scripts/hotlist.py:--fp-calls, the ranking, the summary line.tests/test_prof_symbols.py: a function that is the innermost frame of 2 samples and holds an FP call beats one sampled 8 times without one; the weighted summary is printed; the Zstd decoder is still appended.docs/development.md: theA64_HOT_FUNCSrow, which was missing from the build switches.The list itself is not committed: it is lifted function addresses from a profile of one PC, regenerated per lift (
A64_HOT_FUNCSis in the lift key, so a build given a list is translated again).Measurements
Verdict: on current
main, a build using the list this PR's tool writes measures +1.64 % fps (median of 5; exact permutation p = 0.381, the ranges overlap) and -1.89 % CPU (p = 0.048, the ranges only touch) against main's own build, for +3 min 55 s of a cold build (3:18 to 7:13) and +0.73 % exe size. The same list measured +2.95 % fps and -2.91 % CPU on the previous base (c167ec67); main has since takenalways_inlineheader helpers (#1002) and light-O1passes, which appear to have taken part of that gain and to have made the hot piece the long pole of the compile. So: the tool is useful and its list is small, but the advantage is smaller on today's main than when it was found, and a build that uses a list pays several minutes for it.Baseline: the "Before" column is
mainitself (71b1edd8), built from scratch on this PC in this session. The "After" column is the same commit's source built with the listscripts/hotlist.py --fp-calls 0.79writes: this PR changes only a script, so the game code is identical and the hot list is the whole difference.Hardware: Linux, AMD Ryzen 9 9950X3D (32 threads), RX 9070 XT, RADV (Mesa).
Scene: Kokiri Forest at Link's house, the repo save
kokiri-start, standing still; headless, 1920x1080, the game's own upscaler, V-Sync off, frame generation off,HOS_FPS_MAX=1000, FPS counter off.Method: both builds from the same checkout,
dev_build.py buildwith the object and translation stores off and no shader preparation, so each is a cold lift and full compile; one warm-up run per build, then ten arms through the PC's queue (--mode timing) in the mirrored sequenceA B B A A B B A A B, 40 s each; fps = frames / wall time from the control bridge's counter, CPU =/proc/<pid>/statutime+stime per frame; medians, and an exact permutation test on the difference of medians.How to reproduce:
RX_MAPS=<the profile's build> python -I scripts/hotlist.py <HOS_SAMPLE_ALL file> hot_funcs.txt 0.79 1500 --fp-calls <work>/sa/gen, thenA64_HOT_FUNCS=hot_funcs.txt python -I scripts/dev_build.py buildonmain, and a game run withHOS_FPS_MAX=1000on thekokiri-startsave.71b1edd8, no hot list71b1edd8+ the list this PR's--fp-calls 0.79writeslongshotsizeWhy the list is small (12 s Market profile:
HOS_SAMPLE_ALL, 5,218 samples, 1,137 functions), and what the old time-only cut of the same profile took:c167ec67, time only (share 0.85)0f7a2076,--fp-calls0.79Flags and settings
None. No game switch is added or changed; the measured list was
--fp-calls 0.79on a Market profile (the tool's default share, 0.85, picks 60 functions on that profile), and no list is shipped, so a build with no list behaves exactly asmain.Testing
Tested:
python -I -B tests/test_prof_symbols.pyon Linux (the new--fp-callscase cannot pass onmain: the option does not exist there);python -I -B tests/run_all.pyon Linux, 586 passed, 17 failed, 74 skipped — all 17 also fail onmain @ 71b1edd8(test_audio_output_devices,test_bug_report,test_bug_report_defaults,test_crt_options,test_dlss_feature_size,test_failed_compile_keeps_objects,test_guest_imported,test_hardware_report,test_linux_preflight,test_linux_update,test_make_game_cache,test_missing_nup,test_mods_size_table,test_pending_guest_import,test_run_all_changed,test_vk_occlusion_queries,test_walk_key). The in-game numbers above, with a list this tool wrote, on Linux/RADV.Not tested: the time-only default list (no comparison against it was run); Windows (the tool reads only the lift's C and a sample file); a profile taken in a scene other than the Market (the list was measured in Kokiri).
Must be tested: None.
Risks
What could break: only when the option is used: a poorly chosen list makes the hot chunk bigger and slower to compile. It cannot change what the game computes.
Turning it off: don't pass
--fp-calls(the old behaviour is unchanged), or revert the commit.Compatibility: Nothing changes for a player: no switch is added to the game, no list is shipped, and a build with no list translates exactly as
maindoes.Checklist
Everyone
type(scope): description) of the status line's type.hotlist,A64_HOT_FUNCS,hot_funcs,always_inline; nothing fits).main(71b1edd8).python -I -B tests/run_all.pypasses (the 17 that fail also fail onmain, named above).third_party/untouched (a new folder with aPROVENANCE.mdinstead).When it applies
Assisted-by:/Co-Authored-By:trailers.docs/development.md); no game switch is added.mainand passing here.tests/that reads changed source updated in this PR, not weakened (tests/test_prof_symbols.py).WIP: perf(scripts): hotlist.py ranks a hot list by the FP helper calls it inlinesto perf(scripts): hotlist.py ranks a hot list by the FP helper calls it inlinesPlease include build time measurements, because I had looked at this (I made an issue) and I was able to get the 3% faster CPU but build time was 19% more
Measured the from-scratch build time of the two lists (same checkout, , object and translation stores off, no shader preparation, two runs each, mirrored order old/new/old/new):\n\n| | Before: time-only list (441 functions) | After: --fp-calls list (47 functions) |\n|---|---|---|\n| build, median of 2 | 7 min 37 s (7:30, 7:44) | 7 min 02 s (7:06, 6:58) |\n| compile phase | 391 s | 357 s |\n| lift phase | 53 s | 52 s |\n| hot-chunk C | 72 MB | 25 MB |\n| longshot | 541,310,296 B | 537,922,424 B (-0.63 %) |\n\nThe whole difference is in the compile (the lift is the same): the smaller hot chunk saves ~35 s (-7.6 %) of a cold build on this 32-thread PC, because the hot piece compiles alongside the other chunks rather than on its own. So the list's gain is code size and the hot piece's compile on fewer cores, not a build-time win on a big machine — the description now says that.
Clarified the binary sizes, which are relative to two different baselines: the build-time table compared the two lists (the new one is 0.63% smaller), while the 0.83% figure is against the build with no list. All three, exact (each build is bit-identical, same sha256):\n\n| build | longshot | against no list |\n|---|---|---|\n| no list | 533,476,168 B | - |\n| this PR's list (47 functions) | 537,922,424 B | +0.83% |\n| time-only list (441 functions) | 541,310,296 B | +1.47% |\n\nBoth lists are bigger than no list because a hot call site becomes a gate and an SSE op inline (about 190 bytes) instead of a call; the old list inlines about twice as many sites, so it grows more. The description says this now.
Re-measured against itself (), as asked: both builds from scratch on the same checkout (stores off, no shader preparation), one warm-up each, then the ten-arm mirrored A/B.\n\n| | main @
71b1edd8, no list | main @71b1edd8+ this PR's list |\n|---|---|---|\n| fps, median of 5 | 90.11 | 91.59 (+1.64%, p = 0.381) |\n| CPU ms a frame, median of 5 | 43.311 | 42.492 (-1.89%, p = 0.048) |\n| build from scratch | 3 min 18 s (compile 135 s) | 7 min 13 s (compile 369 s) |\n| longshot | 543,806,840 B | 547,787,416 B (+0.73%) |\n\nTwo things changed against the earlier write-up, and the description now says so: on today's main the same list is worth about half the fps it was on (2.95% then, 1.64% now, and not separable from noise at 5 + 5), and the build cost is now the long pole — main's own compile has become fast enough (135 s) that the 25 MB hot piece at (369 s) doubles the build instead of hiding behind the chunks.\n\nThe CPU saving is still the part that holds up: the slowest list run (42.889) is below the fastest main run's median (43.311), p = 0.048.Typo: 'the an advantage' -> 'the advantage'.
Build-budget data from the follow-up workstreams (relevant to this PR's ranking):\n\n- The list this tool writes at share 0.79 (47 functions, 25 MB of hot C) builds in 3 min 17 s from scratch once the giant hot function is split at compile time (one line in nx2/build.py, PR #1146: the hot pieces were the only ones without LONGSHOT_SPLIT_GIANTS).\n- Widening the same ranking to share 0.90 (633 functions, 88 MB of hot C) costs only +34 s of build (3:51 vs 3:17) and measured -6.72 % CPU (median of 5, exact permutation p = 0.048) against the 0.79 list, with fps +0.47 % (p = 0.12) in Kokiri — i.e. the extra -O2 coverage buys CPU headroom rather than frames in that scene.\n- Compiling whole chunks at -O2 instead is out of the question: all 88 chunks at -O2 is 23 min 25 s from scratch (compile 22:25, 9.6x the chunk CPU) against 3:29, so the cheap way to spend build time on -O2 is a longer hot list, not a higher chunk level.\n\nSo a higher default share (0.9) now looks like the better trade for this tool than 0.79; I have not changed the default here — say the word and I will.
Following up on the build-time question you raised (you measured +19 % on a profile-built list): the cost was never the list as such — it was one translation unit. In a 47-function list, is the single lifted function (4.79 MB of C), and at the hot flags it took 323 s of a 389 s build. The lifted chunks already split such giants at compile time (); the hot pieces did not. With that one line (PR #1146), on this PC, from scratch, stores off, same 47-function list:\n\n| | no list | 47-function list, hot pieces unsplit | 47-function list, hot pieces split (#1146) |\n|---|---|---|---|\n| build | 3:18 | 6:29 | 3:09 |\n| compile phase | 135 s | 331 s | 132 s |\n| | — | 323.0 s / 1.7 GB peak RSS | 54.0 s / 0.4 GB peak RSS |\n\nSo the list's build cost is now zero relative to no list, and more of it is cheap too: widening the same ranking from 0.79 to 0.90 share (47 -> 633 functions, 25 -> 88 MB of hot C) builds in 3:51 (+34 s) and measured -6.72 % CPU (p = 0.048) against the 47-function list in Kokiri, with fps +0.47 % there (p = 0.12 — that scene is not CPU-bound).\n\nFor contrast, spending the same budget on whole chunks at does not work: all 88 chunks at is 23:25 from scratch (9.6x the chunk CPU) against 3:29 (PR #1180, evidence only, WIP).
perf(scripts): hotlist.py ranks a hot list by the FP helper calls it inlinesto WIP: perf(scripts): hotlist.py ranks a hot list by the FP helper calls it inlinesView command line instructions
Checkout
From your project repository, check out a new branch and test the changes.