perf(build): ship a hot list, so every build compiles the game's hottest functions at -O2 #1635
No reviewers
Labels
No labels
area/accessibility
area/audio
area/input
area/install
area/menu
area/renderer
area/saves
area/tooling
area/upscaling
beanstalk: approved
beanstalk: failed
beanstalk: needs approval
beanstalk: running
beanstalk: tested
ci: approved
good first issue
gpu/amd
gpu/igpu
gpu/intel
gpu/nvidia
help wanted
kind
bug
kind
chore
kind
docs
kind
feature
kind
performance
kind
question
needs measurements
platform/linux
platform/macos
platform/steam-deck
platform/windows
preview
priority
critical
priority
high
priority
low
regression
risk
high
risk
low
risk
medium
status
blocked
status
confirmed
status
duplicate
status
invalid
status
needs-info
status
wontfix
ci: approved
No milestone
No project
No assignees
2 participants
Notifications
Due date
No due date set.
Dependencies
No dependencies set
Reference
MagicBeanDev/Longshot!1635
Loading…
Reference in a new issue
No description provided.
Delete branch "snaildev99/Longshot:perf/ship-hot-list"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Type: perf · Risk: medium · Tested on: Windows 11, RTX 3090 · Needs: Linux, a Steam Deck
TL;DR
A player's build compiles all of the game's translated code at the lowest optimisation level, because no hot list ships
with Longshot. This PR commits a hot list of 680 functions recorded with Longshot itself and makes every build use it:
the Windows builder, the AppImage and a checkout. On a 12-core PC,
mainagainst this branch, two interleaved runseach: +24 % fps on Death Mountain Trail (the game's time a frame -19 %) and +10 % in Kakariko, for about 45 s more
build from scratch (8:02 against 6:58, the branch built first). It still needs a run on Linux and on a Steam Deck.
Breakdown
-O1with everything else; only developers with their ownwork/hot_funcs.txtgot them at-O2.-O2in the eight hot pieces, from a list committed inassets/profiles/.work/hot_funcs.txtorA64_HOT_FUNCSstill wins; no play profile of the pieces ships; saves and options are as they were (on Windows the renderer now gets its committed profile too, see Risks).main@35a4a9fcagainst this branch, each built from scratch and played on one route, two interleaved runs each: trail 51.8 -> 64.3 fps, Kakariko 55.4 -> 60.9; 15 new game-free checks;tests/run_all.pyfails only wheremainfails.-O2compiles need no more memory each than an ordinary chunk's, 0.4 to 1.5 GB), and the first build after updating translates the game again and recompiles most lifted files.Why
Issue #505 proposed shipping a hot list with releases; today
make_game.pyuses one only when the checkout'swork/hasit, so a player's build (the Windows builder, the AppImage) compiles every lifted file at
-O1. The data on #505 since:Mountain Trail (one session a side, not an A/B) a list holding the 248 functions hot there gave 27.0 fps against 22.3
without one. His first table (no list 21.9 fps, 319 functions 26.8, 773 functions 26.9) turned out to be the screen
after Link dies under the falling rocks, a frame the game's code limits but not the trail, and "the first 300
functions do as well as the whole list" does not hold in play: that list had 31 of those 248 functions.
between threads), and no list was not measured.
fps on Death Mountain Trail and +9 % in Kakariko. The list got about 87 % of the saving in game time a frame that
building every chunk at
-O2(#1431's switch) gets (84 to 86 % of its fps gain), for 25 s of build instead of 3 min24 s (both incremental builds after a new lift).
On the trail the main thread waits for the actor update about 70 % of the time, and while it waits, at most one of the
six update workers runs about two thirds of the time (Havok physics and cloth): that serial tail is lifted code, which is
what the list speeds up. Where the renderer sets the frame the list does less: on #505 an earlier list gave +3 % walking
in the Market and nothing measurable standing there.
See #505.
What changed
The list is committed as
assets/profiles/hot_funcs.txt, andmake_game.pygives it to the lift unless a developer's ownlist or
LONGSHOT_HOT_LIST=0says otherwise; the AppImage already carriesassets/, and the Windows builder's packagenow carries
assets/profiles/too.assets/profiles/hot_funcs.txt, 680 linesf_<guest address>(guest addresses only, no game code), with itsnote in
assets/profiles/README.md: where and how it was recorded (game 1.1.1, Longshot's own lift of544229a2,adult Link through Kakariko and up Death Mountain Trail played by hand on a build with no list, child Link standing in
the Market and in Hyrule Field, areas weighted equally, cut at 85 %, by
scripts/hotlist.py's rule applied outside therepository), what it covers and what it measured.
nx2/make_game.pyhot_list_for(): the checkout'swork/hot_funcs.txtfirst, then a listA64_HOT_FUNCSnames in the shell, then the committed one, copied to
<work>/hot_funcs.txtwhen it differs (so the lift key hashes itscontents as before and an unchanged list keeps the translation), or read where it is when
--workis the checkout'sown
work/(a copy there would count as the checkout's own list from then on).LONGSHOT_HOT_LIST=0builds with none.A
<work>/hot_funcs.txtleft by an earlier build is removed when the build doesn't use it, and replaced when itdiffers from the committed one, each with a line in the log (p0lux's note on #505: before, after one build with a list
every later build got that list).
hot_list_line()says which list the build takes.lift_key(): no list(
A64_HOT_FUNCSempty in the build's environment) hashes a marker, so it never takes the key of a lift with the list,also when the checkout's own list sits in its own work folder.
scripts/builder_package.pycopies the trackedassets/profiles/into the Windows builder'stools/,where
make_game.pylooks for it (a separate commit: it also gives Windows players' builds the renderer's committedprofile, #569, which the package never carried).
scripts/linux/appimage.shalready copiesassets/whole (commentsonly);
scripts/linux/check_player_build.shchecks that the image carries a well-formed list.docs/development.md(a "The hot list" section, theLONGSHOT_HOT_LISTrow, "A faster build for development",the build times' "No hot pieces" bullet),
docs/design.md(a release no longer ships without one).tests/test_shipped_hot_list.py(new, 15 cases),tests/test_appimage_player_build.py(the image carries thelist).
Measurements
Verdict: faster where the game's own code sets the frame rate:
mainagainst this PR, two interleaved runs each,Death Mountain Trail 51.8 -> 64.3 fps (+24 %, game time a frame 18.1 -> 14.7 ms) and Kakariko 55.4 -> 60.9 fps (+10 %,
game time 17.0 -> 15.0 ms), for 64 s more build from scratch on 12 cores (6:58 -> 8:02), about 45 s of it the list's.
Baseline:
main@35a4a9fcagainst this PR @715b3b4e(its two commits on thatmain), each built from scratch into thesame game folder (a work folder of its own: a new lift, no object cache; the game's files already extracted), their
longshot.exeandlongshot_vk.dllkept and swapped in before each run. No mods, and nowork/hot_funcs.txtin eithercheckout.
Hardware: Windows 11, NVIDIA RTX 3090 (driver 610.88), AMD Ryzen 9 5900X (12 cores, 24 threads), 128 GB
Scene: adult Link from one save: Kakariko Village with three of its houses, then two legs up Death Mountain Trail, played by hand (about 3 minutes); full screen 3840x2160 (output 2048x1152), frame rate cap Uncapped
Method: four runs in the order PR, main, PR, main, with the profiling mode on (
HOS_PROFILE=1, the same cost in every run); the frames in play only: the renderer's own 120-present log windows with at least 500 draws and under 100 fps, the area from a timeline of the control bridge'sstatus, the first window after arriving left out (so loads, menus and the quit menu at the end are out); frame times frompresents.csv; game time a frame is the frame trace'sgame_ms, median of those frames under 50 ms; the verdict gives the median of each side's two runs; build times from each build's last line and itsbuild_times.jsonHow to reproduce: build
mainand this branch (or this branch withLONGSHOT_HOT_LIST=0and without it), play the same route from the same save, compare the in-play frames by area35a4a9fc(runs M1 / M2)715b3b4e(runs P1 / P2)the sides are 12.5 fps apart on the trail (3.4 ms of game time) and 5.5 in Kakariko (2.0 ms).
against 1 s, the shader list 11 s against 5 s). Of the 64 s between the two, the compile is +38 s (330 -> 368 s) and
the lift +7 s (79 -> 86 s): about 45 s for the list on 12 cores. A third build of the branch from scratch, not the
first in line (made for the memory figures in Risks, a sampler running beside it), took 7:50, 52 s more than
main.All 680 listed functions are functions of the lift at
35a4a9fc.mainand inone of this PR's two: it is not the list's. Kakariko's slow frames show no pattern between the sides (41 and 19
against 43 and 14 over 33 ms).
544229a2carrying our unmerged mod pull requests with their switches off, onerun a build, the same route; the list put in
work/hot_funcs.txtby hand), game time a frame on the trail / inKakariko: no list 20.8 / 18.2 ms; this list 16.7 / 16.2 ms; the 849-function list developers had 16.9 / 16.7 ms; no
list and every chunk at
-O2(#1431) 16.1 / 15.9 ms. So the list got about 87 % of what every chunk at-O2saved,for 25 s more build where all
-O2cost 3 min 24 s (incremental builds, 6:17, 6:42 and 9:41), and the old list andthis one came out level.
the field's; the 849-function list developers had (Hookshot's, from an unrecorded route) covers 68.5 to 77 %. A list
made without one of the four areas still covers 79 to 84 % of it (the left-out area's neighbour stays in). On the route
about 12 to 19 % of the samples charged to a lifted function have their innermost frame in host code under it: float
helpers, which a listed function inlines, but also lookups, locks and sleeps, which no list speeds up.
pieces of 4 to 22 MB. The first 300 would be 52.4 MB.
-O1:about +21 % of the lifted compile's CPU time, which a PC with 4 to 8 threads pays in wall time (p0lux's Steam Deck,
--jobs 4: 1,283 s with 319 functions, 1,752 s with 773).Flags and settings
One new build switch,
LONGSHOT_HOT_LIST, off by default (the list is used).LONGSHOT_HOT_LIST=0(build)work/hot_funcs.txtwas there0: none0: every lifted function at-O1, as a player's build was before; the build about 45 s shorter from scratch on 12 cores; the game translated again when switched0only to compareTesting
Tested: on Windows 11, RTX 3090 (driver 610.88), in the game: this branch at
715b3b4ebuilt from scratch with nowork/hot_funcs.txtin the checkout (the log:the hot list: the committed assets/profiles/hot_funcs.txt,hot functions (A64_HOT_FUNCS): 680 of 680 listed found, in code_hot.c, 0 failed compiles) and played twice over the route above(Kakariko Village with three houses, Death Mountain Trail), between two runs of
main@35a4a9fcbuilt the same way: nofault seen, the numbers are in Measurements. Game-free, on this branch:
tests/test_shipped_hot_list.py, 15 cases, all passing: thecommitted list's lines (
f_<8 hex>, none twice, at most 1,500) and its note; which list a build takes (the committed one,a checkout's own first, a shell's
A64_HOT_FUNCSnext, none withLONGSHOT_HOT_LIST=0, an earlier build's copy replaced orremoved and the checkout's own never touched, and with
--workin the checkout's ownwork/the committed list read inplace, a later change of it reaching the build); the lift key (another list translates again, the same one does not, no
list never shares a key with the list, also in the checkout's own work folder); the Windows package carries
assets/profiles/andmake_game.pyrun from itstools/finds the list there; and no listed function is one whose textedit in
nx2/oot_patches.pyfinds its place by a float helper call (a hot function's are rewritten to their inlinecopies, so such an edit would be lost: the step divides' hook is one; none of the 680 is). With the key's marker, the
in-place read, or a listed
f_80706be0, their cases fail. The AppImage's copy block run in Git Bash puts the list in itsassets/profiles/.python -I -B tests/run_all.py --jobs 16(LONGSHOT_MINGWset,opensslonPATHfortests/test_release.py) at715b3b4e: 634 passed, 41 failed, 114 skipped; the 41 are the same checks that fail onmain@
35a4a9fcon this PC (633 passed, 41 failed, 114 skipped in 152 s), among themtests/test_switch_table.py,tests/test_profiling_mode.pyandtests/test_dev_packs.py,which read files this PR changes and fail there with the same output;
tests/test_tested_configurations.pywith the fournew run files: its one failure (a line-ending case of the command's output on Windows) is
main's. The list's nameschecked against lifts: all 680 are functions of the lift at
35a4a9fc(the build above) and of the earlier ones.Not tested: the build time and the memory gate on fewer cores and 8 to 16 GB; child Link in play, dungeons and
fights; Linux; the Windows package as CI makes it (only
package()in a test); the renderer's profile on a Windowsplayer's build.
Must be tested:
--jobs 4, and the trail's fps againstmain(Test Farm: @Beanstalk linux steamdeck, a player's build of this PR's AppImage and the trail's fps against main)LongshotBuilder/tools/assets/profiles/hot_funcs.txtandlongshot_vk.profdata.xz, expecting both there (Test Farm: @Beanstalk windows, list the builder zip's tools/assets/profiles)Risks
What could break: builds take longer: from scratch on 12 cores 8:02 against 6:58, about 45 s of it the list's
(measured), and more on fewer threads, since the eight pieces add about a fifth to the lifted compile's CPU time (on a
Steam Deck with 4 jobs a 773-function list took 469 s longer than a 319-function one; this list has 680): that is the
Steam Deck item under Must be tested. Memory, measured in a third build of this branch from scratch with every
compiler's peak sampled: the eight
-O2pieces peak at 0.4 to 1.5 GB each (417, 440, 518, 916, 1,063, 1,091, 1,287 and1,468 MB of working set; 7.2 GB if all eight run at once, as they did here), no more than an ordinary chunk's
-O1compile, which peaks at 1.5 to 1.6 GB; they hold it longer, since the largest piece compiles for 5 min 44 s and is the
build's critical path. All compilers together peaked at 23.5 GB with 23 jobs; the build's memory gate starts compilers
by free memory, and what that means for a PC with 8 to 16 GB is not measured. A listed function whose text edit in
nx2/oot_patches.pyis anchored on a float helper callwould lose that edit with a WARNING line only (the step divides' hook, the game's speed above 60 fps): none of the 680 is,
and
tests/test_shipped_hot_list.pyfails if a later list adds one. A pieces' play profile left in a work folder(
longshot_pieces.prof) still applies to this list's pieces: a weaker fit, not a fault. On Windows the builder now carriesthe renderer's committed profile as well, so a Windows player's renderer is compiled with it for the first time (#569
measured -3 to -5 % frame time on Linux; a Windows checkout already builds with it); that is the second commit, and can be
its own PR.
Open for review: (1)
scripts/hotlist.pyalways adds the game's Zstd decoder (22 functions; a whole decode is 16 %quicker at the hot flags); this list has 4 of them by their samples, and the other 18 are 0.51 MB of C: add them (698
functions)? (2) The Windows package's commit also brings the renderer's profile to Windows players; keep it here, or copy
only
hot_funcs.txthere and the rest in its own PR? (3) A shorter list for CPUs with few cores, or one list for everyPC? p0lux has withdrawn "the first 300 do as well as the whole" (#issuecomment-49394: a 300 cut missed most of what was
hot at his slow spot), and our data points the same way: coverage grows with size without a knee, a 300-function cut
covers about 73 % of held-out runs' lifted samples against 84 % for 680 and 88 % for 1,000, for roughly 60 % of the
680's extra compile (modelled; the model has one calibration point and reads low for few cores). Two other rankings
were proposed, by samples per byte of C and by time on the frame's critical path. Neither has been built or played, so
this PR keeps the measured list. (4) Record it again over the pipeline tour's route
(#111) for breadth, child Link and the dungeons included. (5)
scripts/hotlist.pycharging a split giant's piecesdirectly, and
--alsofor several runs, are a separate small PR. (6) #1124 (Marcuss2, WIP) ranks a list by samples timesthe float helper calls a function holds (
--fp-calls) and keeps lists out of the repository, regenerated per lift; thisPR commits one, because a player's build has no profile to make one from, and a list of guest addresses stays valid
while the lifted functions' boundaries do (worth a check at each release that every listed name is still a function of
the lift; the build log's "N of M listed found" line shows it too). The two fit together:
the committed list could be re-made with #1124's ranking once it lands. His build numbers agree with ours: with the hot
pieces split (#1146), a 47-function list built in 3:09 against 3:18 with none, and a 633-function list (88 MB of hot C)
in 3:51 from scratch on a 32-thread Linux PC, with -6.7 % CPU in Kokiri Forest standing (a scene the game's code does
not limit, so +0.5 % fps there). (7) #1593 (HumanBean, WIP) pulls the same lever another way:
LONGSHOT_CHUNK_OPT=executorcompiles eight whole chunk files at
-O2(Market +2.5 to +3.4 % on a Ryzen 9 7900X), where this PR moves 680 functionsinto the hot pieces. The two have not been measured in one scene: one build of each on one route would compare them,
and they can also be combined (the hot pieces keep
RX_HOT_OPT, the preset takes the chunks). One thing to check onthe preset, read from
nx2/chunks.pyand not tried: a chunk file's boundaries are kept per work folder(
image/chunks.txt, made from the functions' sizes at that folder's first lift), socode_017.cneed not hold the samefunctions on every install (two of our own work folders, first lifted at different versions, cut their chunks
differently). A list of
f_names (whichLONGSHOT_CHUNK_OPT=<file>also takes) names the same functions everywhere.Turning it off:
LONGSHOT_HOT_LIST=0for a build (the launcher has no setting for it), or revert the commits.Compatibility: the first build after updating translates the game again (the lift key holds the list), and since the
listed functions move out of their chunk files into
code_hot.c(81 of the 92 chunk files hold one), most lifted filescompile again: close to a first build's compile. On Windows the renderer also compiles once more, with its profile
(second commit). Saves,
longshot_options.ini, the shader and pipeline caches and the vk ABI are unchanged. A developer'sown
work/hot_funcs.txtkeeps working as before; a work folder's copy of an earlier list is no longer used when thecheckout has none.
Issues
See #505
Checklist
Everyone
type(scope): description) of the status line's type.can't be run by hand is in Testing's Not testable by hand, with why.
in Risks.
main.python -I -B tests/run_all.pypasses (any failure that also fails onmainis named).third_party/untouched (a new folder with aPROVENANCE.mdinstead).When it applies
Assisted-by:/Co-Authored-By:trailers.docs/tested/for each new measured run (format indocs/tested-configurations.md).mainand passing here.tests/that reads changed source updated in this PR, not weakened.Credits: the measurements on #505 by p0lux (a Steam Deck: no list against a list in play, the build times, and the
stale work-folder list) and waryxxful (a 5600X: list against list), and the proposal itself by HumanBean.
🤖 Generated with Claude Code
Tested in game: Windows 11, NVIDIA RTX 3090, Ryzen 9 5900X: the list this PR commits ran in the game as
work/hot_funcs.txtin a test build of544229a2(v3.0.0-rc.7): adult Link through Kakariko Village, three of its houses and two legs up Death Mountain Trail, about 3 minutes, no fault seen; against the same source built with no list, the trail went 45.3 -> 56.4 fps and Kakariko 51.3 -> 55.7 (one run a side, the table is in the description). This branch's own build path (make_game.py taking the committed list, the Windows package) has not been built or played yet: that is the first item under Must be tested, and why this is WIP.Built and played:
main@35a4a9fcagainst this branch @715b3b4e, each from scratch (a new work folder), then four route runs in the order list, main, list, main (Ryzen 9 5900X, RTX 3090, Windows 11; Kakariko Village with three houses, then Death Mountain Trail).the hot list: the committed assets/profiles/hot_funcs.txt,680 of 680 listed found, 0 failed compiles.maintoo (1.8 and 1.9 s in its two runs, 1.6 s and none in this PR's), so it is not the list's.The description has the full table and the raw lines; the head is now
c4f53a3c, which adds the four run files indocs/tested/and changes no code. Still open: the memory of the eight-O2compiles (the one unticked box; being measured), and a Linux and a Steam Deck run.WIP: perf(build): ship a hot list, so every build compiles the game's hottest functions at -O2to perf(build): ship a hot list, so every build compiles the game's hottest functions at -O2The last open box, the memory of the eight
-O2compiles, is measured: a third build of this branch from scratch with every compiler's peak memory sampled about once a second (Ryzen 9 5900X, 23 jobs).-O1compile peaks at 1.5 to 1.6 GB, so a hot piece needs no more memory than a chunk does; it holds it longer.main's 6:58, which fits the 45 s the first pair gave.The description has it under Risks, and every Everyone box is ticked now. Still wanted from others: a Linux and a Steam Deck run (the request follows).
@Beanstalk steamdeck and linux amd: build
mainand this PR as a player's install does (on the Deck with--jobs 4) and report each build's whole time, and for this PR the two log linesthe hot list: the committed assets/profiles/hot_funcs.txtandhot functions (A64_HOT_FUNCS): 680 of 680 listed found. Then on each build, as adult Link on Death Mountain Trail withHOS_OOT_PROTECT_HEARTS=1(it keeps Link alive under the falling rocks), walk the lower trail for a minute; and the same in Kakariko Village. Expecting no crash, the same picture asmain, and with this PR a higher frame rate and less game time a frame thanmainwhere the game's code limits the frame (the trail), with nothing lost where it doesn't.If a Windows tester is free as well: the builder zip CI makes from this branch should hold
tools/assets/profiles/hot_funcs.txtandlongshot_vk.profdata.xz.@snaildev99 only members of
BigBeanscan start the farm. One of them can approve this request by reacting 👍 to it or to this comment.