perf(build): ship a hot list, so every build compiles the game's hottest functions at -O2 #1635

Merged
HumanBean merged 3 commits from snaildev99/Longshot:perf/ship-hot-list into main 2026-11-18 20:03:14 +00:00
Contributor

Type: perf · Risk: medium · Tested on: Windows 11, RTX 3090 · Needs: Linux, a Steam Deck

TL;DR

A player's build compiles all of the game's translated code at the lowest optimisation level, because no hot list ships
with Longshot. This PR commits a hot list of 680 functions recorded with Longshot itself and makes every build use it:
the Windows builder, the AppImage and a checkout. On a 12-core PC, main against this branch, two interleaved runs
each: +24 % fps on Death Mountain Trail (the game's time a frame -19 %) and +10 % in Kakariko, for about 45 s more
build from scratch (8:02 against 6:58, the branch built first). It still needs a run on Linux and on a Steam Deck.

Breakdown

  • Who it helps: every player whose PC is limited by the game's own work (busy places like Death Mountain Trail and Kakariko, CPUs with few cores such as the Steam Deck).
  • What was wrong: releases ship no hot list, so the functions the game spends most of its time in were built at -O1 with everything else; only developers with their own work/hot_funcs.txt got them at -O2.
  • What changes: every build compiles the 680 hottest lifted functions at -O2 in the eight hot pieces, from a list committed in assets/profiles/.
  • What does not change: a developer's own work/hot_funcs.txt or A64_HOT_FUNCS still wins; no play profile of the pieces ships; saves and options are as they were (on Windows the renderer now gets its committed profile too, see Risks).
  • How it was checked: main @ 35a4a9fc against this branch, each built from scratch and played on one route, two interleaved runs each: trail 51.8 -> 64.3 fps, Kakariko 55.4 -> 60.9; 15 new game-free checks; tests/run_all.py fails only where main fails.
  • What is still open: a build and a run on Linux and on a Steam Deck, and the build time on fewer cores.
  • Risk: medium: a longer build (about +45 s from scratch on 12 cores, more on 4 to 8 threads; the eight -O2 compiles need no more memory each than an ordinary chunk's, 0.4 to 1.5 GB), and the first build after updating translates the game again and recompiles most lifted files.

Why

Issue #505 proposed shipping a hot list with releases; today make_game.py uses one only when the checkout's work/ has
it, so a player's build (the Windows builder, the AppImage) compiles every lifted file at -O1. The data on #505 since:

  • p0lux, Steam Deck OLED (4 cores), as corrected in #issuecomment-49394: in real sessions at the slow spot of Death
    Mountain Trail (one session a side, not an A/B) a list holding the 248 functions hot there gave 27.0 fps against 22.3
    without one. His first table (no list 21.9 fps, 319 functions 26.8, 773 functions 26.9) turned out to be the screen
    after Link dies under the falling rocks, a frame the game's code limits but not the trail, and "the first 300
    functions do as well as the whole list" does not hold in play: that list had 31 of those 248 functions.
  • waryxxful, Ryzen 5 5600X: one list against another moved nothing measurable there (the frame was paced by hand-offs
    between threads), and no list was not measured.
  • Our runs on a Ryzen 9 5900X (comment #issuecomment-48225, corrected): no list against a fresh 680-function list, +25 %
    fps on Death Mountain Trail and +9 % in Kakariko. The list got about 87 % of the saving in game time a frame that
    building every chunk at -O2 (#1431's switch) gets (84 to 86 % of its fps gain), for 25 s of build instead of 3 min
    24 s (both incremental builds after a new lift).

On the trail the main thread waits for the actor update about 70 % of the time, and while it waits, at most one of the
six update workers runs about two thirds of the time (Havok physics and cloth): that serial tail is lifted code, which is
what the list speeds up. Where the renderer sets the frame the list does less: on #505 an earlier list gave +3 % walking
in the Market and nothing measurable standing there.

See #505.

What changed

The list is committed as assets/profiles/hot_funcs.txt, and make_game.py gives it to the lift unless a developer's own
list or LONGSHOT_HOT_LIST=0 says otherwise; the AppImage already carries assets/, and the Windows builder's package
now carries assets/profiles/ too.

  • list: assets/profiles/hot_funcs.txt, 680 lines f_<guest address> (guest addresses only, no game code), with its
    note in assets/profiles/README.md: where and how it was recorded (game 1.1.1, Longshot's own lift of 544229a2,
    adult Link through Kakariko and up Death Mountain Trail played by hand on a build with no list, child Link standing in
    the Market and in Hyrule Field, areas weighted equally, cut at 85 %, by scripts/hotlist.py's rule applied outside the
    repository), what it covers and what it measured.
  • build: nx2/make_game.py hot_list_for(): the checkout's work/hot_funcs.txt first, then a list A64_HOT_FUNCS
    names in the shell, then the committed one, copied to <work>/hot_funcs.txt when it differs (so the lift key hashes its
    contents as before and an unchanged list keeps the translation), or read where it is when --work is the checkout's
    own work/ (a copy there would count as the checkout's own list from then on). LONGSHOT_HOT_LIST=0 builds with none.
    A <work>/hot_funcs.txt left by an earlier build is removed when the build doesn't use it, and replaced when it
    differs from the committed one, each with a line in the log (p0lux's note on #505: before, after one build with a list
    every later build got that list). hot_list_line() says which list the build takes. lift_key(): no list
    (A64_HOT_FUNCS empty in the build's environment) hashes a marker, so it never takes the key of a lift with the list,
    also when the checkout's own list sits in its own work folder.
  • packaging: scripts/builder_package.py copies the tracked assets/profiles/ into the Windows builder's tools/,
    where make_game.py looks for it (a separate commit: it also gives Windows players' builds the renderer's committed
    profile, #569, which the package never carried). scripts/linux/appimage.sh already copies assets/ whole (comments
    only); scripts/linux/check_player_build.sh checks that the image carries a well-formed list.
  • docs: docs/development.md (a "The hot list" section, the LONGSHOT_HOT_LIST row, "A faster build for development",
    the build times' "No hot pieces" bullet), docs/design.md (a release no longer ships without one).
  • tests: tests/test_shipped_hot_list.py (new, 15 cases), tests/test_appimage_player_build.py (the image carries the
    list).

Measurements

Verdict: faster where the game's own code sets the frame rate: main against this PR, two interleaved runs each,
Death Mountain Trail 51.8 -> 64.3 fps (+24 %, game time a frame 18.1 -> 14.7 ms) and Kakariko 55.4 -> 60.9 fps (+10 %,
game time 17.0 -> 15.0 ms), for 64 s more build from scratch on 12 cores (6:58 -> 8:02), about 45 s of it the list's.

Baseline: main @ 35a4a9fc against this PR @ 715b3b4e (its two commits on that main), each built from scratch into the
same game folder (a work folder of its own: a new lift, no object cache; the game's files already extracted), their
longshot.exe and longshot_vk.dll kept and swapped in before each run. No mods, and no work/hot_funcs.txt in either
checkout.

Hardware: Windows 11, NVIDIA RTX 3090 (driver 610.88), AMD Ryzen 9 5900X (12 cores, 24 threads), 128 GB
Scene: adult Link from one save: Kakariko Village with three of its houses, then two legs up Death Mountain Trail, played by hand (about 3 minutes); full screen 3840x2160 (output 2048x1152), frame rate cap Uncapped
Method: four runs in the order PR, main, PR, main, with the profiling mode on (HOS_PROFILE=1, the same cost in every run); the frames in play only: the renderer's own 120-present log windows with at least 500 draws and under 100 fps, the area from a timeline of the control bridge's status, the first window after arriving left out (so loads, menus and the quit menu at the end are out); frame times from presents.csv; game time a frame is the frame trace's game_ms, median of those frames under 50 ms; the verdict gives the median of each side's two runs; build times from each build's last line and its build_times.json
How to reproduce: build main and this branch (or this branch with LONGSHOT_HOT_LIST=0 and without it), play the same route from the same save, compare the in-play frames by area

Before: main @ 35a4a9fc (runs M1 / M2) After: this PR @ 715b3b4e (runs P1 / P2)
Death Mountain Trail fps 51.3 / 52.2 64.0 / 64.6
trail median / p99 frame 18.1 / 38.5 ms and 18.4 / 29.4 ms 14.9 / 25.4 ms and 14.8 / 26.6 ms
trail game time a frame 17.9 / 18.2 ms 14.7 / 14.6 ms
trail frames over 33 / 50 ms 38 / 6 and 4 / 0 (40 s, 41 s) 6 / 0 and 10 / 1 (39 s, 43 s)
Kakariko fps 55.1 / 55.6 60.2 / 61.6
Kakariko median / p99 frame 17.5 / 30.6 ms and 17.5 / 28.6 ms 15.5 / 32.6 ms and 15.6 / 25.6 ms
Kakariko game time a frame 17.0 / 17.0 ms 14.9 / 15.0 ms
Kakariko frames over 33 / 50 ms 41 / 20 and 19 / 3 (94 s, 95 s) 43 / 3 and 14 / 1 (92 s, 95 s)
longest frame at the save's load 1,801 ms / 1,855 ms 1,627 ms / none over 0.4 s
build from scratch: whole (compile, lift) 6:58 (5:30, 1:19) 8:02 (6:08, 1:26)
| area | run | fps | median ms | p95 ms | p99 ms | frames >= 33 ms | >= 50 ms | game_ms median | seconds |
| DeathMountainTrail_Adult | 2026-11-18_133124_P1 | 64.0 | 14.9 | 20.6 | 25.4 | 6 | 0 | 14.7 | 39 |
| DeathMountainTrail_Adult | 2026-11-18_133448_M1 | 51.3 | 18.1 | 28.4 | 38.5 | 38 | 6 | 17.9 | 40 |
| DeathMountainTrail_Adult | 2026-11-18_133950_P2 | 64.6 | 14.8 | 21.1 | 26.6 | 10 | 1 | 14.6 | 43 |
| DeathMountainTrail_Adult | 2026-11-18_134302_M2 | 52.2 | 18.4 | 25.4 | 29.4 | 4 | 0 | 18.2 | 41 |
| KakarikoVillage_Adult | 2026-11-18_133124_P1 | 60.2 | 15.5 | 23.7 | 32.6 | 43 | 3 | 14.9 | 92 |
| KakarikoVillage_Adult | 2026-11-18_133448_M1 | 55.1 | 17.5 | 23.7 | 30.6 | 41 | 20 | 17.0 | 94 |
| KakarikoVillage_Adult | 2026-11-18_133950_P2 | 61.6 | 15.6 | 21.5 | 25.6 | 14 | 1 | 15.0 | 95 |
| KakarikoVillage_Adult | 2026-11-18_134302_M2 | 55.6 | 17.5 | 23.6 | 28.6 | 19 | 3 | 17.0 | 95 |
this PR:  the hot list: the committed assets/profiles/hot_funcs.txt
this PR:  hot functions (A64_HOT_FUNCS): 680 of 680 listed found, in code_hot.c
this PR:  the build took 0:08:02, compiled in 0:06:08
main:     the build took 0:06:58, compiled in 0:05:30
  • The two runs of a side agree to within 0.9 fps on the trail and 1.4 in Kakariko, and to within 0.3 ms of game time;
    the sides are 12.5 fps apart on the trail (3.4 ms of game time) and 5.5 in Kakariko (2.0 ms).
  • The build: the branch was built first, so it also paid the first build's one-time finish steps (the button prompts 14 s
    against 1 s, the shader list 11 s against 5 s). Of the 64 s between the two, the compile is +38 s (330 -> 368 s) and
    the lift +7 s (79 -> 86 s): about 45 s for the list on 12 cores. A third build of the branch from scratch, not the
    first in line (made for the memory figures in Risks, a sampler running beside it), took 7:50, 52 s more than main.
    All 680 listed functions are functions of the lift at 35a4a9fc.
  • The long frame at the save's load (the game's main thread blocked for 1.6 to 1.9 s) is in both runs of main and in
    one of this PR's two: it is not the list's. Kakariko's slow frames show no pattern between the sides (41 and 19
    against 43 and 14 over 33 ms).
  • Earlier, on v3.0.0-rc.7 (test builds of 544229a2 carrying our unmerged mod pull requests with their switches off, one
    run a build, the same route; the list put in work/hot_funcs.txt by hand), game time a frame on the trail / in
    Kakariko: no list 20.8 / 18.2 ms; this list 16.7 / 16.2 ms; the 849-function list developers had 16.9 / 16.7 ms; no
    list and every chunk at -O2 (#1431) 16.1 / 15.9 ms. So the list got about 87 % of what every chunk at -O2 saved,
    for 25 s more build where all -O2 cost 3 min 24 s (incremental builds, 6:17, 6:42 and 9:41), and the old list and
    this one came out level.
  • The list covers 85.0 % of Kakariko's samples of lifted code, 86.7 % of the trail's, 84.6 % of the Market's and 83.8 % of
    the field's; the 849-function list developers had (Hookshot's, from an unrecorded route) covers 68.5 to 77 %. A list
    made without one of the four areas still covers 79 to 84 % of it (the left-out area's neighbour stays in). On the route
    about 12 to 19 % of the samples charged to a lifted function have their innermost frame in host code under it: float
    helpers, which a listed function inlines, but also lookups, locks and sleeps, which no list speeds up.
  • Its 680 functions are 88.8 MB of the lift's 1,449 MB of C (an 11-12 lift); the build above cut them into eight hot
    pieces of 4 to 22 MB. The first 300 would be 52.4 MB.
  • By compile CPU (the rc.7 builds) the eight pieces took 1,452 s where the same functions take about 330 s at -O1:
    about +21 % of the lifted compile's CPU time, which a PC with 4 to 8 threads pays in wall time (p0lux's Steam Deck,
    --jobs 4: 1,283 s with 319 functions, 1,752 s with 773).

Flags and settings

One new build switch, LONGSHOT_HOT_LIST, off by default (the list is used).

Flag What it is Before this PR After this PR Effect Helps / doesn't help / untested Recommendation
LONGSHOT_HOT_LIST=0 (build) Builds with no hot list Not read: no list unless work/hot_funcs.txt was there Unset: the list (a developer's own first); 0: none 0: every lifted function at -O1, as a player's build was before; the build about 45 s shorter from scratch on 12 cores; the game translated again when switched Helps an A/B against the list; untested on a slow PC Leave unset (the list on for everyone); 0 only to compare

Testing

Tested: on Windows 11, RTX 3090 (driver 610.88), in the game: this branch at 715b3b4e built from scratch with no
work/hot_funcs.txt in the checkout (the log: the hot list: the committed assets/profiles/hot_funcs.txt, hot functions (A64_HOT_FUNCS): 680 of 680 listed found, in code_hot.c, 0 failed compiles) and played twice over the route above
(Kakariko Village with three houses, Death Mountain Trail), between two runs of main @ 35a4a9fc built the same way: no
fault seen, the numbers are in Measurements. Game-free, on this branch: tests/test_shipped_hot_list.py, 15 cases, all passing: the
committed list's lines (f_<8 hex>, none twice, at most 1,500) and its note; which list a build takes (the committed one,
a checkout's own first, a shell's A64_HOT_FUNCS next, none with LONGSHOT_HOT_LIST=0, an earlier build's copy replaced or
removed and the checkout's own never touched, and with --work in the checkout's own work/ the committed list read in
place, a later change of it reaching the build); the lift key (another list translates again, the same one does not, no
list never shares a key with the list, also in the checkout's own work folder); the Windows package carries
assets/profiles/ and make_game.py run from its tools/ finds the list there; and no listed function is one whose text
edit in nx2/oot_patches.py finds its place by a float helper call (a hot function's are rewritten to their inline
copies, so such an edit would be lost: the step divides' hook is one; none of the 680 is). With the key's marker, the
in-place read, or a listed f_80706be0, their cases fail. The AppImage's copy block run in Git Bash puts the list in its
assets/profiles/. python -I -B tests/run_all.py --jobs 16 (LONGSHOT_MINGW set, openssl on PATH for
tests/test_release.py) at 715b3b4e: 634 passed, 41 failed, 114 skipped; the 41 are the same checks that fail on main
@ 35a4a9fc on this PC (633 passed, 41 failed, 114 skipped in 152 s), among them tests/test_switch_table.py, tests/test_profiling_mode.py and tests/test_dev_packs.py,
which read files this PR changes and fail there with the same output; tests/test_tested_configurations.py with the four
new run files: its one failure (a line-ending case of the command's output on Windows) is main's. The list's names
checked against lifts: all 680 are functions of the lift at 35a4a9fc (the build above) and of the earlier ones.

Not tested: the build time and the memory gate on fewer cores and 8 to 16 GB; child Link in play, dungeons and
fights; Linux; the Windows package as CI makes it (only package() in a test); the renderer's profile on a Windows
player's build.

Must be tested:

  • Linux, AMD (a Steam Deck): the AppImage of this branch, a player's build, expecting the same two log lines, its build time with --jobs 4, and the trail's fps against main (Test Farm: @Beanstalk linux steamdeck, a player's build of this PR's AppImage and the trail's fps against main)
  • Windows: the builder zip CI makes from this branch holds LongshotBuilder/tools/assets/profiles/hot_funcs.txt and longshot_vk.profdata.xz, expecting both there (Test Farm: @Beanstalk windows, list the builder zip's tools/assets/profiles)

Risks

What could break: builds take longer: from scratch on 12 cores 8:02 against 6:58, about 45 s of it the list's
(measured), and more on fewer threads, since the eight pieces add about a fifth to the lifted compile's CPU time (on a
Steam Deck with 4 jobs a 773-function list took 469 s longer than a 319-function one; this list has 680): that is the
Steam Deck item under Must be tested. Memory, measured in a third build of this branch from scratch with every
compiler's peak sampled: the eight -O2 pieces peak at 0.4 to 1.5 GB each (417, 440, 518, 916, 1,063, 1,091, 1,287 and
1,468 MB of working set; 7.2 GB if all eight run at once, as they did here), no more than an ordinary chunk's -O1
compile, which peaks at 1.5 to 1.6 GB; they hold it longer, since the largest piece compiles for 5 min 44 s and is the
build's critical path. All compilers together peaked at 23.5 GB with 23 jobs; the build's memory gate starts compilers
by free memory, and what that means for a PC with 8 to 16 GB is not measured. A listed function whose text edit in nx2/oot_patches.py is anchored on a float helper call
would lose that edit with a WARNING line only (the step divides' hook, the game's speed above 60 fps): none of the 680 is,
and tests/test_shipped_hot_list.py fails if a later list adds one. A pieces' play profile left in a work folder
(longshot_pieces.prof) still applies to this list's pieces: a weaker fit, not a fault. On Windows the builder now carries
the renderer's committed profile as well, so a Windows player's renderer is compiled with it for the first time (#569
measured -3 to -5 % frame time on Linux; a Windows checkout already builds with it); that is the second commit, and can be
its own PR.

Open for review: (1) scripts/hotlist.py always adds the game's Zstd decoder (22 functions; a whole decode is 16 %
quicker at the hot flags); this list has 4 of them by their samples, and the other 18 are 0.51 MB of C: add them (698
functions)? (2) The Windows package's commit also brings the renderer's profile to Windows players; keep it here, or copy
only hot_funcs.txt here and the rest in its own PR? (3) A shorter list for CPUs with few cores, or one list for every
PC? p0lux has withdrawn "the first 300 do as well as the whole" (#issuecomment-49394: a 300 cut missed most of what was
hot at his slow spot), and our data points the same way: coverage grows with size without a knee, a 300-function cut
covers about 73 % of held-out runs' lifted samples against 84 % for 680 and 88 % for 1,000, for roughly 60 % of the
680's extra compile (modelled; the model has one calibration point and reads low for few cores). Two other rankings
were proposed, by samples per byte of C and by time on the frame's critical path. Neither has been built or played, so
this PR keeps the measured list. (4) Record it again over the pipeline tour's route
(#111) for breadth, child Link and the dungeons included. (5) scripts/hotlist.py charging a split giant's pieces
directly, and --also for several runs, are a separate small PR. (6) #1124 (Marcuss2, WIP) ranks a list by samples times
the float helper calls a function holds (--fp-calls) and keeps lists out of the repository, regenerated per lift; this
PR commits one, because a player's build has no profile to make one from, and a list of guest addresses stays valid
while the lifted functions' boundaries do (worth a check at each release that every listed name is still a function of
the lift; the build log's "N of M listed found" line shows it too). The two fit together:
the committed list could be re-made with #1124's ranking once it lands. His build numbers agree with ours: with the hot
pieces split (#1146), a 47-function list built in 3:09 against 3:18 with none, and a 633-function list (88 MB of hot C)
in 3:51 from scratch on a 32-thread Linux PC, with -6.7 % CPU in Kokiri Forest standing (a scene the game's code does
not limit, so +0.5 % fps there). (7) #1593 (HumanBean, WIP) pulls the same lever another way: LONGSHOT_CHUNK_OPT=executor
compiles eight whole chunk files at -O2 (Market +2.5 to +3.4 % on a Ryzen 9 7900X), where this PR moves 680 functions
into the hot pieces. The two have not been measured in one scene: one build of each on one route would compare them,
and they can also be combined (the hot pieces keep RX_HOT_OPT, the preset takes the chunks). One thing to check on
the preset, read from nx2/chunks.py and not tried: a chunk file's boundaries are kept per work folder
(image/chunks.txt, made from the functions' sizes at that folder's first lift), so code_017.c need not hold the same
functions on every install (two of our own work folders, first lifted at different versions, cut their chunks
differently). A list of f_ names (which
LONGSHOT_CHUNK_OPT=<file> also takes) names the same functions everywhere.

Turning it off: LONGSHOT_HOT_LIST=0 for a build (the launcher has no setting for it), or revert the commits.

Compatibility: the first build after updating translates the game again (the lift key holds the list), and since the
listed functions move out of their chunk files into code_hot.c (81 of the 92 chunk files hold one), most lifted files
compile again: close to a first build's compile. On Windows the renderer also compiles once more, with its profile
(second commit). Saves, longshot_options.ini, the shader and pipeline caches and the vk ABI are unchanged. A developer's
own work/hot_funcs.txt keeps working as before; a work folder's copy of an earlier list is no longer used when the
checkout has none.

Issues

See #505

Checklist

Everyone

  • One topic.
  • The title is a Conventional Commit (type(scope): description) of the status line's type.
  • Why gives the reason plainly and links the issues that fit (the tracker searched).
  • Tested on my own PC where it runs (in the game, for a change the game runs), as Testing's Tested says; what
    can't be run by hand is in Testing's Not testable by hand, with why.
  • Measurements has a verdict, or one line saying why there is nothing to measure.
  • Nothing gets worse that I know of: any cost (speed, latency, memory, a setup that worked before) is measured and
    in Risks.
  • Testing names what was tested, what wasn't, and what must still be tested, where and how.
  • Rebased on current main.
  • python -I -B tests/run_all.py passes (any failure that also fails on main is named).
  • No game files, dumps, build output or generated files.
  • third_party/ untouched (a new folder with a PROVENANCE.md instead).
  • The license is OK: GPL-3.0-or-later, nothing incompatible and nothing decompiled.

When it applies

  • AI assistance recorded with Assisted-by: / Co-Authored-By: trailers.
  • Numbers from the same PC, scene and settings, at least 2 interleaved runs each, the median reported.
  • Switches documented (docs/development.md; the README for a menu row) and shipped at their recommended value.
  • A file in docs/tested/ for each new measured run (format in docs/tested-configurations.md).
  • A game-free test added for the fix, failing on main and passing here.
  • A check under tests/ that reads changed source updated in this PR, not weakened.
  • Docs updated.

Credits: the measurements on #505 by p0lux (a Steam Deck: no list against a list in play, the build times, and the
stale work-folder list) and waryxxful (a 5600X: list against list), and the proposal itself by HumanBean.

🤖 Generated with Claude Code

**Type:** perf · **Risk:** medium · **Tested on:** Windows 11, RTX 3090 · **Needs:** Linux, a Steam Deck ## TL;DR A player's build compiles all of the game's translated code at the lowest optimisation level, because no hot list ships with Longshot. This PR commits a hot list of 680 functions recorded with Longshot itself and makes every build use it: the Windows builder, the AppImage and a checkout. On a 12-core PC, `main` against this branch, two interleaved runs each: +24 % fps on Death Mountain Trail (the game's time a frame -19 %) and +10 % in Kakariko, for about 45 s more build from scratch (8:02 against 6:58, the branch built first). It still needs a run on Linux and on a Steam Deck. ## Breakdown - **Who it helps:** every player whose PC is limited by the game's own work (busy places like Death Mountain Trail and Kakariko, CPUs with few cores such as the Steam Deck). - **What was wrong:** releases ship no hot list, so the functions the game spends most of its time in were built at `-O1` with everything else; only developers with their own `work/hot_funcs.txt` got them at `-O2`. - **What changes:** every build compiles the 680 hottest lifted functions at `-O2` in the eight hot pieces, from a list committed in `assets/profiles/`. - **What does not change:** a developer's own `work/hot_funcs.txt` or `A64_HOT_FUNCS` still wins; no play profile of the pieces ships; saves and options are as they were (on Windows the renderer now gets its committed profile too, see Risks). - **How it was checked:** `main` @ 35a4a9fc against this branch, each built from scratch and played on one route, two interleaved runs each: trail 51.8 -> 64.3 fps, Kakariko 55.4 -> 60.9; 15 new game-free checks; `tests/run_all.py` fails only where `main` fails. - **What is still open:** a build and a run on Linux and on a Steam Deck, and the build time on fewer cores. - **Risk:** medium: a longer build (about +45 s from scratch on 12 cores, more on 4 to 8 threads; the eight `-O2` compiles need no more memory each than an ordinary chunk's, 0.4 to 1.5 GB), and the first build after updating translates the game again and recompiles most lifted files. ## Why Issue #505 proposed shipping a hot list with releases; today `make_game.py` uses one only when the checkout's `work/` has it, so a player's build (the Windows builder, the AppImage) compiles every lifted file at `-O1`. The data on #505 since: - p0lux, Steam Deck OLED (4 cores), as corrected in #issuecomment-49394: in real sessions at the slow spot of Death Mountain Trail (one session a side, not an A/B) a list holding the 248 functions hot there gave 27.0 fps against 22.3 without one. His first table (no list 21.9 fps, 319 functions 26.8, 773 functions 26.9) turned out to be the screen after Link dies under the falling rocks, a frame the game's code limits but not the trail, and "the first 300 functions do as well as the whole list" does not hold in play: that list had 31 of those 248 functions. - waryxxful, Ryzen 5 5600X: one list against another moved nothing measurable there (the frame was paced by hand-offs between threads), and no list was not measured. - Our runs on a Ryzen 9 5900X (comment #issuecomment-48225, corrected): no list against a fresh 680-function list, +25 % fps on Death Mountain Trail and +9 % in Kakariko. The list got about 87 % of the saving in game time a frame that building every chunk at `-O2` (#1431's switch) gets (84 to 86 % of its fps gain), for 25 s of build instead of 3 min 24 s (both incremental builds after a new lift). On the trail the main thread waits for the actor update about 70 % of the time, and while it waits, at most one of the six update workers runs about two thirds of the time (Havok physics and cloth): that serial tail is lifted code, which is what the list speeds up. Where the renderer sets the frame the list does less: on #505 an earlier list gave +3 % walking in the Market and nothing measurable standing there. See #505. ## What changed The list is committed as `assets/profiles/hot_funcs.txt`, and `make_game.py` gives it to the lift unless a developer's own list or `LONGSHOT_HOT_LIST=0` says otherwise; the AppImage already carries `assets/`, and the Windows builder's package now carries `assets/profiles/` too. - list: `assets/profiles/hot_funcs.txt`, 680 lines `f_<guest address>` (guest addresses only, no game code), with its note in `assets/profiles/README.md`: where and how it was recorded (game 1.1.1, Longshot's own lift of 544229a2, adult Link through Kakariko and up Death Mountain Trail played by hand on a build with no list, child Link standing in the Market and in Hyrule Field, areas weighted equally, cut at 85 %, by `scripts/hotlist.py`'s rule applied outside the repository), what it covers and what it measured. - build: `nx2/make_game.py` `hot_list_for()`: the checkout's `work/hot_funcs.txt` first, then a list `A64_HOT_FUNCS` names in the shell, then the committed one, copied to `<work>/hot_funcs.txt` when it differs (so the lift key hashes its contents as before and an unchanged list keeps the translation), or read where it is when `--work` is the checkout's own `work/` (a copy there would count as the checkout's own list from then on). `LONGSHOT_HOT_LIST=0` builds with none. A `<work>/hot_funcs.txt` left by an earlier build is removed when the build doesn't use it, and replaced when it differs from the committed one, each with a line in the log (p0lux's note on #505: before, after one build with a list every later build got that list). `hot_list_line()` says which list the build takes. `lift_key()`: no list (`A64_HOT_FUNCS` empty in the build's environment) hashes a marker, so it never takes the key of a lift with the list, also when the checkout's own list sits in its own work folder. - packaging: `scripts/builder_package.py` copies the tracked `assets/profiles/` into the Windows builder's `tools/`, where `make_game.py` looks for it (a separate commit: it also gives Windows players' builds the renderer's committed profile, #569, which the package never carried). `scripts/linux/appimage.sh` already copies `assets/` whole (comments only); `scripts/linux/check_player_build.sh` checks that the image carries a well-formed list. - docs: `docs/development.md` (a "The hot list" section, the `LONGSHOT_HOT_LIST` row, "A faster build for development", the build times' "No hot pieces" bullet), `docs/design.md` (a release no longer ships without one). - tests: `tests/test_shipped_hot_list.py` (new, 15 cases), `tests/test_appimage_player_build.py` (the image carries the list). ## Measurements **Verdict:** faster where the game's own code sets the frame rate: `main` against this PR, two interleaved runs each, Death Mountain Trail 51.8 -> 64.3 fps (+24 %, game time a frame 18.1 -> 14.7 ms) and Kakariko 55.4 -> 60.9 fps (+10 %, game time 17.0 -> 15.0 ms), for 64 s more build from scratch on 12 cores (6:58 -> 8:02), about 45 s of it the list's. Baseline: `main` @ 35a4a9fc against this PR @ 715b3b4e (its two commits on that `main`), each built from scratch into the same game folder (a work folder of its own: a new lift, no object cache; the game's files already extracted), their `longshot.exe` and `longshot_vk.dll` kept and swapped in before each run. No mods, and no `work/hot_funcs.txt` in either checkout. Hardware: Windows 11, NVIDIA RTX 3090 (driver 610.88), AMD Ryzen 9 5900X (12 cores, 24 threads), 128 GB Scene: adult Link from one save: Kakariko Village with three of its houses, then two legs up Death Mountain Trail, played by hand (about 3 minutes); full screen 3840x2160 (output 2048x1152), frame rate cap Uncapped Method: four runs in the order PR, main, PR, main, with the profiling mode on (`HOS_PROFILE=1`, the same cost in every run); the frames in play only: the renderer's own 120-present log windows with at least 500 draws and under 100 fps, the area from a timeline of the control bridge's `status`, the first window after arriving left out (so loads, menus and the quit menu at the end are out); frame times from `presents.csv`; game time a frame is the frame trace's `game_ms`, median of those frames under 50 ms; the verdict gives the median of each side's two runs; build times from each build's last line and its `build_times.json` How to reproduce: build `main` and this branch (or this branch with `LONGSHOT_HOT_LIST=0` and without it), play the same route from the same save, compare the in-play frames by area | | Before: main @ 35a4a9fc (runs M1 / M2) | After: this PR @ 715b3b4e (runs P1 / P2) | |---|---|---| | Death Mountain Trail fps | 51.3 / 52.2 | 64.0 / 64.6 | | trail median / p99 frame | 18.1 / 38.5 ms and 18.4 / 29.4 ms | 14.9 / 25.4 ms and 14.8 / 26.6 ms | | trail game time a frame | 17.9 / 18.2 ms | 14.7 / 14.6 ms | | trail frames over 33 / 50 ms | 38 / 6 and 4 / 0 (40 s, 41 s) | 6 / 0 and 10 / 1 (39 s, 43 s) | | Kakariko fps | 55.1 / 55.6 | 60.2 / 61.6 | | Kakariko median / p99 frame | 17.5 / 30.6 ms and 17.5 / 28.6 ms | 15.5 / 32.6 ms and 15.6 / 25.6 ms | | Kakariko game time a frame | 17.0 / 17.0 ms | 14.9 / 15.0 ms | | Kakariko frames over 33 / 50 ms | 41 / 20 and 19 / 3 (94 s, 95 s) | 43 / 3 and 14 / 1 (92 s, 95 s) | | longest frame at the save's load | 1,801 ms / 1,855 ms | 1,627 ms / none over 0.4 s | | build from scratch: whole (compile, lift) | 6:58 (5:30, 1:19) | 8:02 (6:08, 1:26) | ``` | area | run | fps | median ms | p95 ms | p99 ms | frames >= 33 ms | >= 50 ms | game_ms median | seconds | | DeathMountainTrail_Adult | 2026-11-18_133124_P1 | 64.0 | 14.9 | 20.6 | 25.4 | 6 | 0 | 14.7 | 39 | | DeathMountainTrail_Adult | 2026-11-18_133448_M1 | 51.3 | 18.1 | 28.4 | 38.5 | 38 | 6 | 17.9 | 40 | | DeathMountainTrail_Adult | 2026-11-18_133950_P2 | 64.6 | 14.8 | 21.1 | 26.6 | 10 | 1 | 14.6 | 43 | | DeathMountainTrail_Adult | 2026-11-18_134302_M2 | 52.2 | 18.4 | 25.4 | 29.4 | 4 | 0 | 18.2 | 41 | | KakarikoVillage_Adult | 2026-11-18_133124_P1 | 60.2 | 15.5 | 23.7 | 32.6 | 43 | 3 | 14.9 | 92 | | KakarikoVillage_Adult | 2026-11-18_133448_M1 | 55.1 | 17.5 | 23.7 | 30.6 | 41 | 20 | 17.0 | 94 | | KakarikoVillage_Adult | 2026-11-18_133950_P2 | 61.6 | 15.6 | 21.5 | 25.6 | 14 | 1 | 15.0 | 95 | | KakarikoVillage_Adult | 2026-11-18_134302_M2 | 55.6 | 17.5 | 23.6 | 28.6 | 19 | 3 | 17.0 | 95 | this PR: the hot list: the committed assets/profiles/hot_funcs.txt this PR: hot functions (A64_HOT_FUNCS): 680 of 680 listed found, in code_hot.c this PR: the build took 0:08:02, compiled in 0:06:08 main: the build took 0:06:58, compiled in 0:05:30 ``` - The two runs of a side agree to within 0.9 fps on the trail and 1.4 in Kakariko, and to within 0.3 ms of game time; the sides are 12.5 fps apart on the trail (3.4 ms of game time) and 5.5 in Kakariko (2.0 ms). - The build: the branch was built first, so it also paid the first build's one-time finish steps (the button prompts 14 s against 1 s, the shader list 11 s against 5 s). Of the 64 s between the two, the compile is +38 s (330 -> 368 s) and the lift +7 s (79 -> 86 s): about 45 s for the list on 12 cores. A third build of the branch from scratch, not the first in line (made for the memory figures in Risks, a sampler running beside it), took 7:50, 52 s more than `main`. All 680 listed functions are functions of the lift at 35a4a9fc. - The long frame at the save's load (the game's main thread blocked for 1.6 to 1.9 s) is in both runs of `main` and in one of this PR's two: it is not the list's. Kakariko's slow frames show no pattern between the sides (41 and 19 against 43 and 14 over 33 ms). - Earlier, on v3.0.0-rc.7 (test builds of 544229a2 carrying our unmerged mod pull requests with their switches off, one run a build, the same route; the list put in `work/hot_funcs.txt` by hand), game time a frame on the trail / in Kakariko: no list 20.8 / 18.2 ms; this list 16.7 / 16.2 ms; the 849-function list developers had 16.9 / 16.7 ms; no list and every chunk at `-O2` (#1431) 16.1 / 15.9 ms. So the list got about 87 % of what every chunk at `-O2` saved, for 25 s more build where all `-O2` cost 3 min 24 s (incremental builds, 6:17, 6:42 and 9:41), and the old list and this one came out level. - The list covers 85.0 % of Kakariko's samples of lifted code, 86.7 % of the trail's, 84.6 % of the Market's and 83.8 % of the field's; the 849-function list developers had (Hookshot's, from an unrecorded route) covers 68.5 to 77 %. A list made without one of the four areas still covers 79 to 84 % of it (the left-out area's neighbour stays in). On the route about 12 to 19 % of the samples charged to a lifted function have their innermost frame in host code under it: float helpers, which a listed function inlines, but also lookups, locks and sleeps, which no list speeds up. - Its 680 functions are 88.8 MB of the lift's 1,449 MB of C (an 11-12 lift); the build above cut them into eight hot pieces of 4 to 22 MB. The first 300 would be 52.4 MB. - By compile CPU (the rc.7 builds) the eight pieces took 1,452 s where the same functions take about 330 s at `-O1`: about +21 % of the lifted compile's CPU time, which a PC with 4 to 8 threads pays in wall time (p0lux's Steam Deck, `--jobs 4`: 1,283 s with 319 functions, 1,752 s with 773). ### Flags and settings One new build switch, `LONGSHOT_HOT_LIST`, off by default (the list is used). | Flag | What it is | Before this PR | After this PR | Effect | Helps / doesn't help / untested | Recommendation | |---|---|---|---|---|---|---| | `LONGSHOT_HOT_LIST=0` (build) | Builds with no hot list | Not read: no list unless `work/hot_funcs.txt` was there | Unset: the list (a developer's own first); `0`: none | `0`: every lifted function at `-O1`, as a player's build was before; the build about 45 s shorter from scratch on 12 cores; the game translated again when switched | Helps an A/B against the list; untested on a slow PC | Leave unset (the list on for everyone); `0` only to compare | ## Testing **Tested:** on Windows 11, RTX 3090 (driver 610.88), in the game: this branch at 715b3b4e built from scratch with no `work/hot_funcs.txt` in the checkout (the log: `the hot list: the committed assets/profiles/hot_funcs.txt`, `hot functions (A64_HOT_FUNCS): 680 of 680 listed found, in code_hot.c`, 0 failed compiles) and played twice over the route above (Kakariko Village with three houses, Death Mountain Trail), between two runs of `main` @ 35a4a9fc built the same way: no fault seen, the numbers are in Measurements. Game-free, on this branch: `tests/test_shipped_hot_list.py`, 15 cases, all passing: the committed list's lines (`f_<8 hex>`, none twice, at most 1,500) and its note; which list a build takes (the committed one, a checkout's own first, a shell's `A64_HOT_FUNCS` next, none with `LONGSHOT_HOT_LIST=0`, an earlier build's copy replaced or removed and the checkout's own never touched, and with `--work` in the checkout's own `work/` the committed list read in place, a later change of it reaching the build); the lift key (another list translates again, the same one does not, no list never shares a key with the list, also in the checkout's own work folder); the Windows package carries `assets/profiles/` and `make_game.py` run from its `tools/` finds the list there; and no listed function is one whose text edit in `nx2/oot_patches.py` finds its place by a float helper call (a hot function's are rewritten to their inline copies, so such an edit would be lost: the step divides' hook is one; none of the 680 is). With the key's marker, the in-place read, or a listed `f_80706be0`, their cases fail. The AppImage's copy block run in Git Bash puts the list in its `assets/profiles/`. `python -I -B tests/run_all.py --jobs 16` (`LONGSHOT_MINGW` set, `openssl` on `PATH` for `tests/test_release.py`) at 715b3b4e: 634 passed, 41 failed, 114 skipped; the 41 are the same checks that fail on `main` @ 35a4a9fc on this PC (633 passed, 41 failed, 114 skipped in 152 s), among them `tests/test_switch_table.py`, `tests/test_profiling_mode.py` and `tests/test_dev_packs.py`, which read files this PR changes and fail there with the same output; `tests/test_tested_configurations.py` with the four new run files: its one failure (a line-ending case of the command's output on Windows) is `main`'s. The list's names checked against lifts: all 680 are functions of the lift at 35a4a9fc (the build above) and of the earlier ones. **Not tested:** the build time and the memory gate on fewer cores and 8 to 16 GB; child Link in play, dungeons and fights; Linux; the Windows package as CI makes it (only `package()` in a test); the renderer's profile on a Windows player's build. **Must be tested:** - Linux, AMD (a Steam Deck): the AppImage of this branch, a player's build, expecting the same two log lines, its build time with `--jobs 4`, and the trail's fps against `main` (Test Farm: @Beanstalk linux steamdeck, a player's build of this PR's AppImage and the trail's fps against main) - Windows: the builder zip CI makes from this branch holds `LongshotBuilder/tools/assets/profiles/hot_funcs.txt` and `longshot_vk.profdata.xz`, expecting both there (Test Farm: @Beanstalk windows, list the builder zip's tools/assets/profiles) ## Risks **What could break:** builds take longer: from scratch on 12 cores 8:02 against 6:58, about 45 s of it the list's (measured), and more on fewer threads, since the eight pieces add about a fifth to the lifted compile's CPU time (on a Steam Deck with 4 jobs a 773-function list took 469 s longer than a 319-function one; this list has 680): that is the Steam Deck item under Must be tested. Memory, measured in a third build of this branch from scratch with every compiler's peak sampled: the eight `-O2` pieces peak at 0.4 to 1.5 GB each (417, 440, 518, 916, 1,063, 1,091, 1,287 and 1,468 MB of working set; 7.2 GB if all eight run at once, as they did here), no more than an ordinary chunk's `-O1` compile, which peaks at 1.5 to 1.6 GB; they hold it longer, since the largest piece compiles for 5 min 44 s and is the build's critical path. All compilers together peaked at 23.5 GB with 23 jobs; the build's memory gate starts compilers by free memory, and what that means for a PC with 8 to 16 GB is not measured. A listed function whose text edit in `nx2/oot_patches.py` is anchored on a float helper call would lose that edit with a WARNING line only (the step divides' hook, the game's speed above 60 fps): none of the 680 is, and `tests/test_shipped_hot_list.py` fails if a later list adds one. A pieces' play profile left in a work folder (`longshot_pieces.prof`) still applies to this list's pieces: a weaker fit, not a fault. On Windows the builder now carries the renderer's committed profile as well, so a Windows player's renderer is compiled with it for the first time (#569 measured -3 to -5 % frame time on Linux; a Windows checkout already builds with it); that is the second commit, and can be its own PR. **Open for review:** (1) `scripts/hotlist.py` always adds the game's Zstd decoder (22 functions; a whole decode is 16 % quicker at the hot flags); this list has 4 of them by their samples, and the other 18 are 0.51 MB of C: add them (698 functions)? (2) The Windows package's commit also brings the renderer's profile to Windows players; keep it here, or copy only `hot_funcs.txt` here and the rest in its own PR? (3) A shorter list for CPUs with few cores, or one list for every PC? p0lux has withdrawn "the first 300 do as well as the whole" (#issuecomment-49394: a 300 cut missed most of what was hot at his slow spot), and our data points the same way: coverage grows with size without a knee, a 300-function cut covers about 73 % of held-out runs' lifted samples against 84 % for 680 and 88 % for 1,000, for roughly 60 % of the 680's extra compile (modelled; the model has one calibration point and reads low for few cores). Two other rankings were proposed, by samples per byte of C and by time on the frame's critical path. Neither has been built or played, so this PR keeps the measured list. (4) Record it again over the pipeline tour's route (#111) for breadth, child Link and the dungeons included. (5) `scripts/hotlist.py` charging a split giant's pieces directly, and `--also` for several runs, are a separate small PR. (6) #1124 (Marcuss2, WIP) ranks a list by samples times the float helper calls a function holds (`--fp-calls`) and keeps lists out of the repository, regenerated per lift; this PR commits one, because a player's build has no profile to make one from, and a list of guest addresses stays valid while the lifted functions' boundaries do (worth a check at each release that every listed name is still a function of the lift; the build log's "N of M listed found" line shows it too). The two fit together: the committed list could be re-made with #1124's ranking once it lands. His build numbers agree with ours: with the hot pieces split (#1146), a 47-function list built in 3:09 against 3:18 with none, and a 633-function list (88 MB of hot C) in 3:51 from scratch on a 32-thread Linux PC, with -6.7 % CPU in Kokiri Forest standing (a scene the game's code does not limit, so +0.5 % fps there). (7) #1593 (HumanBean, WIP) pulls the same lever another way: `LONGSHOT_CHUNK_OPT=executor` compiles eight whole chunk files at `-O2` (Market +2.5 to +3.4 % on a Ryzen 9 7900X), where this PR moves 680 functions into the hot pieces. The two have not been measured in one scene: one build of each on one route would compare them, and they can also be combined (the hot pieces keep `RX_HOT_OPT`, the preset takes the chunks). One thing to check on the preset, read from `nx2/chunks.py` and not tried: a chunk file's boundaries are kept per work folder (`image/chunks.txt`, made from the functions' sizes at that folder's first lift), so `code_017.c` need not hold the same functions on every install (two of our own work folders, first lifted at different versions, cut their chunks differently). A list of `f_` names (which `LONGSHOT_CHUNK_OPT=<file>` also takes) names the same functions everywhere. **Turning it off:** `LONGSHOT_HOT_LIST=0` for a build (the launcher has no setting for it), or revert the commits. **Compatibility:** the first build after updating translates the game again (the lift key holds the list), and since the listed functions move out of their chunk files into `code_hot.c` (81 of the 92 chunk files hold one), most lifted files compile again: close to a first build's compile. On Windows the renderer also compiles once more, with its profile (second commit). Saves, `longshot_options.ini`, the shader and pipeline caches and the vk ABI are unchanged. A developer's own `work/hot_funcs.txt` keeps working as before; a work folder's copy of an earlier list is no longer used when the checkout has none. ## Issues See #505 ## Checklist ### Everyone - [x] One topic. - [x] The title is a Conventional Commit (`type(scope): description`) of the status line's type. - [x] Why gives the reason plainly and links the issues that fit (the tracker searched). - [x] Tested on my own PC where it runs (in the game, for a change the game runs), as Testing's Tested says; what can't be run by hand is in Testing's Not testable by hand, with why. - [x] Measurements has a verdict, or one line saying why there is nothing to measure. - [x] Nothing gets worse that I know of: any cost (speed, latency, memory, a setup that worked before) is measured and in Risks. - [x] Testing names what was tested, what wasn't, and what must still be tested, where and how. - [x] Rebased on current `main`. - [x] `python -I -B tests/run_all.py` passes (any failure that also fails on `main` is named). - [x] No game files, dumps, build output or generated files. - [x] `third_party/` untouched (a new folder with a `PROVENANCE.md` instead). - [x] The license is OK: GPL-3.0-or-later, nothing incompatible and nothing decompiled. ### When it applies - [x] AI assistance recorded with `Assisted-by:` / `Co-Authored-By:` trailers. - [x] Numbers from the same PC, scene and settings, at least 2 interleaved runs each, the median reported. - [x] Switches documented (docs/development.md; the README for a menu row) and shipped at their recommended value. - [x] A file in `docs/tested/` for each new measured run (format in `docs/tested-configurations.md`). - [x] A game-free test added for the fix, failing on `main` and passing here. - [x] A check under `tests/` that reads changed source updated in this PR, not weakened. - [x] Docs updated. Credits: the measurements on #505 by p0lux (a Steam Deck: no list against a list in play, the build times, and the stale work-folder list) and waryxxful (a 5600X: list against list), and the proposal itself by HumanBean. 🤖 Generated with [Claude Code](https://claude.com/claude-code)
A player's build compiled every lifted function at -O1: make_game.py used a hot list only when the checkout's work/
had one, and releases ship none (issue 505). The list is committed now, assets/profiles/hot_funcs.txt: 680 functions
recorded with Longshot's own lift of 544229a2 (adult Link through Kakariko and up Death Mountain Trail, played by hand
on a build with no list; child Link standing in the Market and in Hyrule Field; areas weighted equally, cut at 85 %;
guest addresses only), its note in assets/profiles/README.md. The same list in a build of 544229a2 gave +25 % fps on
Death Mountain Trail and +9 % in Kakariko against no list (one run each, the frames in play), for 25 s more build on a 12-core PC in an
incremental pair (about 55 s from scratch, estimated).

- nx2/make_game.py hot_list_for: the checkout's work/hot_funcs.txt first, then a list A64_HOT_FUNCS names in the shell,
  then the committed one, copied to <work>/hot_funcs.txt when it differs (the lift key hashes it there, as before), or
  read where it is when --work is the checkout's own work/ (a copy there would count as the checkout's own list from
  then on, and a later change of the committed one would never reach it). LONGSHOT_HOT_LIST=0 builds with none. A work
  folder's copy from an earlier build is removed when the build does not use it (before, after one build with a list
  every later build got it: p0lux on #505). hot_list_line: the log's line. lift_key: no list hashes a marker, so it
  never takes the key of a lift with the list.
- scripts/linux/appimage.sh carries assets/ already (comments); check_player_build.sh checks the image's list.
- docs/development.md: "The hot list", the LONGSHOT_HOT_LIST row, the faster build and build times notes;
  docs/design.md.
- tests/test_shipped_hot_list.py (new): the list's lines and note; no listed function whose nx2/oot_patches.py edit
  is anchored on a float helper call (hot_text rewrites those calls); which list a build takes, the checkout's own
  work folder included; the lift key. tests/test_appimage_player_build.py: the image carries the list.

The Windows builder's package carries assets/profiles/ in the next commit.

See #505

Co-Authored-By: Claude Opus 5.5 <[email protected]>
build(windows): the builder's package carries assets/profiles/, the hot list and the renderer's profile
Some checks failed
PR description / Description follows the template (pull_request_target) Successful in 8s
Website / Website on Cloudflare (pull_request_target) Has been skipped
CI / Conventional Commit title (pull_request) Has been cancelled
CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Has been cancelled
CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Has been cancelled
CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Has been cancelled
CI / Workflow lint (actionlint) (pull_request) Has been cancelled
Guide screenshots / Guide images and local UI source review (pull_request) Has been cancelled
Third-party parts / As their notes say (pull_request) Has been cancelled
Third-party parts / The same as upstream (pull_request) Has been cancelled
Website / Website on Cloudflare (pull_request) Has been cancelled
715b3b4eec
scripts/builder_package.py copied the tracked gpu, host, nx2, scripts and third_party and the pipeline state list's
sources into the Windows builder's tools/, but not assets/profiles/, where nx2/make_game.py looks for the committed play
profiles (hot_list_for, vk_profile_for). So a Windows player's build would have had no hot list, and it has never had the
renderer's committed profile (#569: about -3 to -5 % frame time measured on Linux; a Windows checkout already builds the
renderer with it). The package now copies the tracked assets/profiles/ too. The AppImage carries assets/ whole already.

tests/test_shipped_hot_list.py: package() over a small repository puts assets/profiles/ under tools/, and make_game.py
run from that tools folder takes the list there.

See #505

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Author
Contributor

Tested in game: Windows 11, NVIDIA RTX 3090, Ryzen 9 5900X: the list this PR commits ran in the game as work/hot_funcs.txt in a test build of 544229a2 (v3.0.0-rc.7): adult Link through Kakariko Village, three of its houses and two legs up Death Mountain Trail, about 3 minutes, no fault seen; against the same source built with no list, the trail went 45.3 -> 56.4 fps and Kakariko 51.3 -> 55.7 (one run a side, the table is in the description). This branch's own build path (make_game.py taking the committed list, the Windows package) has not been built or played yet: that is the first item under Must be tested, and why this is WIP.

Tested in game: Windows 11, NVIDIA RTX 3090, Ryzen 9 5900X: the list this PR commits ran in the game as `work/hot_funcs.txt` in a test build of 544229a2 (v3.0.0-rc.7): adult Link through Kakariko Village, three of its houses and two legs up Death Mountain Trail, about 3 minutes, no fault seen; against the same source built with no list, the trail went 45.3 -> 56.4 fps and Kakariko 51.3 -> 55.7 (one run a side, the table is in the description). This branch's own build path (make_game.py taking the committed list, the Windows package) has not been built or played yet: that is the first item under Must be tested, and why this is WIP.
docs(tested): the hot list's route runs, main against the list, two each
Some checks are pending
CI / Conventional Commit title (pull_request) Blocked by required conditions
CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Blocked by required conditions
CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Blocked by required conditions
CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Blocked by required conditions
CI / Workflow lint (actionlint) (pull_request) Blocked by required conditions
Guide screenshots / Guide images and local UI source review (pull_request) Blocked by required conditions
Third-party parts / As their notes say (pull_request) Blocked by required conditions
Third-party parts / The same as upstream (pull_request) Blocked by required conditions
Website / Website on Cloudflare (pull_request) Blocked by required conditions
beanstalk/approval waiting for a BigBean's 👍 on the request
PR description / Description follows the template (pull_request_target) Successful in 35s
c4f53a3cc2
Four runs on one PC (Ryzen 9 5900X, RTX 3090, Windows 11), in the order list, main, list, main: Death Mountain Trail
51.3 / 52.2 fps on main 35a4a9fc and 64.0 / 64.6 with the list; Kakariko 55.1 / 55.6 and 60.2 / 61.6.

Co-Authored-By: Claude Fable 5.2 <[email protected]>
Author
Contributor

Built and played: main @ 35a4a9fc against this branch @ 715b3b4e, each from scratch (a new work folder), then four route runs in the order list, main, list, main (Ryzen 9 5900X, RTX 3090, Windows 11; Kakariko Village with three houses, then Death Mountain Trail).

main (M1 / M2) this PR (P1 / P2)
Death Mountain Trail fps 51.3 / 52.2 64.0 / 64.6
trail game time a frame 17.9 / 18.2 ms 14.7 / 14.6 ms
Kakariko fps 55.1 / 55.6 60.2 / 61.6
Kakariko game time a frame 17.0 / 17.0 ms 14.9 / 15.0 ms
build from scratch (compile, lift) 6:58 (5:30, 1:19) 8:02 (6:08, 1:26)
  • The branch took the committed list by itself: the hot list: the committed assets/profiles/hot_funcs.txt, 680 of 680 listed found, 0 failed compiles.
  • Of the 64 s between the two builds about 45 s are the list's (compile +38 s, lift +7 s); the rest is the first build's one-time finish steps, which the branch paid by going first.
  • The long frame at the save's load that the earlier builds with a list showed is on main too (1.8 and 1.9 s in its two runs, 1.6 s and none in this PR's), so it is not the list's.

The description has the full table and the raw lines; the head is now c4f53a3c, which adds the four run files in docs/tested/ and changes no code. Still open: the memory of the eight -O2 compiles (the one unticked box; being measured), and a Linux and a Steam Deck run.

Built and played: `main` @ 35a4a9fc against this branch @ 715b3b4e, each from scratch (a new work folder), then four route runs in the order list, main, list, main (Ryzen 9 5900X, RTX 3090, Windows 11; Kakariko Village with three houses, then Death Mountain Trail). | | main (M1 / M2) | this PR (P1 / P2) | |---|---|---| | Death Mountain Trail fps | 51.3 / 52.2 | 64.0 / 64.6 | | trail game time a frame | 17.9 / 18.2 ms | 14.7 / 14.6 ms | | Kakariko fps | 55.1 / 55.6 | 60.2 / 61.6 | | Kakariko game time a frame | 17.0 / 17.0 ms | 14.9 / 15.0 ms | | build from scratch (compile, lift) | 6:58 (5:30, 1:19) | 8:02 (6:08, 1:26) | - The branch took the committed list by itself: `the hot list: the committed assets/profiles/hot_funcs.txt`, `680 of 680 listed found`, 0 failed compiles. - Of the 64 s between the two builds about 45 s are the list's (compile +38 s, lift +7 s); the rest is the first build's one-time finish steps, which the branch paid by going first. - The long frame at the save's load that the earlier builds with a list showed is on `main` too (1.8 and 1.9 s in its two runs, 1.6 s and none in this PR's), so it is not the list's. The description has the full table and the raw lines; the head is now c4f53a3c, which adds the four run files in `docs/tested/` and changes no code. Still open: the memory of the eight `-O2` compiles (the one unticked box; being measured), and a Linux and a Steam Deck run.
HumanBean changed title from WIP: perf(build): ship a hot list, so every build compiles the game's hottest functions at -O2 to perf(build): ship a hot list, so every build compiles the game's hottest functions at -O2 2026-11-18 19:52:24 +00:00
Author
Contributor

The last open box, the memory of the eight -O2 compiles, is measured: a third build of this branch from scratch with every compiler's peak memory sampled about once a second (Ryzen 9 5900X, 23 jobs).

  • The eight hot pieces peak at 417, 440, 518, 916, 1,063, 1,091, 1,287 and 1,468 MB of working set (7.2 GB if all eight run at once, as they did here).
  • An ordinary chunk's -O1 compile peaks at 1.5 to 1.6 GB, so a hot piece needs no more memory than a chunk does; it holds it longer.
  • The largest piece compiles for 5 min 44 s and is the build's critical path (the compile took 6:18): cutting the pieces smaller is where the list's build cost could still come down.
  • All compilers together peaked at 23.5 GB. What the memory gate makes of that on a PC with 8 to 16 GB is not measured here.
  • That build took 7:50, 52 s more than main's 6:58, which fits the 45 s the first pair gave.

The description has it under Risks, and every Everyone box is ticked now. Still wanted from others: a Linux and a Steam Deck run (the request follows).

The last open box, the memory of the eight `-O2` compiles, is measured: a third build of this branch from scratch with every compiler's peak memory sampled about once a second (Ryzen 9 5900X, 23 jobs). - The eight hot pieces peak at 417, 440, 518, 916, 1,063, 1,091, 1,287 and 1,468 MB of working set (7.2 GB if all eight run at once, as they did here). - An ordinary chunk's `-O1` compile peaks at 1.5 to 1.6 GB, so a hot piece needs no more memory than a chunk does; it holds it longer. - The largest piece compiles for 5 min 44 s and is the build's critical path (the compile took 6:18): cutting the pieces smaller is where the list's build cost could still come down. - All compilers together peaked at 23.5 GB. What the memory gate makes of that on a PC with 8 to 16 GB is not measured here. - That build took 7:50, 52 s more than `main`'s 6:58, which fits the 45 s the first pair gave. The description has it under Risks, and every Everyone box is ticked now. Still wanted from others: a Linux and a Steam Deck run (the request follows).
Author
Contributor

@Beanstalk steamdeck and linux amd: build main and this PR as a player's install does (on the Deck with --jobs 4) and report each build's whole time, and for this PR the two log lines the hot list: the committed assets/profiles/hot_funcs.txt and hot functions (A64_HOT_FUNCS): 680 of 680 listed found. Then on each build, as adult Link on Death Mountain Trail with HOS_OOT_PROTECT_HEARTS=1 (it keeps Link alive under the falling rocks), walk the lower trail for a minute; and the same in Kakariko Village. Expecting no crash, the same picture as main, and with this PR a higher frame rate and less game time a frame than main where the game's code limits the frame (the trail), with nothing lost where it doesn't.

If a Windows tester is free as well: the builder zip CI makes from this branch should hold tools/assets/profiles/hot_funcs.txt and longshot_vk.profdata.xz.

@Beanstalk steamdeck and linux amd: build `main` and this PR as a player's install does (on the Deck with `--jobs 4`) and report each build's whole time, and for this PR the two log lines `the hot list: the committed assets/profiles/hot_funcs.txt` and `hot functions (A64_HOT_FUNCS): 680 of 680 listed found`. Then on each build, as adult Link on Death Mountain Trail with `HOS_OOT_PROTECT_HEARTS=1` (it keeps Link alive under the falling rocks), walk the lower trail for a minute; and the same in Kakariko Village. Expecting no crash, the same picture as `main`, and with this PR a higher frame rate and less game time a frame than `main` where the game's code limits the frame (the trail), with nothing lost where it doesn't. If a Windows tester is free as well: the builder zip CI makes from this branch should hold `tools/assets/profiles/hot_funcs.txt` and `longshot_vk.profdata.xz`.
Member

@snaildev99 only members of BigBeans can start the farm. One of them can approve this request by reacting 👍 to it or to this comment.

@snaildev99 only members of `BigBeans` can start the farm. One of them can approve this request by reacting 👍 to it or to this comment.
HumanBean deleted branch perf/ship-hot-list 2026-11-18 20:03:33 +00:00
Some checks are pending
CI / Conventional Commit title (pull_request) Blocked by required conditions
CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Blocked by required conditions
CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Blocked by required conditions
CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Blocked by required conditions
CI / Workflow lint (actionlint) (pull_request) Blocked by required conditions
Guide screenshots / Guide images and local UI source review (pull_request) Blocked by required conditions
Third-party parts / As their notes say (pull_request) Blocked by required conditions
Third-party parts / The same as upstream (pull_request) Blocked by required conditions
Website / Website on Cloudflare (pull_request) Blocked by required conditions
beanstalk/approval waiting for a BigBean's 👍 on the request
PR description / Description follows the template (pull_request_target) Successful in 35s
Some workflows are waiting to be reviewed.
Sign in to join this conversation.
No reviewers
No milestone
No project
No assignees
2 participants
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
MagicBeanDev/Longshot!1635
No description provided.