perf(build): giant functions with guest locals compile in pieces again, the pieces jumping to each other (LONGSHOT_SPLIT_DIRECT) #1727
No reviewers
Labels
No labels
area/accessibility
area/audio
area/input
area/install
area/menu
area/renderer
area/saves
area/tooling
area/upscaling
beanstalk: approved
beanstalk: failed
beanstalk: needs approval
beanstalk: running
beanstalk: tested
ci: approved
good first issue
gpu/amd
gpu/igpu
gpu/intel
gpu/nvidia
help wanted
kind
bug
kind
chore
kind
docs
kind
feature
kind
performance
kind
question
needs measurements
platform/linux
platform/macos
platform/steam-deck
platform/windows
preview
priority
critical
priority
high
priority
low
regression
risk
high
risk
low
risk
medium
status
blocked
status
confirmed
status
duplicate
status
invalid
status
needs-info
status
wontfix
ci: approved
No milestone
No project
No assignees
2 participants
Notifications
Due date
No due date set.
Dependencies
No dependencies set
Reference
MagicBeanDev/Longshot!1727
Loading…
Reference in a new issue
No description provided.
Delete branch "dyst/Longshot:perf/split-giants-direct"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Type: perf · Risk: medium · Tested on: Windows 11, RTX 3080, Ryzen 9 3900X · Needs: Linux (clang), Steam Deck build time
TL;DR
Since guest registers became local variables (#1576), the game's ~180 biggest functions were no longer cut into pieces for the compiler, so building the game took about twice as long: one file alone took 23 minutes. This PR cuts them again, carrying the locals across each cut, and lets the pieces jump straight to each other. A cold build's game-code compile went from 1,385 s to 728 s (30 to 15.5 minutes for the whole build), and in the game Lake Hylia and the Market run as fast as on main (4 interleaved runs a side).
Breakdown
preserve_noneandmusttail, the pieces jump to each other with the locals in registers instead of going back through a dispatch loop.preserve_noneuses the loop; the build time there is unmeasured).Why
nx2/split_giants.py(#385, #1146) cuts a lifted function of 256 KB of C or more into pieces at its labels, because clang's time on one function grows much faster than its size. Since #1576 turned on guest locals (a function's most used registers and its flags as C locals), every such function starts with their declarations, and the splitter left any function with a top-level declaration whole. On main 2f1c0c9c's cold base that was 181 functions, 92 MB of C:code_hot_7.c(two functions of 1.9 and 1.7 MB, at -O2) took 1,385 s, the whole compile stage, andcode_hot_6.c992 s, while everything else had finished by about 977 s.See #362 (a first build in under 10 minutes), #1576.
What changed
The splitter now takes the locals the kit declares as a function's first lines and carries them across its cuts. Where the compiler can, a goto into another piece becomes a tail call straight to it with the locals as arguments, so a hop costs a jump and a switch rather than a return, a dispatch and a call.
nx2/split_giants.py):locals_of()reads the declarations (uint64_t x8_ = c->x[8], ...;thenuint8_t fn_ = c->n, ...;), only as the body's first lines; a locals line anywhere else (after lift_fixes' entry goto, #1587) still leaves the function whole.split_function():f_<addr>holds the locals in a struct of its own (s_<addr>_l), loaded from the context as the whole function loads them; the context is written exactly where the whole function writes it. (Writing the locals back to the context at each cut would also be correct, but writes dirty scratch registers the whole function never stores.)macros(): each function's pieces are written with five macros (CC,ARGS,LOAD,HOP,CALL), defined twice under#if __has_attribute(preserve_none) && __has_attribute(musttail): direct (musttail return s_<addr>_<m>(c, id, regs..., fl_), the four flags packed into one argument so everything fits preserve_none's 12 argument registers on x86-64) or the loop (the pieces load from and store to the struct, and return the label id tof_'s loop). A function whose locals would not fit in registers uses the loop.flag()/compile_copy():LONGSHOT_SPLIT_DIRECT=0ends the flag with_tand writes the loop alone; the mode is in the object keys.nx2/build.py):split_giants_flags()readsLONGSHOT_SPLIT_DIRECT;largest_function()(the memory estimate) finds the new piece heads byPIECE_HEAD.tests/test_split_giants.py): every compiled program runs in both modes; functions emitted by the real lifter with registers and flags as locals (each alone too) are split and run as the whole function does from 300 states, setjmp resumes included; an assembly check that direct pieces only jump to each other (16 jumps, 0 calls) and loop pieces never name each other; a program that never ends now fails the check instead of raising.docs/development.md'sLONGSHOT_SPLIT_DIRECTrow; eight run files indocs/tested/.On a real lift (main 2f1c0c9c's translation) the splitter now cuts 340 functions instead of 160 (190 MB of C into 14,333 pieces), 180 of them with locals; two are left whole as too small to cut.
Measurements
Verdict: the game-code compile takes 47% less time (1,385 s to 728 s; the whole build 30:18 to 15:31), and the game runs as fast as main: Lake Hylia +1.6%, the Market +1.1%, both within the spread between runs.
Hardware: Windows 11 Pro N (build 22631), RTX 3080 10 GB (NVIDIA 621.14), Ryzen 9 3900X (12 cores), 64 GB
Scene: the repository's
kokiri-startsave, child Link teleported by the control bridge to Lake Hylia (destination 7) and then the Market's gate, standing still; a fresh user folder (default settings, Auto: output 2560x1440, a 1720x720 window, mailbox V-Sync Auto),HOS_FPS_MAX=1000, frame generation off. CPU-bound here (GPU-bound wait 0.1 ms).Method: build time from the build's own summary (23 compilers at once). In the game: the bridge's fps once a second for 40 s after 20 s settled at each place; one warm-up run of each side, then main, PR, PR, main twice (4 runs a side), each a timing job of the PC's queue; the median of each run, and the median of the 4 runs.
How to reproduce:
dev_build.py base,packof this branch,rt --outfor each side; each run starts the game withlongshot_ctl.py launch --repo-save kokiri-start --user <fresh folder> --env HOS_FPS_MAX=1000, presses A until in play, thenteleport 7andteleport Market_Child, samplingstatusonce a second.Build time
2f1c0c9c(cold base, nothing in the object store)code_hot_7.ccode_hot_6.ccode_hot_7.c, 1,385 scode_hot_2.c, 483 sBaseline: the before column is main 2f1c0c9c's own cold base (the last build that compiled every file from nothing); the commits between it and
47430e81do not touch the lifted code or the splitter. Both compiled all 75 lifted files with the same Builder (16ff530a99b1).In the game (median fps of each run; the median of the four)
47430e81The PR's third Lake Hylia run fell to 33-42 fps for about 18 s in the middle of its 40 s, then came back; its log shows nothing the other runs lack (the same core-wait spikes appear in every run, main's too), and the other three are as high as main's or higher. Measured at 99a24b31 on main
47430e81; rebased since onto868f6be8(7921c792, the same change): the ten commits between touch the renderer, scripts and build files, not the splitter, so the comparison stands.Flags and settings
One new build switch;
LONGSHOT_SPLIT_GIANTSkeeps its default and now covers functions with guest locals.LONGSHOT_SPLIT_DIRECT0keeps the loop0: every cut goes back throughf_'s loop, the locals through a struct0for an A/B or a compiler problemLONGSHOT_SPLIT_GIANTSTesting
Tested:
tests/test_split_giants.py: both modes, written shapes and the real lifter's output with and without locals, against the whole function from 300 states; the assembly check; a mutation (the store-back before a hop removed) makes it fail.tests/test_opt_level_exactness.pyrun against this branch: the hot pieces split at -O2 compute the same as whole (0 of 300 states differ).python -I -B tests/run_all.pywith the Builder's llvm-mingw, on the rebased branch: 681 passed, 45 failed, 125 skipped. Every failure fails on main too (the 43 in the team's main baseline, plustest_game_menu_mouse_buttons.pyandtest_linux_preflight.py, which fail on main2f1c0c9cthe same way);test_hot_pieces_split.py,test_light_passes.py,test_stale_hot_pieces.pyandtest_doc_anchors.pypass.Not tested:
preserve_noneand builds the loop (the checks compile both modes, but only with clang 23 on Windows); the Builder's clang on Linux is unmeasured.LONGSHOT_SPLIT_DIRECT=0in the game (the game-free checks only).Must be tested:
compiled intime below main's (Test Farm: @Beanstalk linux, build from scratch and play Kokiri Forest to Hyrule Field).Risks
What could break: the game's largest functions are compiled differently: a wrong cut would be wrong game behaviour in those functions (the checks compare the split code with the whole function in both modes, and the in-game runs showed none). A compiler that handles
musttailacrosspreserve_nonebadly would show as a build error or a crash in those functions.Turning it off:
LONGSHOT_SPLIT_DIRECT=0(the loop) at build time; reverting the commit leaves the functions with locals whole again.Compatibility: the first build after updating recompiles every lifted chunk once (the split flag is in the object keys; about 12 minutes on a Ryzen 9 3900X); the translation is reused (the splitter is outside the lift key). Saves, settings, shader and pipeline caches and the vk ABI: nothing changes.
Checklist
Everyone
type(scope): description) of the status line's type.can't be run by hand is in Testing's Not testable by hand, with why.
in Risks.
main.python -I -B tests/run_all.pypasses (any failure that also fails onmainis named).third_party/untouched (a new folder with aPROVENANCE.mdinstead).When it applies
Assisted-by:/Co-Authored-By:trailers.docs/tested/for each new measured run (format indocs/tested-configurations.md).tests/that reads changed source updated in this PR, not weakened.🤖 Generated with Claude Code
Tested in game: Windows 11 (22631), RTX 3080 (NVIDIA 621.14), Ryzen 9 3900X: Lake Hylia and the Market from the kokiri-start save, 9 runs of this branch (a warm-up and 4 measured) interleaved with 5 of main on one base; no crash, both places drawn as on main, fps level (Lake 46.8 vs 46.0, Market 45.5 vs 45.0)
Why the numbers still hold on main
868f6be8: they were measured at 99a24b31 on main47430e81(the in-game A/B, both sides from that one base) and against main 2f1c0c9c's cold base (the build time's before column). The branch was then rebased onto868f6be8as7921c792, the same change with no conflicts. The ten main commits in between change the renderer (Intel, RDNA 4, compute lanes), host profiling, scripts and nx2's translator file list and version; none touches nx2/split_giants.py or the C the lifter writes for a function, which is what this PR changes how it compiles. The full suite was run again on the rebased branch: 681 passed, 45 failed, every failure also failing on main.WIP: perf(build): giant functions with guest locals compile in pieces again, the pieces jumping to each other (LONGSHOT_SPLIT_DIRECT)to perf(build): giant functions with guest locals compile in pieces again, the pieces jumping to each other (LONGSHOT_SPLIT_DIRECT)A correction to the comment above: one of those commits, #1686, does change the lifter (nx2/insn.py, SCVTF / UCVTF in their scalar fixed-point form). By its own notes the game's code does not use that form, so the C written for the game's functions, and with it what this PR compiles, is the same; the numbers hold for that reason, not because no lifter file changed.
e1ab9461178d4b5cbf91New commits pushed, approval review dismissed automatically according to repository settings
Rebased onto main
7d5fcb28(45 commits on), now8d4b5cbf. One conflict: docs/development.md is gone since #1802, so the LONGSHOT_SPLIT_DIRECT row moved to docs/dev/reference/build.md beside LONGSHOT_SPLIT_GIANTS's. nx2/build.py and nx2/split_giants.py merged without a conflict; tests/test_split_giants.py passes at the new head (both modes, 0 of 300 states differ). Not rebuilt or measured again at this head: the A/B was of the pieces' code, which the 45 commits do not touch (#1782's float paths change the lifted text the pieces are cut from, not how they are cut or called).