perf(build): giant functions with guest locals compile in pieces again, the pieces jumping to each other (LONGSHOT_SPLIT_DIRECT) #1727

Merged
HumanBean merged 2 commits from dyst/Longshot:perf/split-giants-direct into main 2026-11-20 17:14:49 +00:00
Member

Type: perf · Risk: medium · Tested on: Windows 11, RTX 3080, Ryzen 9 3900X · Needs: Linux (clang), Steam Deck build time

TL;DR

Since guest registers became local variables (#1576), the game's ~180 biggest functions were no longer cut into pieces for the compiler, so building the game took about twice as long: one file alone took 23 minutes. This PR cuts them again, carrying the locals across each cut, and lets the pieces jump straight to each other. A cold build's game-code compile went from 1,385 s to 728 s (30 to 15.5 minutes for the whole build), and in the game Lake Hylia and the Market run as fast as on main (4 interleaved runs a side).

Breakdown

  • Who it helps: everyone who builds the game, and every developer who builds a base or a pack: the first build, and every build after an update that changes the lifted code.
  • What was wrong: the splitter refused any function with a top-level declaration, and guest locals add one, so every giant function with locals compiled whole at -O2.
  • What changes: those functions compile as ~8 KB pieces again; where clang has preserve_none and musttail, the pieces jump to each other with the locals in registers instead of going back through a dispatch loop.
  • What does not change: what the game computes (checked against the whole function from 300 random states in both modes), the translation (the splitter is outside the lift key), saves, settings and the renderer.
  • How it was checked: the game-free checks in both modes, #1142's hot-pieces exactness check run against this branch, the full suite, a pack's build time, and an in-game A/B at Lake Hylia and the Market: level with main.
  • What is still open: Linux and the Steam Deck (a system clang without preserve_none uses the loop; the build time there is unmeasured).
  • Risk: medium: the change touches how the game's largest, hottest functions are compiled; a mistake would be wrong game behaviour, which the exactness checks and the in-game runs did not show.

Why

nx2/split_giants.py (#385, #1146) cuts a lifted function of 256 KB of C or more into pieces at its labels, because clang's time on one function grows much faster than its size. Since #1576 turned on guest locals (a function's most used registers and its flags as C locals), every such function starts with their declarations, and the splitter left any function with a top-level declaration whole. On main 2f1c0c9c's cold base that was 181 functions, 92 MB of C: code_hot_7.c (two functions of 1.9 and 1.7 MB, at -O2) took 1,385 s, the whole compile stage, and code_hot_6.c 992 s, while everything else had finished by about 977 s.

See #362 (a first build in under 10 minutes), #1576.

What changed

The splitter now takes the locals the kit declares as a function's first lines and carries them across its cuts. Where the compiler can, a goto into another piece becomes a tail call straight to it with the locals as arguments, so a hop costs a jump and a switch rather than a return, a dispatch and a call.

  • splitter (nx2/split_giants.py):
    • locals_of() reads the declarations (uint64_t x8_ = c->x[8], ...; then uint8_t fn_ = c->n, ...;), only as the body's first lines; a locals line anywhere else (after lift_fixes' entry goto, #1587) still leaves the function whole.
    • split_function(): f_<addr> holds the locals in a struct of its own (s_<addr>_l), loaded from the context as the whole function loads them; the context is written exactly where the whole function writes it. (Writing the locals back to the context at each cut would also be correct, but writes dirty scratch registers the whole function never stores.)
    • macros(): each function's pieces are written with five macros (CC, ARGS, LOAD, HOP, CALL), defined twice under #if __has_attribute(preserve_none) && __has_attribute(musttail): direct (musttail return s_<addr>_<m>(c, id, regs..., fl_), the four flags packed into one argument so everything fits preserve_none's 12 argument registers on x86-64) or the loop (the pieces load from and store to the struct, and return the label id to f_'s loop). A function whose locals would not fit in registers uses the loop.
    • flag() / compile_copy(): LONGSHOT_SPLIT_DIRECT=0 ends the flag with _t and writes the loop alone; the mode is in the object keys.
  • build (nx2/build.py): split_giants_flags() reads LONGSHOT_SPLIT_DIRECT; largest_function() (the memory estimate) finds the new piece heads by PIECE_HEAD.
  • tests (tests/test_split_giants.py): every compiled program runs in both modes; functions emitted by the real lifter with registers and flags as locals (each alone too) are split and run as the whole function does from 300 states, setjmp resumes included; an assembly check that direct pieces only jump to each other (16 jumps, 0 calls) and loop pieces never name each other; a program that never ends now fails the check instead of raising.
  • docs: docs/development.md's LONGSHOT_SPLIT_DIRECT row; eight run files in docs/tested/.

On a real lift (main 2f1c0c9c's translation) the splitter now cuts 340 functions instead of 160 (190 MB of C into 14,333 pieces), 180 of them with locals; two are left whole as too small to cut.

Measurements

Verdict: the game-code compile takes 47% less time (1,385 s to 728 s; the whole build 30:18 to 15:31), and the game runs as fast as main: Lake Hylia +1.6%, the Market +1.1%, both within the spread between runs.

Hardware: Windows 11 Pro N (build 22631), RTX 3080 10 GB (NVIDIA 621.14), Ryzen 9 3900X (12 cores), 64 GB
Scene: the repository's kokiri-start save, child Link teleported by the control bridge to Lake Hylia (destination 7) and then the Market's gate, standing still; a fresh user folder (default settings, Auto: output 2560x1440, a 1720x720 window, mailbox V-Sync Auto), HOS_FPS_MAX=1000, frame generation off. CPU-bound here (GPU-bound wait 0.1 ms).
Method: build time from the build's own summary (23 compilers at once). In the game: the bridge's fps once a second for 40 s after 20 s settled at each place; one warm-up run of each side, then main, PR, PR, main twice (4 runs a side), each a timing job of the PC's queue; the median of each run, and the median of the 4 runs.
How to reproduce: dev_build.py base, pack of this branch, rt --out for each side; each run starts the game with longshot_ctl.py launch --repo-save kokiri-start --user <fresh folder> --env HOS_FPS_MAX=1000, presses A until in play, then teleport 7 and teleport Market_Child, sampling status once a second.

Build time

Before: main @ 2f1c0c9c (cold base, nothing in the object store) After: this PR @ 99a24b31 (pack over main 47430e81's base, every lifted chunk compiled)
Compiling 1,385 s 728 s
The whole build 30:18 15:31
code_hot_7.c 1,385 s 315 s
code_hot_6.c 992 s 304 s
Slowest file code_hot_7.c, 1,385 s code_hot_2.c, 483 s

Baseline: the before column is main 2f1c0c9c's own cold base (the last build that compiled every file from nothing); the commits between it and 47430e81 do not touch the lifted code or the splitter. Both compiled all 75 lifted files with the same Builder (16ff530a99b1).

In the game (median fps of each run; the median of the four)

Before: main @ 47430e81 After: this PR @ 99a24b31
Lake Hylia 46.0, 45.2, 47.1, 46.1: 46.0 47.6, 46.0, 41.0, 48.2: 46.8
Market 45.1, 45.1, 45.0, 42.2: 45.0 45.0, 45.0, 47.1, 46.1: 45.5

The PR's third Lake Hylia run fell to 33-42 fps for about 18 s in the middle of its 40 s, then came back; its log shows nothing the other runs lack (the same core-wait spikes appear in every run, main's too), and the other three are as high as main's or higher. Measured at 99a24b31 on main 47430e81; rebased since onto 868f6be8 (7921c792, the same change): the ten commits between touch the renderer, scripts and build files, not the splitter, so the comparison stands.

before (base-2f1c0c9c): compiled in 1385s, 0 failed / the build took 0:30:18, compiled in 0:23:38
  code_hot_6.c        8 MB     992s  (1002s)
  code_hot_7.c        5 MB    1385s  (1385s)
after (pack split-direct): compiled in 728s, 0 failed / the build took 0:15:31, compiled in 0:12:25
  code_hot_7.c        5 MB     315s  (330s)
  code_hot_2.c       13 MB     483s  (496s)
a run's frame line (main-1): vk: 23880 presents, 45.2 fps; a frame: 2442 draws (0 skipped), ... GPU-bound wait 0.1 ms (recording slots only)

Flags and settings

One new build switch; LONGSHOT_SPLIT_GIANTS keeps its default and now covers functions with guest locals.

Flag What it is Before this PR After this PR Effect Helps / doesn't help / untested Recommendation
LONGSHOT_SPLIT_DIRECT build switch: how a split function's pieces reach each other (none: always the loop) on (unset); 0 keeps the loop direct: pieces tail-call each other with the locals in registers; 0: every cut goes back through f_'s loop, the locals through a struct direct measured above (build and game); the loop covered by the game-free checks only on for everyone; 0 for an A/B or a compiler problem
LONGSHOT_SPLIT_GIANTS build switch: split functions of 256 KB of C or more 8 (functions with locals left whole) 8 (functions with locals split too) 180 more functions split on a real lift helps, measured above unchanged: on at 8

Testing

Tested:

  • In the game, Windows 11, RTX 3080 (NVIDIA 621.14), Ryzen 9 3900X: Lake Hylia and the Market, 9 runs of the PR (a warm-up and 4 measured, each through the title and two teleports) against 5 of main on one base; no crash, both places drawn as on main (each run's screenshot), fps level.
  • tests/test_split_giants.py: both modes, written shapes and the real lifter's output with and without locals, against the whole function from 300 states; the assembly check; a mutation (the store-back before a hop removed) makes it fail.
  • #1142's tests/test_opt_level_exactness.py run against this branch: the hot pieces split at -O2 compute the same as whole (0 of 300 states differ).
  • python -I -B tests/run_all.py with the Builder's llvm-mingw, on the rebased branch: 681 passed, 45 failed, 125 skipped. Every failure fails on main too (the 43 in the team's main baseline, plus test_game_menu_mouse_buttons.py and test_linux_preflight.py, which fail on main 2f1c0c9c the same way); test_hot_pieces_split.py, test_light_passes.py, test_stale_hot_pieces.py and test_doc_anchors.py pass.

Not tested:

  • Linux: a system clang older than 19 has no preserve_none and builds the loop (the checks compile both modes, but only with clang 23 on Windows); the Builder's clang on Linux is unmeasured.
  • LONGSHOT_SPLIT_DIRECT=0 in the game (the game-free checks only).
  • Other CPUs: the change is in how the game code is compiled for x86-64, the same for every GPU.

Must be tested:

  • Linux, any GPU: build this branch from scratch and play a few minutes past the title, expecting the build to finish, the game to behave as on main, and the build's compiled in time below main's (Test Farm: @Beanstalk linux, build from scratch and play Kokiri Forest to Hyrule Field).
  • Steam Deck: the build time with 4 jobs against main, expecting less (by hand or the Test Farm: @Beanstalk steamdeck, a from-scratch build's compile time against main).

Risks

What could break: the game's largest functions are compiled differently: a wrong cut would be wrong game behaviour in those functions (the checks compare the split code with the whole function in both modes, and the in-game runs showed none). A compiler that handles musttail across preserve_none badly would show as a build error or a crash in those functions.

Turning it off: LONGSHOT_SPLIT_DIRECT=0 (the loop) at build time; reverting the commit leaves the functions with locals whole again.

Compatibility: the first build after updating recompiles every lifted chunk once (the split flag is in the object keys; about 12 minutes on a Ryzen 9 3900X); the translation is reused (the splitter is outside the lift key). Saves, settings, shader and pipeline caches and the vk ABI: nothing changes.

Checklist

Everyone

  • One topic.
  • The title is a Conventional Commit (type(scope): description) of the status line's type.
  • Why gives the reason plainly and links the issues that fit (the tracker searched).
  • Tested on my own PC where it runs (in the game, for a change the game runs), as Testing's Tested says; what
    can't be run by hand is in Testing's Not testable by hand, with why.
  • Measurements has a verdict, or one line saying why there is nothing to measure.
  • Nothing gets worse that I know of: any cost (speed, latency, memory, a setup that worked before) is measured and
    in Risks.
  • Testing names what was tested, what wasn't, and what must still be tested, where and how.
  • Rebased on current main.
  • python -I -B tests/run_all.py passes (any failure that also fails on main is named).
  • No game files, dumps, build output or generated files.
  • third_party/ untouched (a new folder with a PROVENANCE.md instead).
  • The license is OK: GPL-3.0-or-later, nothing incompatible and nothing decompiled.

When it applies

  • AI assistance recorded with Assisted-by: / Co-Authored-By: trailers.
  • Numbers from the same PC, scene and settings, at least 2 interleaved runs each, the median reported.
  • Switches documented (docs/development.md; the README for a menu row) and shipped at their recommended value.
  • A file in docs/tested/ for each new measured run (format in docs/tested-configurations.md).
  • A check under tests/ that reads changed source updated in this PR, not weakened.
  • Docs updated.

🤖 Generated with Claude Code

**Type:** perf · **Risk:** medium · **Tested on:** Windows 11, RTX 3080, Ryzen 9 3900X · **Needs:** Linux (clang), Steam Deck build time ## TL;DR Since guest registers became local variables (#1576), the game's ~180 biggest functions were no longer cut into pieces for the compiler, so building the game took about twice as long: one file alone took 23 minutes. This PR cuts them again, carrying the locals across each cut, and lets the pieces jump straight to each other. A cold build's game-code compile went from 1,385 s to 728 s (30 to 15.5 minutes for the whole build), and in the game Lake Hylia and the Market run as fast as on main (4 interleaved runs a side). ## Breakdown - **Who it helps:** everyone who builds the game, and every developer who builds a base or a pack: the first build, and every build after an update that changes the lifted code. - **What was wrong:** the splitter refused any function with a top-level declaration, and guest locals add one, so every giant function with locals compiled whole at -O2. - **What changes:** those functions compile as ~8 KB pieces again; where clang has `preserve_none` and `musttail`, the pieces jump to each other with the locals in registers instead of going back through a dispatch loop. - **What does not change:** what the game computes (checked against the whole function from 300 random states in both modes), the translation (the splitter is outside the lift key), saves, settings and the renderer. - **How it was checked:** the game-free checks in both modes, #1142's hot-pieces exactness check run against this branch, the full suite, a pack's build time, and an in-game A/B at Lake Hylia and the Market: level with main. - **What is still open:** Linux and the Steam Deck (a system clang without `preserve_none` uses the loop; the build time there is unmeasured). - **Risk:** medium: the change touches how the game's largest, hottest functions are compiled; a mistake would be wrong game behaviour, which the exactness checks and the in-game runs did not show. ## Why `nx2/split_giants.py` (#385, #1146) cuts a lifted function of 256 KB of C or more into pieces at its labels, because clang's time on one function grows much faster than its size. Since #1576 turned on guest locals (a function's most used registers and its flags as C locals), every such function starts with their declarations, and the splitter left any function with a top-level declaration whole. On main 2f1c0c9c's cold base that was 181 functions, 92 MB of C: `code_hot_7.c` (two functions of 1.9 and 1.7 MB, at -O2) took 1,385 s, the whole compile stage, and `code_hot_6.c` 992 s, while everything else had finished by about 977 s. See #362 (a first build in under 10 minutes), #1576. ## What changed The splitter now takes the locals the kit declares as a function's first lines and carries them across its cuts. Where the compiler can, a goto into another piece becomes a tail call straight to it with the locals as arguments, so a hop costs a jump and a switch rather than a return, a dispatch and a call. - splitter (`nx2/split_giants.py`): - `locals_of()` reads the declarations (`uint64_t x8_ = c->x[8], ...;` then `uint8_t fn_ = c->n, ...;`), only as the body's first lines; a locals line anywhere else (after lift_fixes' entry goto, #1587) still leaves the function whole. - `split_function()`: `f_<addr>` holds the locals in a struct of its own (`s_<addr>_l`), loaded from the context as the whole function loads them; the context is written exactly where the whole function writes it. (Writing the locals back to the context at each cut would also be correct, but writes dirty scratch registers the whole function never stores.) - `macros()`: each function's pieces are written with five macros (`CC`, `ARGS`, `LOAD`, `HOP`, `CALL`), defined twice under `#if __has_attribute(preserve_none) && __has_attribute(musttail)`: direct (`musttail return s_<addr>_<m>(c, id, regs..., fl_)`, the four flags packed into one argument so everything fits preserve_none's 12 argument registers on x86-64) or the loop (the pieces load from and store to the struct, and return the label id to `f_`'s loop). A function whose locals would not fit in registers uses the loop. - `flag()` / `compile_copy()`: `LONGSHOT_SPLIT_DIRECT=0` ends the flag with `_t` and writes the loop alone; the mode is in the object keys. - build (`nx2/build.py`): `split_giants_flags()` reads `LONGSHOT_SPLIT_DIRECT`; `largest_function()` (the memory estimate) finds the new piece heads by `PIECE_HEAD`. - tests (`tests/test_split_giants.py`): every compiled program runs in both modes; functions emitted by the real lifter with registers and flags as locals (each alone too) are split and run as the whole function does from 300 states, setjmp resumes included; an assembly check that direct pieces only jump to each other (16 jumps, 0 calls) and loop pieces never name each other; a program that never ends now fails the check instead of raising. - docs: `docs/development.md`'s `LONGSHOT_SPLIT_DIRECT` row; eight run files in `docs/tested/`. On a real lift (main 2f1c0c9c's translation) the splitter now cuts 340 functions instead of 160 (190 MB of C into 14,333 pieces), 180 of them with locals; two are left whole as too small to cut. ## Measurements **Verdict:** the game-code compile takes 47% less time (1,385 s to 728 s; the whole build 30:18 to 15:31), and the game runs as fast as main: Lake Hylia +1.6%, the Market +1.1%, both within the spread between runs. Hardware: Windows 11 Pro N (build 22631), RTX 3080 10 GB (NVIDIA 621.14), Ryzen 9 3900X (12 cores), 64 GB Scene: the repository's `kokiri-start` save, child Link teleported by the control bridge to Lake Hylia (destination 7) and then the Market's gate, standing still; a fresh user folder (default settings, Auto: output 2560x1440, a 1720x720 window, mailbox V-Sync Auto), `HOS_FPS_MAX=1000`, frame generation off. CPU-bound here (`GPU-bound wait 0.1 ms`). Method: build time from the build's own summary (23 compilers at once). In the game: the bridge's fps once a second for 40 s after 20 s settled at each place; one warm-up run of each side, then main, PR, PR, main twice (4 runs a side), each a timing job of the PC's queue; the median of each run, and the median of the 4 runs. How to reproduce: `dev_build.py base`, `pack` of this branch, `rt --out` for each side; each run starts the game with `longshot_ctl.py launch --repo-save kokiri-start --user <fresh folder> --env HOS_FPS_MAX=1000`, presses A until in play, then `teleport 7` and `teleport Market_Child`, sampling `status` once a second. **Build time** | | Before: main @ 2f1c0c9c (cold base, nothing in the object store) | After: this PR @ 99a24b31 (pack over main 47430e81's base, every lifted chunk compiled) | |---|---|---| | Compiling | 1,385 s | 728 s | | The whole build | 30:18 | 15:31 | | `code_hot_7.c` | 1,385 s | 315 s | | `code_hot_6.c` | 992 s | 304 s | | Slowest file | `code_hot_7.c`, 1,385 s | `code_hot_2.c`, 483 s | Baseline: the before column is main 2f1c0c9c's own cold base (the last build that compiled every file from nothing); the commits between it and 47430e81 do not touch the lifted code or the splitter. Both compiled all 75 lifted files with the same Builder (16ff530a99b1). **In the game** (median fps of each run; the median of the four) | | Before: main @ 47430e81 | After: this PR @ 99a24b31 | |---|---|---| | Lake Hylia | 46.0, 45.2, 47.1, 46.1: **46.0** | 47.6, 46.0, 41.0, 48.2: **46.8** | | Market | 45.1, 45.1, 45.0, 42.2: **45.0** | 45.0, 45.0, 47.1, 46.1: **45.5** | The PR's third Lake Hylia run fell to 33-42 fps for about 18 s in the middle of its 40 s, then came back; its log shows nothing the other runs lack (the same core-wait spikes appear in every run, main's too), and the other three are as high as main's or higher. Measured at 99a24b31 on main 47430e81; rebased since onto 868f6be8 (7921c792, the same change): the ten commits between touch the renderer, scripts and build files, not the splitter, so the comparison stands. ``` before (base-2f1c0c9c): compiled in 1385s, 0 failed / the build took 0:30:18, compiled in 0:23:38 code_hot_6.c 8 MB 992s (1002s) code_hot_7.c 5 MB 1385s (1385s) after (pack split-direct): compiled in 728s, 0 failed / the build took 0:15:31, compiled in 0:12:25 code_hot_7.c 5 MB 315s (330s) code_hot_2.c 13 MB 483s (496s) a run's frame line (main-1): vk: 23880 presents, 45.2 fps; a frame: 2442 draws (0 skipped), ... GPU-bound wait 0.1 ms (recording slots only) ``` ### Flags and settings One new build switch; `LONGSHOT_SPLIT_GIANTS` keeps its default and now covers functions with guest locals. | Flag | What it is | Before this PR | After this PR | Effect | Helps / doesn't help / untested | Recommendation | |---|---|---|---|---|---|---| | `LONGSHOT_SPLIT_DIRECT` | build switch: how a split function's pieces reach each other | (none: always the loop) | on (unset); `0` keeps the loop | direct: pieces tail-call each other with the locals in registers; `0`: every cut goes back through `f_`'s loop, the locals through a struct | direct measured above (build and game); the loop covered by the game-free checks only | on for everyone; `0` for an A/B or a compiler problem | | `LONGSHOT_SPLIT_GIANTS` | build switch: split functions of 256 KB of C or more | 8 (functions with locals left whole) | 8 (functions with locals split too) | 180 more functions split on a real lift | helps, measured above | unchanged: on at 8 | ## Testing **Tested:** - In the game, Windows 11, RTX 3080 (NVIDIA 621.14), Ryzen 9 3900X: Lake Hylia and the Market, 9 runs of the PR (a warm-up and 4 measured, each through the title and two teleports) against 5 of main on one base; no crash, both places drawn as on main (each run's screenshot), fps level. - `tests/test_split_giants.py`: both modes, written shapes and the real lifter's output with and without locals, against the whole function from 300 states; the assembly check; a mutation (the store-back before a hop removed) makes it fail. - #1142's `tests/test_opt_level_exactness.py` run against this branch: the hot pieces split at -O2 compute the same as whole (0 of 300 states differ). - `python -I -B tests/run_all.py` with the Builder's llvm-mingw, on the rebased branch: 681 passed, 45 failed, 125 skipped. Every failure fails on main too (the 43 in the team's main baseline, plus `test_game_menu_mouse_buttons.py` and `test_linux_preflight.py`, which fail on main 2f1c0c9c the same way); `test_hot_pieces_split.py`, `test_light_passes.py`, `test_stale_hot_pieces.py` and `test_doc_anchors.py` pass. **Not tested:** - Linux: a system clang older than 19 has no `preserve_none` and builds the loop (the checks compile both modes, but only with clang 23 on Windows); the Builder's clang on Linux is unmeasured. - `LONGSHOT_SPLIT_DIRECT=0` in the game (the game-free checks only). - Other CPUs: the change is in how the game code is compiled for x86-64, the same for every GPU. **Must be tested:** - Linux, any GPU: build this branch from scratch and play a few minutes past the title, expecting the build to finish, the game to behave as on main, and the build's `compiled in` time below main's (Test Farm: @Beanstalk linux, build from scratch and play Kokiri Forest to Hyrule Field). - Steam Deck: the build time with 4 jobs against main, expecting less (by hand or the Test Farm: @Beanstalk steamdeck, a from-scratch build's compile time against main). ## Risks **What could break:** the game's largest functions are compiled differently: a wrong cut would be wrong game behaviour in those functions (the checks compare the split code with the whole function in both modes, and the in-game runs showed none). A compiler that handles `musttail` across `preserve_none` badly would show as a build error or a crash in those functions. **Turning it off:** `LONGSHOT_SPLIT_DIRECT=0` (the loop) at build time; reverting the commit leaves the functions with locals whole again. **Compatibility:** the first build after updating recompiles every lifted chunk once (the split flag is in the object keys; about 12 minutes on a Ryzen 9 3900X); the translation is reused (the splitter is outside the lift key). Saves, settings, shader and pipeline caches and the vk ABI: nothing changes. ## Checklist ### Everyone - [x] One topic. - [x] The title is a Conventional Commit (`type(scope): description`) of the status line's type. - [x] Why gives the reason plainly and links the issues that fit (the tracker searched). - [x] Tested on my own PC where it runs (in the game, for a change the game runs), as Testing's Tested says; what can't be run by hand is in Testing's Not testable by hand, with why. - [x] Measurements has a verdict, or one line saying why there is nothing to measure. - [x] Nothing gets worse that I know of: any cost (speed, latency, memory, a setup that worked before) is measured and in Risks. - [x] Testing names what was tested, what wasn't, and what must still be tested, where and how. - [x] Rebased on current `main`. - [x] `python -I -B tests/run_all.py` passes (any failure that also fails on `main` is named). - [x] No game files, dumps, build output or generated files. - [x] `third_party/` untouched (a new folder with a `PROVENANCE.md` instead). - [x] The license is OK: GPL-3.0-or-later, nothing incompatible and nothing decompiled. ### When it applies - [x] AI assistance recorded with `Assisted-by:` / `Co-Authored-By:` trailers. - [x] Numbers from the same PC, scene and settings, at least 2 interleaved runs each, the median reported. - [x] Switches documented (docs/development.md; the README for a menu row) and shipped at their recommended value. - [x] A file in `docs/tested/` for each new measured run (format in `docs/tested-configurations.md`). - [x] A check under `tests/` that reads changed source updated in this PR, not weakened. - [x] Docs updated. 🤖 Generated with [Claude Code](https://claude.com/claude-code)
Since #1576 gave functions guest locals, nx2/split_giants.py refused every one of them (a top-level declaration), so
the ~180 largest functions compiled whole: code_hot_7.c's two (1.9 and 1.7 MB) took 1,385 s at -O2, a cold base's
whole compile stage.

The splitter now takes the locals the kit declares as a function's first lines. f_<addr> keeps them in a struct of its
own, and the context is written exactly where the whole function writes it. Where the compiler has preserve_none and
musttail, the pieces jump straight to each other with the locals as arguments in registers (the four flags packed into
one), and come back to f_'s loop only for a guest return or tail call. LONGSHOT_SPLIT_DIRECT=0 keeps the loop, and the
struct carries the locals there.

build.py's largest_function finds the new piece heads. tests/test_split_giants.py runs both modes against the whole
function, with functions emitted with locals, and checks that direct pieces only jump to each other.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
docs(tested): the split-giants A/B in Lake Hylia and the Market, main against perf/split-giants-direct (RTX 3080, Ryzen 9 3900X)
Some checks failed
CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Has been skipped
CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Has been skipped
CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Has been skipped
Third-party parts / The same as upstream (pull_request) Has been skipped
Website / Website on Cloudflare (pull_request) Has been skipped
CI / Conventional Commit title (pull_request) Successful in 7s
Guide screenshots / Guide images and local UI source review (pull_request) Failing after 11s
CI / Workflow lint (actionlint) (pull_request) Successful in 11s
Third-party parts / As their notes say (pull_request) Successful in 11s
Website / Website on Cloudflare (pull_request_target) Has been skipped
PR description / Description follows the template (pull_request_target) Successful in 3s
e1ab946117
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Author
Member

Tested in game: Windows 11 (22631), RTX 3080 (NVIDIA 621.14), Ryzen 9 3900X: Lake Hylia and the Market from the kokiri-start save, 9 runs of this branch (a warm-up and 4 measured) interleaved with 5 of main on one base; no crash, both places drawn as on main, fps level (Lake 46.8 vs 46.0, Market 45.5 vs 45.0)

Tested in game: Windows 11 (22631), RTX 3080 (NVIDIA 621.14), Ryzen 9 3900X: Lake Hylia and the Market from the kokiri-start save, 9 runs of this branch (a warm-up and 4 measured) interleaved with 5 of main on one base; no crash, both places drawn as on main, fps level (Lake 46.8 vs 46.0, Market 45.5 vs 45.0)
Author
Member

Why the numbers still hold on main 868f6be8: they were measured at 99a24b31 on main 47430e81 (the in-game A/B, both sides from that one base) and against main 2f1c0c9c's cold base (the build time's before column). The branch was then rebased onto 868f6be8 as 7921c792, the same change with no conflicts. The ten main commits in between change the renderer (Intel, RDNA 4, compute lanes), host profiling, scripts and nx2's translator file list and version; none touches nx2/split_giants.py or the C the lifter writes for a function, which is what this PR changes how it compiles. The full suite was run again on the rebased branch: 681 passed, 45 failed, every failure also failing on main.

Why the numbers still hold on main 868f6be8: they were measured at 99a24b31 on main 47430e81 (the in-game A/B, both sides from that one base) and against main 2f1c0c9c's cold base (the build time's before column). The branch was then rebased onto 868f6be8 as 7921c792, the same change with no conflicts. The ten main commits in between change the renderer (Intel, RDNA 4, compute lanes), host profiling, scripts and nx2's translator file list and version; none touches nx2/split_giants.py or the C the lifter writes for a function, which is what this PR changes how it compiles. The full suite was run again on the rebased branch: 681 passed, 45 failed, every failure also failing on main.
dyst changed title from WIP: perf(build): giant functions with guest locals compile in pieces again, the pieces jumping to each other (LONGSHOT_SPLIT_DIRECT) to perf(build): giant functions with guest locals compile in pieces again, the pieces jumping to each other (LONGSHOT_SPLIT_DIRECT) 2026-11-19 08:48:22 +00:00
Author
Member

A correction to the comment above: one of those commits, #1686, does change the lifter (nx2/insn.py, SCVTF / UCVTF in their scalar fixed-point form). By its own notes the game's code does not use that form, so the C written for the game's functions, and with it what this PR compiles, is the same; the numbers hold for that reason, not because no lifter file changed.

A correction to the comment above: one of those commits, #1686, does change the lifter (nx2/insn.py, SCVTF / UCVTF in their scalar fixed-point form). By its own notes the game's code does not use that form, so the C written for the game's functions, and with it what this PR compiles, is the same; the numbers hold for that reason, not because no lifter file changed.
HumanBean approved these changes 2026-11-20 17:00:41 +00:00
Dismissed
dyst force-pushed perf/split-giants-direct from e1ab946117
Some checks failed
CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Has been skipped
CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Has been skipped
CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Has been skipped
Third-party parts / The same as upstream (pull_request) Has been skipped
Website / Website on Cloudflare (pull_request) Has been skipped
CI / Conventional Commit title (pull_request) Successful in 7s
Guide screenshots / Guide images and local UI source review (pull_request) Failing after 11s
CI / Workflow lint (actionlint) (pull_request) Successful in 11s
Third-party parts / As their notes say (pull_request) Successful in 11s
Website / Website on Cloudflare (pull_request_target) Has been skipped
PR description / Description follows the template (pull_request_target) Successful in 3s
to 8d4b5cbf91
Some checks failed
CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Has been skipped
CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Has been skipped
CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Has been skipped
Third-party parts / The same as upstream (pull_request) Has been skipped
Website / Website on Cloudflare (pull_request) Has been skipped
CI / Conventional Commit title (pull_request) Successful in 22s
PR description / Description follows the template (pull_request_target) Failing after 25s
Guide screenshots / Guide images and local UI source review (pull_request) Failing after 29s
CI / Workflow lint (actionlint) (pull_request) Successful in 30s
Third-party parts / As their notes say (pull_request) Successful in 31s
CI / Website build and checks (pull_request) Failing after 2m2s
2026-11-20 17:03:15 +00:00
Compare
dyst dismissed HumanBean's review 2026-11-20 17:03:16 +00:00
Reason:

New commits pushed, approval review dismissed automatically according to repository settings

Author
Member

Rebased onto main 7d5fcb28 (45 commits on), now 8d4b5cbf. One conflict: docs/development.md is gone since #1802, so the LONGSHOT_SPLIT_DIRECT row moved to docs/dev/reference/build.md beside LONGSHOT_SPLIT_GIANTS's. nx2/build.py and nx2/split_giants.py merged without a conflict; tests/test_split_giants.py passes at the new head (both modes, 0 of 300 states differ). Not rebuilt or measured again at this head: the A/B was of the pieces' code, which the 45 commits do not touch (#1782's float paths change the lifted text the pieces are cut from, not how they are cut or called).

Rebased onto main 7d5fcb28 (45 commits on), now 8d4b5cbf. One conflict: docs/development.md is gone since #1802, so the LONGSHOT_SPLIT_DIRECT row moved to docs/dev/reference/build.md beside LONGSHOT_SPLIT_GIANTS's. nx2/build.py and nx2/split_giants.py merged without a conflict; tests/test_split_giants.py passes at the new head (both modes, 0 of 300 states differ). Not rebuilt or measured again at this head: the A/B was of the pieces' code, which the 45 commits do not touch (#1782's float paths change the lifted text the pieces are cut from, not how they are cut or called).
HumanBean deleted branch perf/split-giants-direct 2026-11-20 17:14:49 +00:00
Sign in to join this conversation.
No reviewers
No milestone
No project
No assignees
2 participants
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
MagicBeanDev/Longshot!1727
No description provided.