perf(build): the hot pieces compile with their giant functions split #1146

Merged
HumanBean merged 1 commit from Marcuss2/Longshot:perf/build-split-hot-giants into main 2026-11-16 17:44:00 +00:00
Contributor

Type: perf · Risk: low · Tested on: Linux (RADV, RX 9070 XT): the split build played in Kokiri Forest; game-free checks; the build measured from scratch · Needs: none

Why

A development build that carries a hot list (A64_HOT_FUNCS) spends most of its compile on the hot pieces, and one of them is a single lifted function: code_hot_0.c is f_81b8c000, 4.79 MB of C, compiled at the hot -O2 flags. On this PC it took 438 s of a from-scratch build's 446 s compile, so a build that carries a hot list took 8 min 26 s where one without takes about 3 min 20 s.

The rest of the lift has had this fixed since LONGSHOT_SPLIT_GIANTS (nx2/split_giants.py): a lifted function of 256 KB or more compiles as pieces cut at its labels, with a dispatcher under its own name, exact by construction. But the hot pieces' compile flags are built separately in nx2/build.py (hot_fl) and never got split_giants_flags(), so the hot giants were still compiled whole.

What changed

The hot pieces now carry the same split flag as the lifted chunks. A 47-function hot-list build from scratch goes 8 min 26 s -> 3 min 19 s, and the exe is 896,056 bytes smaller.

  • nx2/build.py: hot_flags(fl) (new) builds the hot pieces' flags: fl at RX_HOT_OPT (default O2) plus split_giants_flags(). main() calls it instead of the inline list, and split_fl's comment no longer says the hot pieces are left out.
  • tests/test_hot_pieces_split.py (new): the source-level guard on those flags. It checks that hot_flags() carries split_giants.FLAG (default, RX_HOT_OPT=O1, LONGSHOT_SPLIT_GIANTS=0 and =16), that main() takes its hot_fl from it (so an edit back to the inline list fails), and that source_flags() sends a code_hot* file to it. The splitter's exactness is not re-proven: tests/test_split_giants.py (unchanged) proves it, and #1142 covers the hot path at -O2 and -O1 against -O2 compiled.
  • docs/development.md: the LONGSHOT_SPLIT_GIANTS row says the hot pieces are split too, with the numbers; "A faster build for development" says the "about a third of the compile time" was before this.

Measurements

Verdict: yes. From scratch with the real 47-function hot list: wall 8 min 26 s -> 3 min 19 s (-61%), compile 446 s -> 137 s (-69%), the single giant translation unit 438 s -> 56 s (-87%); the game exe 547,787,416 -> 546,891,360 bytes. An earlier pair the Lead measured on the same PC with the same list and the same one-line change gave 7 min 13 s -> 3 min 17 s (compile 369 s -> 138 s) and the same two exe sizes.

Hardware: Linux, AMD Ryzen 9 9950X3D (16 cores, 32 threads), 62 GB, NVMe
Scene: none (the build alone)
Method: rm -rf build-linux && python3 -I scripts/dev_build.py build --out $PWD/build-linux/out-game, from scratch, with LONGSHOT_DEV_PYTHON, LONGSHOT_DEV_ROMFS, LONGSHOT_DEV_EXEFS, A64_HOT_FUNCS (the 47-function list) and LONGSHOT_OBJ_STORE=0 LONGSHOT_LIFT_STORE=0, shader preparation skipped (as dev_build.py build does), through the PC's job queue. Before: main @ 71b1edd8, one from-scratch build. After: this PR @ d44ffb4c, two from-scratch builds (3 min 35 s and 3 min 19 s; the second is the table's). Three from-scratch builds in all, not interleaved (each is 3-8 minutes), and nothing else was building under the queue in those windows.
How to reproduce: the command above, then the log's code_hot_0.c line and the build took ... compiled in ....

Before: main @ 71b1edd8 After: this PR @ d44ffb4c
whole build, wall 8 min 26 s 3 min 19 s
compile stage 446 s 137 s
compiling 438 s 128 s
code_hot_0.c alone 438 s 56 s
longshot exe 547,787,416 bytes 546,891,360 bytes
Before:   code_hot_0.c        4 MB     438s  (438s)
          the build took 0:08:26, compiled in 0:07:26
After:    code_hot_0.c        4 MB      56s  (60s)
          the build took 0:03:19, compiled in 0:02:17

Flags and settings

None. LONGSHOT_SPLIT_GIANTS (on by default at 8 KB) now covers code_hot*.c as well; 0 still turns it off. Its row in docs/development.md is updated.

Testing

Tested:

  • tests/test_hot_pieces_split.py: passes here; fails on main @ 71b1edd8 with nx2/build.py has no hot_flags(): the hot pieces' flags carry no giant split and main() builds the hot flags with hot_flags(fl).
  • tests/test_split_giants.py (unchanged): passes; it is the exactness proof for the splitter (random lifter-shaped functions, the real emitter's output, and compile_one's copy, comparing the context, guest memory, page marks, calls, setjmp resumes and polls). #1142 adds the compiled hot path at -O2 and -O1 against -O2; this PR does not duplicate it.
  • python3 -I -B tests/run_all.py --jobs 8: 587 passed, 17 failed, 74 skipped (8 at once, in 86 s). The 17 fail on main @ 71b1edd8 on this PC too: test_audio_output_devices, test_bug_report, test_bug_report_defaults, test_crt_options, test_dlss_feature_size, test_failed_compile_keeps_objects, test_guest_imported, test_hardware_report, test_linux_preflight, test_linux_update, test_make_game_cache, test_missing_nup, test_mods_size_table, test_pending_guest_import, test_run_all_changed, test_vk_occlusion_queries, test_walk_key.
  • In the game (the Lead's run, Linux, RADV, RX 9070 XT, Ryzen 9 9950X3D): the split build with the 47-function hot list, Kokiri Forest at Link's house from the repo save, 40 s headless, 676 draws a frame, no crash, no assertion, 0 failed programs; split against unsplit over 10 interleaved arms: +0.24% fps (exact permutation p = 0.43) and -0.38% CPU (p = 0.12), no measurable change.

Not tested: a Windows build with a hot list (the splitter and the flags are platform-neutral, and the game run above is on this PC's GPU and driver), and a release or player build (there is no hot list, so this path does not run). The exe is 0.9 MB smaller, so the code layout differs.

Must be tested: none. This is a build script's flags: the only thing that runs it is a home build with a hot list, which built and linked here and ran in the game above; releases ship no hot list.

Risks

What could break: a hot function whose text the splitter will not cut (a label inside a block, a computed goto, a top-level declaration) is left whole exactly as before, so a build cannot fail this way; the split's rules are the lifted chunks' already. The flag is in the object key, so the next build compiles the hot pieces again instead of taking an old object.

Turning it off: LONGSHOT_SPLIT_GIANTS=0.

Compatibility: nothing for players (a release has no hot list). A development build compiles its hot pieces again once, and the exe is 896,056 bytes smaller.

Issues

See #1142 (the compiled proof of the hot path at -O2, and of -O1 against -O2, open against main).

Checklist

Everyone

  • One topic.
  • The title is a Conventional Commit (type(scope): description) of the status line's type.
  • Why gives the reason plainly and links the issues that fit (the tracker searched).
  • Measurements has a verdict, or one line saying why there is nothing to measure.
  • Testing names what was tested, what wasn't, and what must still be tested, where and how.
  • Rebased on current main (71b1edd8).
  • python -I -B tests/run_all.py passes (the 17 failures also fail on main; they are named above).
  • No game files, dumps, build output or generated files.
  • third_party/ untouched (a new folder with a PROVENANCE.md instead).
  • The license is OK: GPL-3.0-or-later, nothing incompatible and nothing decompiled.

When it applies

  • AI assistance recorded with Assisted-by: / Co-Authored-By: trailers.
  • Numbers from the same PC, scene and settings, at least 2 interleaved runs each, the median reported. (From-scratch builds cannot be interleaved cheaply: Before is one build of 8 min 26 s, After two of 3 min 35 s and 3 min 19 s; the effect is far outside the run-to-run spread.)
  • Switches documented (docs/development.md) and shipped at their recommended value.
  • A game-free test added for the fix, failing on main and passing here.
  • A check under tests/ that reads changed source updated in this PR, not weakened. (tests/test_hot_pieces_split.py reads nx2/build.py's hot_flags and main; no existing check needed updating.)
  • Docs updated.
**Type:** perf · **Risk:** low · **Tested on:** Linux (RADV, RX 9070 XT): the split build played in Kokiri Forest; game-free checks; the build measured from scratch · **Needs:** none ## Why A development build that carries a hot list (`A64_HOT_FUNCS`) spends most of its compile on the hot pieces, and one of them is a single lifted function: `code_hot_0.c` is `f_81b8c000`, 4.79 MB of C, compiled at the hot `-O2` flags. On this PC it took 438 s of a from-scratch build's 446 s compile, so a build that carries a hot list took 8 min 26 s where one without takes about 3 min 20 s. The rest of the lift has had this fixed since `LONGSHOT_SPLIT_GIANTS` (`nx2/split_giants.py`): a lifted function of 256 KB or more compiles as pieces cut at its labels, with a dispatcher under its own name, exact by construction. But the hot pieces' compile flags are built separately in `nx2/build.py` (`hot_fl`) and never got `split_giants_flags()`, so the hot giants were still compiled whole. ## What changed The hot pieces now carry the same split flag as the lifted chunks. A 47-function hot-list build from scratch goes 8 min 26 s -> 3 min 19 s, and the exe is 896,056 bytes smaller. - `nx2/build.py`: `hot_flags(fl)` (new) builds the hot pieces' flags: `fl` at `RX_HOT_OPT` (default `O2`) plus `split_giants_flags()`. `main()` calls it instead of the inline list, and `split_fl`'s comment no longer says the hot pieces are left out. - `tests/test_hot_pieces_split.py` (new): the source-level guard on those flags. It checks that `hot_flags()` carries `split_giants.FLAG` (default, `RX_HOT_OPT=O1`, `LONGSHOT_SPLIT_GIANTS=0` and `=16`), that `main()` takes its `hot_fl` from it (so an edit back to the inline list fails), and that `source_flags()` sends a `code_hot*` file to it. The splitter's exactness is not re-proven: `tests/test_split_giants.py` (unchanged) proves it, and #1142 covers the hot path at `-O2` and `-O1` against `-O2` compiled. - `docs/development.md`: the `LONGSHOT_SPLIT_GIANTS` row says the hot pieces are split too, with the numbers; "A faster build for development" says the "about a third of the compile time" was before this. ## Measurements **Verdict:** yes. From scratch with the real 47-function hot list: wall 8 min 26 s -> 3 min 19 s (-61%), compile 446 s -> 137 s (-69%), the single giant translation unit 438 s -> 56 s (-87%); the game exe 547,787,416 -> 546,891,360 bytes. An earlier pair the Lead measured on the same PC with the same list and the same one-line change gave 7 min 13 s -> 3 min 17 s (compile 369 s -> 138 s) and the same two exe sizes. Hardware: Linux, AMD Ryzen 9 9950X3D (16 cores, 32 threads), 62 GB, NVMe Scene: none (the build alone) Method: `rm -rf build-linux && python3 -I scripts/dev_build.py build --out $PWD/build-linux/out-game`, from scratch, with `LONGSHOT_DEV_PYTHON`, `LONGSHOT_DEV_ROMFS`, `LONGSHOT_DEV_EXEFS`, `A64_HOT_FUNCS` (the 47-function list) and `LONGSHOT_OBJ_STORE=0 LONGSHOT_LIFT_STORE=0`, shader preparation skipped (as `dev_build.py build` does), through the PC's job queue. Before: main @ 71b1edd8, one from-scratch build. After: this PR @ d44ffb4c, two from-scratch builds (3 min 35 s and 3 min 19 s; the second is the table's). Three from-scratch builds in all, not interleaved (each is 3-8 minutes), and nothing else was building under the queue in those windows. How to reproduce: the command above, then the log's `code_hot_0.c` line and `the build took ... compiled in ...`. | | Before: main @ 71b1edd8 | After: this PR @ d44ffb4c | |---|---|---| | whole build, wall | 8 min 26 s | 3 min 19 s | | compile stage | 446 s | 137 s | | compiling | 438 s | 128 s | | `code_hot_0.c` alone | 438 s | 56 s | | `longshot` exe | 547,787,416 bytes | 546,891,360 bytes | ``` Before: code_hot_0.c 4 MB 438s (438s) the build took 0:08:26, compiled in 0:07:26 After: code_hot_0.c 4 MB 56s (60s) the build took 0:03:19, compiled in 0:02:17 ``` ### Flags and settings None. `LONGSHOT_SPLIT_GIANTS` (on by default at 8 KB) now covers `code_hot*.c` as well; `0` still turns it off. Its row in docs/development.md is updated. ## Testing **Tested:** - `tests/test_hot_pieces_split.py`: passes here; fails on main @ 71b1edd8 with `nx2/build.py has no hot_flags(): the hot pieces' flags carry no giant split` and `main() builds the hot flags with hot_flags(fl)`. - `tests/test_split_giants.py` (unchanged): passes; it is the exactness proof for the splitter (random lifter-shaped functions, the real emitter's output, and `compile_one`'s copy, comparing the context, guest memory, page marks, calls, setjmp resumes and polls). #1142 adds the compiled hot path at `-O2` and `-O1` against `-O2`; this PR does not duplicate it. - `python3 -I -B tests/run_all.py --jobs 8`: 587 passed, 17 failed, 74 skipped (8 at once, in 86 s). The 17 fail on main @ 71b1edd8 on this PC too: test_audio_output_devices, test_bug_report, test_bug_report_defaults, test_crt_options, test_dlss_feature_size, test_failed_compile_keeps_objects, test_guest_imported, test_hardware_report, test_linux_preflight, test_linux_update, test_make_game_cache, test_missing_nup, test_mods_size_table, test_pending_guest_import, test_run_all_changed, test_vk_occlusion_queries, test_walk_key. - In the game (the Lead's run, Linux, RADV, RX 9070 XT, Ryzen 9 9950X3D): the split build with the 47-function hot list, Kokiri Forest at Link's house from the repo save, 40 s headless, 676 draws a frame, no crash, no assertion, 0 failed programs; split against unsplit over 10 interleaved arms: +0.24% fps (exact permutation p = 0.43) and -0.38% CPU (p = 0.12), no measurable change. **Not tested:** a Windows build with a hot list (the splitter and the flags are platform-neutral, and the game run above is on this PC's GPU and driver), and a release or player build (there is no hot list, so this path does not run). The exe is 0.9 MB smaller, so the code layout differs. **Must be tested:** none. This is a build script's flags: the only thing that runs it is a home build with a hot list, which built and linked here and ran in the game above; releases ship no hot list. ## Risks **What could break:** a hot function whose text the splitter will not cut (a label inside a block, a computed goto, a top-level declaration) is left whole exactly as before, so a build cannot fail this way; the split's rules are the lifted chunks' already. The flag is in the object key, so the next build compiles the hot pieces again instead of taking an old object. **Turning it off:** `LONGSHOT_SPLIT_GIANTS=0`. **Compatibility:** nothing for players (a release has no hot list). A development build compiles its hot pieces again once, and the exe is 896,056 bytes smaller. ## Issues See #1142 (the compiled proof of the hot path at `-O2`, and of `-O1` against `-O2`, open against main). ## Checklist ### Everyone - [x] One topic. - [x] The title is a Conventional Commit (`type(scope): description`) of the status line's type. - [x] Why gives the reason plainly and links the issues that fit (the tracker searched). - [x] Measurements has a verdict, or one line saying why there is nothing to measure. - [x] Testing names what was tested, what wasn't, and what must still be tested, where and how. - [x] Rebased on current `main` (71b1edd8). - [x] `python -I -B tests/run_all.py` passes (the 17 failures also fail on `main`; they are named above). - [x] No game files, dumps, build output or generated files. - [x] `third_party/` untouched (a new folder with a `PROVENANCE.md` instead). - [x] The license is OK: GPL-3.0-or-later, nothing incompatible and nothing decompiled. ### When it applies - [x] AI assistance recorded with `Assisted-by:` / `Co-Authored-By:` trailers. - [x] Numbers from the same PC, scene and settings, at least 2 interleaved runs each, the median reported. (From-scratch builds cannot be interleaved cheaply: Before is one build of 8 min 26 s, After two of 3 min 35 s and 3 min 19 s; the effect is far outside the run-to-run spread.) - [x] Switches documented (docs/development.md) and shipped at their recommended value. - [x] A game-free test added for the fix, failing on `main` and passing here. - [x] A check under `tests/` that reads changed source updated in this PR, not weakened. (`tests/test_hot_pieces_split.py` reads `nx2/build.py`'s `hot_flags` and `main`; no existing check needed updating.) - [x] Docs updated.
perf(build): the hot pieces compile with their giant functions split
Some checks are pending
CI / Conventional Commit title (pull_request) Blocked by required conditions
CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Blocked by required conditions
CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Blocked by required conditions
CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Blocked by required conditions
CI / Workflow lint (actionlint) (pull_request) Blocked by required conditions
Guide screenshots / Guide images and local UI source review (pull_request) Blocked by required conditions
Third-party parts / As their notes say (pull_request) Blocked by required conditions
Third-party parts / The same as upstream (pull_request) Blocked by required conditions
Website / Website on Cloudflare (pull_request) Blocked by required conditions
Website / Website on Cloudflare (pull_request_target) Has been skipped
PR description / Description follows the template (pull_request_target) Successful in 3s
d44ffb4cb8
A hot-list build's hot pieces are built at -O2 by their own flags, and those flags never carried
LONGSHOT_SPLIT_GIANTS: code_hot_0.c, the single lifted function f_81b8c000 (4.79 MB of C), was one
translation unit and took 438 s of a from-scratch build's 446 s compile. The lifted chunks already
compile their giants as pieces cut at their labels (nx2/split_giants.py, exact by construction), so
hot_flags() in nx2/build.py now adds the same split flag to the hot pieces' flags.

From scratch with the 47-function hot list and the stores off: 8 min 26 s -> 3 min 19 s wall, compile
446 s -> 137 s, code_hot_0.c 438 s -> 56 s; the exe 547,787,416 -> 546,891,360 bytes.
tests/test_hot_pieces_split.py is the source-level guard that the hot flags carry the split (it fails
on main); the splitter's exactness is tests/test_split_giants.py, unchanged.

Assisted-by: DeepSeek Harness
Author
Contributor

Tested in game: Linux (RADV, RX 9070 XT, Ryzen 9 9950X3D), the split build with the 47-function hot list: Kokiri Forest at Link's house from the repo save, 40 s headless, 676 draws a frame, no crash, no assertion, 0 failed programs; split vs unsplit A/B over 10 interleaved arms: +0.24 % fps (exact permutation p = 0.43), -0.38 % CPU (p = 0.12), i.e. no measurable change.

Tested in game: Linux (RADV, RX 9070 XT, Ryzen 9 9950X3D), the split build with the 47-function hot list: Kokiri Forest at Link's house from the repo save, 40 s headless, 676 draws a frame, no crash, no assertion, 0 failed programs; split vs unsplit A/B over 10 interleaved arms: +0.24 % fps (exact permutation p = 0.43), -0.38 % CPU (p = 0.12), i.e. no measurable change.
Collaborator

Possibly related, for context:

  • PR #791 perf(build): the hot code compiles within memory with no profile, its giant functions first — Open build PR also changes hot-piece compile behavior in nx2/build.py; check for overlap
Possibly related, for context: - PR #791 perf(build): the hot code compiles within memory with no profile, its giant functions first — Open build PR also changes hot-piece compile behavior in nx2/build.py; check for overlap
Marcuss2 changed title from WIP: perf(build): the hot pieces compile with their giant functions split to perf(build): the hot pieces compile with their giant functions split 2026-11-16 15:13:07 +00:00
Author
Contributor

Bot context (BeanIssues) names two open build PRs worth a line here:

  • #791 (dyst, open) changes the same in : it adds (memory guard, all modes) and a 2.2x cost weight in so the piece holding a giant function starts first. This PR extracts — that is the place their flag goes if they land first, and their should then read instead of the inline list. Composition is one line, no design overlap.
  • #756 (siantm, WIP) adds a separate flag path for / () through an optional extra parameter of ; this PR keeps calling with the existing 7 arguments, so the two do not collide.
  • Worth noting for #791's motivation: the split also removes the memory pressure it guards against. The monster hot function (, 4.79 MB of C) at the hot flags measured just now on this PC: whole 248.1 s / 1.7 GB peak RSS, split 27.0 s / 0.4 GB. A build has more room to grow, and I have not measured that mode, so this is context, not a claim that #791 is unnecessary.
Bot context (BeanIssues) names two open build PRs worth a line here: - **#791** (dyst, open) changes the same in : it adds (memory guard, all modes) and a 2.2x cost weight in so the piece holding a giant function starts first. This PR extracts — that is the place their flag goes if they land first, and their should then read instead of the inline list. Composition is one line, no design overlap. - **#756** (siantm, WIP) adds a separate flag path for / () through an optional extra parameter of ; this PR keeps calling with the existing 7 arguments, so the two do not collide. - Worth noting for #791's motivation: the split also removes the memory pressure it guards against. The monster hot function (, 4.79 MB of C) at the hot flags measured just now on this PC: whole 248.1 s / **1.7 GB** peak RSS, split 27.0 s / **0.4 GB**. A build has more room to grow, and I have not measured that mode, so this is context, not a claim that #791 is unnecessary.
HumanBean deleted branch perf/build-split-hot-giants 2026-11-16 17:44:00 +00:00
Some checks are pending
CI / Conventional Commit title (pull_request) Blocked by required conditions
CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Blocked by required conditions
CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Blocked by required conditions
CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Blocked by required conditions
CI / Workflow lint (actionlint) (pull_request) Blocked by required conditions
Guide screenshots / Guide images and local UI source review (pull_request) Blocked by required conditions
Third-party parts / As their notes say (pull_request) Blocked by required conditions
Third-party parts / The same as upstream (pull_request) Blocked by required conditions
Website / Website on Cloudflare (pull_request) Blocked by required conditions
Website / Website on Cloudflare (pull_request_target) Has been skipped
PR description / Description follows the template (pull_request_target) Successful in 3s
Some workflows are waiting to be reviewed.
Sign in to join this conversation.
No reviewers
No milestone
No project
No assignees
2 participants
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
MagicBeanDev/Longshot!1146
No description provided.