perf(build): a first build in under 10 minutes: what was measured, and a proposal (a clang-free cold tier) #362

New issue

Open

opened 2026-11-14 04:25:32 +00:00 by madebyjordan · 1 comment

madebyjordan commented

2026-11-14 04:25:32 +00:00

Contributor

1. A clang-free cold tier (the big one)

Clang compiles only the hot list (the functions play actually runs). Every other function becomes threaded code: a list of calls to small helpers, one helper per statement form the lift emits, generated by the lifter and compiled on the player's PC. So nothing made from the game ships.

	today	cold tier (spike)
clang compiler-seconds	6,478	1,157 (hot list 3,680 + 77 functions it can't take)
assembling the threaded code	0	163 s (Python spike; a native emitter's inner loop does the whole game in ~0.2 s)
image	617 MB	648 MB
fps in play (3 interleaved runs, medians)	40.2	42.5
the start's loading	12.0 s	18.2 s
first save load	23.9 s	30.8 s

Coverage: the lift emits about 2,240 distinct statement forms. Helpers for all of them compile in about 2.5 s, so every function is covered except 96, which stay on clang: game patches, setjmp callers and a few odd statements.
Equivalence: per form, 477,468 cases against both the lifted C and Unicorn, with no differences. Per whole function, 2,797 real functions on 68,160 states, clang-built against threaded: no differences after one jump-table fix in the spike.
The cost: loads are slower until the code they run is on the hot list. We judged about 6 s acceptable for a much shorter first build. A background -O1 rebuild swapped in at the next launch, a bigger shipped hot list, or true copy-and-patch would remove it.
Projected Build click to title: about 8.7 minutes at 4 compilers, with the emitter built into the lift and the items below (an estimate; shader preparation not included).

What it would take:

dyst: the emitter in the lifter.
HumanBean: the launcher shipping the hot list, and optionally the background rebuild and swap.

2. More compilers, smaller pieces (#31)

Plain compilation: with #31's ~4 MB pieces, the same 204 MB of C took 63 s at 8 compilers against 127 s as whole chunks at 4 (3 reps). A piece peaks at ~280 MB against ~1.45 GB for a whole chunk.
The catch: mem_need counts a 4 MB piece at about 2.4 times its real peak, so with ~5 GB free the gate starts only about 5. Across 70 measured compiles, 40 + 72 × MB of C + 550 × MB of the largest function never under-counted, and starts 8 with about 2.5 GB to spare.
Why 4 compilers ran 1.5x slower: not paging. Four compilers on a quiet PC slow each other by about 1.05x; memory bandwidth taken by other programs costs up to 2.4x.

3. Splitting the giant functions

The giant cluster, code_023-025, is one function per file at up to 6 MB of C. Split at its labels into 8 KB pieces with a dispatcher, it compiles 1,462 → 559 compiler-seconds (0.38x), with its peak memory down from 2.26 to 0.59 GB. Equivalence: 1,200 random states on made-up functions in the lifter's shapes, and 160 states recorded in play for each of the two hottest giants, all with no differences. On those real states the split code is 2-3% faster per call. Seven 300 s game runs, split and unsplit alternated, showed no fps difference. This overlaps #38's fast float regions, which change the same emitter: the splitter would need to learn their shape.

4. Lighter compiler passes

Without -O1's module passes, ordinary chunks compile 21-34% faster for 3-5% more code; the inliner barely inlines lifted code anyway. On the giants, Machine LICM off saves 16-18%. Runtime speed with these isn't measured yet.

Already up as PRs

#357: the lift on Windows, one worker per 0.75 GB free: 246.7 -> 150.6 s, output byte-identical.
#358: the button prompts, 158.9 -> 21.3 s, the same files.
#359: Translate's progress bar moves through the lift.
#360 (+ LongshotLauncher MagicBeanDev/LongshotLauncher#58): a new version's game files hard-linked from the last one: 26 GB not written for each version.
#356: a 2 GB image guard, which a bigger code tier would need.

A lifter bug found on the way

#361: SDIV of the most negative number by -1 kills the game on x86 (found by the equivalence harness of section 1).

Happy to share any of the harnesses or numbers.

A player's first build (Build click to title) takes about 41 minutes on a Ryzen 5 5600 with 16 GB under Windows. Most of it is clang compiling the lifted C: 6,478 compiler-seconds, about 27 minutes at 4 compilers. Over the last day I measured what can take it down. All of this was on that PC, in local experiments; nothing has been pushed apart from the PRs linked below. ## 1. A clang-free cold tier (the big one) Clang compiles only the hot list (the functions play actually runs). Every other function becomes threaded code: a list of calls to small helpers, one helper per statement form the lift emits, generated by the lifter and compiled on the player's PC. So nothing made from the game ships. | | today | cold tier (spike) | |---|---|---| | clang compiler-seconds | 6,478 | **1,157** (hot list 3,680 + 77 functions it can't take) | | assembling the threaded code | 0 | 163 s (Python spike; a native emitter's inner loop does the whole game in ~0.2 s) | | image | 617 MB | 648 MB | | fps in play (3 interleaved runs, medians) | 40.2 | 42.5 | | the start's loading | 12.0 s | 18.2 s | | first save load | 23.9 s | 30.8 s | - **Coverage:** the lift emits about 2,240 distinct statement forms. Helpers for all of them compile in about 2.5 s, so every function is covered except 96, which stay on clang: game patches, setjmp callers and a few odd statements. - **Equivalence:** per form, 477,468 cases against both the lifted C and Unicorn, with no differences. Per whole function, 2,797 real functions on 68,160 states, clang-built against threaded: no differences after one jump-table fix in the spike. - **The cost:** loads are slower until the code they run is on the hot list. We judged about 6 s acceptable for a much shorter first build. A background -O1 rebuild swapped in at the next launch, a bigger shipped hot list, or true copy-and-patch would remove it. - **Projected Build click to title:** about 8.7 minutes at 4 compilers, with the emitter built into the lift and the items below (an estimate; shader preparation not included). What it would take: - **dyst:** the emitter in the lifter. - **HumanBean:** the launcher shipping the hot list, and optionally the background rebuild and swap. ## 2. More compilers, smaller pieces (#31) - **Plain compilation:** with #31's ~4 MB pieces, the same 204 MB of C took 63 s at 8 compilers against 127 s as whole chunks at 4 (3 reps). A piece peaks at ~280 MB against ~1.45 GB for a whole chunk. - **The catch:** `mem_need` counts a 4 MB piece at about 2.4 times its real peak, so with ~5 GB free the gate starts only about 5. Across 70 measured compiles, `40 + 72 × MB of C + 550 × MB of the largest function` never under-counted, and starts 8 with about 2.5 GB to spare. - **Why 4 compilers ran 1.5x slower:** not paging. Four compilers on a quiet PC slow each other by about 1.05x; memory bandwidth taken by other programs costs up to 2.4x. ## 3. Splitting the giant functions The giant cluster, code_023-025, is one function per file at up to 6 MB of C. Split at its labels into 8 KB pieces with a dispatcher, it compiles 1,462 → 559 compiler-seconds (0.38x), with its peak memory down from 2.26 to 0.59 GB. Equivalence: 1,200 random states on made-up functions in the lifter's shapes, and 160 states recorded in play for each of the two hottest giants, all with no differences. On those real states the split code is 2-3% faster per call. Seven 300 s game runs, split and unsplit alternated, showed no fps difference. This overlaps #38's fast float regions, which change the same emitter: the splitter would need to learn their shape. ## 4. Lighter compiler passes Without -O1's module passes, ordinary chunks compile 21-34% faster for 3-5% more code; the inliner barely inlines lifted code anyway. On the giants, Machine LICM off saves 16-18%. Runtime speed with these isn't measured yet. ## Already up as PRs - #357: the lift on Windows, one worker per 0.75 GB free: 246.7 -> 150.6 s, output byte-identical. - #358: the button prompts, 158.9 -> 21.3 s, the same files. - #359: Translate's progress bar moves through the lift. - #360 (+ LongshotLauncher https://git.magicbean.dev/MagicBeanDev/LongshotLauncher/pulls/58): a new version's game files hard-linked from the last one: 26 GB not written for each version. - #356: a 2 GB image guard, which a bigger code tier would need. ## A lifter bug found on the way #361: SDIV of the most negative number by -1 kills the game on x86 (found by the equivalence harness of section 1). Happy to share any of the harnesses or numbers.

madebyjordan referenced this issue

2026-11-14 07:25:55 +00:00

perf(build): compile giant lifted functions in pieces cut at their labels, on by default (LONGSHOT_SPLIT_GIANTS) #385

madebyjordan referenced this issue

2026-11-14 08:07:53 +00:00

fix(vk): seed_keys is filled only for RX_VK_SEED_CHECK, its one reader #393

madebyjordan commented

2026-11-14 11:56:28 +00:00

Author

Contributor

How big the hot list needs to be: one 100-minute play session

To size section 1's hot list, I recorded which lifted functions run in a real session: nightly-130 (0caf0618) on Windows (Ryzen 5 5600, RX 6700 XT), played by a person for 100 minutes from boot, about 2 minutes to reach play. A small DLL marks each function the first time it runs. It's the coverage tool on the local branch bold/cover, not pushed. Compile costs use the build's own model (nx2/chunks.py, size + size²/350000), scaled to the 6,478 -O1 compiler-seconds of nightly-34's first build. The C sizes are from a recent main lift, not nightly-130's own.

	functions	lifted C	est. -O1 compiler-s
the whole game	295,270	1,376 MB	6,478
ran in the 100 minutes	77,725 (26%)	570 MB (41%)	~2,500 (39%)
ran by the title screen (29 s)	34,520	206 MB	~840
ran by the time play started (108 s)	39,326	235 MB	~940
ran in the first 10 minutes	59,315	463 MB	~2,120
ran in the first 30 minutes	68,959	529 MB	~2,360
the current hot list	3,682	171 MB	~1,010
the current hot list + all that ran by the start of play	41,457	354 MB	~1,710

What it says:

About 60% of the compile is for code that didn't run once in 100 minutes. That's the cold tier's case.
The current hot list is accurate but narrow: 3,644 of its 3,682 functions ran (99%), but it holds only 4.7% of the functions that ran (38% of their host code). The other ~74,000 would run as threaded code, which fits the slower loads measured in section 1 (12.0 → 18.2 s, 23.9 → 30.8 s).
Those loads can probably be bought back: adding everything that ran before play started costs ~700 compiler-seconds, about 3 minutes at 4 compilers (an estimate; not measured as a build or a load). Everyone boots to the title and starts or loads a file, so that part shouldn't depend on the route.
After the first two minutes, new code comes in small amounts, except for one burst 4m23s in: 6,125 new functions in 10 s, including several of the giants. Otherwise the rest of the first 10 minutes adds ~20,000 functions, and the next 90 minutes add ~18,400.

The limits: one player, one route, one session. Later play and other routes will reach code this didn't. A shipped hot list would want several sessions merged, from boot to a few areas in.

## How big the hot list needs to be: one 100-minute play session To size section 1's hot list, I recorded which lifted functions run in a real session: nightly-130 (0caf0618) on Windows (Ryzen 5 5600, RX 6700 XT), played by a person for 100 minutes from boot, about 2 minutes to reach play. A small DLL marks each function the first time it runs. It's the coverage tool on the local branch bold/cover, not pushed. Compile costs use the build's own model (`nx2/chunks.py`, size + size²/350000), scaled to the 6,478 -O1 compiler-seconds of nightly-34's first build. The C sizes are from a recent main lift, not nightly-130's own. | | functions | lifted C | est. -O1 compiler-s | |---|---|---|---| | the whole game | 295,270 | 1,376 MB | 6,478 | | ran in the 100 minutes | 77,725 (26%) | 570 MB (41%) | ~2,500 (39%) | | ran by the title screen (29 s) | 34,520 | 206 MB | ~840 | | ran by the time play started (108 s) | 39,326 | 235 MB | ~940 | | ran in the first 10 minutes | 59,315 | 463 MB | ~2,120 | | ran in the first 30 minutes | 68,959 | 529 MB | ~2,360 | | the current hot list | 3,682 | 171 MB | ~1,010 | | the current hot list + all that ran by the start of play | 41,457 | 354 MB | ~1,710 | What it says: - **About 60% of the compile is for code that didn't run once in 100 minutes.** That's the cold tier's case. - **The current hot list is accurate but narrow:** 3,644 of its 3,682 functions ran (99%), but it holds only 4.7% of the functions that ran (38% of their host code). The other ~74,000 would run as threaded code, which fits the slower loads measured in section 1 (12.0 → 18.2 s, 23.9 → 30.8 s). - **Those loads can probably be bought back:** adding everything that ran before play started costs ~700 compiler-seconds, about 3 minutes at 4 compilers (an estimate; not measured as a build or a load). Everyone boots to the title and starts or loads a file, so that part shouldn't depend on the route. - **After the first two minutes, new code comes in small amounts**, except for one burst 4m23s in: 6,125 new functions in 10 s, including several of the giants. Otherwise the rest of the first 10 minutes adds ~20,000 functions, and the next 90 minutes add ~18,400. The limits: one player, one route, one session. Later play and other routes will reach code this didn't. A shipped hot list would want several sessions merged, from boot to a few areas in.

madebyjordan referenced this issue

2026-11-14 12:00:38 +00:00

WIP: perf(build): lifted chunks of ~4 MB, by guest address, besides chunks.txt's boundaries #31

madebyjordan referenced this issue

2026-11-14 12:08:09 +00:00

chore(scripts): cover.py records which lifted functions a play session runs, and when #419