Build: cleanups after #728 (one free-memory reader, one function-start pattern, no behaviour change) #735

Merged

hckrman101 merged 7 commits from build-speed-cleanup into main

2026-11-08 21:37:42 +00:00

hckrman101 commented

2026-11-08 19:11:53 +00:00

(Migrated from github.com)

Follow-up to #728: tidy-ups, no behaviour change.

This also fixes the Windows test failures #728 introduced on main ("Python source checks (windows-2022)": test_chunk_split, test_mem_gate). On Windows, Path.write_text writes \r\n, so the fixtures differed from the lifted C, which is always written with \n:

The old memory estimate found no definitions, because it wanted a line ending in {.
The chunk test's sizes moved by a byte per line, so the part after the giant function fell just under its floor.

The second commit makes the fixtures write \n, and lets a note line above a function end in \r\n. I checked both by simulating CRLF writes locally.

This branch also merges main. Main's own fix for the same two tests is kept, except that test_chunk_split reads its parts without read_text(newline=), which Python 3.12 (CI's version) doesn't have.

With main's repaired workflow (#760) merged in, CI passes on Ubuntu and Windows.

One free-memory reader. make_game.default_jobs read the free memory only to print it, now that the memory gate decides. That read and its second copy of the Windows MEMORYSTATUSEX structure are gone, and build.free_mb is the one reader.
The "build: up to N compilers" line says the count once: up to 24 compilers at once (one per CPU thread), as the memory allows.
One pattern for a lifted function's first line, in the small new module nx2/lifted_c.py. It replaces three copies:
- the hand loop in build.mem_need, through a bytes version of the pattern;
- chunks.HEAD, which keeps the optional /* ... */ line above a function;
- split_hot.HEAD, which captures the name.
- It lives in its own module so build.py doesn't have to import the lifter.
COMPILER_MB: the 1.5 GB counted for one compiler is now one constant, used for a hot piece's need and for the countdown's estimate (MemoryGate.expected).
threading and the Windows ctypes structures are now at module level in build.py.
chunks.write uses nonlocal instead of a one-item dict.
write_lift_key uses "\n".
lift_key no longer inserts into sys.path on every call: make_game already sets it once at import.
The split_hot.py docstring is wrapped to the file's width, and the measured machine is called "a 24-thread desktop" in comments.

No behaviour change. I checked this against the C from a real lift (81 files, including code_hot.c.whole): mem_need, the function starts that chunks and split_hot find, and chunks.text_cost give the same results as before. Only the lift key changes, because nx2/ changed, so the first build after this lifts once more.

Tests

python -I -B tests/run_all.py --strict: 87 passed, 1 failed, 4 skipped. Plain main gives the same result here. The one failure is test_make_game_cache.py, which fails on this machine with or without this change.
test_build_times, test_linux_builder_jobs (updated: default_jobs no longer reads the memory), test_mem_gate, test_chunk_split, test_parallel_emit and test_lift_cache pass.
A short build (the lift and the first objects) matches the same build before this change: same chunk count, same @@plan/@@eta, longest file first.

Follow-up to #728: tidy-ups, no behaviour change. This also fixes the Windows test failures #728 introduced on main ("Python source checks (windows-2022)": `test_chunk_split`, `test_mem_gate`). On Windows, `Path.write_text` writes `\r\n`, so the fixtures differed from the lifted C, which is always written with `\n`: - The old memory estimate found no definitions, because it wanted a line ending in `{`. - The chunk test's sizes moved by a byte per line, so the part after the giant function fell just under its floor. The second commit makes the fixtures write `\n`, and lets a note line above a function end in `\r\n`. I checked both by simulating CRLF writes locally. This branch also merges main. Main's own fix for the same two tests is kept, except that `test_chunk_split` reads its parts without `read_text(newline=)`, which Python 3.12 (CI's version) doesn't have. With main's repaired workflow (#760) merged in, CI passes on Ubuntu and Windows. - **One free-memory reader.** `make_game.default_jobs` read the free memory only to print it, now that the memory gate decides. That read and its second copy of the Windows `MEMORYSTATUSEX` structure are gone, and `build.free_mb` is the one reader. - **The "build: up to N compilers" line** says the count once: `up to 24 compilers at once (one per CPU thread), as the memory allows`. - **One pattern for a lifted function's first line**, in the small new module `nx2/lifted_c.py`. It replaces three copies: - the hand loop in `build.mem_need`, through a bytes version of the pattern; - `chunks.HEAD`, which keeps the optional `/* ... */` line above a function; - `split_hot.HEAD`, which captures the name. - It lives in its own module so `build.py` doesn't have to import the lifter. - **`COMPILER_MB`**: the 1.5 GB counted for one compiler is now one constant, used for a hot piece's need and for the countdown's estimate (`MemoryGate.expected`). - `threading` and the Windows `ctypes` structures are now at module level in `build.py`. - `chunks.write` uses `nonlocal` instead of a one-item dict. - `write_lift_key` uses `"\n"`. - `lift_key` no longer inserts into `sys.path` on every call: `make_game` already sets it once at import. - The `split_hot.py` docstring is wrapped to the file's width, and the measured machine is called "a 24-thread desktop" in comments. **No behaviour change.** I checked this against the C from a real lift (81 files, including `code_hot.c.whole`): `mem_need`, the function starts that `chunks` and `split_hot` find, and `chunks.text_cost` give the same results as before. Only the lift key changes, because `nx2/` changed, so the first build after this lifts once more. **Tests** - `python -I -B tests/run_all.py --strict`: 87 passed, 1 failed, 4 skipped. Plain main gives the same result here. The one failure is `test_make_game_cache.py`, which fails on this machine with or without this change. - `test_build_times`, `test_linux_builder_jobs` (updated: `default_jobs` no longer reads the memory), `test_mem_gate`, `test_chunk_split`, `test_parallel_emit` and `test_lift_cache` pass. - A short build (the lift and the first objects) matches the same build before this change: same chunk count, same `@@plan`/`@@eta`, longest file first.

Commenting is not possible because the repository is archived.

No reviewers