build: complete Phase 1 build unblockers #527

Closed

nicholasgower wants to merge 20 commits from phase1 into main

nicholasgower commented

2026-11-08 08:35:38 +00:00

(Migrated from github.com)

Summary

Completes the Phase 1 - Build unblockers code work from ROADMAP.md. It is stacked on #518 (the roadmap PR), which should be reviewed and merged first. This branch is rebased directly onto origin/main at 3601726; it does not modify #518 or #522.

Phase 1 makes a clean build predictable again, gives a failed build an actionable message, and removes the 7-10 hour compile tail. No game code or generated game content is included.

What is included

Worker count: #505 makes the Linux builder honour HOOKSHOT_JOBS; #517 adds the Windows Build workers choice. --jobs and HOOKSHOT_JOBS remain the CLI overrides.
Build scheduling: #429 cuts chunks of giant functions into parts of about equal compile cost and compiles the longest first; #428 emits the lift in parallel and skips it when its inputs are unchanged.
Installer and updates: #514 retains installed and partial builder versions so a build started by another version can finish; #331 tracks tag plus source revision, verifies the archive and binds reused GUI metadata to it.
Failure diagnostics: #499 keeps the child exit status and signal (already merged upstream and kept here); #434 reports a missing compiler or Python through the builder protocol instead of "the game files could not be read"; #453 adds --log; #469's staging-permission explanation is kept where it is not already in main.
Progress: nx2/build.py now emits @@progress weighted by each file's estimated compile cost, so a 20-minute file no longer leaves the ring at 95% for minutes. The file count remains visible.
Portable paths: #451 launches a game from a relative folder.
CPU target: #414 reports the AppImage's AVX2 requirement before a 40-minute build.
Docs and roadmap: the readme explains workers, stopped builds, --log, the no-update-during-build rule and Defender false positives; the Phase 1 section of ROADMAP.md records what landed.

Rebase notes

The branch was rebased from the old roadmap base onto current origin/main:

#499 and #469 were already represented in main, so their duplicate merge work was dropped.
#505/#517 were reconciled with main's newer worker-policy hotfix; the explicit overrides and Windows UI remain.
#331 needed its final files restored after the PR's own merge commits were flattened: BuilderDownload.cs, SetupManifest.cs, UpdatePreparation.cs, UpdateRequest.cs, the final identity/archive/contents/updater files, and the final make_setup_parts.py.
MainWindow now reads Tools.Identity.Tag after #331 removed the old Version property.

Checks

Passed after the rebase:

Phase 1 Python tests: build log, failure reason, Linux jobs, chunk split, lift cache, parallel emit, finish output, CPU target, builder Python, missing compiler, CPM discovery, builder status, child Unicode, release identity and version
dotnet build for HookshotBuilder
Windows builder worker tests (69 assertions)
Update-identity .NET tests
check_update_identity_c.py with MSVC, including host compilation
test_setup_pruning.py with MSVC
Workflow and issue-template YAML parsing, and git diff --check

What the owner still needs to do for Phase 1

Run a full build on the reference hardware and record the wall time and worker count: 16-thread/32 GB under 60 minutes, 8-thread/16 GB under 90 minutes.
Submit the current Windows binary to Microsoft and sign the installer when the signing identity is available (#433, #393).
Persist per-stage timings for a previous-build ETA (#491).
Collect the exact file/log for #501, #507, #457 and #478 before changing path or ADR handling.
Decide whether staging-path reuse should be added on top of #514/#331.

Previous roadmap PR: #518.

## Summary Completes the **Phase 1 - Build unblockers** code work from [ROADMAP.md](https://git.magicbean.dev/MagicBeanDev/Hookshot/src/branch/main/ROADMAP.md). It is stacked on **#518** (the roadmap PR), which should be reviewed and merged first. This branch is rebased directly onto `origin/main` at `3601726`; it does not modify #518 or #522. Phase 1 makes a clean build predictable again, gives a failed build an actionable message, and removes the 7-10 hour compile tail. No game code or generated game content is included. ## What is included - **Worker count:** #505 makes the Linux builder honour `HOOKSHOT_JOBS`; #517 adds the Windows **Build workers** choice. `--jobs` and `HOOKSHOT_JOBS` remain the CLI overrides. - **Build scheduling:** #429 cuts chunks of giant functions into parts of about equal compile cost and compiles the longest first; #428 emits the lift in parallel and skips it when its inputs are unchanged. - **Installer and updates:** #514 retains installed and partial builder versions so a build started by another version can finish; #331 tracks tag plus source revision, verifies the archive and binds reused GUI metadata to it. - **Failure diagnostics:** #499 keeps the child exit status and signal (already merged upstream and kept here); #434 reports a missing compiler or Python through the builder protocol instead of "the game files could not be read"; #453 adds `--log`; #469's staging-permission explanation is kept where it is not already in main. - **Progress:** `nx2/build.py` now emits `@@progress` weighted by each file's estimated compile cost, so a 20-minute file no longer leaves the ring at 95% for minutes. The file count remains visible. - **Portable paths:** #451 launches a game from a relative folder. - **CPU target:** #414 reports the AppImage's AVX2 requirement before a 40-minute build. - **Docs and roadmap:** the readme explains workers, stopped builds, `--log`, the no-update-during-build rule and Defender false positives; the Phase 1 section of `ROADMAP.md` records what landed. ## Rebase notes The branch was rebased from the old roadmap base onto current `origin/main`: - #499 and #469 were already represented in main, so their duplicate merge work was dropped. - #505/#517 were reconciled with main's newer worker-policy hotfix; the explicit overrides and Windows UI remain. - #331 needed its final files restored after the PR's own merge commits were flattened: `BuilderDownload.cs`, `SetupManifest.cs`, `UpdatePreparation.cs`, `UpdateRequest.cs`, the final identity/archive/contents/updater files, and the final `make_setup_parts.py`. - `MainWindow` now reads `Tools.Identity.Tag` after #331 removed the old `Version` property. ## Checks Passed after the rebase: - Phase 1 Python tests: build log, failure reason, Linux jobs, chunk split, lift cache, parallel emit, finish output, CPU target, builder Python, missing compiler, CPM discovery, builder status, child Unicode, release identity and version - `dotnet build` for `HookshotBuilder` - Windows builder worker tests (69 assertions) - Update-identity .NET tests - `check_update_identity_c.py` with MSVC, including host compilation - `test_setup_pruning.py` with MSVC - Workflow and issue-template YAML parsing, and `git diff --check` ## What the owner still needs to do for Phase 1 1. Run a full build on the reference hardware and record the wall time and worker count: 16-thread/32 GB under 60 minutes, 8-thread/16 GB under 90 minutes. 2. Submit the current Windows binary to Microsoft and sign the installer when the signing identity is available (#433, #393). 3. Persist per-stage timings for a previous-build ETA (#491). 4. Collect the exact file/log for #501, #507, #457 and #478 before changing path or ADR handling. 5. Decide whether staging-path reuse should be added on top of #514/#331. Previous roadmap PR: **#518**.

NotRexed commented

2026-11-09 05:29:43 +00:00

(Migrated from github.com)

Thank you for the roll-up! Most of its parts are on main now (the lift key, the build worker choice, the chunk split). Closing to keep the queue clear — please rebase on current main and reopen; thanks! Smaller PRs with what is still new would be easiest to review.

Commenting is not possible because the repository is archived.

No reviewers