MagicBeanDev/Longshot

Watch

Fork 98

fix(build): one build at a time in a work folder, so a second build doesn't compile again what the first is compiling #630

Merged

HumanBean merged 2 commits from HumanBean/Longshot:fix/work-lock into main

2026-11-15 03:15:18 +00:00

HumanBean commented

2026-11-15 01:15:26 +00:00

Owner

Type: fix · Risk: low · Tested on: Linux, i7-9700K lab · Needs: Windows: a second build started while one runs

Why

Two builds could run in one work folder at the same time. The second finds the first's lift done, compiles again every object the first hasn't cached yet, and then the two race over the game folder's files. That happened on a player's PC: a second build started about 7.5 minutes into a first one and compiled 197 of 480 files over again (451 s). On the lab, the second build then crashed renaming game/longshot.new, which the first had already moved. A normal update compiles none of the lifted code (below), so this was the only cause found for that rebuild.

How to reproduce

Seen on: Linux (a player's PC through the launcher; reproduced on the lab)
Settings and switches: defaults
Steps: start python -I nx2/make_game.py ... --work <w>, and while it compiles start a second one with the same --work <w>.
Expected: the second stops at once, saying a build is using the folder.
Actual: the second build compiles what the first hasn't cached yet. On the lab it then stopped with FileNotFoundError: game/longshot.new -> game/longshot.

What changed

make_game.py now holds a lock on the work folder for the whole build, and a second build into it stops at once with "Another build is using the work folder … Wait for it to end (or stop it), then build again." The lock is an OS lock, not a marker file, so a stopped or killed build never keeps the next one out.

build: lock_work() in nx2/make_game.py holds <work>/make_game.lock (fcntl.flock on Linux, msvcrt.locking on Windows). It is called in main() right after work is set, before stop_stray_compilers, drop_stale_objects and the lift. make_game --check takes no lock. A second main() in the same process keeps the lock it holds (the tests do this).
tests: new tests/test_work_lock.py: a second build into a held folder stops, a dead holder frees the lock, and main() takes the lock before anything in the folder.

Measurements

Verdict: The bug is gone on the lab: with the fix, the second of two simultaneous updates was refused in 1 s. Without it, the second compiled 2 of the first's 3 files again and crashed.

Hardware: Linux, i7-9700K, LONGSHOT_JOBS=6, under the lab's build lock
Scene: none (the build)
Method: launcher-style updates. Two checkouts at different paths share one --work, and each job's key parts (source, flag text, deps, each header) were dumped and compared per file.
How to reproduce: the steps above

	Before: main @ `2b1d38c6` / `225c37f8`	After: this PR @ `a947cae3`
First build, `2b1d38c6`	375 of 488 compiled, compile 997 s	—
Update to `225c37f8` (`host/control.c` only), lift kept	3 of 488 compiled, 0 `code_*`, 29 s	—
Update with a comment-only `nx2/lift.py` edit (full re-lift, 104 s)	2 of 488 compiled, 0 `code_*`, 29 s	—
Two updates at once	second compiled 2 of the same files again, then crashed on `game/longshot.new`	first: 3 compiled (23 s); second refused in 1 s

Flags, headers and deps were identical between the builds. The native CPU tag was stable, and a fresh lift gave the same lifted-code fingerprint, so a normal update recompiles no code_* file.
The player's records (build_times.json): the two builds ended 19 s apart but took 950.9 s and 521.2 s. The second one's lift took 0.1 s, so it started while the first was compiling.

Flags and settings

None

Testing

Tested: on the lab (Linux), the runs above. tests/test_work_lock.py passes. The 52 tests/test_*.py that mention make_game fail identically with and without the patch: 9 also fail on main there, because the lab lacks --cc/--cxx arguments, dotnet and the like. Nothing was run on the author's PC.

Not tested: Windows (msvcrt.locking); the full tests/run_all.py.

Must be tested:

Windows, by hand: start a build from the Builder or make_game.py, start a second with the same work folder, and expect the "Another build is using the work folder" message. Then, once the first has ended, a build in the folder must run.

Risks

A tool that starts two make_game.py builds into one work folder on purpose now gets the second refused. Nothing in the tree does that. The lock file stays in the work folder (empty). Reverting the commit restores the old behaviour.

What started the second build on the player's PC isn't known. The launcher doesn't stop a build when its window closes, and a new build step always starts another. That is a launcher follow-up.

Checklist

Everyone

One topic.
The title is a Conventional Commit (type(scope): description) of the status line's type.
Why gives the reason plainly and links the issues that fit (the tracker searched).
Measurements has a verdict, or one line saying why there is nothing to measure.
Testing names what was tested, what wasn't, and what must still be tested, where and how.
Rebased on current main.
python -I -B tests/run_all.py passes (any failure that also fails on main is named). (not run: left to CI)
No game files, dumps, build output or generated files.
third_party/ untouched (a new folder with a PROVENANCE.md instead).
The license is OK: GPL-3.0-or-later, nothing incompatible and nothing decompiled.

When it applies

AI assistance recorded with Assisted-by: / Co-Authored-By: trailers.
A game-free test added for the fix, failing on main and passing here.

🤖 Generated with Claude Code

https://claude.ai/code/session_017vw8e3MfaADZobPGhWJk77

**Type:** fix · **Risk:** low · **Tested on:** Linux, i7-9700K lab · **Needs:** Windows: a second build started while one runs ## Why Two builds could run in one work folder at the same time. The second finds the first's lift done, compiles again every object the first hasn't cached yet, and then the two race over the game folder's files. That happened on a player's PC: a second build started about 7.5 minutes into a first one and compiled 197 of 480 files over again (451 s). On the lab, the second build then crashed renaming `game/longshot.new`, which the first had already moved. A normal update compiles none of the lifted code (below), so this was the only cause found for that rebuild. ## How to reproduce **Seen on:** Linux (a player's PC through the launcher; reproduced on the lab) **Settings and switches:** defaults **Steps:** start `python -I nx2/make_game.py ... --work <w>`, and while it compiles start a second one with the same `--work <w>`. **Expected:** the second stops at once, saying a build is using the folder. **Actual:** the second build compiles what the first hasn't cached yet. On the lab it then stopped with `FileNotFoundError: game/longshot.new -> game/longshot`. ## What changed `make_game.py` now holds a lock on the work folder for the whole build, and a second build into it stops at once with "Another build is using the work folder … Wait for it to end (or stop it), then build again." The lock is an OS lock, not a marker file, so a stopped or killed build never keeps the next one out. - build: `lock_work()` in `nx2/make_game.py` holds `<work>/make_game.lock` (`fcntl.flock` on Linux, `msvcrt.locking` on Windows). It is called in `main()` right after `work` is set, before `stop_stray_compilers`, `drop_stale_objects` and the lift. `make_game --check` takes no lock. A second `main()` in the same process keeps the lock it holds (the tests do this). - tests: new `tests/test_work_lock.py`: a second build into a held folder stops, a dead holder frees the lock, and `main()` takes the lock before anything in the folder. ## Measurements **Verdict:** The bug is gone on the lab: with the fix, the second of two simultaneous updates was refused in 1 s. Without it, the second compiled 2 of the first's 3 files again and crashed. Hardware: Linux, i7-9700K, `LONGSHOT_JOBS=6`, under the lab's build lock Scene: none (the build) Method: launcher-style updates. Two checkouts at different paths share one `--work`, and each job's key parts (source, flag text, deps, each header) were dumped and compared per file. How to reproduce: the steps above | | Before: main @ 2b1d38c6 / 225c37f8 | After: this PR @ a947cae3 | |---|---|---| | First build, `2b1d38c6` | 375 of 488 compiled, compile 997 s | — | | Update to `225c37f8` (`host/control.c` only), lift kept | 3 of 488 compiled, 0 `code_*`, 29 s | — | | Update with a comment-only `nx2/lift.py` edit (full re-lift, 104 s) | 2 of 488 compiled, 0 `code_*`, 29 s | — | | Two updates at once | second compiled 2 of the same files again, then crashed on `game/longshot.new` | first: 3 compiled (23 s); second refused in 1 s | - Flags, headers and deps were identical between the builds. The native CPU tag was stable, and a fresh lift gave the same lifted-code fingerprint, so a normal update recompiles no `code_*` file. - The player's records (`build_times.json`): the two builds ended 19 s apart but took 950.9 s and 521.2 s. The second one's lift took 0.1 s, so it started while the first was compiling. ### Flags and settings None ## Testing **Tested:** on the lab (Linux), the runs above. `tests/test_work_lock.py` passes. The 52 `tests/test_*.py` that mention `make_game` fail identically with and without the patch: 9 also fail on main there, because the lab lacks `--cc`/`--cxx` arguments, dotnet and the like. Nothing was run on the author's PC. **Not tested:** Windows (`msvcrt.locking`); the full `tests/run_all.py`. **Must be tested:** - Windows, by hand: start a build from the Builder or `make_game.py`, start a second with the same work folder, and expect the "Another build is using the work folder" message. Then, once the first has ended, a build in the folder must run. ## Risks A tool that starts two `make_game.py` builds into one work folder on purpose now gets the second refused. Nothing in the tree does that. The lock file stays in the work folder (empty). Reverting the commit restores the old behaviour. What started the second build on the player's PC isn't known. The launcher doesn't stop a build when its window closes, and a new build step always starts another. That is a launcher follow-up. ## Checklist ### Everyone - [x] One topic. - [x] The title is a Conventional Commit (`type(scope): description`) of the status line's type. - [x] Why gives the reason plainly and links the issues that fit (the tracker searched). - [x] Measurements has a verdict, or one line saying why there is nothing to measure. - [x] Testing names what was tested, what wasn't, and what must still be tested, where and how. - [x] Rebased on current `main`. - [ ] `python -I -B tests/run_all.py` passes (any failure that also fails on `main` is named). (not run: left to CI) - [x] No game files, dumps, build output or generated files. - [x] `third_party/` untouched (a new folder with a `PROVENANCE.md` instead). - [x] The license is OK: GPL-3.0-or-later, nothing incompatible and nothing decompiled. ### When it applies - [x] AI assistance recorded with `Assisted-by:` / `Co-Authored-By:` trailers. - [x] A game-free test added for the fix, failing on `main` and passing here. 🤖 Generated with [Claude Code](https://claude.com/claude-code) https://claude.ai/code/session_017vw8e3MfaADZobPGhWJk77

HumanBean added 1 commit

2026-11-15 01:15:26 +00:00

fix(build): one build at a time in a work folder, so a second build doesn't compile again what the first is compiling

CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Has been skipped

Details

CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Has been skipped

Details

CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Has been skipped

Details

Third-party parts / The same as upstream (pull_request) Has been skipped

Details

CI / Conventional Commit title (pull_request) Successful in 3s

Details

CI / Workflow lint (actionlint) (pull_request) Successful in 6s

Details

Third-party parts / As their notes say (pull_request) Successful in 8s

Details

PR description / Description follows the template (pull_request_target) Has been cancelled

Details

a947cae37c

Two make_game.py runs in one --work folder shared its lift, object cache and
lists: a player's second build, started 7.5 minutes after the first, found the
lift done and compiled again the 197 of 480 files the first had not cached yet
(451 s), and the two then raced over the game folder's files (on the lab the
second stopped renaming longshot.new, which the first had moved). make_game.py
now holds a lock on <work>/make_game.lock for the whole build (flock on Linux,
msvcrt.locking on Windows) and a second build stops at once, saying another
build is using the folder. The lock goes with the process, however it ended.
make_game --check takes no lock.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_017vw8e3MfaADZobPGhWJk77

HumanBean changed title from ~~WIP: fix(build): one build at a time in a work folder, so a second build doesn't compile again what the first is compiling~~ to fix(build): one build at a time in a work folder, so a second build doesn't compile again what the first is compiling

2026-11-15 03:14:59 +00:00

HumanBean added 1 commit

2026-11-15 03:15:07 +00:00

Merge branch 'main' into fix/work-lock

CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Has been skipped

Details

CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Has been skipped

Details

CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Has been skipped

Details

Third-party parts / The same as upstream (pull_request) Has been skipped

Details