MagicBeanDev/Longshot

Watch

Fork 71

fix(setup): a failed update gives the installed Builder its tools back #447

Open

jarynclouatre wants to merge 1 commit from jarynclouatre/Longshot:fix/setup-reused-tools-back into main

jarynclouatre commented

2026-11-14 14:29:48 +00:00

Contributor

Why

On an ordinary (not portable) Builder install, an update that fails part way (a download or unpack error, or the setup closed) leaves the installed Builder without its tools: it can no longer build, and the next update downloads them all again (llvm-mingw alone is 191 MB).

reuse_part in builder/setup/setup.c moves the folders of the parts that did not change (Python, llvm-mingw, emsdk) out of the installed version into versions\<tag>.part (if (!g_home.portable) return MoveFileW(from, to) != 0;). Nothing moves them back when the new version is not published, and the next start removes the staging folder with them (rmtree(install) in work).

What changed

setup (builder/setup/setup.c): the move stays (it keeps an update small and quick).
- note_reused: reuse_part names each folder in the staging folder's reused.txt before it moves it. A setup stopped between the two leaves a name whose folder is still in the installed version, and nothing moves. When the name cannot be written the part is downloaded instead.
- restore_reused: moves the folders named in reused.txt back into the installed version (current.txt), only where that version has none, so a part this attempt downloaded and unpacked (perhaps half) never goes into it. Names are checked with list_name_ok, as the setup list's are.
- fail calls it once the staging has begun (g_staged), and work calls it before it removes an old staging folder, which covers a setup that was closed. After the version is published reused.txt is deleted from it.
- Portable homes copy as before and write no list.
tests (tests/test_setup_tool_merge.py): case 5 runs the real reuse_part and restore_reused over a POSIX stand-in: an ordinary update moves an unchanged Python folder over and lists it; (a) after a failure the installed version has it again, bytes the same, and the staging has none; (b) a second attempt moves it over again, no download; (c) a stopped setup: the next start's restore and removal of the staging leave the installed version whole; (d) a folder this attempt unpacked, not listed, stays in the staging; (e) a listed folder whose move failed is not replaced; (f) portable still copies and lists nothing. The stand-in's CreateFileW gained an append mode and WriteFile; the extractor skips a declaration ahead of its definition.
tests, Windows-only fixtures kept in step: tests/test_portable_setup_transaction.py extracts note_reused with reuse_part and checks the ordinary move's reused.txt; tests/test_setup_runtime_paths.py declares g_staged for the publication block it extracts.

Flags and settings

None.

Measurements

main: 5e072fc8 (2026-11-14)
this PR: fix/setup-reused-tools-back @ 80f44a4f
Hardware: none used (game-free checks on a Linux x86-64 PC, gcc 15.2, Python 3.14)
Scene: none, the game was not run
Method: python3 -I -B tests/run_all.py, two runs each, interleaved (main, PR, the other branch, twice). Case 5 also runs against main's setup.c, with no-op stand-ins for the two new functions.
How to reproduce: the commands above in a checkout of each commit

	main @ `5e072fc8`	this PR @ `80f44a4f`
`tests/run_all.py` (run 1, run 2)	474 passed, 4 failed, 83 skipped (both)	474 passed, 4 failed, 83 skipped (both)
failing checks	`test_bake_portability.py`, `test_menu_keys.py`, `test_module_ids.py`, `test_pad_layout.py`	the same four
`tests/test_setup_tool_merge.py` case 5	fails: `after a failed update the installed version has no Python`	passes

test_bake_portability.py fails here for a missing Python module (zstandard). test_menu_keys.py doesn't build with g++ and test_pad_layout.py's fixture lacks host/hold_toggle.h, both on main since #137 (#331 and #338 fix them). test_module_ids.py looks for a source marker that main no longer has.

Testing

Tested: the game-free suite on a Linux PC, as in Measurements. setup.c compiles (-fsyntax-only -Wall -Wextra) with llvm-mingw 20261104's x86_64-w64-mingw32-clang with the same 6 warnings as main's. The Windows-only setup checks (test_portable_setup_transaction, test_setup_runtime_paths, test_portable_setup_start, test_setup_runtime, test_portable_setup_paths, test_setup_tar_diagnostics) and the Windows variant of test_setup_tool_merge's fixture build with that compiler (-Werror).

Not tested: those Windows-only checks were built but not run (no Windows PC for them here); the setup itself on Windows: an update of an ordinary install with the network cut after the first parts.

Needs other testers: None.

Screenshots

Nothing on screen.

Risk and rollback

A setup that was closed (not failed) gives the folders back at its next start, before anything is downloaded; the installed Builder started on its own before then still lacks them. A failed move back leaves the folder in the staging, where the next start tries again before removing it (as before this change, it would then be downloaded). Reverting the commit is enough.

Issues

None filed.

Checklist

For everyone

One topic.
The title is a Conventional Commit (type(scope): description).
Rebased on current main.
python -I -B tests/run_all.py run and passing (any failure that also fails on main is named above).
Tested in the game, or what wasn't is named under Testing.
Testing filled in: what was tested, what wasn't, and what needs other testers (with test requests).
Measured: main and this PR at named commits, on the same PC, scene and settings, at least 2 interleaved runs each, the median reported.
No game files, dumps, build output or generated files.
third_party/ untouched (a new folder with a PROVENANCE.md instead).
The license is OK: GPL-3.0-or-later, nothing incompatible and nothing decompiled.

If applicable

A game-free test added for the fix, failing on main and passing here.
A check under tests/ that reads changed source updated in this PR, not weakened.

## Why On an ordinary (not portable) Builder install, an update that fails part way (a download or unpack error, or the setup closed) leaves the installed Builder without its tools: it can no longer build, and the next update downloads them all again (llvm-mingw alone is 191 MB). `reuse_part` in `builder/setup/setup.c` moves the folders of the parts that did not change (Python, llvm-mingw, emsdk) out of the installed version into `versions\<tag>.part` (`if (!g_home.portable) return MoveFileW(from, to) != 0;`). Nothing moves them back when the new version is not published, and the next start removes the staging folder with them (`rmtree(install)` in `work`). ## What changed - setup (`builder/setup/setup.c`): the move stays (it keeps an update small and quick). - `note_reused`: `reuse_part` names each folder in the staging folder's `reused.txt` before it moves it. A setup stopped between the two leaves a name whose folder is still in the installed version, and nothing moves. When the name cannot be written the part is downloaded instead. - `restore_reused`: moves the folders named in `reused.txt` back into the installed version (`current.txt`), only where that version has none, so a part this attempt downloaded and unpacked (perhaps half) never goes into it. Names are checked with `list_name_ok`, as the setup list's are. - `fail` calls it once the staging has begun (`g_staged`), and `work` calls it before it removes an old staging folder, which covers a setup that was closed. After the version is published `reused.txt` is deleted from it. - Portable homes copy as before and write no list. - tests (`tests/test_setup_tool_merge.py`): case 5 runs the real `reuse_part` and `restore_reused` over a POSIX stand-in: an ordinary update moves an unchanged Python folder over and lists it; (a) after a failure the installed version has it again, bytes the same, and the staging has none; (b) a second attempt moves it over again, no download; (c) a stopped setup: the next start's restore and removal of the staging leave the installed version whole; (d) a folder this attempt unpacked, not listed, stays in the staging; (e) a listed folder whose move failed is not replaced; (f) portable still copies and lists nothing. The stand-in's `CreateFileW` gained an append mode and `WriteFile`; the extractor skips a declaration ahead of its definition. - tests, Windows-only fixtures kept in step: `tests/test_portable_setup_transaction.py` extracts `note_reused` with `reuse_part` and checks the ordinary move's `reused.txt`; `tests/test_setup_runtime_paths.py` declares `g_staged` for the publication block it extracts. ## Flags and settings None. ## Measurements main: 5e072fc8 (2026-11-14) this PR: fix/setup-reused-tools-back @ 80f44a4f Hardware: none used (game-free checks on a Linux x86-64 PC, gcc 15.2, Python 3.14) Scene: none, the game was not run Method: `python3 -I -B tests/run_all.py`, two runs each, interleaved (main, PR, the other branch, twice). Case 5 also runs against main's `setup.c`, with no-op stand-ins for the two new functions. How to reproduce: the commands above in a checkout of each commit | | main @ 5e072fc8 | this PR @ 80f44a4f | |---|---|---| | `tests/run_all.py` (run 1, run 2) | 474 passed, 4 failed, 83 skipped (both) | 474 passed, 4 failed, 83 skipped (both) | | failing checks | `test_bake_portability.py`, `test_menu_keys.py`, `test_module_ids.py`, `test_pad_layout.py` | the same four | | `tests/test_setup_tool_merge.py` case 5 | fails: `after a failed update the installed version has no Python` | passes | `test_bake_portability.py` fails here for a missing Python module (zstandard). `test_menu_keys.py` doesn't build with g++ and `test_pad_layout.py`'s fixture lacks `host/hold_toggle.h`, both on main since #137 (#331 and #338 fix them). `test_module_ids.py` looks for a source marker that main no longer has. ## Testing **Tested:** the game-free suite on a Linux PC, as in Measurements. `setup.c` compiles (`-fsyntax-only -Wall -Wextra`) with llvm-mingw 20261104's x86_64-w64-mingw32-clang with the same 6 warnings as main's. The Windows-only setup checks (`test_portable_setup_transaction`, `test_setup_runtime_paths`, `test_portable_setup_start`, `test_setup_runtime`, `test_portable_setup_paths`, `test_setup_tar_diagnostics`) and the Windows variant of `test_setup_tool_merge`'s fixture build with that compiler (`-Werror`). **Not tested:** those Windows-only checks were built but not run (no Windows PC for them here); the setup itself on Windows: an update of an ordinary install with the network cut after the first parts. **Needs other testers:** None. ## Screenshots Nothing on screen. ## Risk and rollback A setup that was closed (not failed) gives the folders back at its next start, before anything is downloaded; the installed Builder started on its own before then still lacks them. A failed move back leaves the folder in the staging, where the next start tries again before removing it (as before this change, it would then be downloaded). Reverting the commit is enough. ## Issues None filed. ## Checklist ### For everyone - [x] One topic. - [x] The title is a Conventional Commit (`type(scope): description`). - [x] Rebased on current `main`. - [x] `python -I -B tests/run_all.py` run and passing (any failure that also fails on `main` is named above). - [x] Tested in the game, or what wasn't is named under Testing. - [x] Testing filled in: what was tested, what wasn't, and what needs other testers (with test requests). - [x] Measured: main and this PR at named commits, on the same PC, scene and settings, at least 2 interleaved runs each, the median reported. - [x] No game files, dumps, build output or generated files. - [x] `third_party/` untouched (a new folder with a `PROVENANCE.md` instead). - [x] The license is OK: GPL-3.0-or-later, nothing incompatible and nothing decompiled. ### If applicable - [x] A game-free test added for the fix, failing on `main` and passing here. - [x] A check under `tests/` that reads changed source updated in this PR, not weakened.

jarynclouatre added 1 commit

2026-11-14 14:29:48 +00:00

fix(setup): a failed update gives the installed Builder its tools back

CI / Conventional Commit title (pull_request) Blocked by required conditions

Details

CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Blocked by required conditions

Details

CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Blocked by required conditions

Details

CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Blocked by required conditions

Details

CI / Workflow lint (actionlint) (pull_request) Blocked by required conditions

Details

Third-party parts / As their notes say (pull_request) Blocked by required conditions

Details

Third-party parts / The same as upstream (pull_request) Blocked by required conditions

Details

PR description / Description follows the template (pull_request_target) Successful in 4s

Details

Website / Website on Cloudflare (pull_request_target) Has been skipped

Details

80f44a4f38

On an ordinary (not portable) install, reuse_part moves the folders of
the parts that did not change (Python, llvm-mingw, emsdk) out of the
installed version into versions\<tag>.part. When a later part's download
or unpack failed, or the setup was closed, nothing moved them back: the
installed Builder had lost its tools, and the next start deleted the
moved copies with the staging folder, so they were downloaded again
(llvm-mingw alone is 191 MB).

reuse_part now names each folder in the staging folder's reused.txt
before it moves it. restore_reused moves the named folders back into the
installed version, only where it has none, so a part this attempt
downloaded and unpacked never goes into it. fail() calls it once the
staging has begun, and work() calls it before removing an old staging
folder, which covers a setup that was closed. The move itself stays, and
portable homes still copy.

jarynclouatre changed title from ~~WIP: fix(setup): a failed update gives the installed Builder its tools back~~ to fix(setup): a failed update gives the installed Builder its tools back

2026-11-14 14:31:42 +00:00