feat(build): Longshot's object and lift stores in the build files folder, so switching versions reuses earlier builds' work #72

Merged

HumanBean merged 2 commits from HumanBean/LongshotLauncher:build/stores into main 2026-11-15 08:06:17 +00:00

Conversation 2 Commits 2 Files changed 6 +151 -6

HumanBean commented 2026-11-14 23:38:27 +00:00

Owner

• Copy link
• Copy Markdown

Switching between builds (stable and nightly, or going back a version) compiles almost the whole game again, because after each build the work folder keeps only that build's compiled objects. Longshot's object and lift stores (MagicBeanDev/Longshot#371, MagicBeanDev/Longshot#372) keep earlier builds' objects and translations; this turns them on for launcher builds.

What changed

• Core/MakeGameCommand.cs For, both systems: LONGSHOT_OBJ_STORE=<work>/obj-store and LONGSHOT_LIFT_STORE=<work>/lift-store, where <work> is the player's chosen build files folder or the launcher's own. Inside the work folder, so always on its drive (the objects are hard links there; a copy across drives).
  • LONGSHOT_OBJ_STORE_GB=4 (Longshot's default is 8) and LONGSHOT_LIFT_STORE_KEEP=2 (default 3); Longshot's 14-day expiry is kept.
  • Tools from before the stores (before nightly-144) read none of these variables.
• Core/Disk.cs: obj-store and lift-store are in WorkParts, so Free up space counts and clears them. WorkBytes counts an object that is in both objcache and the store once (one hard-linked file), matched by name in the store's folder for the work folder's system_headers.txt, as nx2/object_store.py folder names it.
• Tests: tests/launcher_core/Program.cs checks the variables on Windows (default work folder) and Linux (a chosen one); ScreenChecks.cs has a Free up space case with both stores (an object in both counted once, another compiler's counted, both stores deleted, the player's other files kept).
• README: the stores and what they cost; Free up space clears them.

Disk cost

• Objects: one a build still uses is the same file as objcache's, so it costs nothing. Objects no build holds go after 14 days, and oldest first past 4 GB, about four versions' own objects (a build's are ~1 GB).
• Translations: a kept one is ~1.6 GB (its lifted C whole, which the work folder itself deletes after a build). Two are kept: the version built and the one before. While a build saves a new one there are briefly three.
• So at most about 7 GB more than today once settled, all of it cleared by Free up space. Disk.BuildBytes stays 45 GB: what the stores keep is on the disk already, and a build adds at most one translation while it runs.
• The ~1 GB and ~1.6 GB are from make_game.py's comments, not measured.

Testing

Not compiled or run yet: neither dotnet build nor python -I tests/test_launcher_core.py (the author runs no checks on their PC, and the lab machine has no .NET). Checked by reading against Longshot's nx2/object_store.py, lift_store.py and make_game.py on main: make_game deletes only exefs, probe and sa after a build and trims objcache, so a store inside the work folder is left alone; the lift store's path is not part of the lift key.

Must be tested: python -I tests/test_launcher_core.py --strict --longshot <Longshot main> on Linux and Windows; a build, then a build of another version, then the first again on one PC: the third should compile almost nothing (its log's compile count) and translate nothing.

Risks

The disk use above. Reverting the commit turns the stores off; Free up space clears what they hold.

🤖 Generated with Claude Code

https://claude.ai/code/session_017vw8e3MfaADZobPGhWJk77

Switching between builds (stable and nightly, or going back a version) compiles almost the whole game again, because after each build the work folder keeps only that build's compiled objects. Longshot's object and lift stores (https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/371, https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/372) keep earlier builds' objects and translations; this turns them on for launcher builds. ## What changed - `Core/MakeGameCommand.cs` `For`, both systems: `LONGSHOT_OBJ_STORE=<work>/obj-store` and `LONGSHOT_LIFT_STORE=<work>/lift-store`, where `<work>` is the player's chosen build files folder or the launcher's own. Inside the work folder, so always on its drive (the objects are hard links there; a copy across drives). - `LONGSHOT_OBJ_STORE_GB=4` (Longshot's default is 8) and `LONGSHOT_LIFT_STORE_KEEP=2` (default 3); Longshot's 14-day expiry is kept. - Tools from before the stores (before nightly-144) read none of these variables. - `Core/Disk.cs`: `obj-store` and `lift-store` are in `WorkParts`, so Free up space counts and clears them. `WorkBytes` counts an object that is in both `objcache` and the store once (one hard-linked file), matched by name in the store's folder for the work folder's `system_headers.txt`, as `nx2/object_store.py` `folder` names it. - Tests: `tests/launcher_core/Program.cs` checks the variables on Windows (default work folder) and Linux (a chosen one); `ScreenChecks.cs` has a Free up space case with both stores (an object in both counted once, another compiler's counted, both stores deleted, the player's other files kept). - README: the stores and what they cost; Free up space clears them. ## Disk cost - Objects: one a build still uses is the same file as `objcache`'s, so it costs nothing. Objects no build holds go after 14 days, and oldest first past 4 GB, about four versions' own objects (a build's are ~1 GB). - Translations: a kept one is ~1.6 GB (its lifted C whole, which the work folder itself deletes after a build). Two are kept: the version built and the one before. While a build saves a new one there are briefly three. - So at most about 7 GB more than today once settled, all of it cleared by Free up space. `Disk.BuildBytes` stays 45 GB: what the stores keep is on the disk already, and a build adds at most one translation while it runs. - The ~1 GB and ~1.6 GB are from `make_game.py`'s comments, not measured. ## Testing **Not compiled or run yet**: neither `dotnet build` nor `python -I tests/test_launcher_core.py` (the author runs no checks on their PC, and the lab machine has no .NET). Checked by reading against Longshot's `nx2/object_store.py`, `lift_store.py` and `make_game.py` on main: make_game deletes only `exefs`, `probe` and `sa` after a build and trims `objcache`, so a store inside the work folder is left alone; the lift store's path is not part of the lift key. **Must be tested:** `python -I tests/test_launcher_core.py --strict --longshot <Longshot main>` on Linux and Windows; a build, then a build of another version, then the first again on one PC: the third should compile almost nothing (its log's compile count) and translate nothing. ## Risks The disk use above. Reverting the commit turns the stores off; Free up space clears what they hold. 🤖 Generated with [Claude Code](https://claude.com/claude-code) https://claude.ai/code/session_017vw8e3MfaADZobPGhWJk77

HumanBean added 1 commit 2026-11-14 23:38:27 +00:00

feat(build): Longshot's object and lift stores in the build files folder, so switching versions reuses earlier builds' work ... d102db3724

Every build sets LONGSHOT_OBJ_STORE and LONGSHOT_LIFT_STORE to obj-store and
lift-store in its work folder (on its drive, whichever the player chose: the
objects are hard links there), with LONGSHOT_OBJ_STORE_GB=4 and
LONGSHOT_LIFT_STORE_KEEP=2. Going between stable and nightly, or back to a
version built before, then compiles only what no earlier build compiled and
copies the translation instead of making it again. Tools from before the stores
read none of the variables.

Free up space counts and clears both stores; an object objcache holds too is
one hard-linked file and is counted once.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_017vw8e3MfaADZobPGhWJk77

HumanBean commented 2026-11-15 07:18:21 +00:00

Author

Owner

• Copy link
• Copy Markdown

Review notes (C# compiled and launcher-core checks passed on a scratch copy with this applied: 1048 checks):

1. A zeroed system_headers.txt crashes Disk.WorkBytes (Core/Disk.cs, ObjectStoreBytes). A file of NUL bytes, which a crash or power loss can leave behind, passes Trim() and the length check. Path.Combine(store, stamp[..16]) then throws ArgumentException, which isn't caught. In SettingsViewModel.MeasureWork that leaves measuring true for good, so the note stays on "Measuring…" and Free up space stays disabled, the very button that would clear the bad file. Fix: require 16 lowercase hex characters, as object_store.folder does, and catch ArgumentException.
2. Space shown too low where hard links don't work. The store is inside the work folder, so a copy never means "across drives". _place copies when os.link fails, which happens on exFAT/FAT drives and some SMB shares. WorkBytes still subtracts any store object named like one in objcache, so it under-counts by about 1 GB there. The new test writes two separate files, so it asserts this under-count rather than a hard link. Only subtract when the two are the same file (link count or file ID).
3. The first switch back may translate again. lift_share_key hashes the work folder's current image/chunks.txt and hot_pieces.txt before the lift, and the other version's build has rewritten them. The planned third build will likely translate again on the first round. That's Longshot's behaviour (#372), but the README and test plan should say so.
4. Wording:
   • "a copy across drives" (comment and README) is wrong: the store is always on the work folder's drive.
   • The stores came in v3.0.0-rc.4 / nightly-164, not nightly-144.

🤖 Review drafted with Claude Code

Review notes (C# compiled and launcher-core checks passed on a scratch copy with this applied: 1048 checks): 1. **A zeroed `system_headers.txt` crashes `Disk.WorkBytes`** (`Core/Disk.cs`, `ObjectStoreBytes`). A file of NUL bytes, which a crash or power loss can leave behind, passes `Trim()` and the length check. `Path.Combine(store, stamp[..16])` then throws `ArgumentException`, which isn't caught. In `SettingsViewModel.MeasureWork` that leaves `measuring` true for good, so the note stays on "Measuring…" and Free up space stays disabled, the very button that would clear the bad file. Fix: require 16 lowercase hex characters, as `object_store.folder` does, and catch `ArgumentException`. 2. **Space shown too low where hard links don't work.** The store is inside the work folder, so a copy never means "across drives". `_place` copies when `os.link` fails, which happens on exFAT/FAT drives and some SMB shares. `WorkBytes` still subtracts any store object named like one in `objcache`, so it under-counts by about 1 GB there. The new test writes two separate files, so it asserts this under-count rather than a hard link. Only subtract when the two are the same file (link count or file ID). 3. **The first switch back may translate again.** `lift_share_key` hashes the work folder's current `image/chunks.txt` and `hot_pieces.txt` before the lift, and the other version's build has rewritten them. The planned third build will likely translate again on the first round. That's Longshot's behaviour (#372), but the README and test plan should say so. 4. **Wording:** - "a copy across drives" (comment and README) is wrong: the store is always on the work folder's drive. - The stores came in v3.0.0-rc.4 / nightly-164, not nightly-144. 🤖 Review drafted with [Claude Code](https://claude.com/claude-code)

HumanBean referenced this pull request 2026-11-15 07:18:21 +00:00

WIP: fix(update): an update stays in its build's folder, and a new build's is named after its channel #57

HumanBean force-pushed build/stores from d102db3724 to 63760b51fa 2026-11-15 07:35:16 +00:00 Compare

HumanBean commented 2026-11-15 07:40:16 +00:00

Author

Owner

• Copy link
• Copy Markdown

Rebased onto main (4bab213, #82, no conflicts), and the review's fixes are in 63760b51:

1. ObjectStoreBytes takes the stamp only when its first 16 characters are lowercase hex (as object_store.folder), and also catches ArgumentException. A NUL-filled or junk system_headers.txt no longer hangs "Measuring…".
2. A store object is subtracted only when it really is objcache's file: new FileLinks.SameFile (link count > 1 and the same file: statx device and inode on Linux, GetFileInformationByHandle volume serial and file index on Windows). When it can't tell, the file counts twice. The test uses a real hard link and checks that a copy counts twice.
3. "A copy across drives" corrected: copies happen only on file systems without hard links (exFAT, FAT, some network shares).
4. The README says the stores came with v3.0.0-rc.4 / nightly-164.
5. The README notes that the first switch back to a version may translate again (the lift key hashes the work folder's current chunks.txt / hot_pieces.txt).

Not compiled or run yet. The statx P/Invoke (a fixed 256-byte layout) needs a build and a run on Linux before this leaves WIP.

🤖 Drafted with Claude Code

Rebased onto main (`4bab213`, #82, no conflicts), and the review's fixes are in `63760b51`: 1. `ObjectStoreBytes` takes the stamp only when its first 16 characters are lowercase hex (as `object_store.folder`), and also catches `ArgumentException`. A NUL-filled or junk `system_headers.txt` no longer hangs "Measuring…". 2. A store object is subtracted only when it really is objcache's file: new `FileLinks.SameFile` (link count > 1 and the same file: `statx` device and inode on Linux, `GetFileInformationByHandle` volume serial and file index on Windows). When it can't tell, the file counts twice. The test uses a real hard link and checks that a copy counts twice. 3. "A copy across drives" corrected: copies happen only on file systems without hard links (exFAT, FAT, some network shares). 4. The README says the stores came with v3.0.0-rc.4 / nightly-164. 5. The README notes that the first switch back to a version may translate again (the lift key hashes the work folder's current `chunks.txt` / `hot_pieces.txt`). Not compiled or run yet. The `statx` P/Invoke (a fixed 256-byte layout) needs a build and a run on Linux before this leaves WIP. 🤖 Drafted with [Claude Code](https://claude.com/claude-code)

HumanBean changed title from WIP: feat(build): Longshot's object and lift stores in the build files folder, so switching versions reuses earlier builds' work to feat(build): Longshot's object and lift stores in the build files folder, so switching versions reuses earlier builds' work 2026-11-15 07:46:20 +00:00

HumanBean merged commit c257c08774 into main 2026-11-15 08:06:17 +00:00

HumanBean deleted branch build/stores 2026-11-15 08:06:18 +00:00

HumanBean referenced this pull request from a commit 2026-11-15 08:06:18 +00:00

feat(build): Longshot's object and lift stores in the build files folder, so switching versions reuses earlier builds' work (#72)

Sign in to join this conversation.

Reviewers

No reviewers

Labels

Clear labels   

area/build

Building Longshot from your game

  

area/install

Installing Longshot

  

area/saves

Moving and backing up saves

  

area/ui

The launcher's windows and screens

  

area/updates

Finding and applying updates

  

ci: approved

A maintainer approved CircleCI for this fork PR (sticky; removed after 14 quiet days)

  

good first issue

A good start for a new contributor

  

gpu/amd

AMD GPUs

  

gpu/intel

Intel GPUs

  

gpu/nvidia

NVIDIA GPUs

  

help wanted

Extra hands or knowledge wanted

  

kind

bug

Something doesn't work as it should

  

kind

chore

Build, CI, dependencies, tidying

  

kind

docs

Documentation

  

kind

feature

A new feature or an improvement

  

kind

performance

Frame rate, frame time, loading, memory

  

kind

question

A question rather than a change

  

platform/linux

Linux only

  

platform/steam-deck

Steam Deck

  

platform/windows

Windows only

  

priority

critical

Crashes, data loss, or blocks most players

  

priority

high

Important: next in line

  

priority

low

Nice to have

  

status

blocked

Waits on something else first

  

status

confirmed

Reproduced or agreed on

  

status

duplicate

Already reported or proposed

  

status

invalid

Not a problem in this project

  

status

needs-info

Waiting for more information from the reporter

  

status

wontfix

Will not be worked on

  

ci: approved

No labels

area/build

area/install

area/saves

area/ui

area/updates

ci: approved

good first issue

gpu/amd

gpu/intel

gpu/nvidia

help wanted

kind

bug

kind

chore

kind

docs

kind

feature

kind

performance

kind

question

platform/linux

platform/steam-deck

platform/windows

priority

critical

priority

high

priority

low

status

blocked

status

confirmed

status

duplicate

status

invalid

status

needs-info

status

wontfix

ci: approved

Milestone

Clear milestone

No items

No milestone

Projects

Clear projects

No items

No project

Assignees

Clear assignees

No assignees

1 participant

Notifications

Subscribe

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference

MagicBeanDev/LongshotLauncher!72

No description provided.