perf(build): an update with nothing changed copies no unchanged exe into the game folder, and the finish says where its time went #631

Merged

HumanBean merged 1 commit from HumanBean/Longshot:perf/finish-skip-unchanged into main 2026-11-15 18:49:23 +00:00

Conversation 1 Commits 1 Files changed 4 +157 -2

HumanBean commented 2026-11-15 01:18:09 +00:00

Owner

• Copy link
• Copy Markdown

Type: perf · Risk: low · Tested on: Linux, i7-9700K lab (btrfs, SATA SSD) · Needs: Windows: a no-change update's finish

Why

Every update copies the game's exe (about 490 MB) and image.bin into the game folder and swaps them in, even when the link produced the same bytes. The link is deterministic, so on an update with nothing changed this is pure waste: 4-8 s of the finish on the lab, and more on Windows, where the copy is a real write and an antivirus scans each new exe. The finish's time also showed as one number, so a player's record couldn't say where 70 s went.

What changed

The finish skips a file the last finish installed when it is still the installed one and the new file has the same bytes. It also times each of its parts, so the build-times table and build_times.json show where the finish's time goes.

• build: installed_already() in nx2/make_game.py drops a file from staging when all of these hold:

  • the game folder's new .longshot_installed.json (INSTALLED_NOTE) records it;
  • its stat (size, mtime_ns, file id, mode) still matches the note;
  • the new source has the recorded SHA-1, and the same mode.

  The installed file is never read: comparing it with filecmp took 10 s for the exe on a cold cache, slower than the copy. Hashing the freshly linked source takes about 0.4 s.

• build: note_installed() writes the note after the swap, only for the files actually copied. A changed file is still staged as .new and swapped, every file is staged before any swap, and end_stale_games still runs on Windows. A leftover <dst>.new of a skipped file is removed. If the note is missing or broken, the files are copied as before.

• build: PhaseClock.part() splits the finish into copying, swap and runtimes, shader list, button prompts, pipeline lists/icon/stamps, build files deleted, and object store.

• docs: docs/development.md, Build times: the parts and the note.

• tests: tests/test_finish_output.py: five new checks:

  • an installed file is neither copied nor swapped (same inode and mtime), its .new is removed, and a changed image is still copied;
  • identical bytes with no note, or a broken note, are copied, and the installed file is never read;
  • a changed build of the same size is copied;
  • an installed file replaced since is copied;
  • a new mode is copied.

  Also tests/test_build_times.py: part().

The button prompts, the largest part left (14-18 s on every packaged build), are left to #314, which keeps unchanged prompts in packaged builds. The two don't overlap.

Measurements

Verdict: Helps on every update with nothing changed: the copy and swap go from 4-8 s to about 0.5-1.5 s of hashing on the lab's btrfs (two interleaved runs each way). The finish total went 26 → 20 s and 19 → 15 s, with the button prompts' 14-18 s left unchanged. Windows should save more and is not measured.

Hardware: Linux, i7-9700K, btrfs on a SATA SSD (the copy is a clone there)
Scene: none (the build)
Method: make_game.py --update with nothing changed, as a packaged builder (LONGSHOT_TOOLS set, game in bin/), --no-prepare-shaders. Interleaved B/A/B/A in one locked job after a warm-up that wrote the note. "Before" forces the copy path back on with a lab-only switch (it also wrote the note, +0.5 s, which main doesn't). The parts come from the new per-part timing.
How to reproduce: two --update runs in a row with nothing changed; the second's "where the time went" table

	Before: main @ 2b1d38c6 (copy forced)	After: this PR @ 499e0c8a (same change as 6d053652, rebased)
Copying the new files in, run 1	3.2 s	1.5 s (hashing)
Swap and runtimes, run 1	5.0 s	0 s
Finish total, run 1	26 s (prompts 16)	20 s (prompts 18)
Copying the new files in, run 2	0.4 s	0.5 s (hashing)
Swap and runtimes, run 2	3.8 s	0 s
Finish total, run 2	19 s (prompts 14)	15 s (prompts 14)

Two more runs on main with only the timing added: finish 24 s and 30 s, copy 3 s, swap 3 s and 8 s. The exe had the same hash over three no-change runs.

Where a no-change update's finish goes on main (lab):

Finish part	No-change update
copying the new files in	0-3 s
swap and runtimes	3-8 s
shader list	0 s (already skipped when current)
button prompts	14-18 s
pipeline lists, icon, language.txt, version	0-1 s
build files deleted	0 s
object store	0 s

Flags and settings

None

Testing

Tested: on the lab (Linux): the runs above. tests/test_finish_output.py (18 OK, 3 Windows-only skipped), tests/test_build_times.py (20 OK) and tests/test_prompt_skip.py (41 OK, it reads the finish's source) pass. Nothing was run on the author's PC.

Not tested: Windows; the full tests/run_all.py.

Must be tested:

• Windows, by hand: two Builder updates in a row with nothing changed. The second's build log must say nothing was copied in for the exe (its "copying the new files in" part near 0, "swap and runtimes" with no exe swap), and the game must start. Then change a file, update, and expect the exe copied and swapped.

Risks

• A stale file could be kept if something rewrites an installed exe or image.bin in place and keeps its size, mtime and file id. Nothing in Longshot does that, but a tool that restores timestamps could. On a file system with no stable file id (FAT/exFAT), only size, mtime and mode protect it.
• A game folder copied elsewhere gets new file ids, so its files are copied as before.
• Windows: an unchanged exe is no longer replaced while the game is running. The stale-game check still runs first, as before.
• A new dotfile, .longshot_installed.json, sits in the game folder.
• Reverting the commit restores the old behaviour.

Compatibility: saves, longshot_options.ini, the shader and pipeline caches, the vk ABI and the builder's tools are untouched. The game folder gains .longshot_installed.json; a folder without it (every one before this PR) has all its files copied once more, as before.

Checklist

Everyone

• One topic.
• The title is a Conventional Commit (type(scope): description) of the status line's type.
• Why gives the reason plainly and links the issues that fit (the tracker searched).
• Measurements has a verdict, or one line saying why there is nothing to measure.
• Testing names what was tested, what wasn't, and what must still be tested, where and how.
• Rebased on current main.
• python -I -B tests/run_all.py passes (any failure that also fails on main is named). (not run: left to CI)
• No game files, dumps, build output or generated files.
• third_party/ untouched (a new folder with a PROVENANCE.md instead).
• The license is OK: GPL-3.0-or-later, nothing incompatible and nothing decompiled.

When it applies

• AI assistance recorded with Assisted-by: / Co-Authored-By: trailers.
• A check under tests/ that reads changed source updated in this PR, not weakened.
• Docs updated.

🤖 Generated with Claude Code

https://claude.ai/code/session_017vw8e3MfaADZobPGhWJk77

**Type:** perf · **Risk:** low · **Tested on:** Linux, i7-9700K lab (btrfs, SATA SSD) · **Needs:** Windows: a no-change update's finish ## Why Every update copies the game's exe (about 490 MB) and image.bin into the game folder and swaps them in, even when the link produced the same bytes. The link is deterministic, so on an update with nothing changed this is pure waste: 4-8 s of the finish on the lab, and more on Windows, where the copy is a real write and an antivirus scans each new exe. The finish's time also showed as one number, so a player's record couldn't say where 70 s went. ## What changed The finish skips a file the last finish installed when it is still the installed one and the new file has the same bytes. It also times each of its parts, so the build-times table and `build_times.json` show where the finish's time goes. - build: `installed_already()` in `nx2/make_game.py` drops a file from staging when all of these hold: - the game folder's new `.longshot_installed.json` (`INSTALLED_NOTE`) records it; - its stat (size, mtime_ns, file id, mode) still matches the note; - the new source has the recorded SHA-1, and the same mode. The installed file is never read: comparing it with `filecmp` took 10 s for the exe on a cold cache, slower than the copy. Hashing the freshly linked source takes about 0.4 s. - build: `note_installed()` writes the note after the swap, only for the files actually copied. A changed file is still staged as `.new` and swapped, every file is staged before any swap, and `end_stale_games` still runs on Windows. A leftover `<dst>.new` of a skipped file is removed. If the note is missing or broken, the files are copied as before. - build: `PhaseClock.part()` splits the finish into copying, swap and runtimes, shader list, button prompts, pipeline lists/icon/stamps, build files deleted, and object store. - docs: `docs/development.md`, Build times: the parts and the note. - tests: `tests/test_finish_output.py`: five new checks: - an installed file is neither copied nor swapped (same inode and mtime), its `.new` is removed, and a changed image is still copied; - identical bytes with no note, or a broken note, are copied, and the installed file is never read; - a changed build of the same size is copied; - an installed file replaced since is copied; - a new mode is copied. Also `tests/test_build_times.py`: `part()`. The button prompts, the largest part left (14-18 s on every packaged build), are left to https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/314, which keeps unchanged prompts in packaged builds. The two don't overlap. ## Measurements **Verdict:** Helps on every update with nothing changed: the copy and swap go from 4-8 s to about 0.5-1.5 s of hashing on the lab's btrfs (two interleaved runs each way). The finish total went 26 → 20 s and 19 → 15 s, with the button prompts' 14-18 s left unchanged. Windows should save more and is not measured. Hardware: Linux, i7-9700K, btrfs on a SATA SSD (the copy is a clone there) Scene: none (the build) Method: `make_game.py --update` with nothing changed, as a packaged builder (`LONGSHOT_TOOLS` set, game in `bin/`), `--no-prepare-shaders`. Interleaved B/A/B/A in one locked job after a warm-up that wrote the note. "Before" forces the copy path back on with a lab-only switch (it also wrote the note, +0.5 s, which main doesn't). The parts come from the new per-part timing. How to reproduce: two `--update` runs in a row with nothing changed; the second's "where the time went" table | | Before: main @ 2b1d38c6 (copy forced) | After: this PR @ 499e0c8a (same change as 6d053652, rebased) | |---|---|---| | Copying the new files in, run 1 | 3.2 s | 1.5 s (hashing) | | Swap and runtimes, run 1 | 5.0 s | 0 s | | Finish total, run 1 | 26 s (prompts 16) | 20 s (prompts 18) | | Copying the new files in, run 2 | 0.4 s | 0.5 s (hashing) | | Swap and runtimes, run 2 | 3.8 s | 0 s | | Finish total, run 2 | 19 s (prompts 14) | 15 s (prompts 14) | Two more runs on main with only the timing added: finish 24 s and 30 s, copy 3 s, swap 3 s and 8 s. The exe had the same hash over three no-change runs. Where a no-change update's finish goes on main (lab): | Finish part | No-change update | |---|---| | copying the new files in | 0-3 s | | swap and runtimes | 3-8 s | | shader list | 0 s (already skipped when current) | | button prompts | 14-18 s | | pipeline lists, icon, language.txt, version | 0-1 s | | build files deleted | 0 s | | object store | 0 s | ### Flags and settings None ## Testing **Tested:** on the lab (Linux): the runs above. `tests/test_finish_output.py` (18 OK, 3 Windows-only skipped), `tests/test_build_times.py` (20 OK) and `tests/test_prompt_skip.py` (41 OK, it reads the finish's source) pass. Nothing was run on the author's PC. **Not tested:** Windows; the full `tests/run_all.py`. **Must be tested:** - Windows, by hand: two Builder updates in a row with nothing changed. The second's build log must say nothing was copied in for the exe (its "copying the new files in" part near 0, "swap and runtimes" with no exe swap), and the game must start. Then change a file, update, and expect the exe copied and swapped. ## Risks - A stale file could be kept if something rewrites an installed exe or image.bin in place and keeps its size, mtime and file id. Nothing in Longshot does that, but a tool that restores timestamps could. On a file system with no stable file id (FAT/exFAT), only size, mtime and mode protect it. - A game folder copied elsewhere gets new file ids, so its files are copied as before. - Windows: an unchanged exe is no longer replaced while the game is running. The stale-game check still runs first, as before. - A new dotfile, `.longshot_installed.json`, sits in the game folder. - Reverting the commit restores the old behaviour. **Compatibility:** saves, `longshot_options.ini`, the shader and pipeline caches, the vk ABI and the builder's tools are untouched. The game folder gains `.longshot_installed.json`; a folder without it (every one before this PR) has all its files copied once more, as before. ## Checklist ### Everyone - [x] One topic. - [x] The title is a Conventional Commit (`type(scope): description`) of the status line's type. - [x] Why gives the reason plainly and links the issues that fit (the tracker searched). - [x] Measurements has a verdict, or one line saying why there is nothing to measure. - [x] Testing names what was tested, what wasn't, and what must still be tested, where and how. - [x] Rebased on current `main`. - [ ] `python -I -B tests/run_all.py` passes (any failure that also fails on `main` is named). (not run: left to CI) - [x] No game files, dumps, build output or generated files. - [x] `third_party/` untouched (a new folder with a `PROVENANCE.md` instead). - [x] The license is OK: GPL-3.0-or-later, nothing incompatible and nothing decompiled. ### When it applies - [x] AI assistance recorded with `Assisted-by:` / `Co-Authored-By:` trailers. - [x] A check under `tests/` that reads changed source updated in this PR, not weakened. - [x] Docs updated. 🤖 Generated with [Claude Code](https://claude.com/claude-code) https://claude.ai/code/session_017vw8e3MfaADZobPGhWJk77

HumanBean added 1 commit 2026-11-15 01:18:09 +00:00

perf(build): an update with nothing changed copies no unchanged exe into the game folder, and the finish says where its time went ... 6d053652f8

The finish stage copied the game's exe (460-490 MB), its renderer and image.bin in as .new and swapped them at every
build, even when the update linked the same bytes again (the link is deterministic: the same hash run after run).
On the lab's btrfs the copy is a clone, but the swap that frees the old exe took 4-8 s of a no-change update's finish;
elsewhere the copy itself costs, and an antivirus reads each new exe.

installed_already() leaves out of the staging a file the last finish put there: INSTALLED_NOTE
(.longshot_installed.json in the game folder) has each copied file's SHA-1 and stat (size, mtime, file id, mode),
written after the swap (note_installed). A file whose stat still matches and whose new source has the noted hash and
the same mode is kept; anything else (no note, another file in its place, a changed build, a new mode) is staged and
swapped as before, all staged before any swap. The installed file is never read: on a cold cache comparing it took 10 s
for the exe, longer than the copy. Hashing the freshly linked source is ~0.4 s.

PhaseClock.part() cuts the finish into its parts in the where-the-time-went table and build_times.json: copying,
swap and runtimes, shader list, button prompts, pipeline lists and stamps, build files deleted, object store.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_017vw8e3MfaADZobPGhWJk77

mooki added the

area/install

kind

performance

platform/linux

platform/windows

priority

high

labels 2026-11-15 13:56:42 +00:00

HumanBean force-pushed perf/finish-skip-unchanged from 6d053652f8 to ddd371233d 2026-11-15 18:27:22 +00:00 Compare

HumanBean commented 2026-11-15 18:27:59 +00:00

Author

Owner

• Copy link
• Copy Markdown

Rebased on main @ dec8aa67 (now ddd37123). One conflict in nx2/make_game.py: main's removal of an old wgpu_native.dll (#577) and this PR's note_installed call were added at the same spot; both kept, note_installed first. Added the Compatibility line under Risks.

Rebased on main @ dec8aa67 (now ddd37123). One conflict in `nx2/make_game.py`: main's removal of an old `wgpu_native.dll` (#577) and this PR's `note_installed` call were added at the same spot; both kept, `note_installed` first. Added the Compatibility line under Risks.

HumanBean changed title from WIP: perf(build): an update with nothing changed copies no unchanged exe into the game folder, and the finish says where its time went to perf(build): an update with nothing changed copies no unchanged exe into the game folder, and the finish says where its time went 2026-11-15 18:29:30 +00:00

HumanBean merged commit a5fd123b73 into main 2026-11-15 18:49:23 +00:00

HumanBean deleted branch perf/finish-skip-unchanged 2026-11-15 18:49:23 +00:00

HumanBean referenced this pull request from a commit 2026-11-15 18:49:25 +00:00

perf(build): an update with nothing changed copies no unchanged exe into the game folder, and the finish says where its time went (#631)

nyannoying referenced this pull request 2026-11-15 19:08:58 +00:00

feat(vk): Intel XeSS Super Resolution as a temporal upscaler (Vulkan, the player's libxess.dll) #459

madebyjordan referenced this pull request 2026-11-15 19:45:30 +00:00

feat(build): a new version's game files hard-linked from another game folder of the same .nup #360

Sign in to join this conversation.

Reviewers

No reviewers

Labels

Clear labels   

area/accessibility

Barriers for people with disabilities

  

area/audio

Sound and music

  

area/input

Controllers, keyboard, mouse

  

area/install

Installing, updating, building from your game

  

area/menu

In-game menu and settings

  

area/renderer

Vulkan renderer, shaders, effects

  

area/saves

Save files

  

area/tooling

Developer tools: scripts, CI, build and release tooling

  

area/upscaling

DLSS, FSR, frame generation

  

ci: approved

A maintainer approved CircleCI for this fork PR (sticky; removed after 14 quiet days)

  

good first issue

A good start for a new contributor

  

gpu/amd

AMD GPUs

  

gpu/igpu

Integrated GPUs

  

gpu/intel

Intel GPUs

  

gpu/nvidia

NVIDIA GPUs

  

help wanted

Extra hands or knowledge wanted

  

kind

bug

Something doesn't work as it should

  

kind

chore

Build, CI, dependencies, tidying

  

kind

docs

Documentation

  

kind

feature

A new feature or an improvement

  

kind

performance

Frame rate, frame time, loading, memory

  

kind

question

A question rather than a change

  

needs measurements

Waiting for before/after measurements

  

platform/linux

Affects Linux (a change for every system carries both Windows and Linux)

  

platform/macos

macOS (Apple silicon or Intel Macs)

  

platform/steam-deck

Steam Deck

  

platform/windows

Affects Windows (a change for every system carries both Windows and Linux)

  

preview

Maintainers: deploy a preview of this PR's website (read its changes first). Removed after the run.

  

priority

critical

Crashes, data loss, or blocks most players

  

priority

high

Important: next in line

  

priority

low

Nice to have

  

regression

Worked in an earlier build, broken in a newer one: highest priority, above other bugs

  

risk

high

Broad or core change: the renderer, defaults on many systems, saves, installs, security

  

risk

low

Small, contained change: little can break

  

risk

medium

Changes behaviour in one area, or a default on some systems

  

status

blocked

Waits on something else first

  

status

confirmed

Reproduced or agreed on

  

status

duplicate

Already reported or proposed

  

status

invalid

Not a problem in this project

  

status

needs-info

Waiting for more information from the reporter

  

status

wontfix

Will not be worked on

  

testing wanted

Wanted: a test on other hardware (posts a test request in #test-requests)

  

ci: approved

No labels

area/accessibility

area/audio

area/input

area/install

area/menu

area/renderer

area/saves

area/tooling

area/upscaling

ci: approved

good first issue

gpu/amd

gpu/igpu

gpu/intel

gpu/nvidia

help wanted

kind

bug

kind

chore

kind

docs

kind

feature

kind

performance

kind

question

needs measurements

platform/linux

platform/macos

platform/steam-deck

platform/windows

preview

priority

critical

priority

high

priority

low

regression

risk

high

risk

low

risk

medium

status

blocked

status

confirmed

status

duplicate

status

invalid

status

needs-info

status

wontfix

testing wanted

ci: approved

Milestone

Clear milestone

No items

No milestone

Projects

Clear projects

No items

No project

Assignees

Clear assignees

No assignees

1 participant

Notifications

Subscribe

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference

MagicBeanDev/Longshot!631

No description provided.