fix(release): reject duplicate checksum manifest names #1039

Open
NitnekLeBorgne wants to merge 1 commit from NitnekLeBorgne/Longshot-contributions:fix/release-checksum-names-20261116 into main
Contributor

Type: fix · Risk: low · Tested on: Linux and native Windows CLI, game-free Python checks (GPU unused) · Needs: no remaining Windows public-fixture gate; production use remains untested

Why

Release assets from different directories can share a filename. The checksum writer currently writes both under that one name, while the reader silently keeps only the last checksum. A manifest should identify each asset unambiguously, not depend on entry order.

This is a locally reproduced filename-identity defect, not a reported player incident or a signature-verification bypass. The parent admitted this scope using the public corpus and complete PR file lists; no matching issue is claimed. PR #685 changes the same file's .nx blacklist regex only; that line is untouched here.

How to reproduce

Seen on: Linux, no GPU or game needed.
Settings and switches: defaults; no new switches.
Steps: create left/Builder.zip and right/Builder.zip with different bytes. Run python -I -B scripts/release.py sums left/Builder.zip right/Builder.zip --out SHA256SUMS. Separately pass two checksum lines naming Builder.zip to read_sums. The new tests/test_release_manifest_names.py exercises both using temporary fixtures.
Expected: reject duplicate names, even when hashes are identical; leave an existing output untouched and do not create a new output on rejection.
Actual: on the pinned main: writer exits successfully and emits duplicate names; reader silently overwrites the earlier dictionary entry. These were independently observed as RED assertion failures before the respective production edits.

What changed

Repeated filenames now fail explicitly instead of producing or accepting ambiguous checksum manifests. Unique-name output stays byte-for-byte unchanged.

  • scripts/release.py, sums: check asset basenames before opening the output file; keep the existing exclusions for the manifest and its signature.
  • scripts/release.py, read_sums: reject an already-seen filename before dictionary assignment. The policy rejects all duplicates, including identical hashes and alternate sha256sum separators.
  • New tests/test_release_manifest_names.py: five automatically discovered game-free tests cover both regressions, identical/repeated inputs, preserving an output that is itself an input, absent-output preservation, sorted unique output, manifest/signature exclusions, and unchanged malformed-line diagnostics.
  • No cryptographic, API, network, publish or blacklist-regex code changed. No other source or existing test changed.

Measurements

Both reproduced regressions are fixed; this is a script correctness change with tests, not a performance change. No runtime or throughput measurements or median claims.

Flags and settings

None.

Testing

  • Windows CLI fixture check completed: python -I -B tests/test_release_manifest_names.py -v passes all five methods; unchanged python -I -B tests/test_release.py -v passes all 34 methods. Every native command exited 0, including real checksum CLI subprocesses. Tests ran in an accented path (épreuve) with private D: temporary files. No GUI or release publication was exercised.

Tested: commit 7bb689d619250bc2a0d791875a3a6555aebc72f3, tree 2bc9f5059021b469d334f7a996393d8eebf47446, direct parent d9486a0846e90905e2ee0bdc4a0d6df96c547c87.

  • Strict vertical TDD: writer RED → minimal writer GREEN, then reader RED → minimal reader GREEN. Unique-name control passed in the initial RED run. All five new tests pass.
  • All 34 unchanged tests/test_release.py tests pass. Its verbose run emitted ResourceWarnings from existing fixtures; none suppressed.
  • Exact committed source-only checkout: normal python -I -B tests/run_all.py --jobs 3, with no selection or skip overrides, isolated HOME/XDG/TMP and baseline-identical PATH. 538 passed, 22 failed, 92 skipped, 652 checks, native exit 1, 234.172 seconds wall time. Four discovered C++ checks pass.
  • Frozen baseline reused, not rerun: 537 passed, 22 failed, 92 skipped, 651 checks. Every existing check status, failure case ID and skip reason matches; only the new passing test was added.
  • Strict failure-signature comparator remains false: test_bug_report.py and test_hardware_report.py report a missing sibling Tooling directory using checkout-specific absolute paths. A separate literal substitution of only those exact paths proves otherwise identical diagnostics; the original strict result and diffs are retained. This is not CI-green.
  • All tracked source bytes and POSIX/Git executable modes preserved across qualification, full parent tree unchanged outside the two allowed files, own process group absent after exit, shared heavy flock held and 3 GiB RAM/disk reserves monitored.
  • No AGENTS.md materialized. Two unchanged upstream SwitchesRemoved methods additionally pass with the public blob provided in memory. This is native normal-runner qualification of a source-only checkout, not exact-tree CI.

Existing baseline failures retained:

  • tests/test_appimage_player_build.py
  • tests/test_bake_portability.py
  • tests/test_bug_report.py
  • tests/test_crash_dialog.py
  • tests/test_crt_options.py
  • tests/test_env_on.py
  • tests/test_guest_imported.py
  • tests/test_hardware_report.py
  • tests/test_lift_cleanup_error.py
  • tests/test_link_skip.py
  • tests/test_linux_update.py
  • tests/test_make_game_cache.py
  • tests/test_missing_nup.py
  • tests/test_mods_size_table.py
  • tests/test_options_budget_once.py
  • tests/test_pending_guest_import.py
  • tests/test_pipeline_tour.py
  • tests/test_readback_late.py
  • tests/test_run_all_changed.py
  • tests/test_vk_occlusion_queries.py
  • tests/test_vk_pgo.py
  • tests/test_walk_key.py

Not tested: real release publishing, production manifests, or any game. No API call, publication, or test-farm request was made.

Must be tested: None for the public-fixture regression; the Windows CLI checks above are complete. Production use and the other exclusions under Not tested remain unqualified.

Risks

What could break: callers that intentionally repeat an asset path or manifest name now receive an error, even with identical bytes. This is the chosen consistent duplicate policy. Filename comparison remains exact and case-sensitive; filesystem alias normalization is not added.

Turning it off: revert the commit.

Compatibility: valid unique-name manifests keep their format and sorted output. No flags, saves, runtime, ABI, shaders, signature format or cryptographic checks change.

Checklist

Everyone

  • One topic.
  • The title is a Conventional Commit (type(scope): description) of the status line's type.
  • Why gives the reason plainly and links the issues that fit (the public tracker and complete public PR file lists searched; parent scope adopted, no matching issue claimed).
  • Tested on my own PC where it runs (in the game, for a change the game runs), as Testing's Tested says; what
    can't be run by hand is in Testing's Not testable by hand, with why.
  • Measurements has a verdict, or one line saying why there is nothing to measure.
  • Nothing gets worse that I know of: any cost (speed, latency, memory, a setup that worked before) is measured and
    in Risks.
  • Testing names what was tested, what wasn't, and what must still be tested, where and how.
  • Rebased on current main at frozen intake (d9486a0846e90905e2ee0bdc4a0d6df96c547c87). Public main has since advanced; no rebase to that later tip is claimed. Relevant source closure is checked again before publication, without transferring full-suite results to another tree.
  • python -I -B tests/run_all.py qualified against its exact pinned-main baseline: native exit 1 is preserved, all 22 existing failures are named above, and no new failure or changed skip status was found. This does not assert a wholly passing suite.
  • No game files, dumps, build output or generated files.
  • third_party/ untouched (a new folder with a PROVENANCE.md instead).
  • The license is OK: GPL-3.0-or-later, nothing incompatible and nothing decompiled.

When it applies

  • AI assistance recorded with Assisted-by: / Co-Authored-By: trailers.
  • A game-free test added for the fix, failing on main and passing here.
  • A check under tests/ that reads changed source updated in this PR, not weakened (new independent test file).

Source export scope: unchanged gpu/build-shadertool.bat has the recorded .gitattributes LF-to-CRLF conversion. Candidate production and test files match native Git blobs; source/modes are unchanged across each run. Tooling is absent in the isolated runner context: two corresponding failure messages require only an exact sibling-path substitution; the original strict comparison remains false. This is not hosted-CI or literal exact-tree equivalence.

Current-main context: at the fresh read of main 6fb13fe527, scripts/release.py adds an unrelated nightly-age command and its standard-library datetime import. The sums/read_sums fix remains on the exact reviewed and tested commit above; full-suite results are not claimed for the later main tip or a merged tree. The initial whole-file closure guard stopped before any push or PR creation, and that stop is retained.

Windows qualification caveats: Native CPython 3.14.7 used a new independent copy of an existing pinned-dependency runtime, not a venv or a replacement product runner. Each candidate used its own full frozen-head public Git-blob export (2652 tree entries; 2651 files materialized; AGENTS.md kept opaque and never materialized). Production/test SHA-256 and all exported file bytes were verified before/after; Git modes remain metadata, not a claim of POSIX permission semantics on NTFS. Normal public test sources and commands were unchanged. Full stdout/stderr and exact native executable/argv/cwd/exit records are retained. A post-test harness assertion exited 1 because five CPython encoding bytecode cache files appeared only in the new owned runtime copy; independent readback confirmed no existing runtime file changed, the original runtime stayed byte-identical, and all test invocations exited 0. No full Windows suite, real compiler, game, GUI, hosted CI or publication qualification is claimed. Existing Linux full-suite results above are unchanged and are not transferred to Windows.

**Type:** fix · **Risk:** low · **Tested on:** Linux and native Windows CLI, game-free Python checks (GPU unused) · **Needs:** no remaining Windows public-fixture gate; production use remains untested ## Why Release assets from different directories can share a filename. The checksum writer currently writes both under that one name, while the reader silently keeps only the last checksum. A manifest should identify each asset unambiguously, not depend on entry order. This is a locally reproduced filename-identity defect, not a reported player incident or a signature-verification bypass. The parent admitted this scope using the public corpus and complete PR file lists; no matching issue is claimed. PR #685 changes the same file's `.nx` blacklist regex only; that line is untouched here. ## How to reproduce **Seen on:** Linux, no GPU or game needed. **Settings and switches:** defaults; no new switches. **Steps:** create `left/Builder.zip` and `right/Builder.zip` with different bytes. Run `python -I -B scripts/release.py sums left/Builder.zip right/Builder.zip --out SHA256SUMS`. Separately pass two checksum lines naming `Builder.zip` to `read_sums`. The new `tests/test_release_manifest_names.py` exercises both using temporary fixtures. **Expected:** reject duplicate names, even when hashes are identical; leave an existing output untouched and do not create a new output on rejection. **Actual:** on the pinned main: writer exits successfully and emits duplicate names; reader silently overwrites the earlier dictionary entry. These were independently observed as RED assertion failures before the respective production edits. ## What changed Repeated filenames now fail explicitly instead of producing or accepting ambiguous checksum manifests. Unique-name output stays byte-for-byte unchanged. - `scripts/release.py`, `sums`: check asset basenames before opening the output file; keep the existing exclusions for the manifest and its signature. - `scripts/release.py`, `read_sums`: reject an already-seen filename before dictionary assignment. The policy rejects all duplicates, including identical hashes and alternate sha256sum separators. - New `tests/test_release_manifest_names.py`: five automatically discovered game-free tests cover both regressions, identical/repeated inputs, preserving an output that is itself an input, absent-output preservation, sorted unique output, manifest/signature exclusions, and unchanged malformed-line diagnostics. - No cryptographic, API, network, publish or blacklist-regex code changed. No other source or existing test changed. ## Measurements Both reproduced regressions are fixed; this is a script correctness change with tests, not a performance change. No runtime or throughput measurements or median claims. ### Flags and settings None. ## Testing - Windows CLI fixture check completed: `python -I -B tests/test_release_manifest_names.py -v` passes all five methods; unchanged `python -I -B tests/test_release.py -v` passes all 34 methods. Every native command exited 0, including real checksum CLI subprocesses. Tests ran in an accented path (`épreuve`) with private D: temporary files. No GUI or release publication was exercised. **Tested:** commit `7bb689d619250bc2a0d791875a3a6555aebc72f3`, tree `2bc9f5059021b469d334f7a996393d8eebf47446`, direct parent `d9486a0846e90905e2ee0bdc4a0d6df96c547c87`. - Strict vertical TDD: writer RED → minimal writer GREEN, then reader RED → minimal reader GREEN. Unique-name control passed in the initial RED run. All five new tests pass. - All 34 unchanged `tests/test_release.py` tests pass. Its verbose run emitted ResourceWarnings from existing fixtures; none suppressed. - Exact committed source-only checkout: normal `python -I -B tests/run_all.py --jobs 3`, with no selection or skip overrides, isolated HOME/XDG/TMP and baseline-identical PATH. **538 passed, 22 failed, 92 skipped**, 652 checks, native exit **1**, 234.172 seconds wall time. Four discovered C++ checks pass. - Frozen baseline reused, not rerun: **537 passed, 22 failed, 92 skipped**, 651 checks. Every existing check status, failure case ID and skip reason matches; only the new passing test was added. - Strict failure-signature comparator remains **false**: `test_bug_report.py` and `test_hardware_report.py` report a missing sibling Tooling directory using checkout-specific absolute paths. A separate literal substitution of only those exact paths proves otherwise identical diagnostics; the original strict result and diffs are retained. This is not CI-green. - All tracked source bytes and POSIX/Git executable modes preserved across qualification, full parent tree unchanged outside the two allowed files, own process group absent after exit, shared heavy flock held and 3 GiB RAM/disk reserves monitored. - No `AGENTS.md` materialized. Two unchanged upstream SwitchesRemoved methods additionally pass with the public blob provided in memory. This is native normal-runner qualification of a source-only checkout, not exact-tree CI. Existing baseline failures retained: - `tests/test_appimage_player_build.py` - `tests/test_bake_portability.py` - `tests/test_bug_report.py` - `tests/test_crash_dialog.py` - `tests/test_crt_options.py` - `tests/test_env_on.py` - `tests/test_guest_imported.py` - `tests/test_hardware_report.py` - `tests/test_lift_cleanup_error.py` - `tests/test_link_skip.py` - `tests/test_linux_update.py` - `tests/test_make_game_cache.py` - `tests/test_missing_nup.py` - `tests/test_mods_size_table.py` - `tests/test_options_budget_once.py` - `tests/test_pending_guest_import.py` - `tests/test_pipeline_tour.py` - `tests/test_readback_late.py` - `tests/test_run_all_changed.py` - `tests/test_vk_occlusion_queries.py` - `tests/test_vk_pgo.py` - `tests/test_walk_key.py` **Not tested:** real release publishing, production manifests, or any game. No API call, publication, or test-farm request was made. **Must be tested:** None for the public-fixture regression; the Windows CLI checks above are complete. Production use and the other exclusions under Not tested remain unqualified. ## Risks **What could break:** callers that intentionally repeat an asset path or manifest name now receive an error, even with identical bytes. This is the chosen consistent duplicate policy. Filename comparison remains exact and case-sensitive; filesystem alias normalization is not added. **Turning it off:** revert the commit. **Compatibility:** valid unique-name manifests keep their format and sorted output. No flags, saves, runtime, ABI, shaders, signature format or cryptographic checks change. ## Checklist ### Everyone - [x] One topic. - [x] The title is a Conventional Commit (`type(scope): description`) of the status line's type. - [x] Why gives the reason plainly and links the issues that fit (the public tracker and complete public PR file lists searched; parent scope adopted, no matching issue claimed). - [x] Tested on my own PC where it runs (in the game, for a change the game runs), as Testing's Tested says; what can't be run by hand is in Testing's Not testable by hand, with why. - [x] Measurements has a verdict, or one line saying why there is nothing to measure. - [x] Nothing gets worse that I know of: any cost (speed, latency, memory, a setup that worked before) is measured and in Risks. - [x] Testing names what was tested, what wasn't, and what must still be tested, where and how. - [x] Rebased on current `main` at frozen intake (`d9486a0846e90905e2ee0bdc4a0d6df96c547c87`). Public main has since advanced; no rebase to that later tip is claimed. Relevant source closure is checked again before publication, without transferring full-suite results to another tree. - [x] `python -I -B tests/run_all.py` qualified against its exact pinned-main baseline: native exit 1 is preserved, all 22 existing failures are named above, and no new failure or changed skip status was found. This does not assert a wholly passing suite. - [x] No game files, dumps, build output or generated files. - [x] `third_party/` untouched (a new folder with a `PROVENANCE.md` instead). - [x] The license is OK: GPL-3.0-or-later, nothing incompatible and nothing decompiled. ### When it applies - [x] AI assistance recorded with `Assisted-by:` / `Co-Authored-By:` trailers. - [x] A game-free test added for the fix, failing on `main` and passing here. - [x] A check under `tests/` that reads changed source updated in this PR, not weakened (new independent test file). Source export scope: unchanged gpu/build-shadertool.bat has the recorded .gitattributes LF-to-CRLF conversion. Candidate production and test files match native Git blobs; source/modes are unchanged across each run. Tooling is absent in the isolated runner context: two corresponding failure messages require only an exact sibling-path substitution; the original strict comparison remains false. This is not hosted-CI or literal exact-tree equivalence. Current-main context: at the fresh read of main 6fb13fe52766a3f4d3ec21cb508de6715d252693, scripts/release.py adds an unrelated nightly-age command and its standard-library datetime import. The sums/read_sums fix remains on the exact reviewed and tested commit above; full-suite results are not claimed for the later main tip or a merged tree. The initial whole-file closure guard stopped before any push or PR creation, and that stop is retained. **Windows qualification caveats:** Native CPython 3.14.7 used a new independent copy of an existing pinned-dependency runtime, not a venv or a replacement product runner. Each candidate used its own full frozen-head public Git-blob export (2652 tree entries; 2651 files materialized; `AGENTS.md` kept opaque and never materialized). Production/test SHA-256 and all exported file bytes were verified before/after; Git modes remain metadata, not a claim of POSIX permission semantics on NTFS. Normal public test sources and commands were unchanged. Full stdout/stderr and exact native executable/argv/cwd/exit records are retained. A post-test harness assertion exited 1 because five CPython encoding bytecode cache files appeared only in the new owned runtime copy; independent readback confirmed no existing runtime file changed, the original runtime stayed byte-identical, and all test invocations exited 0. No full Windows suite, real compiler, game, GUI, hosted CI or publication qualification is claimed. Existing Linux full-suite results above are unchanged and are not transferred to Windows.
fix(release): reject duplicate checksum manifest names
Some checks are pending
CI / Conventional Commit title (pull_request) Blocked by required conditions
CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Blocked by required conditions
CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Blocked by required conditions
CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Blocked by required conditions
CI / Workflow lint (actionlint) (pull_request) Blocked by required conditions
Guide screenshots / Guide images and local UI source review (pull_request) Blocked by required conditions
Third-party parts / As their notes say (pull_request) Blocked by required conditions
Third-party parts / The same as upstream (pull_request) Blocked by required conditions
Website / Website on Cloudflare (pull_request_target) Has been skipped
PR description / Description follows the template (pull_request_target) Successful in 5s
7bb689d619
Reject repeated asset basenames before opening the output manifest and
reject repeated parsed names before assigning a checksum. Identical
checksums are duplicates too. Preserve existing outputs and diagnostics.

Assisted-by: Hermes Agent (OpenAI GPT-6.1-sol)
Co-Authored-By: Claudia (OpenAI GPT-6.1-sol)
NitnekLeBorgne changed title from WIP: fix(release): reject duplicate checksum manifest names to fix(release): reject duplicate checksum manifest names 2026-11-16 04:08:29 +00:00
Member

Code-only review of 7bb689d619; informational findings, with final approval left to another reviewer.

No new correctness issue found in the duplicate-name checks or their regression tests. The writer rejects repeated asset basenames before opening the output, and the reader rejects repeated parsed names before overwriting a dictionary entry, including identical hashes and alternate checksum separators. Unique-name output and the manifest/signature exclusions remain unchanged.

The new tests exercise the real CLI and parser rather than matching source text, including existing-output preservation, absent-output preservation, repeated input paths, and distinct files sharing a basename. Removing either production guard would make the corresponding regression fail. The stated policy remains exact, case-sensitive names; this does not add filesystem alias normalization.

The author reports five new tests and 34 existing release tests passing on Linux and Windows. The Linux full-suite report retains 22 baseline failures and its strict-comparison caveat; this is not a green full-suite claim. I read the full diff, relevant release/signature call sites, test coverage, release documentation and main's intervening changes; I did not run tests, builds, scripts from the PR, or a publication.

The shared-file overlap with PR 685 is a separate release-file exclusion edit, not a duplicate fix or an apparent merge-order dependency. Final integration checks and approval remain with another reviewer.

Code-only review of 7bb689d619250bc2a0d791875a3a6555aebc72f3; informational findings, with final approval left to another reviewer. No new correctness issue found in the duplicate-name checks or their regression tests. The writer rejects repeated asset basenames before opening the output, and the reader rejects repeated parsed names before overwriting a dictionary entry, including identical hashes and alternate checksum separators. Unique-name output and the manifest/signature exclusions remain unchanged. The new tests exercise the real CLI and parser rather than matching source text, including existing-output preservation, absent-output preservation, repeated input paths, and distinct files sharing a basename. Removing either production guard would make the corresponding regression fail. The stated policy remains exact, case-sensitive names; this does not add filesystem alias normalization. The author reports five new tests and 34 existing release tests passing on Linux and Windows. The Linux full-suite report retains 22 baseline failures and its strict-comparison caveat; this is not a green full-suite claim. I read the full diff, relevant release/signature call sites, test coverage, release documentation and main's intervening changes; I did not run tests, builds, scripts from the PR, or a publication. The shared-file overlap with [PR 685](https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/685) is a separate release-file exclusion edit, not a duplicate fix or an apparent merge-order dependency. Final integration checks and approval remain with another reviewer.
Owner

Brought up to the ready-for-review standard (#1454): the checklist's Everyone list has two new boxes, added here from what this PR's description and comments show (please check they match what you know).

  • Tested on my own PC: ticked: Testing's Tested says the author ran the new test and the release tests on the native Windows CLI.
  • Nothing gets worse that I know of: ticked: Measurements says it is a script correctness change with no runtime cost, and Risks names only the intended error.
Brought up to the [ready-for-review standard](https://git.magicbean.dev/MagicBeanDev/Longshot/src/branch/main/CONTRIBUTING.md#ready-for-review-out-of-wip) (#1454): the checklist's Everyone list has two new boxes, added here from what this PR's description and comments show (please check they match what you know). - *Tested on my own PC*: ticked: Testing's Tested says the author ran the new test and the release tests on the native Windows CLI. - *Nothing gets worse that I know of*: ticked: Measurements says it is a script correctness change with no runtime cost, and Risks names only the intended error.
Some checks are pending
CI / Conventional Commit title (pull_request) Blocked by required conditions
CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Blocked by required conditions
CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Blocked by required conditions
CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Blocked by required conditions
CI / Workflow lint (actionlint) (pull_request) Blocked by required conditions
Guide screenshots / Guide images and local UI source review (pull_request) Blocked by required conditions
Third-party parts / As their notes say (pull_request) Blocked by required conditions
Third-party parts / The same as upstream (pull_request) Blocked by required conditions
Website / Website on Cloudflare (pull_request_target) Has been skipped
PR description / Description follows the template (pull_request_target) Successful in 5s
This pull request doesn't have enough approvals yet. 0 of 1 approvals granted.
Some workflows are waiting to be reviewed.
This branch is out-of-date with the base branch
You are not authorized to merge this pull request.
View command line instructions

Checkout

From your project repository, check out a new branch and test the changes.
git fetch -u 'fix/release-checksum-names-20261116:NitnekLeBorgne-fix/release-checksum-names-20261116'
git switch 'NitnekLeBorgne-fix/release-checksum-names-20261116'
Sign in to join this conversation.
No reviewers
No milestone
No project
No assignees
3 participants
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
MagicBeanDev/Longshot!1039
No description provided.