fix(tests): test_run_all_changed's by-name case uses longshot_saves, as team_chat left main #954

Open
dyst wants to merge 1 commit from dyst/Longshot:fix/run-all-changed-team-chat into main
Member

Type: fix · Risk: low · Tested on: Windows 11, game-free checks · Needs: none

Why

#915 moved scripts/team_chat.py and tests/test_team_chat.py to MagicBeanDev/Tooling. Since then tests/test_run_all_changed.py (from #32) fails on main, because its by-name case changes a file that is no longer there and expects a check that is no longer there. The two merged without a conflict, so it only showed once both were on main.

How to reproduce

Seen on: Windows 11 (game-free; any OS)
Settings and switches: defaults
Steps: on main @ 3dec7681, run python -I -B tests/test_run_all_changed.py
Expected: all 8 tests pass
Actual: test_by_name_too_and_not_inside_a_longer_name fails: 'tests/test_team_chat.py' not found in ['tests/test_run_all_changed.py', 'tests/test_test_requests.py']

What changed

The check's by-name case now uses a script and check that are still on main: it changes scripts/longshot_saves.py and expects tests/test_longshot_saves.py, whose text names longshot_saves.py. --changed itself is unchanged.

  • tests: tests/test_run_all_changed.py, test_by_name_too_and_not_inside_a_longer_name: scripts/longshot_saves.py picks tests/test_longshot_saves.py, and nothing named longshot_saves_x. test_the_checks_that_name_a_changed_file asserts that tests/test_longshot_saves.py is not picked for nx2/chunks.py, in place of the deleted tests/test_team_chat.py.
  • tests: tests/run_all.py, comments only: two comments that said scripts/forge.py now say forge.py, from MagicBeanDev/Tooling (main's scripts/forge.py forwards there since #915).

Measurements

Verdict: nothing to measure: a check's fixture and two comments, with no runtime or speed effect.

Flags and settings

None

Testing

Tested: on this PR @ 72d85e24 (on main @ 3dec7681), Windows 11 with the Builder's llvm-mingw clang (6 at once), the 15 checks that read this diff (the runner's own checks, the ones that name tests/run_all.py or CONTRIBUTING.md, and test_longshot_saves, which the fixture now names), not the whole suite: 12 passed, 1 skipped (no SPIR-V headers on this PC), and 2 failed. Both failures, test_make_game_cache and test_longshot_saves, also fail on main @ 3dec7681 in a clean worktree. test_run_all_changed fails on main @ 3dec7681 and passes here.

Not tested: on Linux (Python only; the fixture names files that are on both OSes).

Must be tested: None

Risks

What could break: only tests/test_run_all_changed.py. If scripts/longshot_saves.py or its check is renamed later, this case would need the same update.

Turning it off: revert the commit.

Compatibility: Nothing changes.

Issues

See #32 (the check) and #915 (the move to Tooling).

Checklist

Everyone

  • One topic.
  • The title is a Conventional Commit (type(scope): description) of the status line's type.
  • Why gives the reason plainly and links the issues that fit (the tracker searched).
  • Measurements has a verdict, or one line saying why there is nothing to measure.
  • Testing names what was tested, what wasn't, and what must still be tested, where and how.
  • Rebased on current main.
  • python -I -B tests/run_all.py passes (any failure that also fails on main is named).
  • No game files, dumps, build output or generated files.
  • third_party/ untouched (a new folder with a PROVENANCE.md instead).
  • The license is OK: GPL-3.0-or-later, nothing incompatible and nothing decompiled.

When it applies

  • AI assistance recorded with Assisted-by: / Co-Authored-By: trailers.
  • A check under tests/ that reads changed source updated in this PR, not weakened.

🤖 Generated with Claude Code

**Type:** fix · **Risk:** low · **Tested on:** Windows 11, game-free checks · **Needs:** none ## Why #915 moved `scripts/team_chat.py` and `tests/test_team_chat.py` to MagicBeanDev/Tooling. Since then `tests/test_run_all_changed.py` (from #32) fails on main, because its by-name case changes a file that is no longer there and expects a check that is no longer there. The two merged without a conflict, so it only showed once both were on main. ## How to reproduce **Seen on:** Windows 11 (game-free; any OS) **Settings and switches:** defaults **Steps:** on main @ 3dec7681, run `python -I -B tests/test_run_all_changed.py` **Expected:** all 8 tests pass **Actual:** `test_by_name_too_and_not_inside_a_longer_name` fails: `'tests/test_team_chat.py' not found in ['tests/test_run_all_changed.py', 'tests/test_test_requests.py']` ## What changed The check's by-name case now uses a script and check that are still on main: it changes `scripts/longshot_saves.py` and expects `tests/test_longshot_saves.py`, whose text names `longshot_saves.py`. `--changed` itself is unchanged. - tests: `tests/test_run_all_changed.py`, `test_by_name_too_and_not_inside_a_longer_name`: `scripts/longshot_saves.py` picks `tests/test_longshot_saves.py`, and nothing named `longshot_saves_x`. `test_the_checks_that_name_a_changed_file` asserts that `tests/test_longshot_saves.py` is not picked for `nx2/chunks.py`, in place of the deleted `tests/test_team_chat.py`. - tests: `tests/run_all.py`, comments only: two comments that said `scripts/forge.py` now say forge.py, from MagicBeanDev/Tooling (main's `scripts/forge.py` forwards there since #915). ## Measurements **Verdict:** nothing to measure: a check's fixture and two comments, with no runtime or speed effect. ### Flags and settings None ## Testing **Tested:** on this PR @ 72d85e24 (on main @ 3dec7681), Windows 11 with the Builder's llvm-mingw clang (6 at once), the 15 checks that read this diff (the runner's own checks, the ones that name `tests/run_all.py` or CONTRIBUTING.md, and `test_longshot_saves`, which the fixture now names), not the whole suite: 12 passed, 1 skipped (no SPIR-V headers on this PC), and 2 failed. Both failures, `test_make_game_cache` and `test_longshot_saves`, also fail on main @ 3dec7681 in a clean worktree. `test_run_all_changed` fails on main @ 3dec7681 and passes here. **Not tested:** on Linux (Python only; the fixture names files that are on both OSes). **Must be tested:** None ## Risks **What could break:** only `tests/test_run_all_changed.py`. If `scripts/longshot_saves.py` or its check is renamed later, this case would need the same update. **Turning it off:** revert the commit. **Compatibility:** Nothing changes. ## Issues See #32 (the check) and #915 (the move to Tooling). ## Checklist ### Everyone - [x] One topic. - [x] The title is a Conventional Commit (`type(scope): description`) of the status line's type. - [x] Why gives the reason plainly and links the issues that fit (the tracker searched). - [x] Measurements has a verdict, or one line saying why there is nothing to measure. - [x] Testing names what was tested, what wasn't, and what must still be tested, where and how. - [x] Rebased on current `main`. - [x] `python -I -B tests/run_all.py` passes (any failure that also fails on `main` is named). - [x] No game files, dumps, build output or generated files. - [x] `third_party/` untouched (a new folder with a `PROVENANCE.md` instead). - [x] The license is OK: GPL-3.0-or-later, nothing incompatible and nothing decompiled. ### When it applies - [x] AI assistance recorded with `Assisted-by:` / `Co-Authored-By:` trailers. - [x] A check under `tests/` that reads changed source updated in this PR, not weakened. 🤖 Generated with [Claude Code](https://claude.com/claude-code)
fix(tests): test_run_all_changed's by-name case uses longshot_saves, as team_chat left main
Some checks failed
CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Has been skipped
CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Has been skipped
CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Has been skipped
Third-party parts / The same as upstream (pull_request) Has been skipped
CI / Conventional Commit title (pull_request) Successful in 3s
Website / Website on Cloudflare (pull_request_target) Has been skipped
CI / Workflow lint (actionlint) (pull_request) Successful in 5s
Guide screenshots / Guide images and local UI source review (pull_request) Failing after 4s
Third-party parts / As their notes say (pull_request) Successful in 8s
PR description / Description follows the template (pull_request_target) Successful in 3s
72d85e246f
#915 moved scripts/team_chat.py and tests/test_team_chat.py to
MagicBeanDev/Tooling, so test_run_all_changed (from #32) fails on main: its
by-name case changes scripts/team_chat.py and expects tests/test_team_chat.py.
It now changes scripts/longshot_saves.py and expects
tests/test_longshot_saves.py. The two comments in tests/run_all.py that said
scripts/forge.py say forge.py, from Tooling.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
dyst changed title from WIP: fix(tests): test_run_all_changed's by-name case uses longshot_saves, as team_chat left main to fix(tests): test_run_all_changed's by-name case uses longshot_saves, as team_chat left main 2026-11-15 22:41:45 +00:00
Member

Code-only review of 72d85e246f; informational findings, with final approval left to another reviewer.

No introduced correctness issue found. The replacement fixture exists, its test names longshot_saves.py, and the positive selection assertion remains in place. The negative nx2/chunks.py case now checks an existing unrelated test instead of a deleted one. The runner changes are comments only; selection behavior and executable side effects are unchanged.

Coverage limit: the existing “not inside a longer name” assertion still has no deliberately supplied longer-name fixture, so it does not independently prove boundary matching. That weakness predates this PR and does not block this fixture repair.

The author reports the affected check failing on main and passing at this head on Windows, with two other failures reproduced on main. Linux was not run. I read the full diff, selector implementation, test fixture and reported validation; I ran no tests or builds. No conflicting implementation was identified in the cached open-PR overlaps.

Code-only review of 72d85e246f8df2a742ad5453779df44a8b4c49b1; informational findings, with final approval left to another reviewer. No introduced correctness issue found. The replacement fixture exists, its test names `longshot_saves.py`, and the positive selection assertion remains in place. The negative `nx2/chunks.py` case now checks an existing unrelated test instead of a deleted one. The runner changes are comments only; selection behavior and executable side effects are unchanged. Coverage limit: the existing “not inside a longer name” assertion still has no deliberately supplied longer-name fixture, so it does not independently prove boundary matching. That weakness predates this PR and does not block this fixture repair. The author reports the affected check failing on main and passing at this head on Windows, with two other failures reproduced on main. Linux was not run. I read the full diff, selector implementation, test fixture and reported validation; I ran no tests or builds. No conflicting implementation was identified in the cached open-PR overlaps.
mooki approved these changes 2026-11-16 13:29:28 +00:00
mooki left a comment

Thanks for this. #915 and #32 each merged cleanly and only failed together, and this is the right size of repair: the by-name case now changes a script that is still on main (scripts/longshot_saves.py) and expects its check (tests/test_longshot_saves.py, whose first line and loader name it), and the same assertions are kept, so nothing is weakened. The two tests/run_all.py edits are a comment and a docstring only: I compared the syntax trees of the file before and after and they are identical apart from the docstring text. The description is clear, the failing message is quoted, and you named the two failures that also fail on main.

Your description says Linux was not run, so I ran it there, in clean worktrees (Linux, Python 3.12):

  • python -I -B tests/test_run_all_changed.py: on the base 3dec7681 it fails at test_by_name_too_and_not_inside_a_longer_name ('tests/test_team_chat.py' not found in ['tests/test_run_all_changed.py', 'tests/test_test_requests.py']). On current main 71b1edd8 it fails the same way. On this head 72d85e24 it passes, 8 of 8, and it also passes on the head merged into main 71b1edd8 (the merge is clean).
  • The whole tests/run_all.py --jobs 6 --strict: base 605 passed, 21 failed, 17 skipped; this head 606 passed, 20 failed, 17 skipped. The only difference between the two lists is test_run_all_changed, so nothing new fails. test_make_game_cache fails on both here too, as you found, and test_longshot_saves passes on Linux, so the failure you saw there is Windows only.
  • No other open pull request gets a new conflict from this one. #284 and #29 already conflict with main in that same block of tests/run_all.py, with or without this change.

Approving. Three small things, none of which needs to hold this up:

  1. The "not inside a longer name" line cannot fail (tests/test_run_all_changed.py line 33). It looks for longshot_saves_x in the picked test paths, and no such path exists (it was the same with team_chat_x, and bibiburl mentioned this above). I broke the selector in a scratch copy three ways: no trailing boundary (?![\w-]), no leading boundary (?<![\w.-]), and a plain substring match. All three still pass 8 of 8. This 12-line test uses a temporary folder, passes on this head, and fails under each of the three:
    def test_a_longer_name_is_not_the_name(self):
        texts = {"tests/test_exact.py": "run('longshot_saves.py')\n",
                 "tests/test_prefixed.py": "run('test_longshot_saves.py')\n",
                 "tests/test_suffixed.py": "run('longshot_saves.pyc')\n"}
        with tempfile.TemporaryDirectory() as root, mock.patch.object(run_all, "ROOT", root):
            os.mkdir(os.path.join(root, "tests"))
            for rel, text in texts.items():
                with open(os.path.join(root, rel), "w", encoding="utf-8") as f:
                    f.write(text)
            picked, _ = run_all.reads_changed(sorted(texts), ["scripts/longshot_saves.py"])
        self.assertEqual(picked, ["tests/test_exact.py"])

It needs import tempfile. The same idea would also stop this case from depending on a real script and its check, which is what turned it red here. Fine as a follow-up if you want this one to stay small.
2. tests/run_all.py line 445 still says scripts/team_chat.py (moved by #915, like the two comments you reworded), and so does scripts/tested_configurations.py line 109.
3. In the description, the Everyone box for run_all.py is ticked while Testing says 15 checks and not the whole suite, and Needs says none while Linux is listed as not tested. The numbers above cover both, so you can quote them or reword. "Rebased on current main" is also out of date (main is 68 commits ahead), but none of them touch these files and it merges clean, so I would not ask for a rebase.

About CI: the red Guide screenshots check is not from this change. scripts/check_guide_screenshots.py --review-sources prints the same 19 issues on 3dec7681. CI does not run the Python source checks on pull requests right now, so this test was only ever going to be caught by someone running it, and the only Windows run is yours.

Thanks for this. #915 and #32 each merged cleanly and only failed together, and this is the right size of repair: the by-name case now changes a script that is still on main (`scripts/longshot_saves.py`) and expects its check (`tests/test_longshot_saves.py`, whose first line and loader name it), and the same assertions are kept, so nothing is weakened. The two `tests/run_all.py` edits are a comment and a docstring only: I compared the syntax trees of the file before and after and they are identical apart from the docstring text. The description is clear, the failing message is quoted, and you named the two failures that also fail on main. Your description says Linux was not run, so I ran it there, in clean worktrees (Linux, Python 3.12): - `python -I -B tests/test_run_all_changed.py`: on the base 3dec7681 it fails at `test_by_name_too_and_not_inside_a_longer_name` (`'tests/test_team_chat.py' not found in ['tests/test_run_all_changed.py', 'tests/test_test_requests.py']`). On current main 71b1edd8 it fails the same way. On this head 72d85e24 it passes, 8 of 8, and it also passes on the head merged into main 71b1edd8 (the merge is clean). - The whole `tests/run_all.py --jobs 6 --strict`: base 605 passed, 21 failed, 17 skipped; this head 606 passed, 20 failed, 17 skipped. The only difference between the two lists is `test_run_all_changed`, so nothing new fails. `test_make_game_cache` fails on both here too, as you found, and `test_longshot_saves` passes on Linux, so the failure you saw there is Windows only. - No other open pull request gets a new conflict from this one. #284 and #29 already conflict with main in that same block of `tests/run_all.py`, with or without this change. Approving. Three small things, none of which needs to hold this up: 1. **The "not inside a longer name" line cannot fail** (`tests/test_run_all_changed.py` line 33). It looks for `longshot_saves_x` in the picked test paths, and no such path exists (it was the same with `team_chat_x`, and bibiburl mentioned this above). I broke the selector in a scratch copy three ways: no trailing boundary `(?![\w-])`, no leading boundary `(?<![\w.-])`, and a plain substring match. All three still pass 8 of 8. This 12-line test uses a temporary folder, passes on this head, and fails under each of the three: ```python def test_a_longer_name_is_not_the_name(self): texts = {"tests/test_exact.py": "run('longshot_saves.py')\n", "tests/test_prefixed.py": "run('test_longshot_saves.py')\n", "tests/test_suffixed.py": "run('longshot_saves.pyc')\n"} with tempfile.TemporaryDirectory() as root, mock.patch.object(run_all, "ROOT", root): os.mkdir(os.path.join(root, "tests")) for rel, text in texts.items(): with open(os.path.join(root, rel), "w", encoding="utf-8") as f: f.write(text) picked, _ = run_all.reads_changed(sorted(texts), ["scripts/longshot_saves.py"]) self.assertEqual(picked, ["tests/test_exact.py"]) ``` It needs `import tempfile`. The same idea would also stop this case from depending on a real script and its check, which is what turned it red here. Fine as a follow-up if you want this one to stay small. 2. `tests/run_all.py` line 445 still says `scripts/team_chat.py` (moved by #915, like the two comments you reworded), and so does `scripts/tested_configurations.py` line 109. 3. In the description, the Everyone box for `run_all.py` is ticked while Testing says 15 checks and not the whole suite, and Needs says none while Linux is listed as not tested. The numbers above cover both, so you can quote them or reword. "Rebased on current main" is also out of date (main is 68 commits ahead), but none of them touch these files and it merges clean, so I would not ask for a rebase. About CI: the red Guide screenshots check is not from this change. `scripts/check_guide_screenshots.py --review-sources` prints the same 19 issues on 3dec7681. CI does not run the Python source checks on pull requests right now, so this test was only ever going to be caught by someone running it, and the only Windows run is yours.
Some checks failed
CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Has been skipped
CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Has been skipped
CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Has been skipped
Third-party parts / The same as upstream (pull_request) Has been skipped
CI / Conventional Commit title (pull_request) Successful in 3s
Website / Website on Cloudflare (pull_request_target) Has been skipped
CI / Workflow lint (actionlint) (pull_request) Successful in 5s
Guide screenshots / Guide images and local UI source review (pull_request) Failing after 4s
Third-party parts / As their notes say (pull_request) Successful in 8s
PR description / Description follows the template (pull_request_target) Successful in 3s
This pull request can be merged automatically.
This branch is out-of-date with the base branch
You are not authorized to merge this pull request.
View command line instructions

Checkout

From your project repository, check out a new branch and test the changes.
git fetch -u 'fix/run-all-changed-team-chat:dyst-fix/run-all-changed-team-chat'
git switch 'dyst-fix/run-all-changed-team-chat'
Sign in to join this conversation.
No reviewers
No milestone
No project
No assignees
3 participants
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
MagicBeanDev/Longshot!954
No description provided.