fix(tests): test_run_all_changed's by-name case uses longshot_saves, as team_chat left main #954
No reviewers
Labels
No labels
area/accessibility
area/audio
area/input
area/install
area/menu
area/renderer
area/saves
area/tooling
area/upscaling
beanstalk: approved
beanstalk: failed
beanstalk: needs approval
beanstalk: running
beanstalk: tested
ci: approved
good first issue
gpu/amd
gpu/igpu
gpu/intel
gpu/nvidia
help wanted
kind
bug
kind
chore
kind
docs
kind
feature
kind
performance
kind
question
needs measurements
platform/linux
platform/macos
platform/steam-deck
platform/windows
preview
priority
critical
priority
high
priority
low
regression
risk
high
risk
low
risk
medium
status
blocked
status
confirmed
status
duplicate
status
invalid
status
needs-info
status
wontfix
ci: approved
No milestone
No project
No assignees
3 participants
Notifications
Due date
No due date set.
Dependencies
No dependencies set
Reference
MagicBeanDev/Longshot!954
Loading…
Reference in a new issue
No description provided.
Delete branch "dyst/Longshot:fix/run-all-changed-team-chat"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Type: fix · Risk: low · Tested on: Windows 11, game-free checks · Needs: none
Why
#915 moved
scripts/team_chat.pyandtests/test_team_chat.pyto MagicBeanDev/Tooling. Since thentests/test_run_all_changed.py(from #32) fails on main, because its by-name case changes a file that is no longer there and expects a check that is no longer there. The two merged without a conflict, so it only showed once both were on main.How to reproduce
Seen on: Windows 11 (game-free; any OS)
Settings and switches: defaults
Steps: on main @
3dec7681, runpython -I -B tests/test_run_all_changed.pyExpected: all 8 tests pass
Actual:
test_by_name_too_and_not_inside_a_longer_namefails:'tests/test_team_chat.py' not found in ['tests/test_run_all_changed.py', 'tests/test_test_requests.py']What changed
The check's by-name case now uses a script and check that are still on main: it changes
scripts/longshot_saves.pyand expectstests/test_longshot_saves.py, whose text nameslongshot_saves.py.--changeditself is unchanged.tests/test_run_all_changed.py,test_by_name_too_and_not_inside_a_longer_name:scripts/longshot_saves.pypickstests/test_longshot_saves.py, and nothing namedlongshot_saves_x.test_the_checks_that_name_a_changed_fileasserts thattests/test_longshot_saves.pyis not picked fornx2/chunks.py, in place of the deletedtests/test_team_chat.py.tests/run_all.py, comments only: two comments that saidscripts/forge.pynow say forge.py, from MagicBeanDev/Tooling (main'sscripts/forge.pyforwards there since #915).Measurements
Verdict: nothing to measure: a check's fixture and two comments, with no runtime or speed effect.
Flags and settings
None
Testing
Tested: on this PR @
72d85e24(on main @3dec7681), Windows 11 with the Builder's llvm-mingw clang (6 at once), the 15 checks that read this diff (the runner's own checks, the ones that nametests/run_all.pyor CONTRIBUTING.md, andtest_longshot_saves, which the fixture now names), not the whole suite: 12 passed, 1 skipped (no SPIR-V headers on this PC), and 2 failed. Both failures,test_make_game_cacheandtest_longshot_saves, also fail on main @3dec7681in a clean worktree.test_run_all_changedfails on main @3dec7681and passes here.Not tested: on Linux (Python only; the fixture names files that are on both OSes).
Must be tested: None
Risks
What could break: only
tests/test_run_all_changed.py. Ifscripts/longshot_saves.pyor its check is renamed later, this case would need the same update.Turning it off: revert the commit.
Compatibility: Nothing changes.
Issues
See #32 (the check) and #915 (the move to Tooling).
Checklist
Everyone
type(scope): description) of the status line's type.main.python -I -B tests/run_all.pypasses (any failure that also fails onmainis named).third_party/untouched (a new folder with aPROVENANCE.mdinstead).When it applies
Assisted-by:/Co-Authored-By:trailers.tests/that reads changed source updated in this PR, not weakened.🤖 Generated with Claude Code
WIP: fix(tests): test_run_all_changed's by-name case uses longshot_saves, as team_chat left mainto fix(tests): test_run_all_changed's by-name case uses longshot_saves, as team_chat left mainCode-only review of
72d85e246f; informational findings, with final approval left to another reviewer.No introduced correctness issue found. The replacement fixture exists, its test names
longshot_saves.py, and the positive selection assertion remains in place. The negativenx2/chunks.pycase now checks an existing unrelated test instead of a deleted one. The runner changes are comments only; selection behavior and executable side effects are unchanged.Coverage limit: the existing “not inside a longer name” assertion still has no deliberately supplied longer-name fixture, so it does not independently prove boundary matching. That weakness predates this PR and does not block this fixture repair.
The author reports the affected check failing on main and passing at this head on Windows, with two other failures reproduced on main. Linux was not run. I read the full diff, selector implementation, test fixture and reported validation; I ran no tests or builds. No conflicting implementation was identified in the cached open-PR overlaps.
Thanks for this. #915 and #32 each merged cleanly and only failed together, and this is the right size of repair: the by-name case now changes a script that is still on main (
scripts/longshot_saves.py) and expects its check (tests/test_longshot_saves.py, whose first line and loader name it), and the same assertions are kept, so nothing is weakened. The twotests/run_all.pyedits are a comment and a docstring only: I compared the syntax trees of the file before and after and they are identical apart from the docstring text. The description is clear, the failing message is quoted, and you named the two failures that also fail on main.Your description says Linux was not run, so I ran it there, in clean worktrees (Linux, Python 3.12):
python -I -B tests/test_run_all_changed.py: on the base3dec7681it fails attest_by_name_too_and_not_inside_a_longer_name('tests/test_team_chat.py' not found in ['tests/test_run_all_changed.py', 'tests/test_test_requests.py']). On current main71b1edd8it fails the same way. On this head72d85e24it passes, 8 of 8, and it also passes on the head merged into main71b1edd8(the merge is clean).tests/run_all.py --jobs 6 --strict: base 605 passed, 21 failed, 17 skipped; this head 606 passed, 20 failed, 17 skipped. The only difference between the two lists istest_run_all_changed, so nothing new fails.test_make_game_cachefails on both here too, as you found, andtest_longshot_savespasses on Linux, so the failure you saw there is Windows only.tests/run_all.py, with or without this change.Approving. Three small things, none of which needs to hold this up:
tests/test_run_all_changed.pyline 33). It looks forlongshot_saves_xin the picked test paths, and no such path exists (it was the same withteam_chat_x, and bibiburl mentioned this above). I broke the selector in a scratch copy three ways: no trailing boundary(?![\w-]), no leading boundary(?<![\w.-]), and a plain substring match. All three still pass 8 of 8. This 12-line test uses a temporary folder, passes on this head, and fails under each of the three:It needs
import tempfile. The same idea would also stop this case from depending on a real script and its check, which is what turned it red here. Fine as a follow-up if you want this one to stay small.2.
tests/run_all.pyline 445 still saysscripts/team_chat.py(moved by #915, like the two comments you reworded), and so doesscripts/tested_configurations.pyline 109.3. In the description, the Everyone box for
run_all.pyis ticked while Testing says 15 checks and not the whole suite, and Needs says none while Linux is listed as not tested. The numbers above cover both, so you can quote them or reword. "Rebased on current main" is also out of date (main is 68 commits ahead), but none of them touch these files and it merges clean, so I would not ask for a rebase.About CI: the red Guide screenshots check is not from this change.
scripts/check_guide_screenshots.py --review-sourcesprints the same 19 issues on3dec7681. CI does not run the Python source checks on pull requests right now, so this test was only ever going to be caught by someone running it, and the only Windows run is yours.View command line instructions
Checkout
From your project repository, check out a new branch and test the changes.