fix(tests): run_all uses the clone's llvm-mingw folder, as make_game does #1032
No reviewers
Labels
No labels
area/accessibility
area/audio
area/input
area/install
area/menu
area/renderer
area/saves
area/tooling
area/upscaling
beanstalk: approved
beanstalk: failed
beanstalk: needs approval
beanstalk: running
beanstalk: tested
ci: approved
good first issue
gpu/amd
gpu/igpu
gpu/intel
gpu/nvidia
help wanted
kind
bug
kind
chore
kind
docs
kind
feature
kind
performance
kind
question
needs measurements
platform/linux
platform/macos
platform/steam-deck
platform/windows
preview
priority
critical
priority
high
priority
low
regression
risk
high
risk
low
risk
medium
status
blocked
status
confirmed
status
duplicate
status
invalid
status
needs-info
status
wontfix
ci: approved
No milestone
No project
No assignees
3 participants
Notifications
Due date
No due date set.
Dependencies
No dependencies set
Reference
MagicBeanDev/Longshot!1032
Loading…
Reference in a new issue
No description provided.
Delete branch "enum/Longshot:fix/run-all-clone-mingw"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Type: fix · Risk: low · Tested on: Windows 11, llvm-mingw 23.1.5 (the clone's
llvm-mingwfolder) · Needs: noneWhy
CONTRIBUTING.md and docs/development.md give two ways to set up the compiler on Windows: set
LONGSHOT_MINGW, or putllvm-mingw in an
llvm-mingwfolder in the clone. The build (nx2/make_game.py'sbundled_mingw()) takes either, buttests/run_all.pytook only the variable. With the folder alone, three game-free checks fail on Windows for a reasonthat has nothing to do with the code they test:
test_spawn_emit,test_fpx_stickyandtest_fastfp_fma. Acontributor who set up the second way sees red checks on a clean
main.How to reproduce
Seen on: Windows 11, llvm-mingw 23.1.5 in
<clone>\llvm-mingw, itsbinon PATH,LONGSHOT_MINGWunsetSettings and switches: defaults
Steps:
python -I -B tests/run_all.pyExpected: the three checks pass, as they do with
LONGSHOT_MINGWset to the same folderActual: they fail with
fatal error: 'algorithm' file not found. They find the folder'sclang++on PATH, buttoolchain.mingw()(third_party/a64-kit/tools/toolchain.py) reads onlyLONGSHOT_MINGW, so they compile forx86_64-pc-windows-msvc, where llvm-mingw has no C++ library.What changed
run_all now does what make_game does: on Windows, when
LONGSHOT_MINGWis unset and the clone'sllvm-mingw/bin/clang.exeexists, it setsLONGSHOT_MINGWto that folder for the checks it starts.bundled_mingw()intests/run_all.py, called inmain()beforefind_tools(). ALONGSHOT_MINGWthat isalready set wins.
LONGSHOT_MSVC=1(the Visual Studio route) and a folder without its clang leave it unset. It doesnothing on Linux.
tests/test_run_all_mingw.py(new, the first commit) runs a copy of run_all.py in a scratch repository witha stand-in
llvm-mingw/bin/clang.exeand a probe check that records theLONGSHOT_MINGWit was started with. Its 4cases: the folder with the variable unset, a variable already set,
LONGSHOT_MSVC=1, and a folder without clang.third_party/is untouched: the fix sits outsidetoolchain.py.Measurements
Verdict: passes. The new test fails on
mainand passes here; the full suite on Windows goes from 23 failures to19: the three checks above and the new test, and nothing else differs.
Hardware: Windows 11 Pro (26300), llvm-mingw 23.1.5 in the clone's folder, Python 3.12.10
Scene: none (game-free checks)
Method:
python -I -B tests/test_run_all_mingw.py, andpython -I -B tests/run_all.pywithLONGSHOT_MINGWunset, onthe same PC and shell
How to reproduce: the commands above, with llvm-mingw in
<clone>\llvm-mingwandLONGSHOT_MINGWunset5f3bb44b(run_all.py from main, the new test added)e1ab75fetest_run_all_mingw.pyrun_all.pyFlags and settings
None. No switch is added. run_all now sets the existing
LONGSHOT_MINGWfor its own checks, the way make_game does.Testing
Tested: On Windows 11, the new test (fails on main, passes here) and the full
run_all.pywithLONGSHOT_MINGWunset, before and after. I also ran each check that reads run_all.py on its own (
test_run_all_*,test_bake_*,test_check_pr_description,test_cpm_discovery,test_portable_builder_startup,test_program_cache_cleanup,test_relocations,test_split_giants,test_spawn_emit). All pass excepttest_run_all_changed, which fails onmain too (#954). The new test also passes in WSL Linux, where the function does nothing. To check that the test
catches a broken fix: removing the assignment makes it fail, and dropping the
LONGSHOT_MSVCcondition makestest_the_visual_studio_route_leaves_it_unsetfail.Still failing here, the same on main:
test_auto_handheld,test_bug_report_defaults(#773),test_crash_dialog,test_crt_options,test_fsr_uniform_ownership,test_guest_imported,test_import_reach,test_linux_preflight(#999, #592),
test_make_game_cache(#592),test_missing_nup(#999),test_pads_after_menu,test_parallel_init_order,test_pending_guest_import,test_replay_write_order(#937),test_run_all_changed(#954),
test_tested_configurations(#493),test_vk_occlusion_queries,test_walk_key(#537).test_gpu_hang_watchfails on main too and is flaky on this PC: run on its own 3 times each, it passed and failed about equally with
LONGSHOT_MINGWset or unset.Not tested: a full run_all on Linux, where the new function returns before doing anything; the Visual Studio route
end to end (the test covers its condition).
Must be tested: None. This is a game-free change to the test runner: no in-game run applies.
Risks
What could break: only run_all on Windows with the folder and no variable, which was the failing setup. A check
that expected
LONGSHOT_MINGWunset there now sees it set, the same as with the first setup route.Turning it off: set
LONGSHOT_MINGWorLONGSHOT_MSVC=1, or revert the commit.Compatibility: Nothing changes for players.
Out of scope: a check run on its own, outside run_all, still needs
LONGSHOT_MINGWwith the folder setup; and run_alldoesn't fall back to
llvm-mingw\bin\clang++when it isn't on PATH.Checklist
Everyone
type(scope): description) of the status line's type.main.python -I -B tests/run_all.pypasses (any failure that also fails onmainis named).third_party/untouched (a new folder with aPROVENANCE.mdinstead).When it applies
Assisted-by:/Co-Authored-By:trailers.mainand passing here.🤖 Generated with Claude Code
Possibly related, for context:
Re the possibly related #893: checked, no overlap in what they do. #893 caches the checks' compiles; this PR picks which llvm-mingw they compile with (LONGSHOT_MINGW from the clone's folder). They touch different parts of tests/run_all.py and the branches merge cleanly (git merge-tree). With both in, the checks still inherit the LONGSHOT_MINGW set here, and #893 keys a compile by its compiler and flags, so the target this fixes gets its own entries.
Also since opening: Before is now measured on main @
5f3bb44b, the PR's base (598 passed / 23 failed / 42 skipped; this PR 602 / 19 / 42; only the three checks and the new test differ).WIP: fix(tests): run_all uses the clone's llvm-mingw folder, as make_game doesto fix(tests): run_all uses the clone's llvm-mingw folder, as make_game doesCode-only review of
e68b073045; informational findings, with final approval left to another reviewer.No introduced correctness issue found in the complete diff. The runner sets the clone's compiler folder before starting checks, preserves an existing
LONGSHOT_MINGW, and leaves Linux and the explicit Visual Studio route alone. The environment value is passed directly to child processes, without shell interpolation.The new regression checks what a real child check inherits, rather than matching source text. Its four cases cover discovery, an explicit override, Visual Studio opt-out, and a missing compiler. The PR records Windows before/after runs of the three affected checks and a WSL pass of the new regression; I did not rerun them. The fixture verifies selection and propagation, not whether a real compiler can link, which the author's Windows runs cover for the reported setup.
Read the runner, regression, affected checks,
make_game.pydiscovery and the shared toolchain's compiler/target selection. No tests, builds, generators or game code were executed, and no source changes were made. Final approval remains with another reviewer.View command line instructions
Checkout
From your project repository, check out a new branch and test the changes.