fix(build): preserve quoted third-party driver paths #1021
No reviewers
Labels
No labels
area/accessibility
area/audio
area/input
area/install
area/menu
area/renderer
area/saves
area/tooling
area/upscaling
beanstalk: approved
beanstalk: failed
beanstalk: needs approval
beanstalk: running
beanstalk: tested
ci: approved
good first issue
gpu/amd
gpu/igpu
gpu/intel
gpu/nvidia
help wanted
kind
bug
kind
chore
kind
docs
kind
feature
kind
performance
kind
question
needs measurements
platform/linux
platform/macos
platform/steam-deck
platform/windows
preview
priority
critical
priority
high
priority
low
regression
risk
high
risk
low
risk
medium
status
blocked
status
confirmed
status
duplicate
status
invalid
status
needs-info
status
wontfix
ci: approved
No milestone
No project
No assignees
2 participants
Notifications
Due date
No due date set.
Dependencies
No dependencies set
Reference
MagicBeanDev/Longshot!1021
Loading…
Reference in a new issue
No description provided.
Delete branch "NitnekLeBorgne/Longshot-contributions:fix/third-party-driver-paths-20261116"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Type: fix · Risk: low · Tested on: Linux/native Windows game-free Python fixtures; native Windows Clang 23.1.5 and Linux Clang 18.1.3 driver transcripts (GPU not used) · Needs: full stamp/preprocessing and object-reuse validation
Why
The release's precompiled third-party objects are matched using compiler metadata that should not depend on where the build tools are installed. Splitting a quoted Clang argument at every space leaves bare directory fragments in that metadata. With
C:/Program Files/Longshot Build Tools, unchanged main retainsBuildfragments, so otherwise identical installations do not match.This is a reproduced build-metadata defect, not a measured runtime or player incident. The parent's public corpus and complete file-list ownership checks admitted this source and new test; no matching issue is claimed.
How to reproduce
Seen on: Linux using explicitly synthetic Clang
-###replies; no GPU or compiler involved.Settings and switches: no new switches; same cc1 options in each fixture.
Steps: run
python -I -B tests/test_third_party_driver_paths.pywith the new test on maind9486a0846e90905e2ee0bdc4a0d6df96c547c87and on this candidate.Expected:
C:/ToolsandC:/Program Files/Longshot Build Toolsyield identical driver-stamp bytes.Actual: main fails byte equality; the spaced installation leaves directory fragments. The initial focused RED was one assertion failure, no errors.
What changed
Quoted compiler arguments are now kept together before the existing path filter runs. Identical options produce path-independent metadata even when a tool installation contains spaces.
nx2/third_party_objects.py,_driver(): tokenize whole double-quoted Clang arguments before the existing path filter, retaining Windows backslashes rather than applying POSIX shell unescaping.tests/test_third_party_driver_paths.py: seven game-free tests cover the regression, Linux/Unicode and Windows slash forms, escaped quotes, unchanged no-space bytes, distinct CPU/target/feature/default values, quoted semantic values, and existing failed/missing-cc1 behavior.tests/test_third_party_objects.pyremains unchanged (neighbor PR #966 owns it).Measurements
The reproduced byte-equality regression is fixed. No game or build-time performance was measured; this changes only compiler metadata parsing. No FPS, player startup, compilation, linking or object-cache hit-rate claim. Native driver-only invocations are described below.
Flags and settings
None.
Testing
Additional native Windows checks: CPython 3.14.7 on the exact head above:
python -I -B tests/test_third_party_driver_paths.py -vpasses seven new methods andpython -I -B tests/test_third_party_objects.py -vpasses all fifteen unchanged methods; both native exits 0. Those fixture compiler replies remain synthetic. A separate, real Windows Clang driver differential is described below.Tested: candidate
2370dd31fdc1aa330dff666108cdadde3159984b, tree880e51bb55af63f1683c62c1e768f7c328c453c8, direct parentd9486a0846e90905e2ee0bdc4a0d6df96c547c87.nx2/build.pywhen run verbosely; no warnings were suppressed.python -I -B tests/run_all.py --jobs 3, no selection or skip overrides, Linux, isolated HOME/XDG/TMP and baseline-identical PATH. Result: 538 passed, 22 failed, 92 skipped, 652 checks, exit 1, 230.272 seconds wall time. Four discovered C++ checks pass.test_bug_report.py,test_hardware_report.py). Their text is identical after substituting only those exact paths; a separate no-regression assessment passes. Original strict outcome and diff evidence retained; this is not CI-green.AGENTS.mdwas never materialized; two unchanged upstream SwitchesRemoved checks additionally passed with the public blob supplied only in memory. This is normal native-runner qualification of a source-only checkout, not exact-tree CI.Existing baseline failures retained:
tests/test_appimage_player_build.pytests/test_bake_portability.pytests/test_bug_report.pytests/test_crash_dialog.pytests/test_crt_options.pytests/test_env_on.pytests/test_guest_imported.pytests/test_hardware_report.pytests/test_lift_cleanup_error.pytests/test_link_skip.pytests/test_linux_update.pytests/test_make_game_cache.pytests/test_missing_nup.pytests/test_mods_size_table.pytests/test_options_budget_once.pytests/test_pending_guest_import.pytests/test_pipeline_tour.pytests/test_readback_late.pytests/test_run_all_changed.pytests/test_vk_occlusion_queries.pytests/test_vk_pgo.pytests/test_walk_key.pyEarlier real native Windows resource-argument differential (2026-11-16; not physical relocation): Exact published head
2370dd31fdc1aa330dff666108cdadde3159984b, unchangednx2/third_party_objects.pySHA-256105397297ee1d8a6b4dbab981febb77f5a24d2ba2d56087e8ae2d05ad369e4dd, normally imported after static confirmation that its top-level closure is stdlib-only and does not build/download. Explicit existing CPython 3.14.7 and Clang 23.1.5 were used; compiler SHA-256d05fee2c3f79b15d998d3c3413c76aabd416ec789edff89c8f698a8be44a3970._driver()invoked Clang-###with--target=x86_64-w64-windows-gnu -march=x86-64-v3 -nostdinc, comparing-resource-dir D:/PublicTools/resourcewith-resource-dir "D:/Public Tools épreuve/resource". The latter contains one three-word Unicode directory component. This is an actual resource-directory argument axis, not two relocated compiler installations; the directories need not contain headers for this driver-only check.x86-64-v2), feature (-mno-avx2) and target-triple mutations remain distinct, and the actual quoted nonpath macro valuePUBLIC_LABEL=hello wide worldis preserved.--versioncall, all native exit 0; harness exit 0. An earlier four-word Unicode path run independently passed the same controls and is retained separately. Complete raw native stdout/stderr, argv, return codes, stamp bytes and SHA-256 readbacks are retained, including the literal parent RED differential. Instrumentation records actual compiler output; it supplies no synthetic compiler reply.-I -Bwas used, without child Python invocations. Resource checks sampled before/after native commands retained the prescribed reserves; this is not a continuous-monitoring claim. Subsequent process readback found no matching owned test Python/driver process._driver()parsing of actual Clang output.-###does not compile, link, check headers, exercise the complete preprocessingstamp(), run a game or use a GPU. No installation, download, compiler copy, game input or product source edit was needed. Existing Linux full-suite counts, exit 1 and strict-comparator false above remain unchanged; the suite was not rerun.Earlier real native Linux resource-argument differential (2026-11-16; not physical relocation): The same exact published head and source SHA-256 above were checked against the same frozen direct parent using the existing Ubuntu Clang 18.1.3 (1ubuntu1), compiler SHA-256
8ef402d453d1ba4902e4ee0f0f847f6cfa01400c95aa43c24e97818b9c0e3f45, and native CPython 3.12.3. The existing ELF compiler matched the prior host-659 frozen artifact manifest; nothing was installed, downloaded or copied from the compiler installation. The complete public production module was normally imported after its stdlib-only top-level closure was read; the unchanged real_driver()executed every native driver call.-###used--target=x86_64-unknown-linux-gnu -march=x86-64-v3 -nostdincand two actual-resource-dirarguments under the new evidence root:PublicTools/resourceandPublic Tools épreuve/resource(one three-word Unicode component). This is the resource-directory argument axis, not a relocated compiler installation, and does not require headers to exist.Tools. Candidate metadata is equal across those paths; no-space C/C++ bytes remain byte-identical to the parent's. Separate real CPU (x86-64-v2), feature (-mno-avx2), target (x86_64-w64-windows-gnu) and option (-fno-stack-protector -fno-pic) mutations remain distinct. The actual quoted nonpath macro valuePUBLIC_LABEL=hello wide worldsurvives.--versioncall, all native exit 0; harness exit 0. Two read-only Git blob reads and oneldddependency discovery also exited 0 (22 subprocesses total). Complete argv, UTC timing, raw stdout/stderr, native return codes, stamp bytes and hashes, and the real parent differential are retained in the locallinux-driverevidence packet. The recording adapter delegates to real subprocesses; it supplies no mock compiler response.python -I -Bwas used; child commands had isolated HOME/XDG/TMP, a process-scoped library search path and individual owned process groups with 20-second timeouts. The shared heavy lock was held. Before/after samples for every invocation stayed above 3 GiB disk and MemAvailable; these are samples, not continuous monitoring. All owned child groups were absent after exit, with an independent post-run process readback retained.-###does not compile, link, validate headers, invoke the complete preprocessingstamp(), run a game or use a GPU.New physical Linux placement checks (2026-11-16): On the same unchanged candidate and frozen parent, real ELF Clang 18.1.3 was independently copied into
NoSpaceClangandTrois Mots épreuve, with 17 resolved libraries and 237 existing resource files per prefix (510 files, 487926162 bytes total). These are distinct physical executable/dependency/resource copies, not resource-argument substitutions; paired runs used identical cwd, environment and options, without a-resource-diroverride. Across eight configurations, 32 real driver calls plus 11 other native calls all exited 0. C and C++ path baselines are candidate-equal, parent-RED and no-space byte-unchanged; C-only CPU, feature, target, default-option and quoted-value controls remain distinct. Source and original/copied tools were unchanged. The OS loader and kernel remain shared; copied resource/header existence is not header validation.New physical Windows placement checks (2026-11-16): Existing LLVM-mingw Clang 23.1.5's real wrapper,
clang-23.exebackend, recursive DLL/config closure and resources were independently copied into no-space and three-word Unicode prefixes (924 files, 469882234 bytes total). The 11 actual config files and 10 same-folder includes were kept unchanged. Initial 32 driver calls plus four version/resource calls all exited 0; C and C++ path baselines are candidate-equal, parent-RED and no-space byte-unchanged, while C-only CPU, feature, target and quoted-value controls remain distinct. The original initial PRODUCT_RED is retained: two default-option oracle assertions expected-fno-stack-protector -fno-picto change metadata, but the raw cc1 arguments were already identical before parsing. A separate additional C-only-fstack-protector-strongcontrol made four real calls and confirmed meaningful raw and normalized differences at both placements, candidate path equality, parent RED and unchanged no-space bytes. It did not recopy tools, replay initial cases or modify original evidence/inputs. Combined Windows totals are 36 driver calls and 40 native wrapper invocations, not a claim about actual process count; the separate three benign Job controls are not product tests.Scope and retained limits: These new physical-placement checks are separate from the earlier resource-argument checks, not replacements or full-suite reruns. Linux's original global-process audit FAIL remains retained because some
/procentries were unreadable; independent SPEC and QUALITY adopted only the scoped evidence, including absence of all 43 owned child groups. Windows owned-Job and readable own-namespace readbacks were empty; there is no global host process-absence or runtime DLL-load-trace claim. The combined Windows assessment is scoped to these retained receipts, not maintainer approval or a full-suite pass. Neither OS result establishes installation/SDK completeness, fullstamp(), transitive preprocessing, header usability, compilation, linking, object reuse or gameplay. Different Clang versions are qualified separately; no cross-OS/version metadata equality is claimed. The existing Linux full-suite exit 1, all 22 failures and strict-comparator false remain unchanged.Retained harness history: The first Linux placement harness stopped on a lexical
ldd-path membership assertion; a scoped continuation reused the same verified physical copies and corrected only that bookkeeping check, without a product edit or recopy. Before Windows compiler execution, review corrected launch containment (suspended creation, owned-Job assignment, then resume) and lossless diagnostic-byte packaging; benign exit/nonzero/descendant-timeout controls exercised that harness separately. The first Windows inspection also stopped on same-folder config includes until the actual finite config closure was reviewed and pinned. The separate four-call control had a failure-path raw-packaging correction reviewed before execution, and an invocation without its required arming flag stopped before any native call. Those review, harness and fail-closed preflight receipts remain retained alongside the initial results; none is erased, counted as a compiler pass or used to change product code.Not tested: full
stamp()preprocessing/header behavior, a native Windows build, shipped third-party-object reuse, or any game. Compiler replies in the original focused tests remain labelled synthetic fixtures; the separate native driver checks above are not mocked.Must be tested:
stamp()preprocessing with usable headers and shipped third-party-object reuse, expecting unchanged inputs/options to select compatible metadata and objects and semantic changes to remain distinguishable. Physical driver placement is now exercised within the scoped checks above; that does not establish these remaining build-level behaviors. No test-farm request has been made.Risks
What could break: parsing a Clang argument transcript unlike the quoted/escaped form covered here could still prevent reuse. The path filter's existing semantics are unchanged, not broadened to remove semantic options.
Turning it off: revert the commit.
Compatibility: no new flags, defaults, manifest format, runtime code, saves, renderer ABI or shader/pipeline format. Previously contaminated spaced-path metadata changes, intentionally, to the existing path-independent value; no-space metadata is tested byte-for-byte unchanged.
Checklist
Everyone
type(scope): description) of the status line's type.mainat frozen intake (d9486a0846e90905e2ee0bdc4a0d6df96c547c87). Public main has since advanced; no rebase to that later tip is claimed. Relevant source closure is checked again before publication, without transferring full-suite results to another tree.python -I -B tests/run_all.pyqualified against its exact pinned-main baseline: native exit 1 is preserved, all 22 existing failures are named above, and no new failure or changed skip status was found. This does not assert a wholly passing suite.third_party/untouched (a new folder with aPROVENANCE.mdinstead).When it applies
Assisted-by:/Co-Authored-By:trailers.mainand passing here.tests/that reads changed source updated in this PR, not weakened (new independent file).Source export scope: unchanged gpu/build-shadertool.bat has the recorded .gitattributes LF-to-CRLF conversion. Candidate production and test files match native Git blobs; source/modes are unchanged across each run. Tooling is absent in the isolated runner context: two corresponding failure messages require only an exact sibling-path substitution; the original strict comparison remains false. This is not hosted-CI or literal exact-tree equivalence.
Windows qualification caveats: Native CPython 3.14.7 used a new independent copy of an existing pinned-dependency runtime, not a venv or a replacement product runner. Each candidate used its own full frozen-head public Git-blob export (2652 tree entries; 2651 files materialized;
AGENTS.mdkept opaque and never materialized). Production/test SHA-256 and all exported file bytes were verified before/after; Git modes remain metadata, not a claim of POSIX permission semantics on NTFS. Normal public test sources and commands were unchanged. Full stdout/stderr and exact native executable/argv/cwd/exit records are retained. A post-test harness assertion exited 1 because five CPython encoding bytecode cache files appeared only in the new owned runtime copy; independent readback confirmed no existing runtime file changed, the original runtime stayed byte-identical, and all test invocations exited 0. That earlier fixture run did not qualify a real compiler. The separate native driver-only checks above do; no full Windows suite, compilation, game, GUI, hosted CI or publication qualification is claimed. Existing Linux full-suite results above are unchanged and are not transferred to Windows.WIP: fix(build): preserve quoted third-party driver pathsto fix(build): preserve quoted third-party driver pathsCode-only review of
2370dd31fd; informational findings, with final approval left to another reviewer.No introduced correctness defect found in the two-file diff. Keeping each quoted argument together prevents words inside spaced installation paths from surviving the existing path filter. The subprocess invocation and cache-copy behavior are unchanged. The new tests exercise the actual
_driver()parser using mocked compiler replies, including escaped quotes, Windows separators, Unicode, semantic values and failed/missing compiler output; restoring the old whitespace split would break the spaced-path regression.The remaining validation is already identified in the description: capture real Clang
-###output on Windows and Linux from equivalent installations with and without spaces, confirm equal metadata for identical options, and confirm a CPU/target change still changes it. Reported native Windows and Linux fixture passes do not establish that transcript coverage. No unrelated game run is needed for this parser change.Read the full diff, surrounding stamp/cache code, existing tests and reported validation. No tests, builds, compiler invocations or game runs performed in this review. No unexplained script side effects found.
View command line instructions
Checkout
From your project repository, check out a new branch and test the changes.