fix(build): check Linux prerequisites before reading the dump #2793
Closed
github-sync
wants to merge 3 commits from
github/pr/2793 into main
pull from: github/pr/2793
merge into: MagicBeanDev:main
MagicBeanDev:main
MagicBeanDev:github/pr/2893
MagicBeanDev:github/pr/2890
MagicBeanDev:github/pr/2887
MagicBeanDev:github/pr/2878
MagicBeanDev:github/pr/2875
MagicBeanDev:github/pr/2874
MagicBeanDev:github/pr/2873
MagicBeanDev:github/pr/2871
MagicBeanDev:github/pr/2756
MagicBeanDev:github/pr/2897
MagicBeanDev:github/pr/2896
MagicBeanDev:github/pr/2718
MagicBeanDev:github/pr/2842
MagicBeanDev:github/pr/2691
MagicBeanDev:github/pr/2895
MagicBeanDev:github/pr/2743
MagicBeanDev:github/pr/2765
MagicBeanDev:github/pr/2708
MagicBeanDev:github/pr/2732
MagicBeanDev:github/pr/2755
MagicBeanDev:github/pr/2817
MagicBeanDev:github/pr/2876
MagicBeanDev:release-please--branches--main
MagicBeanDev:release-please--branches--main--release-notes
MagicBeanDev:github/pr/2778
MagicBeanDev:github/pr/2888
MagicBeanDev:github/pr/2866
MagicBeanDev:github/pr/2877
MagicBeanDev:github/pr/2800
MagicBeanDev:github/pr/2829
MagicBeanDev:github/pr/2850
MagicBeanDev:github/pr/2889
MagicBeanDev:github/pr/2882
MagicBeanDev:github/pr/2860
MagicBeanDev:github/pr/2849
MagicBeanDev:github/pr/2886
MagicBeanDev:github/pr/2794
MagicBeanDev:github/pr/2840
MagicBeanDev:github/pr/2830
MagicBeanDev:github/pr/2834
MagicBeanDev:github/pr/2719
MagicBeanDev:github/pr/2742
MagicBeanDev:github/pr/2741
MagicBeanDev:github/pr/2820
MagicBeanDev:github/pr/2771
MagicBeanDev:github/pr/2859
MagicBeanDev:github/pr/2858
MagicBeanDev:github/pr/2823
MagicBeanDev:github/pr/2757
MagicBeanDev:github/pr/2769
MagicBeanDev:github/pr/2844
MagicBeanDev:github/pr/2843
MagicBeanDev:github/pr/2673
MagicBeanDev:github/pr/2855
MagicBeanDev:github/pr/2739
MagicBeanDev:github/pr/2807
MagicBeanDev:github/pr/2851
MagicBeanDev:github/pr/2789
MagicBeanDev:github/pr/2814
MagicBeanDev:github/pr/2818
MagicBeanDev:github/pr/2774
MagicBeanDev:github/pr/2772
MagicBeanDev:github/pr/2750
MagicBeanDev:github/pr/2786
MagicBeanDev:github/pr/2785
MagicBeanDev:github/pr/2782
MagicBeanDev:github/pr/2779
MagicBeanDev:github/pr/2775
MagicBeanDev:github/pr/2706
MagicBeanDev:github/pr/2766
MagicBeanDev:github/pr/2747
MagicBeanDev:github/pr/2759
MagicBeanDev:github/pr/2745
MagicBeanDev:github/pr/2740
MagicBeanDev:github/pr/2733
MagicBeanDev:github/pr/2725
MagicBeanDev:github/pr/2727
MagicBeanDev:github/pr/2728
MagicBeanDev:github/pr/2729
MagicBeanDev:github/pr/2721
MagicBeanDev:github/pr/2722
MagicBeanDev:github/pr/2715
MagicBeanDev:github/pr/2714
MagicBeanDev:github/pr/2707
MagicBeanDev:github/pr/2703
MagicBeanDev:github/pr/2705
MagicBeanDev:github/pr/2697
MagicBeanDev:github/pr/2679
MagicBeanDev:github/pr/2676
MagicBeanDev:github/pr/2669
MagicBeanDev:docs/what-hookshot-connects-to
MagicBeanDev:fix/hdr-encode-test
MagicBeanDev:fix/main-tests
MagicBeanDev:fix/play-cache-files
MagicBeanDev:perf/game-barrier-order
MagicBeanDev:feat/heap-lazy
MagicBeanDev:feat/texture-detail
MagicBeanDev:perf/astc-opaque-bc1
MagicBeanDev:fix/fs-dir-lock
MagicBeanDev:fix/framegen-native-kept
MagicBeanDev:fix/update-no-downgrade
MagicBeanDev:port/k10-gcn-f16-mirrors
MagicBeanDev:fix/framegen-mult-cap
MagicBeanDev:chore/release-as-2.1.1
MagicBeanDev:perf/pcache-load-switch
MagicBeanDev:fix/prepare-progress-log
MagicBeanDev:fix/update-closes-stale-game
MagicBeanDev:fix/hdr-scaled-sampled
MagicBeanDev:perf/first-start-preset
MagicBeanDev:perf/zero-upload
MagicBeanDev:fix/vcore-escape-2s
MagicBeanDev:test/fake-gh-atomic
MagicBeanDev:perf/batch-idle-flush
MagicBeanDev:test/finish-stub-a
MagicBeanDev:chore/record-2.1.0
MagicBeanDev:fix/main-red-tests
MagicBeanDev:fix/test-display-startup-driver-cache
MagicBeanDev:feat/mesh-shader-mode
MagicBeanDev:fix/menu-probes-at-startup
MagicBeanDev:fix/fps-counter-classic
MagicBeanDev:fix/test-log-batch-flake
MagicBeanDev:fix/narrow-screen-hud
MagicBeanDev:ci/prereleases
MagicBeanDev:release/v2.0.15
MagicBeanDev:hotfix/v2.0.15
MagicBeanDev:fix/present-ahead-replay
MagicBeanDev:release/v2.0.13
MagicBeanDev:fix/bracket-paths-2013
MagicBeanDev:fix/hdr-overlays-2013
MagicBeanDev:fix/fps-counter-scale-2013
MagicBeanDev:fix/stick-seen-at-rest
MagicBeanDev:hotfix/menu-fits-screen
MagicBeanDev:fix/f1-always-opens-menu
MagicBeanDev:fix/ini-keep-lines
MagicBeanDev:claude/cranky-pike-7cf247
MagicBeanDev:perf/texture-answer-flat
MagicBeanDev:fix/stamp-test-modules
MagicBeanDev:fix/appimage-fonts
MagicBeanDev:fix/display-awake
MagicBeanDev:release/v2.0.12
MagicBeanDev:fix/discord-windows
MagicBeanDev:rc/v2.0.12-tests
MagicBeanDev:fix/dlss-keep-feature
MagicBeanDev:rc/v2.0.12
MagicBeanDev:button-icons
No reviewers
Labels
Clear labels
accessibility
Barrier affecting people with disabilities
autorelease: pending
autorelease: tagged
bug
Something isn't working
dependencies
Pull requests that update a dependency file
documentation
Improvements or additions to documentation
duplicate
This issue or pull request already exists
enhancement
New feature or request
github_actions
Pull requests that update GitHub Actions code
good first issue
Good for newcomers
help wanted
Extra attention is needed
invalid
This doesn't seem right
question
Further information is requested
testing wanted
wontfix
This will not be worked on
No labels
accessibility
autorelease: pending
autorelease: tagged
bug
dependencies
documentation
duplicate
enhancement
github_actions
good first issue
help wanted
invalid
question
testing wanted
wontfix
Milestone
Clear milestone
No items
No milestone
Assignees
Clear assignees
No assignees
1 participant
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".
No due date set.
Dependencies
No dependencies set
Reference
MagicBeanDev/Hookshot!2793
Loading…
Reference in a new issue
No description provided.
Delete branch "github/pr/2793"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
What this is for: Linux source builders should see all missing build prerequisites before their game files are read or translated, rather than fixing separate errors minutes into the build. What is measured: real builder starts on Arch Linux (the missing-FFmpeg error appeared after 0.141 s, and a normal start passed preflight after 0.141 s), one complete source build with the preflight in place (493 files, 0 failed) whose game then reached the title screen, and authored dependency and ordering tests. Not measured: gameplay beyond the title, an installed AppImage, or Windows execution.
Refs #1000, specifically the source-build prerequisite report. This does not fix or close the tracker's other Linux/Steam Deck reports.
Problem
make_game.main()currently reaches extraction and translation before the Linux compiler check.What changed
nx2/make_game.py: check Linux prerequisites before the read stage and report the complete missing set in one@@errormessage. Mandatory Python module names come from the top-level pins inrequirements.txt; the included button-prompt requirements remain optional.nx2/build.py: share renderer-header and prebuilt-renderer selection with the preflight. Keep the existing lookup precedence: CPM caches, explicit header roots, then system prefixes. Extend the existing FFmpeg resolver to explain missing pkg-config entries or files in its selected static prefix.docs/development.md.The check installs nothing and calls no package manager.
How it was tested
Environment: 2026-11-12, Arch-based Omarchy 4.5.0, Linux 7.2.15-1-omarchy, i9-13900K; GCC 16.2.1 and Clang 23.1.3. Base
51da0464b4cf0c7f56b81ebd6bb309325835e175; head9fc8038b. The real builder starts below ranc8bccacedb9e5f5306aee0d8dac889086873fd1b; the only subsequent commit changes tests, not product code. The PC has an RTX 4090 with NVIDIA 615.78.08, and the builder starts below use only the CPU; the complete build's title run further down is the one part that used the card. Load averages at all three real starts were 34.45 / 56.47 / 51.74; these are startup observations under other work, not frame-rate benchmarks.Real builder starts
Each start ran the unmodified
make_game.pyfrom a temporary archive of its named commit, with separate output/work folders and the existing extracted-game paths. For the missing case,HOOKSHOT_FFMPEG,PKG_CONFIG_LIBDIRandPKG_CONFIG_PATHpointed to an empty temporary directory. Nothing was removed from the system.Reading your game fileswithout reporting the missing librariesc8bccace, FFmpeg hiddenc8bccace, normal dependenciesLinux build prerequisites: OK, then the read stageThe failing head start emitted neither a read nor a lift stage and created neither the extracted-code directory nor translation output. No game data was read or translated on that failure path. The normal start's total time through successful preflight was below one second. The controls were deliberately stopped before doing a full build.
The error was:
A complete source build, and the game it made at the title
Built
9fc8038bcompletely from source into a fresh game folder (Clang 23.1.3, the renderer compiled too, eight lift and eight compile workers). The build log begins:The preflight passed 0.094 s after the start and the read stage began at 0.104 s. All 493 planned files compiled (
compiled in 746s, 0 failed), both binaries linked, exit 0, 14 min 55 s in all. This is one build under other work on the PC (load average 11.5 falling to 11.0), not a build-speed figure.That game then ran 90 seconds without a window on the RTX 4090 (NVIDIA 615.78.08) with a fresh profile and
RX_VK_PRECOMPILE=0. It reached the title and was still there at the end; the log has 0 fatal, crash, device-loss or driver-fault lines. No frame rate is claimed.Game-free checks
('game', ['read', 'lift']) != (1, []). Main requests the authored extraction and lift before checking the deliberately missing module, headers, pkg-config entries and compilers. The head reports the missing set first.HOOKSHOT_FFMPEG,RX_CPMand the VMA override are unset, and only CI's pinned Vulkan/SPIR-V include overrides remain. The first strict whole-suite comparison had no preflight-caused failures: main475 passed, 0 failed, 20 skipped;c8bccace475 passed, 1 failed, 20 skipped, with the sole failure a shader-fixture compiler killed by SIGKILL innx2/test_depth_sample_types.py, which never invokes the builder. Its cause is not established, and the failed output is retained.9fc8038b, four workers: CI-like476 passed, 0 failed, 20 skipped in 289 s; normal dependencies476 passed, 0 failed, 20 skipped in 274 s. Neither final run has a failure, including the shader fixture that was killed in the earlier eight-worker run.x86_64-w64-mingw32, preserving its compile flags: one translation unit, no diagnostics. Automatic Windows-selected fixture capture reached no compile because the real Linux probe is explicitly skipped there; the separate capture exercised that exact case. This is not Windows execution.Not tested
RX_VK_PRECOMPILE=0); no game area or visual comparison is claimed.Risks
The new rejection happens earlier, so an incorrectly shared resolution rule could reject a working environment. The tests use the real build resolvers and cover their alternate roots and prebuilt/cross-build routes. Module presence is checked, not package-version compatibility; this does not attempt to certify every possible SDK or library incompatibility.
Overlaps
The open-PR comparison found one additional textual conflict: #1872, the macOS draft, changes FFmpeg's package list/static-library handling and renderer selection in
nx2/build.py. Its thread and hunks were read; it is not a competing Linux prerequisite fix. An eventual integration must retain its platform-specific package choices. Other PRs sharing these code files merge cleanly, including the build-loop changes #2675, #2686, #2693 and #2706.Assisted-by: GPT-6 Astra
Co-Authored-By: GPT-6 Astra [email protected]
Looks good to me. Failing in 0.14 s with one message instead of minutes into a build is a real improvement for source builders, and the evidence is the kind I'd want: the same hidden-FFmpeg start on base (goes straight to "Reading your game files") versus the head (one error, no read or lift stage), a normal start passing the preflight, a full source build from the head with the game reaching the title, and CI green with no conflicts.
What I checked in the code:
header_package,ffmpeg_linux,toolchain.find,target()), with the same precedence (CPM, thenHOOKSHOT_*_HEADERS, then/usr/localand/usr), and the CPM lookup now takes the--workfolder, so it matches what the compile subprocess sees. Anything the preflight refuses, the compile would have stopped on later, so Arch, Fedora, Ubuntu, SteamOS and NixOS (pkg-config,EMSDK,HOOKSHOT_FFMPEGandHOOKSHOT_SYSROOTall honored) behave as before, just earlier. The bundledbuild-linux/depssetup works throughEMSDK,RX_CPMandHOOKSHOT_FFMPEGthe same way. The AppImage ships norequirements.txtand sets a prebuilt renderer, so it skips the Python module and header checks. That is only tested through fixtures, as the description says.fetch_ngx_sdkbefore the preflight is needed, sincebuild.NGX_SDKis read whenbuildis first imported, and the preflight now does that import.test_make_game_ngxonly follows the refactor: it asserts the sameNGX_SDKexpression, now insideprebuilt_renderer(), plus the call site. Behavior is unchanged.check_windows_sdkis now called unconditionally, but on a non-Linux target it takes exactly the old branches.RX_TARGET=windowshasLINUXfalse, so the Linux checks don't apply.Nits, none blocking:
;), and the FFmpeg entry ends with the rawCommand '[...]' returned non-zero exit status 1.text, which adds noise after an already clear message. The GUI protocol needs one line, so this is fine, just wordy.check()(the--check --nuppath) callscheck_windows_sdkfirst, so on Linux checking that a dump is the right game now also requires FFmpeg and the renderer headers, not only a compiler. That is probably fine, since it can't be built without them, but worth knowing if the launcher uses--checkon its own.Evidence is Arch only (the description says so); I'd welcome a Fedora, Ubuntu or SteamOS start from a tester.
For ordering: #1872 (macOS draft) touches the same
build.pyFFmpeg code and will need to rebase on this.