MagicBeanDev/Longshot
MagicBeanDev/Longshot
Watch
55
68
Fork
You've already forked Longshot
 71
Code Issues 125 Pull requests 195 Projects Releases 8 Packages Wiki Activity Actions 445

WIP: build(macos): discover dual-architecture toolchains and form Mach-O commands #453

Draft
mooki wants to merge 4 commits from mooki/Longshot:mac/build-flags into main
pull from: mooki/Longshot:mac/build-flags
merge into: MagicBeanDev:main
Conversation 0 Commits 4 Files changed 24 +2087 -94
mooki commented 2026-11-14 14:41:51 +00:00
Member

What this is for: anyone building LS on a Mac. Today the builder takes macOS for Linux, so Apple's tools get ELF
linker options and x86 CPU flags, and nothing links. This PR gives the build scripts a macOS command path of their own:
Mach-O, native arm64 or x86_64 under Rosetta, macOS 14.4 as the oldest system. It is the build-command floor of the macOS
port (#420), not a working game on a Mac. What is measured: on an Apple M4 Max with macOS 26.7.1, a new game-free test
fails on main (13 failures in 15 tests) and passes here (15 of 15), two runs each, interleaved. The 36 Windows and
Linux command arrays are identical to main's. Real C/C++ links with Apple clang 21 and LLVM 23.1.5, for arm64 and
x86_64, load and run. The whole game-free suite is compared by test name with main. Not measured: Windows and Linux
runs, a complete LS build or a game run on a Mac, macOS 14.4 itself, and an Intel Mac.

Why

On a Mac, toolchain.windows() is false, so the builder takes the Linux route. It passes ELF options (-fuse-ld=lld,
--gc-sections, -Bsymbolic, -static-libgcc), -march=x86-64-v3 and -mcx16 on Apple silicon, and .so names.
None of that builds on macOS.

What changed

  • nx2/build.py:

    • DARWIN is set beside LINUX, and macOS is no longer LINUX. RX_TARGET=windows works as before.
    • The command arrays move into callable functions: host_flags, renderer_flags, executable_link and
      renderer_link. The Windows and Linux arrays are unchanged, element for element (Measurements).
    • On macOS:
      • compiles take -arch, -mmacosx-version-min=14.4 and -isysroot;
      • links pass linker options as separate -Xlinker pairs: -map, -dead_strip, -exported_symbol _hos_posix_*
        and -rpath @executable_path;
      • the renderer is longshot_vk.dylib with -install_name @rpath/longshot_vk.dylib, -rpath @loader_path,
        -undefined error and the system's libc++;
      • arm64 gets no -mcx16, and no -march unless RX_MARCH names an ARM level. An x86 level is refused there;
      • work/hot_order.txt is left as it is, and a copy with Mach-O's leading underscores, hot_order.txt.macos,
        orders the link;
      • flag_text adds the compiler and SDK identity, so a new compiler rebuilds the objects;
      • NVIDIA NGX, AMD's FSR runtime and the WebGPU renderer are refused with a message: none has a macOS build here.

  • nx2/darwin_build.py (new): the macOS build policy:

    • the file names;
    • the host's POSIX sources: every host/posix/darwin_*.c, but not Linux's prof.c;
    • the order file;
    • FFmpeg and SDL2 through pkg-config, or the prefix in LONGSHOT_FFMPEG;
    • compiler_key, and the post-link note (Mach-O's strict link stands in for ELF's nm -D check).

  • nx2/darwin_toolchain.py (new): a64-kit's tools/toolchain.py and lift/build_emit.py, made to form Mach-O
    commands, from outside third_party/:

    • the output system and architecture: RX_ARCH chooses the game's, never the helpers' that Python loads (a64emit,
      zbic), which follow the Python process;
    • tools from an explicit EMSDK prefix (bin/ or emsdk's upstream/bin/), then PATH, then xcrun;
    • Apple's SDK (SDKROOT, else xcrun's), and a Mach-O linker: ld64.lld when there is one, else Apple's ld;
    • a64emit as liba64emit-<process arch>.dylib, rebuilt when the compiler, its version or the flags change.

    kit.py and make_game.py install it on macOS only, over four of toolchain's functions (find, need,
    native_target, native_link) and over build_emit's OUT and build. This is the way nx2/insn.py overrides
    a64emit.translate. On Windows and Linux nothing imports the file.

  • nx2/make_game.py:

    • check_darwin_prerequisites checks the Python modules, the renderer headers (or a prebuilt renderer's Mach-O slice),
      FFmpeg and SDL2 with a test link, and each compiler's actual version;
    • on a Mac, no Windows or AppImage probe runs, and there is no launcher;
    • NGX and FSR are refused.

  • nx2/host_cpu.py: arm64 gets no -march, an x86_64 Mac gets x86-64-v2, and neither gets -mtune.

  • nx2/zbic.py: zbic-<process arch>.dylib, rebuilt when the compiler or the flags change.

  • nx2/kit.py: three lines that install darwin_toolchain on macOS.

  • .gitignore: *.dylib and *.dylib.sha256.

  • tests/test_macos_build_flags.py (new), with tests/build_flags_main.json:

    • the Windows/MSVC, Windows/MinGW and Linux arrays, each plain and with every option (RX_RELEASE, RX_PGO=gen,
      RX_MTUNE, LONGSHOT_SYSROOT, NGX, FFmpeg, wgpu): 6 configurations of 6 arrays (target, lifted C, host C,
      renderer C++, executable link, renderer link). They were captured from main's own expressions
      (--capture-main), and are compared on a POSIX and on a Windows path host;
    • the simulated Mac, arm64 and x86_64: Mach-O, strict links, separate -Xlinker arguments;
    • tool discovery, the host/target architecture split and pkg-config quoting;
    • that kit.py and make_game.py install darwin_toolchain on a Mac (a child Python each);
    • on macOS, real C and C++ dylibs and a host executable, linked with the production commands, loaded and run. They
      check exports, symbol order, install names, minos 14.4, local strong-symbol binding, and that an unresolved
      symbol is refused. The default is the Python process's architecture; --arch (repeatable) and --llvm-prefix
      add routes.

  • 13 existing tests that run code cut from build.py, make_game.py or host_cpu.py now select Windows or Linux
    explicitly (DARWIN False, or sys.platform set to linux), so on a Mac they still test what they tested. No
    assertion was removed. They are test_build_cleanup_scope, test_cpm_discovery, test_cpp_preflight,
    test_cpu_target, test_dlss_sdk_cache, test_driver_cache_folder, test_ffx_sdk, test_finish_output,
    test_link_path_comma, test_linux_preflight, test_make_game_ngx, test_shader_stamp_cache and
    test_system_header_cache.

  • docs/development.md: a macOS paragraph, and rows for RX_ARCH, EMSDK/SDKROOT,
    LONGSHOT_FFMPEG/LONGSHOT_SDL2, RX_MARCH and LONGSHOT_VK_PREBUILT.

  • Unchanged: third_party/ (byte for byte main's; scripts/check_third_party.py passes), every Windows and Linux
    command, and every source under host/ and gpu/.

Flags and settings

Flag What it does Default now Recommendation Helps on Doesn't help on Untested on
RX_ARCH=arm64 / x86_64 (new) The macOS game's and renderer's architecture; never that of the helpers Python loads arm64 on Apple silicon, x86_64 on an Intel Mac Native arm64; x86_64 for the Rosetta route macOS Windows and Linux: not read A complete build on either; an Intel Mac
RX_MARCH (changed on macOS) The CPU level Mac arm64: none; Mac x86_64: x86-64-v2; Windows and Linux unchanged Leave the Mac defaults macOS A complete build
EMSDK (on macOS) An LLVM prefix (bin/ or upstream/bin/), before PATH and xcrun PATH, then xcrun Set it to qualify a pinned LLVM macOS The whole game with LLVM 23.1.5
SDKROOT (on macOS) Apple's SDK xcrun --sdk macosx --show-sdk-path Leave it macOS
LONGSHOT_FFMPEG, LONGSHOT_SDL2 (on macOS) An FFmpeg prefix with .pc files; an SDL2 .dylib pkg-config (PATH, then Homebrew) Libraries of the target's architecture macOS They don't bundle the dependencies An x86_64 dependency set
LONGSHOT_VK_PREBUILT (on macOS) A renderer built already, longshot_vk.dylib; its Mach-O slice is checked Unset macOS A Mac renderer
Test: --arch (repeatable), --llvm-prefix, --source, --capture-main Real-link routes; a second compiler; another tree; recapture main's arrays The process's architecture, Apple's compiler the test Not a build switch

Measurements

main: 5e072fc8c2adadba5ea610148be1c47800ba0cca (2026-11-14)
this PR: mac/build-flags @ 4723ee36a494cfafb82b18ccc685feaf2caeb964
Hardware: Apple M4 Max, 16 cores, 64 GiB, macOS 26.7.1 (25G241); Apple clang 21.0.0, LLVM 23.1.5, SDK 27.0. No GPU.
Scene: none. Game-free test and suite; no game, no window.
Method: tests/test_macos_build_flags.py on this PR and against main's tree (--source), interleaved, two runs each.
The 36 arrays: main's recaptured from its own expressions, compared with the frozen file, which this PR's builders
equal. Real links once with both compilers and both architectures. The whole suite on this PR, compared by test name
with a main run on the same Mac.
How to reproduce:

python -I -B tests/test_macos_build_flags.py
python -I -B tests/test_macos_build_flags.py --source <a checkout of main>
python -I -B tests/test_macos_build_flags.py --arch arm64 --arch x86_64 --llvm-prefix <LLVM 23.1.5>
python -I -B tests/test_macos_build_flags.py --capture-main --source <a checkout of main>   # equals tests/build_flags_main.json
python -I -B scripts/check_third_party.py
main @ 5e072fc8 this PR @ 4723ee36
tests/test_macos_build_flags.py (run 1 / run 2) FAILED (failures=13), 15 tests / the same OK, 15 tests / the same
Windows and Linux command arrays 36, recaptured from main's expressions: equal to the frozen file 36, from the new functions: equal to the frozen file
macOS commands (build arrays and helper compiles, arm64 and x86_64) macOS is Linux: ELF and x86 flags Mach-O; 40 of 40 identical to this work's earlier head, which edited a64-kit (see Reviewer notes)
Real Mach-O links: Apple clang 21 and LLVM 23.1.5, arm64 and x86_64 no Mach-O command path 4 of 4 rows: 3 links each, load, exports, order and binding pass; the unresolved symbol refused
third_party/ identical to main; third_party: 11 parts as their notes say 
main @ 5e072fc8 (--source), cut to 4 of the 13 failures:
FAIL: test_mac_is_not_linux (__main__.CommandContracts.test_mac_is_not_linux) [arm64]
AssertionError: True is not false : macOS is classified as Linux and receives ELF/x86 flags
FAIL: test_mac_link_paths_are_separate_and_strict (__main__.CommandContracts.test_mac_link_paths_are_separate_and_strict)
AssertionError: False is not true : no Mach-O target
FAIL: test_kit_and_maker_install_the_macos_toolchain (__main__.DiscoveryContracts.test_kit_and_maker_install_the_macos_toolchain)
AssertionError: None is not true : main has no macOS toolchain for kit.py and make_game.py to install
FAIL: test_c_cpp_links_load_exports_order_and_local_binding (__main__.RealMachO.test_c_cpp_links_load_exports_order_and_local_binding)
AssertionError: False is not true : main has no callable Mach-O command path
Ran 15 tests in 0.450s
FAILED (failures=13)

this PR @ 4723ee36, --arch arm64 --arch x86_64 --llvm-prefix LLVM-23.1.5:
MACHO compiler=Apple21 arch=arm64 min=14.4 links=3 load=PASS exports=PASS order=PASS binding=PASS undefined=rejected
MACHO compiler=Apple21 arch=x86_64 min=14.4 links=3 load=PASS exports=PASS order=PASS binding=PASS undefined=rejected
MACHO compiler=LLVM23 arch=arm64 min=14.4 links=3 load=PASS exports=PASS order=PASS binding=PASS undefined=rejected
MACHO compiler=LLVM23 arch=x86_64 min=14.4 links=3 load=PASS exports=PASS order=PASS binding=PASS undefined=rejected
binding own_a=31 own_b=42 host_export=99 distinct_data=1 result=PASS   (printed by each of the 4 rows)
Ran 15 tests in 16.131s
OK

frozen JSON: 6 configurations x 6 arrays = 36 arrays (base 514591a30d707fe9f3ad31eae148c3cc1e750bff)
main 5e072fc8 recaptured from its own expressions equals the frozen JSON: True
nx2/build.py sha256 at 5e072fc8 == frozen record: True
third_party/a64-kit/tools/toolchain.py sha256 at 5e072fc8 == frozen record: True

The other 9 failures on main name the same missing pieces: no Darwin source selection, no compiler identity, no
host/target architecture split, no pkg-config parser. The three tests that pass on main are the two frozen-array
comparisons and the architecture-selection helper. The frozen file says base 514591a3: it was captured there, and
nx2/build.py and a64-kit's toolchain.py are byte-identical at 5e072fc8.

Testing

Tested: the new test both ways, the dual route and the arrays, as above. And the whole suite, tests/run_all.py, 6 at
once, on the same Mac (the runner labels macOS linux):

main @ 5e072fc8:     525 passed, 19 failed, 17 skipped in 177 s (linux, 6 at once)
this PR @ 4723ee36:  526 passed, 19 failed, 17 skipped in 243 s (linux, 6 at once)
  • By test name, the only difference is the new tests/test_macos_build_flags.py (pass). No other check changed
    status, and no crash report was left by this run.
  • The 19 failures are the same on both. They are macOS portability failures of main, which the port's other PRs
    address: test_appimage_package_retry, test_collect_wide, test_control, test_framegen_resources,
    test_game_layout, test_low_memory_notice, test_mem_gate, test_mem_gate_children, test_mirror_trim,
    test_module_ids, test_oot_state_tap, test_pad_layout, test_pcache_header_read, test_pipeline_cache_file,
    test_pipeline_cache_save, test_pipeline_play_cache, test_prof_symbols, test_program_cache_cleanup,
    test_program_cache_one_variant.
  • The checks this PR touches, and the ones near it, pass at this head (named, 17 of 17): the 13 changed tests,
    test_macos_build_flags, test_lift_cache (the lift's sources), and test_spawn_emit and test_fpx_sticky, which
    replace build_emit.build themselves.

Not tested:

  • Windows and Linux. Their arrays are compared, not run. On a Mac the 13 changed tests run their Windows and Linux
    cases as before.
  • A complete LS build or a game run on a Mac. The runtime port (host/, gpu/) is in other PRs.
  • macOS 14.4 itself, and an Intel Mac. minos 14.4 in a binary is not a run on 14.4.
  • LLVM's x86_64 compiler runtime. LLVM 23.1.5's arm64 archive warns that its libclang_rt.osx.a has no x86_64
    slice. The fixtures need none of it; a larger program may.
  • python third_party/a64-kit/lift/build_emit.py, run by hand on a Mac, builds a64-kit's own liba64emit.so, as on
    main. The build loads liba64emit-<arch>.dylib, through kit.py.

Needs other testers:

  • Windows, llvm-mingw and MSVC: the whole game-free suite and a standalone build. Expect the same commands and results as
    main (request to open).
  • Linux: the same, and an AppImage build (request to open).
  • An Intel Mac, or macOS 14.4: python -I -B tests/test_macos_build_flags.py, expecting 15 of 15 (request to open).

Screenshots

Nothing on screen.

Risk and rollback

  • Windows and Linux: their command arrays now come from functions. They are compared element for element with
    main's, but not run there in this PR.
  • Every platform lifts the game again once after updating: nx2/kit.py and the new nx2/darwin_toolchain.py are
    among the lift's sources (make_game.py's lift_sources follows every import of the lift's, inside an if too), so
    the lift key changes. It costs one lift, minutes. Off macOS the lift runs none of the new code, so its output should be
    the same; that lift was not run here.
  • macOS: the toolchain functions are replaced when kit.py or make_game.py is imported. A script that imports
    a64-kit's toolchain without either gets main's behaviour on a Mac, as today.
  • Rollback: revert the PR. No switch is needed: no command changes off macOS.

Reviewer notes

  • Start with nx2/darwin_toolchain.py and its two install sites (kit.py, make_game.py), then build.py's four new
    functions and nx2/darwin_build.py.
  • The commits: da3132bc, 06c20446 and 37587502 are this work as first written (4766930e, 9de9a7f2 and
    7ed96d16), which edited third_party/a64-kit with an updated PROVENANCE.md. 4723ee36 restores a64-kit to
    main's bytes and moves those edits into nx2/darwin_toolchain.py (the owner's decision, after CONTRIBUTING). So the
    first commit's message still describes the a64-kit edit; the squash has none.
  • The move changes no macOS command: each tree read through its own test gives the same 40 macOS arrays and helper
    compiles at 7ed96d16 and at 4723ee36, for arm64 and x86_64, plain and with every option.
  • Review all four commits, da3132bc to 4723ee36: the whole branch is this PR's own (it depends on no other PR).

Issues

See #420.

Checklist

For everyone

  • One topic.
  • The title is a Conventional Commit (type(scope): description).
  • Rebased on current main. (Branched from 5e072fc8.)
  • python -I -B tests/run_all.py run and passing (any failure that also fails on main is named above). (526 passed,
    17 skipped, and the same 19 failures as main on this Mac, named under Testing.)
  • Tested in the game, or what wasn't is named under Testing. (No game run: build commands; named under Testing.)
  • Testing filled in: what was tested, what wasn't, and what needs other testers (with test requests). The test
    requests are still to open.
  • Measured: main and this PR at named commits, on the same PC, scene and settings, at least 2 interleaved runs each, the median reported. (For a build: the test's output on main and here, two interleaved runs each.)
  • No game files, dumps, build output or generated files. (tests/build_flags_main.json is command data from the
    source's own expressions.)
  • third_party/ untouched (a new folder with a PROVENANCE.md instead).
  • The license is OK: GPL-3.0-or-later, nothing incompatible and nothing decompiled.

If applicable

  • AI assistance recorded with Assisted-by: / Co-Authored-By: trailers.
  • Flags noted, and the switch reference updated (docs/development.md, the README). (Build switches only: no README
    row.)
  • A game-free test added for the fix, failing on main and passing here.
  • A check under tests/ that reads changed source updated in this PR, not weakened.
  • Docs updated.
**What this is for:** anyone building LS on a Mac. Today the builder takes macOS for Linux, so Apple's tools get ELF linker options and x86 CPU flags, and nothing links. This PR gives the build scripts a macOS command path of their own: Mach-O, native arm64 or x86_64 under Rosetta, macOS 14.4 as the oldest system. It is the build-command floor of the macOS port (#420), not a working game on a Mac. **What is measured:** on an Apple M4 Max with macOS 26.7.1, a new game-free test fails on `main` (13 failures in 15 tests) and passes here (15 of 15), two runs each, interleaved. The 36 Windows and Linux command arrays are identical to `main`'s. Real C/C++ links with Apple clang 21 and LLVM 23.1.5, for arm64 and x86_64, load and run. The whole game-free suite is compared by test name with `main`. **Not measured:** Windows and Linux runs, a complete LS build or a game run on a Mac, macOS 14.4 itself, and an Intel Mac. ## Why On a Mac, `toolchain.windows()` is false, so the builder takes the Linux route. It passes ELF options (`-fuse-ld=lld`, `--gc-sections`, `-Bsymbolic`, `-static-libgcc`), `-march=x86-64-v3` and `-mcx16` on Apple silicon, and `.so` names. None of that builds on macOS. ## What changed - **`nx2/build.py`**: - `DARWIN` is set beside `LINUX`, and macOS is no longer `LINUX`. `RX_TARGET=windows` works as before. - The command arrays move into callable functions: `host_flags`, `renderer_flags`, `executable_link` and `renderer_link`. The Windows and Linux arrays are unchanged, element for element (Measurements). - On macOS: - compiles take `-arch`, `-mmacosx-version-min=14.4` and `-isysroot`; - links pass linker options as separate `-Xlinker` pairs: `-map`, `-dead_strip`, `-exported_symbol _hos_posix_*` and `-rpath @executable_path`; - the renderer is `longshot_vk.dylib` with `-install_name @rpath/longshot_vk.dylib`, `-rpath @loader_path`, `-undefined error` and the system's libc++; - arm64 gets no `-mcx16`, and no `-march` unless `RX_MARCH` names an ARM level. An x86 level is refused there; - `work/hot_order.txt` is left as it is, and a copy with Mach-O's leading underscores, `hot_order.txt.macos`, orders the link; - `flag_text` adds the compiler and SDK identity, so a new compiler rebuilds the objects; - NVIDIA NGX, AMD's FSR runtime and the WebGPU renderer are refused with a message: none has a macOS build here. - **`nx2/darwin_build.py` (new):** the macOS build policy: - the file names; - the host's POSIX sources: every `host/posix/darwin_*.c`, but not Linux's `prof.c`; - the order file; - FFmpeg and SDL2 through `pkg-config`, or the prefix in `LONGSHOT_FFMPEG`; - `compiler_key`, and the post-link note (Mach-O's strict link stands in for ELF's `nm -D` check). - **`nx2/darwin_toolchain.py` (new):** a64-kit's `tools/toolchain.py` and `lift/build_emit.py`, made to form Mach-O commands, from outside `third_party/`: - the output system and architecture: `RX_ARCH` chooses the game's, never the helpers' that Python loads (`a64emit`, `zbic`), which follow the Python process; - tools from an explicit `EMSDK` prefix (`bin/` or emsdk's `upstream/bin/`), then `PATH`, then `xcrun`; - Apple's SDK (`SDKROOT`, else `xcrun`'s), and a Mach-O linker: `ld64.lld` when there is one, else Apple's `ld`; - `a64emit` as `liba64emit-<process arch>.dylib`, rebuilt when the compiler, its version or the flags change. `kit.py` and `make_game.py` install it on macOS only, over four of toolchain's functions (`find`, `need`, `native_target`, `native_link`) and over build_emit's `OUT` and `build`. This is the way `nx2/insn.py` overrides `a64emit.translate`. On Windows and Linux nothing imports the file. - **`nx2/make_game.py`:** - `check_darwin_prerequisites` checks the Python modules, the renderer headers (or a prebuilt renderer's Mach-O slice), FFmpeg and SDL2 with a test link, and each compiler's actual version; - on a Mac, no Windows or AppImage probe runs, and there is no launcher; - NGX and FSR are refused. - **`nx2/host_cpu.py`:** arm64 gets no `-march`, an x86_64 Mac gets `x86-64-v2`, and neither gets `-mtune`. - **`nx2/zbic.py`:** `zbic-<process arch>.dylib`, rebuilt when the compiler or the flags change. - **`nx2/kit.py`:** three lines that install `darwin_toolchain` on macOS. - **`.gitignore`:** `*.dylib` and `*.dylib.sha256`. - **`tests/test_macos_build_flags.py` (new), with `tests/build_flags_main.json`:** - the Windows/MSVC, Windows/MinGW and Linux arrays, each plain and with every option (`RX_RELEASE`, `RX_PGO=gen`, `RX_MTUNE`, `LONGSHOT_SYSROOT`, NGX, FFmpeg, wgpu): 6 configurations of 6 arrays (target, lifted C, host C, renderer C++, executable link, renderer link). They were captured from `main`'s own expressions (`--capture-main`), and are compared on a POSIX and on a Windows path host; - the simulated Mac, arm64 and x86_64: Mach-O, strict links, separate `-Xlinker` arguments; - tool discovery, the host/target architecture split and `pkg-config` quoting; - that `kit.py` and `make_game.py` install `darwin_toolchain` on a Mac (a child Python each); - on macOS, real C and C++ dylibs and a host executable, linked with the production commands, loaded and run. They check exports, symbol order, install names, `minos 14.4`, local strong-symbol binding, and that an unresolved symbol is refused. The default is the Python process's architecture; `--arch` (repeatable) and `--llvm-prefix` add routes. - **13 existing tests** that run code cut from `build.py`, `make_game.py` or `host_cpu.py` now select Windows or Linux explicitly (`DARWIN` False, or `sys.platform` set to `linux`), so on a Mac they still test what they tested. No assertion was removed. They are `test_build_cleanup_scope`, `test_cpm_discovery`, `test_cpp_preflight`, `test_cpu_target`, `test_dlss_sdk_cache`, `test_driver_cache_folder`, `test_ffx_sdk`, `test_finish_output`, `test_link_path_comma`, `test_linux_preflight`, `test_make_game_ngx`, `test_shader_stamp_cache` and `test_system_header_cache`. - **`docs/development.md`:** a macOS paragraph, and rows for `RX_ARCH`, `EMSDK`/`SDKROOT`, `LONGSHOT_FFMPEG`/`LONGSHOT_SDL2`, `RX_MARCH` and `LONGSHOT_VK_PREBUILT`. - **Unchanged:** `third_party/` (byte for byte `main`'s; `scripts/check_third_party.py` passes), every Windows and Linux command, and every source under `host/` and `gpu/`. ## Flags and settings | Flag | What it does | Default now | Recommendation | Helps on | Doesn't help on | Untested on | |---|---|---|---|---|---|---| | `RX_ARCH=arm64` / `x86_64` (new) | The macOS game's and renderer's architecture; never that of the helpers Python loads | arm64 on Apple silicon, x86_64 on an Intel Mac | Native arm64; x86_64 for the Rosetta route | macOS | Windows and Linux: not read | A complete build on either; an Intel Mac | | `RX_MARCH` (changed on macOS) | The CPU level | Mac arm64: none; Mac x86_64: `x86-64-v2`; Windows and Linux unchanged | Leave the Mac defaults | macOS | | A complete build | | `EMSDK` (on macOS) | An LLVM prefix (`bin/` or `upstream/bin/`), before `PATH` and `xcrun` | `PATH`, then `xcrun` | Set it to qualify a pinned LLVM | macOS | | The whole game with LLVM 23.1.5 | | `SDKROOT` (on macOS) | Apple's SDK | `xcrun --sdk macosx --show-sdk-path` | Leave it | macOS | | | | `LONGSHOT_FFMPEG`, `LONGSHOT_SDL2` (on macOS) | An FFmpeg prefix with `.pc` files; an SDL2 `.dylib` | `pkg-config` (PATH, then Homebrew) | Libraries of the target's architecture | macOS | They don't bundle the dependencies | An x86_64 dependency set | | `LONGSHOT_VK_PREBUILT` (on macOS) | A renderer built already, `longshot_vk.dylib`; its Mach-O slice is checked | Unset | | macOS | | A Mac renderer | | Test: `--arch` (repeatable), `--llvm-prefix`, `--source`, `--capture-main` | Real-link routes; a second compiler; another tree; recapture main's arrays | The process's architecture, Apple's compiler | | the test | Not a build switch | | ## Measurements main: `5e072fc8c2adadba5ea610148be1c47800ba0cca` (2026-11-14) this PR: `mac/build-flags` @ `4723ee36a494cfafb82b18ccc685feaf2caeb964` Hardware: Apple M4 Max, 16 cores, 64 GiB, macOS 26.7.1 (25G241); Apple clang 21.0.0, LLVM 23.1.5, SDK 27.0. No GPU. Scene: none. Game-free test and suite; no game, no window. Method: `tests/test_macos_build_flags.py` on this PR and against `main`'s tree (`--source`), interleaved, two runs each. The 36 arrays: `main`'s recaptured from its own expressions, compared with the frozen file, which this PR's builders equal. Real links once with both compilers and both architectures. The whole suite on this PR, compared by test name with a `main` run on the same Mac. How to reproduce: ```sh python -I -B tests/test_macos_build_flags.py python -I -B tests/test_macos_build_flags.py --source <a checkout of main> python -I -B tests/test_macos_build_flags.py --arch arm64 --arch x86_64 --llvm-prefix <LLVM 23.1.5> python -I -B tests/test_macos_build_flags.py --capture-main --source <a checkout of main> # equals tests/build_flags_main.json python -I -B scripts/check_third_party.py ``` | | main @ 5e072fc8 | this PR @ 4723ee36 | |---|---|---| | `tests/test_macos_build_flags.py` (run 1 / run 2) | `FAILED (failures=13)`, 15 tests / the same | `OK`, 15 tests / the same | | Windows and Linux command arrays | 36, recaptured from `main`'s expressions: equal to the frozen file | 36, from the new functions: equal to the frozen file | | macOS commands (build arrays and helper compiles, arm64 and x86_64) | macOS is Linux: ELF and x86 flags | Mach-O; 40 of 40 identical to this work's earlier head, which edited a64-kit (see Reviewer notes) | | Real Mach-O links: Apple clang 21 and LLVM 23.1.5, arm64 and x86_64 | no Mach-O command path | 4 of 4 rows: 3 links each, load, exports, order and binding pass; the unresolved symbol refused | | `third_party/` | | identical to `main`; `third_party: 11 parts as their notes say` | ``` main @ 5e072fc8 (--source), cut to 4 of the 13 failures: FAIL: test_mac_is_not_linux (__main__.CommandContracts.test_mac_is_not_linux) [arm64] AssertionError: True is not false : macOS is classified as Linux and receives ELF/x86 flags FAIL: test_mac_link_paths_are_separate_and_strict (__main__.CommandContracts.test_mac_link_paths_are_separate_and_strict) AssertionError: False is not true : no Mach-O target FAIL: test_kit_and_maker_install_the_macos_toolchain (__main__.DiscoveryContracts.test_kit_and_maker_install_the_macos_toolchain) AssertionError: None is not true : main has no macOS toolchain for kit.py and make_game.py to install FAIL: test_c_cpp_links_load_exports_order_and_local_binding (__main__.RealMachO.test_c_cpp_links_load_exports_order_and_local_binding) AssertionError: False is not true : main has no callable Mach-O command path Ran 15 tests in 0.450s FAILED (failures=13) this PR @ 4723ee36, --arch arm64 --arch x86_64 --llvm-prefix LLVM-23.1.5: MACHO compiler=Apple21 arch=arm64 min=14.4 links=3 load=PASS exports=PASS order=PASS binding=PASS undefined=rejected MACHO compiler=Apple21 arch=x86_64 min=14.4 links=3 load=PASS exports=PASS order=PASS binding=PASS undefined=rejected MACHO compiler=LLVM23 arch=arm64 min=14.4 links=3 load=PASS exports=PASS order=PASS binding=PASS undefined=rejected MACHO compiler=LLVM23 arch=x86_64 min=14.4 links=3 load=PASS exports=PASS order=PASS binding=PASS undefined=rejected binding own_a=31 own_b=42 host_export=99 distinct_data=1 result=PASS (printed by each of the 4 rows) Ran 15 tests in 16.131s OK frozen JSON: 6 configurations x 6 arrays = 36 arrays (base 514591a30d707fe9f3ad31eae148c3cc1e750bff) main 5e072fc8 recaptured from its own expressions equals the frozen JSON: True nx2/build.py sha256 at 5e072fc8 == frozen record: True third_party/a64-kit/tools/toolchain.py sha256 at 5e072fc8 == frozen record: True ``` The other 9 failures on `main` name the same missing pieces: no Darwin source selection, no compiler identity, no host/target architecture split, no `pkg-config` parser. The three tests that pass on `main` are the two frozen-array comparisons and the architecture-selection helper. The frozen file says `base 514591a3`: it was captured there, and `nx2/build.py` and a64-kit's `toolchain.py` are byte-identical at `5e072fc8`. ## Testing **Tested:** the new test both ways, the dual route and the arrays, as above. And the whole suite, `tests/run_all.py`, 6 at once, on the same Mac (the runner labels macOS `linux`): ``` main @ 5e072fc8: 525 passed, 19 failed, 17 skipped in 177 s (linux, 6 at once) this PR @ 4723ee36: 526 passed, 19 failed, 17 skipped in 243 s (linux, 6 at once) ``` - **By test name**, the only difference is the new `tests/test_macos_build_flags.py` (pass). No other check changed status, and no crash report was left by this run. - **The 19 failures are the same on both.** They are macOS portability failures of `main`, which the port's other PRs address: `test_appimage_package_retry`, `test_collect_wide`, `test_control`, `test_framegen_resources`, `test_game_layout`, `test_low_memory_notice`, `test_mem_gate`, `test_mem_gate_children`, `test_mirror_trim`, `test_module_ids`, `test_oot_state_tap`, `test_pad_layout`, `test_pcache_header_read`, `test_pipeline_cache_file`, `test_pipeline_cache_save`, `test_pipeline_play_cache`, `test_prof_symbols`, `test_program_cache_cleanup`, `test_program_cache_one_variant`. - **The checks this PR touches, and the ones near it**, pass at this head (named, 17 of 17): the 13 changed tests, `test_macos_build_flags`, `test_lift_cache` (the lift's sources), and `test_spawn_emit` and `test_fpx_sticky`, which replace `build_emit.build` themselves. **Not tested:** - **Windows and Linux.** Their arrays are compared, not run. On a Mac the 13 changed tests run their Windows and Linux cases as before. - **A complete LS build or a game run on a Mac.** The runtime port (`host/`, `gpu/`) is in other PRs. - **macOS 14.4 itself, and an Intel Mac.** `minos 14.4` in a binary is not a run on 14.4. - **LLVM's x86_64 compiler runtime.** LLVM 23.1.5's arm64 archive warns that its `libclang_rt.osx.a` has no x86_64 slice. The fixtures need none of it; a larger program may. - **`python third_party/a64-kit/lift/build_emit.py`, run by hand on a Mac**, builds a64-kit's own `liba64emit.so`, as on `main`. The build loads `liba64emit-<arch>.dylib`, through `kit.py`. **Needs other testers:** - Windows, llvm-mingw and MSVC: the whole game-free suite and a standalone build. Expect the same commands and results as `main` (request to open). - Linux: the same, and an AppImage build (request to open). - An Intel Mac, or macOS 14.4: `python -I -B tests/test_macos_build_flags.py`, expecting 15 of 15 (request to open). ## Screenshots Nothing on screen. ## Risk and rollback - **Windows and Linux:** their command arrays now come from functions. They are compared element for element with `main`'s, but not run there in this PR. - **Every platform lifts the game again once** after updating: `nx2/kit.py` and the new `nx2/darwin_toolchain.py` are among the lift's sources (`make_game.py`'s `lift_sources` follows every import of the lift's, inside an `if` too), so the lift key changes. It costs one lift, minutes. Off macOS the lift runs none of the new code, so its output should be the same; that lift was not run here. - **macOS:** the toolchain functions are replaced when `kit.py` or `make_game.py` is imported. A script that imports a64-kit's `toolchain` without either gets `main`'s behaviour on a Mac, as today. - Rollback: revert the PR. No switch is needed: no command changes off macOS. ## Reviewer notes - Start with `nx2/darwin_toolchain.py` and its two install sites (`kit.py`, `make_game.py`), then `build.py`'s four new functions and `nx2/darwin_build.py`. - The commits: `da3132bc`, `06c20446` and `37587502` are this work as first written (`4766930e`, `9de9a7f2` and `7ed96d16`), which edited `third_party/a64-kit` with an updated `PROVENANCE.md`. `4723ee36` restores a64-kit to `main`'s bytes and moves those edits into `nx2/darwin_toolchain.py` (the owner's decision, after CONTRIBUTING). So the first commit's message still describes the a64-kit edit; the squash has none. - The move changes no macOS command: each tree read through its own test gives the same 40 macOS arrays and helper compiles at `7ed96d16` and at `4723ee36`, for arm64 and x86_64, plain and with every option. - Review all four commits, `da3132bc` to `4723ee36`: the whole branch is this PR's own (it depends on no other PR). ## Issues See #420. ## Checklist ### For everyone - [x] One topic. - [x] The title is a Conventional Commit (`type(scope): description`). - [x] Rebased on current `main`. (Branched from 5e072fc8.) - [x] `python -I -B tests/run_all.py` run and passing (any failure that also fails on `main` is named above). (526 passed, 17 skipped, and the same 19 failures as `main` on this Mac, named under Testing.) - [x] Tested in the game, or what wasn't is named under Testing. (No game run: build commands; named under Testing.) - [ ] Testing filled in: what was tested, what wasn't, and what needs other testers (with test requests). The test requests are still to open. - [x] Measured: main and this PR at named commits, on the same PC, scene and settings, at least 2 interleaved runs each, the median reported. (For a `build`: the test's output on `main` and here, two interleaved runs each.) - [x] No game files, dumps, build output or generated files. (`tests/build_flags_main.json` is command data from the source's own expressions.) - [x] `third_party/` untouched (a new folder with a `PROVENANCE.md` instead). - [x] The license is OK: GPL-3.0-or-later, nothing incompatible and nothing decompiled. ### If applicable - [x] AI assistance recorded with `Assisted-by:` / `Co-Authored-By:` trailers. - [x] Flags noted, and the switch reference updated (docs/development.md, the README). (Build switches only: no README row.) - [x] A game-free test added for the fix, failing on `main` and passing here. - [x] A check under `tests/` that reads changed source updated in this PR, not weakened. - [x] Docs updated. 
Separate macOS from Linux, select arm64 or x86_64 without changing the
Windows/Linux command arrays, and discover LLVM/Apple tools through the
existing prefix, PATH and xcrun. Use the Apple SDK, a 14.4 deployment
target, Mach-O export/map/order/rpath options and system libc++.

Keep the runtime port outside this change. Pick up Darwin host helpers,
exclude the Linux sampler, and test command construction, prefix quoting,
missing tools and real C/C++ dylib loading on native ARM and Rosetta.
The a64-kit adaptation and refreshed tree digest are recorded in its
PROVENANCE.md. No game or generated game content is included.

Assisted-by: GPT-6 Astra
Co-Authored-By: GPT-6 Astra <[email protected]>
Assisted-by: Claude Opus 5.5
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Do not require Rosetta in an ordinary Apple Silicon game-free suite or
skip all native x86_64 evidence on Intel. Keep both simulated target
contracts everywhere and add repeatable --arch options for explicit
dual-route qualification. Keep every binding, export, symbol-order,
C++ runtime and undefined-symbol assertion and all production flags.

Assisted-by: GPT-6 Astra
Co-Authored-By: GPT-6 Astra <[email protected]>
Assisted-by: Claude Opus 5.5
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Use canonical POSIX paths only for the virtual /fixture command inputs.
Keep host paths for real discovery, temporary-directory helpers and real
Mach-O runs, and patch exists on the selected namespace path object.

Exercise the real frozen-array comparison under an ntpath host view for
all six configurations. Leave the baseline JSON and production flags
unchanged.

Assisted-by: GPT-6 Astra
Co-Authored-By: GPT-6 Astra <[email protected]>
Assisted-by: Claude Opus 5.5
Co-Authored-By: Claude Opus 5.5 <[email protected]>
build(macos): keep third_party/a64-kit as it is; nx2/darwin_toolchain.py makes its commands Mach-O
All checks were successful
CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Has been skipped
Details
CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Has been skipped
Details
CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Has been skipped
Details
Third-party parts / The same as upstream (pull_request) Has been skipped
Details
CI / Conventional Commit title (pull_request) Successful in 4s
Details
PR description / Description follows the template (pull_request_target) Successful in 5s
Details
CI / Workflow lint (actionlint) (pull_request) Successful in 7s
Details
Third-party parts / As their notes say (pull_request) Successful in 10s
Details
Website / Website on Cloudflare (pull_request_target) Has been skipped
Details
4723ee36a4 
CONTRIBUTING keeps third_party/ as its upstream copies. Restore a64-kit's
toolchain.py, build_emit.py and PROVENANCE.md to main's bytes and move
their macOS behaviour into nx2/darwin_toolchain.py, an LS file.

On macOS, kit.py and make_game.py install it: toolchain's find, need,
native_target and native_link become its own (tools from EMSDK, PATH,
then xcrun; -arch, a 14.4 target and Apple's SDK; a Mach-O linker), and
build_emit builds liba64emit-<process arch>.dylib, rebuilt when the
compiler, its version or the flags change, as nx2/insn.py overrides
a64emit.translate. system(), architecture(), darwin_sdk() and version()
move with it; build.py, make_game.py, darwin_build.py, host_cpu.py and
zbic.py call them there. Windows and Linux never import the file.

kit.py's import makes the new file one of the lift's sources
(make_game.py lift_sources), so every platform lifts the game again
once, as the a64-kit edit did; build_emit is imported by name for that
walk (tests/test_lift_cache.py).

The command contracts now load main's own toolchain.py, so the 36
Windows/Linux arrays are compared with a64-kit untouched. A new check
proves kit.py and make_game.py install the functions on a Mac. The
test_cpm_discovery stub no longer needs toolchain.system().

Assisted-by: Claude Opus 5.5
Co-Authored-By: Claude Opus 5.5 <[email protected]>
All checks were successful
CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Has been skipped
Details
CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Has been skipped
Details
CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Has been skipped
Details
Third-party parts / The same as upstream (pull_request) Has been skipped
Details
CI / Conventional Commit title (pull_request) Successful in 4s
Details
PR description / Description follows the template (pull_request_target) Successful in 5s
Details
CI / Workflow lint (actionlint) (pull_request) Successful in 7s
Details
Third-party parts / As their notes say (pull_request) Successful in 10s
Details
Website / Website on Cloudflare (pull_request_target) Has been skipped
Details
This pull request has changes conflicting with the target branch.
  • docs/development.md
  • nx2/build.py
  • nx2/make_game.py
View command line instructions

Manual merge helper

Use this merge commit message when completing the merge manually.

Checkout

From your project repository, check out a new branch and test the changes.
git fetch -u 'mac/build-flags:mooki-mac/build-flags'
git switch 'mooki-mac/build-flags'
Sign in to join this conversation.
Reviewers
No reviewers
Labels
Clear labels   
area/accessibility
Barriers for people with disabilities

  
area/audio
Sound and music

  
area/input
Controllers, keyboard, mouse

  
area/install
Installing, updating, building from your game

  
area/menu
In-game menu and settings

  
area/renderer
Vulkan renderer, shaders, effects

  
area/saves
Save files

  
area/upscaling
DLSS, FSR, frame generation

  
good first issue
A good start for a new contributor

  
gpu/amd
AMD GPUs

  
gpu/igpu
Integrated GPUs

  
gpu/intel
Intel GPUs

  
gpu/nvidia
NVIDIA GPUs

  
help wanted
Extra hands or knowledge wanted

  
kind
bug
 Something doesn't work as it should

  
kind
chore
 Build, CI, dependencies, tidying

  
kind
docs
 Documentation

  
kind
feature
 A new feature or an improvement

  
kind
performance
 Frame rate, frame time, loading, memory

  
kind
question
 A question rather than a change

  
needs measurements
Waiting for before/after measurements

  
platform/linux
Linux only

  
platform/steam-deck
Steam Deck

  
platform/windows
Windows only

  
preview
Maintainers: deploy a preview of this PR's website (read its changes first). Removed after the run.

  
priority
critical
 Crashes, data loss, or blocks most players

  
priority
high
 Important: next in line

  
priority
low
 Nice to have

  
status
blocked
 Waits on something else first

  
status
confirmed
 Reproduced or agreed on

  
status
duplicate
 Already reported or proposed

  
status
invalid
 Not a problem in this project

  
status
needs-info
 Waiting for more information from the reporter

  
status
wontfix
 Will not be worked on

  
testing wanted
Wanted: a test on other hardware (posts a test request in #test-requests)

No labels
area/accessibility
area/audio
area/input
area/install
area/menu
area/renderer
area/saves
area/upscaling
good first issue
gpu/amd
gpu/igpu
gpu/intel
gpu/nvidia
help wanted
kind
bug
kind
chore
kind
docs
kind
feature
kind
performance
kind
question
needs measurements
platform/linux
platform/steam-deck
platform/windows
preview
priority
critical
priority
high
priority
low
status
blocked
status
confirmed
status
duplicate
status
invalid
status
needs-info
status
wontfix
testing wanted
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
MagicBeanDev/Longshot!453
No description provided.