feat(dev): dev_build.py on Windows too, with tools.lock's Windows build tools #1095

Merged
HumanBean merged 1 commit from dyst/Longshot:feat/dev-build-windows into main 2026-11-16 08:48:45 +00:00
Member

Type: feat · Risk: low · Tested on: Windows 11, Ryzen 5 (Zen 2); Linux (python:3.12 container, game-free checks) · Needs: Linux, a real dev_build.py env / vk run

Why

scripts/dev_build.py is how agents and people build on a shared PC (one base per main commit, packs, runtimes, all through the job queue), but it refused Windows outright, so on a Windows PC everyone fell back to hand-set dev_packs.py and make_game.py runs with their own tool folders and stores, and no two checkouts shared a base. This makes the same commands work on Windows, with the build tools a player's Windows build uses.

What changed

dev_build.py now runs on Windows as well as Linux: the same commands, with Windows' folders, tools.lock's windows-x64 tools set as the Windows builder sets them, and a Windows check of the renderer. Linux behaves as before (the same folders, tools, Python, messages and checks).

  • folders (home, cache, own_dir, config_file): LONGSHOT_DEV_HOME defaults to %LOCALAPPDATA%\Longshot\dev, the stores to dev_packs.py's Windows defaults (%LOCALAPPDATA%\Longshot\objects, lifts), the settings file is %APPDATA%\Longshot\dev_build.json (beside job_queue.json), and a checkout's own files go in build-windows\ (added to .gitignore). The tmpfs check is already a no-op on Windows (job_queue.on_ram).
  • build tools (tools_lock_key, builder_dir, windows_toolchain, windows_tool_dirs): tools.lock's lines for this system. On Windows scripts/tools.py install (in-process, under a lock, downloads into LONGSHOT_DEV_HOME\builder\downloads) installs the windows-x64 set once per tools.lock, and the build gets LONGSHOT_MINGW, EMSDK and RX_CPM set to it, as builder/LongshotBuilder/Runner.cs and make_game.py bundled_mingw set them. LONGSHOT_DEV_DEPS (a build-linux/deps folder) stays Linux-only and is refused on Windows with a message.
  • Python (python_candidates, find_python): on Windows the build tools' own python\python.exe (which has capstone, unicorn, zstandard, numpy), unless LONGSHOT_DEV_PYTHON names another.
  • prebuilt files (windows_prebuilt): SDL2.dll and SDL3.dll from the tools' third_party are copied into the tree a base, pack or build reads (third_party\sdl2) when they are not there, as the Windows builder's tools folder has them. Nothing in the repository builds them; git ignores *.dll, so dev_packs.py sees no change in the tree.
  • renderer (cmd_vk, check_dll_exports): longshot_vk.dll on Windows. An undefined symbol already fails the link there, so instead of nm -D it checks with llvm-objdump (llvm-mingw's) that the DLL exports mkwvk_init, which host/rx_backend.c looks up. --copy-to copies the DLL.
  • launcher and game files (launcher_ngx_sdk, game_roots): the Longshot Launcher's NGX SDK at %LOCALAPPDATA%\LongshotLauncher\ngx-sdk (nvsdk_ngx_s.lib, its LauncherHome.NgxSdk), and game folders in ~\Games and ~\Games\Longshot (the launcher's default builds folder). LONGSHOT_DEV_ROMFS / LONGSHOT_DEV_EXEFS and the config's romfs / exefs are still the explicit way.
  • BaseLock: msvcrt's lock on Windows (fcntl on Linux, as before). env prints PowerShell $env: lines on Windows (export lines on Linux, as before).
  • tests/test_dev_build.py (new): both systems' folders, tools.lock keys, tool variables against Runner.cs and tools.py REQUIRED, the Python, NGX, shell lines, the prebuilt copy, the export parser against host/rx_backend.c, and the base lock.
  • docs/development.md, Sharing a PC: the job queue: a paragraph on Windows.

Measurements

Nothing to measure: a development script and its tests, and the game is not touched.

Flags and settings

None (no new switch: LONGSHOT_DEV_HOME, LONGSHOT_DEV_PYTHON, LONGSHOT_DEV_DEPS, LONGSHOT_DEV_ROMFS and LONGSHOT_DEV_EXEFS were already there, and only their Windows defaults are new. LONGSHOT_MINGW, EMSDK and RX_CPM are the build's existing variables, which dev_build.py now sets on Windows as the Windows builder does).

Testing

Tested:

  • Windows 11, from a clean dev-home on D: (LONGSHOT_DEV_HOME=D:/HookshotDev/dev-home): python -I scripts/dev_build.py env installed tools.lock's five Windows tools (llvm-mingw, python, emsdk, cpm, third-party) in 52 s and printed LONGSHOT_MINGW, EMSDK, RX_CPM and PY pointing at them. A second run found them installed and printed the same. --json printed the same.
  • Windows 11: python -I scripts/dev_build.py vk through the job queue (a light job, beside another agent's base build): it compiled the renderer's 207 files with the installed llvm-mingw and cpm (1109 s, 1 job at a time beside the other build), linked build-windows\vk-work\sa\build\longshot_vk.dll (13 MB) and passed the export check (renderer: ... (symbols checked)). The same check on that DLL, asked for a name it does not export, fails with longshot_vk.dll does not export mkwvk_nope: the game would find no renderer.
  • Game-free checks: tests/test_dev_build.py passes on Windows and on Linux (python:3.12 container). The checks that read the changed files (run_all.py --changed, plus test_job_queue, test_tools_lock, test_linux_tools), run on this branch and on main c167ec67: the same results on both. On Windows test_vk_occlusion_queries, test_crash_dialog and test_walk_key fail on main too, and test_job_queue and test_linux_tools need fcntl. In the read-only Linux container test_dev_packs, test_check_third_party, test_vk_occlusion_queries and test_walk_key fail on main too (no git in a mounted worktree). Everything else passes.

Not tested:

  • dev_build.py base, pack, rt and build on Windows: they run dev_packs.py and make_game.py, which already build on Windows, with this environment. A base would compile the whole game here: the PC's object store was filled with another tool folder, and the objects' flags hold its paths, so nothing would be taken from it. That is well over half an hour beside the timed runs on this PC.
  • A real Linux run of env / vk: only the game-free checks ran on Linux.

Must be tested:

  • Linux, any GPU: python -I scripts/dev_build.py env and python -I scripts/dev_build.py vk in a checkout of this branch, expecting the same output and folders as on main (by hand: a Linux agent on a shared PC)

Risks

What could break: Linux runs of dev_build.py if a path changed by mistake (the tests pin home, stores, build-linux/, the Python candidates, the tools.lock key and the export lines to their old values). On Windows, a tree's third_party\sdl2 gains the builder's SDL2.dll / SDL3.dll, which git ignores.

Turning it off: revert the commit.

Compatibility: Nothing changes for players: only a development script, its test, .gitignore and the docs.

Checklist

Everyone

  • One topic.
  • The title is a Conventional Commit (type(scope): description) of the status line's type.
  • Why gives the reason plainly and links the issues that fit (the tracker searched).
  • Measurements has a verdict, or one line saying why there is nothing to measure.
  • Testing names what was tested, what wasn't, and what must still be tested, where and how.
  • Rebased on current main.
  • python -I -B tests/run_all.py passes (any failure that also fails on main is named).
  • No game files, dumps, build output or generated files.
  • third_party/ untouched (a new folder with a PROVENANCE.md instead).
  • The license is OK: GPL-3.0-or-later, nothing incompatible and nothing decompiled.

When it applies

  • AI assistance recorded with Assisted-by: / Co-Authored-By: trailers.
  • Switches documented (docs/development.md; the README for a menu row) and shipped at their recommended value.
  • A check under tests/ that reads changed source updated in this PR, not weakened.
  • Docs updated.

🤖 Generated with Claude Code

**Type:** feat · **Risk:** low · **Tested on:** Windows 11, Ryzen 5 (Zen 2); Linux (python:3.12 container, game-free checks) · **Needs:** Linux, a real `dev_build.py env` / `vk` run ## Why `scripts/dev_build.py` is how agents and people build on a shared PC (one base per main commit, packs, runtimes, all through the job queue), but it refused Windows outright, so on a Windows PC everyone fell back to hand-set `dev_packs.py` and `make_game.py` runs with their own tool folders and stores, and no two checkouts shared a base. This makes the same commands work on Windows, with the build tools a player's Windows build uses. ## What changed `dev_build.py` now runs on Windows as well as Linux: the same commands, with Windows' folders, tools.lock's `windows-x64` tools set as the Windows builder sets them, and a Windows check of the renderer. Linux behaves as before (the same folders, tools, Python, messages and checks). - folders (`home`, `cache`, `own_dir`, `config_file`): `LONGSHOT_DEV_HOME` defaults to `%LOCALAPPDATA%\Longshot\dev`, the stores to `dev_packs.py`'s Windows defaults (`%LOCALAPPDATA%\Longshot\objects`, `lifts`), the settings file is `%APPDATA%\Longshot\dev_build.json` (beside `job_queue.json`), and a checkout's own files go in `build-windows\` (added to `.gitignore`). The tmpfs check is already a no-op on Windows (`job_queue.on_ram`). - build tools (`tools_lock_key`, `builder_dir`, `windows_toolchain`, `windows_tool_dirs`): tools.lock's lines for this system. On Windows `scripts/tools.py install` (in-process, under a lock, downloads into `LONGSHOT_DEV_HOME\builder\downloads`) installs the `windows-x64` set once per tools.lock, and the build gets `LONGSHOT_MINGW`, `EMSDK` and `RX_CPM` set to it, as `builder/LongshotBuilder/Runner.cs` and `make_game.py bundled_mingw` set them. `LONGSHOT_DEV_DEPS` (a `build-linux/deps` folder) stays Linux-only and is refused on Windows with a message. - Python (`python_candidates`, `find_python`): on Windows the build tools' own `python\python.exe` (which has capstone, unicorn, zstandard, numpy), unless `LONGSHOT_DEV_PYTHON` names another. - prebuilt files (`windows_prebuilt`): `SDL2.dll` and `SDL3.dll` from the tools' `third_party` are copied into the tree a base, pack or build reads (`third_party\sdl2`) when they are not there, as the Windows builder's tools folder has them. Nothing in the repository builds them; git ignores `*.dll`, so `dev_packs.py` sees no change in the tree. - renderer (`cmd_vk`, `check_dll_exports`): `longshot_vk.dll` on Windows. An undefined symbol already fails the link there, so instead of `nm -D` it checks with llvm-objdump (llvm-mingw's) that the DLL exports `mkwvk_init`, which `host/rx_backend.c` looks up. `--copy-to` copies the DLL. - launcher and game files (`launcher_ngx_sdk`, `game_roots`): the Longshot Launcher's NGX SDK at `%LOCALAPPDATA%\LongshotLauncher\ngx-sdk` (`nvsdk_ngx_s.lib`, its `LauncherHome.NgxSdk`), and game folders in `~\Games` and `~\Games\Longshot` (the launcher's default builds folder). `LONGSHOT_DEV_ROMFS` / `LONGSHOT_DEV_EXEFS` and the config's `romfs` / `exefs` are still the explicit way. - `BaseLock`: msvcrt's lock on Windows (fcntl on Linux, as before). `env` prints PowerShell `$env:` lines on Windows (`export` lines on Linux, as before). - `tests/test_dev_build.py` (new): both systems' folders, tools.lock keys, tool variables against `Runner.cs` and `tools.py REQUIRED`, the Python, NGX, shell lines, the prebuilt copy, the export parser against `host/rx_backend.c`, and the base lock. - `docs/development.md`, *Sharing a PC: the job queue*: a paragraph on Windows. ## Measurements Nothing to measure: a development script and its tests, and the game is not touched. ### Flags and settings None (no new switch: `LONGSHOT_DEV_HOME`, `LONGSHOT_DEV_PYTHON`, `LONGSHOT_DEV_DEPS`, `LONGSHOT_DEV_ROMFS` and `LONGSHOT_DEV_EXEFS` were already there, and only their Windows defaults are new. `LONGSHOT_MINGW`, `EMSDK` and `RX_CPM` are the build's existing variables, which `dev_build.py` now sets on Windows as the Windows builder does). ## Testing **Tested:** - Windows 11, from a clean `dev-home` on D: (`LONGSHOT_DEV_HOME=D:/HookshotDev/dev-home`): `python -I scripts/dev_build.py env` installed tools.lock's five Windows tools (llvm-mingw, python, emsdk, cpm, third-party) in 52 s and printed `LONGSHOT_MINGW`, `EMSDK`, `RX_CPM` and `PY` pointing at them. A second run found them installed and printed the same. `--json` printed the same. - Windows 11: `python -I scripts/dev_build.py vk` through the job queue (a `light` job, beside another agent's base build): it compiled the renderer's 207 files with the installed llvm-mingw and cpm (1109 s, 1 job at a time beside the other build), linked `build-windows\vk-work\sa\build\longshot_vk.dll` (13 MB) and passed the export check (`renderer: ... (symbols checked)`). The same check on that DLL, asked for a name it does not export, fails with `longshot_vk.dll does not export mkwvk_nope: the game would find no renderer`. - Game-free checks: `tests/test_dev_build.py` passes on Windows and on Linux (python:3.12 container). The checks that read the changed files (`run_all.py --changed`, plus `test_job_queue`, `test_tools_lock`, `test_linux_tools`), run on this branch and on main c167ec67: the same results on both. On Windows `test_vk_occlusion_queries`, `test_crash_dialog` and `test_walk_key` fail on main too, and `test_job_queue` and `test_linux_tools` need fcntl. In the read-only Linux container `test_dev_packs`, `test_check_third_party`, `test_vk_occlusion_queries` and `test_walk_key` fail on main too (no git in a mounted worktree). Everything else passes. **Not tested:** - `dev_build.py base`, `pack`, `rt` and `build` on Windows: they run `dev_packs.py` and `make_game.py`, which already build on Windows, with this environment. A base would compile the whole game here: the PC's object store was filled with another tool folder, and the objects' flags hold its paths, so nothing would be taken from it. That is well over half an hour beside the timed runs on this PC. - A real Linux run of `env` / `vk`: only the game-free checks ran on Linux. **Must be tested:** - Linux, any GPU: `python -I scripts/dev_build.py env` and `python -I scripts/dev_build.py vk` in a checkout of this branch, expecting the same output and folders as on main (by hand: a Linux agent on a shared PC) ## Risks **What could break:** Linux runs of `dev_build.py` if a path changed by mistake (the tests pin home, stores, `build-linux/`, the Python candidates, the tools.lock key and the `export` lines to their old values). On Windows, a tree's `third_party\sdl2` gains the builder's `SDL2.dll` / `SDL3.dll`, which git ignores. **Turning it off:** revert the commit. **Compatibility:** Nothing changes for players: only a development script, its test, `.gitignore` and the docs. ## Checklist ### Everyone - [x] One topic. - [x] The title is a Conventional Commit (`type(scope): description`) of the status line's type. - [x] Why gives the reason plainly and links the issues that fit (the tracker searched). - [x] Measurements has a verdict, or one line saying why there is nothing to measure. - [x] Testing names what was tested, what wasn't, and what must still be tested, where and how. - [x] Rebased on current `main`. - [x] `python -I -B tests/run_all.py` passes (any failure that also fails on `main` is named). - [x] No game files, dumps, build output or generated files. - [x] `third_party/` untouched (a new folder with a `PROVENANCE.md` instead). - [x] The license is OK: GPL-3.0-or-later, nothing incompatible and nothing decompiled. ### When it applies - [x] AI assistance recorded with `Assisted-by:` / `Co-Authored-By:` trailers. - [x] Switches documented (docs/development.md; the README for a menu row) and shipped at their recommended value. - [x] A check under `tests/` that reads changed source updated in this PR, not weakened. - [x] Docs updated. 🤖 Generated with [Claude Code](https://claude.com/claude-code)
feat(dev): dev_build.py on Windows too, with tools.lock's Windows build tools
Some checks failed
CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Has been skipped
CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Has been skipped
CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Has been skipped
Third-party parts / The same as upstream (pull_request) Has been skipped
Website / Website on Cloudflare (pull_request) Has been skipped
CI / Conventional Commit title (pull_request) Successful in 6s
Guide screenshots / Guide images and local UI source review (pull_request) Failing after 11s
CI / Workflow lint (actionlint) (pull_request) Successful in 12s
Website / Website on Cloudflare (pull_request_target) Has been skipped
Third-party parts / As their notes say (pull_request) Successful in 14s
PR description / Description follows the template (pull_request_target) Successful in 3s
754e9f4a45
dev_build.py refused Windows, so a shared Windows PC had no shared bases and no common build environment. It now runs
there with Windows' folders (LONGSHOT_DEV_HOME %LOCALAPPDATA%\Longshot\dev, dev_packs.py's stores, %APPDATA%\Longshot\
dev_build.json, build-windows\), tools.lock's windows-x64 tools installed by scripts/tools.py and set as the Windows
builder sets them (LONGSHOT_MINGW, EMSDK, RX_CPM; its Python runs the build; its prebuilt SDL2.dll and SDL3.dll in the
tree), the launcher's NGX SDK and builds folder, an msvcrt base lock, PowerShell lines from `env`, and a check that
longshot_vk.dll exports mkwvk_init in place of nm's undefined-symbol check. Linux is as it was.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
dyst changed title from WIP: feat(dev): dev_build.py on Windows too, with tools.lock's Windows build tools to feat(dev): dev_build.py on Windows too, with tools.lock's Windows build tools 2026-11-16 08:45:18 +00:00
HumanBean deleted branch feat/dev-build-windows 2026-11-16 08:48:45 +00:00
Sign in to join this conversation.
No reviewers
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
MagicBeanDev/Longshot!1095
No description provided.