refactor(dev): docs, tools and process revamp: one dev.py, dev.py ab, one dev folder, a short PR template, faster dev builds #1802

Merged
HumanBean merged 1 commit from HumanBean/Longshot:refactor/dev-revamp into main 2026-11-20 00:34:15 +00:00
Owner

Summary

A revamp of the developer docs, tools and process, so development is simpler and faster. Nothing a player sees changes,
apart from the run name now also showing on the start-up screen of a development run.

  • Docs. New docs/dev/ (setup, developing, standards, per-area reference pages, CI). README is a landing page with
    features and current goals; the player guide in docs/user/ takes its content (one requirements page, an advanced
    setup page, the settings table the website builds from). docs/old/, the tested-configurations catalogue and the
    in-repo worklist are gone (work is picked from the open issues by the triage bot's labels). AGENTS.md, CONTRIBUTING.md
    and RELEASING.md are short.
  • Process. A short pull request template (Summary, Testing, Measurements, Risks, and the checklists Ready for
    review
    and Ready to merge) and a checker for the rules that matter (807 → ~480 lines); the three skills follow it.
  • Tools removed. The offline shader bake, nine unused nx2/ tools, shadertool, VULKAN.md, committed MSBuild
    output.
  • Tools rebuilt.
    • scripts/dev.py: one flat command set (base, pack, rt, game, vk, env, bases, queue, paths,
      migrate, ab); dev.py packs stays the Test Farm's folder-level layer with the same command lines.
    • All developer data in one folder (~/.local/share/longshot-dev, %LOCALAPPDATA%\longshot-dev) apart from the
      player's; dev.py migrate moves an existing setup.
    • dev.py ab: one command for a correct A/B (skips the title, loads the scene, warms up, interleaves the runs in the
      queue in timing mode, refuses when the sides differ, writes the Measurements section).
    • dev.py queue run --desc "<what it tests>": the run's name shows under the version in the menu and on the start-up
      screen.
    • The control bridge client is a library (scripts/longshot_bridge.py); the tour is split (scripts/tours/);
      profiling has one data flow; one scripts/helpers.py for downloads, hashes, archives; release.py comes from
      Tooling at the commit in .tooling-rev.
    • mise run setup does the whole setup (Tooling, hooks, deps, plugin check, toolchain check); mise run ci runs what
      CI runs; mise run test-changed; ruff in lint. dev.py works in jj workspaces.
  • Faster dev builds. Comment and docstring edits to the translator don't re-lift; tools/CI/docs changes build
    nothing; the version lives in two tiny files so a version change doesn't recompile the menu; game-only packs skip the
    renderer; packs keep unchanged outputs; runtimes are cloned and rt refreshes in place; the Windows header stamp is by
    content. #1792's romfs-in-place is ported onto the new devtools.
  • Comments. Code comments state the rule and its reason, without PR numbers and history (comments only; checked
    mechanically).

Testing

Tested: not run in the game, and no tests run on this PC (the maintainer asked to open it without a game run).
Static checks only: every changed Python file compiles, --help of every changed tool, bash -n, actionlint, a link
and anchor check over all docs (none broken; every // doc: anchor resolves), and two cold reviews by agents that did
not write the code, whose findings are fixed. The first real check is CI.

Still to test:

  • Linux and Windows, any GPU: the CI checks (run_all, the website build), by CI.
  • Linux and Windows: a full build and a start of the game (the version split into host/version.c and
    gpu/vk/vk_version.cpp, the start-up screen's run name, the comments-only sweep compiles), by hand.
  • dev.py ab --env HOS_COLLECT_WIDE=0 --runs 1 --seconds 20 --scene kokiri-house on a runtime, expecting a write-up, by
    hand.
  • dev.py migrate --dry-run, then dev.py migrate on a PC with the old layout, expecting everything moved and links
    left, by hand.
  • The Test Farm builds a base, a pack and a runtime with this branch's dev.py packs (@Beanstalk, after Beanstalk's
    change below is deployed).

Measurements

Nothing to measure: docs, dev tools and comments; the game's code is unchanged apart from the version's file and the
start-up screen's name line.

Risks

  • Order: Beanstalk's change (dev.py packs in place of scripts/dev_packs.py; scripts/dev.py,
    scripts/devtools/, scripts/helpers.py need two approvers) must be deployed first, or the Test Farm breaks.
    Tooling's change (forge.py for the new template, the shared release.py) lands together with this one, and
    .tooling-rev gets its commit (a placeholder now: release jobs fail at checkout until it is set).
  • One relift on every machine and for players' next update (the lift key now hashes translator modules by code).
  • Open PRs on the old template fail the description check until their description is moved to the new one.
  • Old dev commands (job_queue.py, dev_build.py, dev_packs.py, dev.py build ...) are gone; existing dev PCs run
    python -I scripts/dev.py migrate once while the queue is empty.

Checklist

Ready for review

  • Tested on my own PC where it runs (in the game, for a change the game runs), as Testing's Tested says.
  • Measured against main, or one line saying why there is nothing to measure.
  • The checks pass (CI).
  • Rebased on current main.
  • No game files, dumps, build output or generated files.

When it applies

  • AI assistance recorded with Co-Authored-By: trailers.
  • Switches documented (their rows in gpu/vk/switches.def) and shipped at their recommended value.
  • A check under tests/ that reads changed source updated in this PR, not weakened.
  • Docs updated.

Ready to merge

  • Every Still to test item has run, its result in Tested.
  • Approved in review, every comment answered.
  • CI passes on the head commit.
  • On current main with no conflicts.

🤖 Generated with Claude Code

## Summary A revamp of the developer docs, tools and process, so development is simpler and faster. Nothing a player sees changes, apart from the run name now also showing on the start-up screen of a development run. - **Docs.** New `docs/dev/` (setup, developing, standards, per-area reference pages, CI). README is a landing page with features and current goals; the player guide in `docs/user/` takes its content (one requirements page, an advanced setup page, the settings table the website builds from). `docs/old/`, the tested-configurations catalogue and the in-repo worklist are gone (work is picked from the open issues by the triage bot's labels). AGENTS.md, CONTRIBUTING.md and RELEASING.md are short. - **Process.** A short pull request template (Summary, Testing, Measurements, Risks, and the checklists *Ready for review* and *Ready to merge*) and a checker for the rules that matter (807 → ~480 lines); the three skills follow it. - **Tools removed.** The offline shader bake, nine unused `nx2/` tools, `shadertool`, `VULKAN.md`, committed MSBuild output. - **Tools rebuilt.** - `scripts/dev.py`: one flat command set (`base`, `pack`, `rt`, `game`, `vk`, `env`, `bases`, `queue`, `paths`, `migrate`, `ab`); `dev.py packs` stays the Test Farm's folder-level layer with the same command lines. - All developer data in one folder (`~/.local/share/longshot-dev`, `%LOCALAPPDATA%\longshot-dev`) apart from the player's; `dev.py migrate` moves an existing setup. - `dev.py ab`: one command for a correct A/B (skips the title, loads the scene, warms up, interleaves the runs in the queue in timing mode, refuses when the sides differ, writes the Measurements section). - `dev.py queue run --desc "<what it tests>"`: the run's name shows under the version in the menu and on the start-up screen. - The control bridge client is a library (`scripts/longshot_bridge.py`); the tour is split (`scripts/tours/`); profiling has one data flow; one `scripts/helpers.py` for downloads, hashes, archives; `release.py` comes from Tooling at the commit in `.tooling-rev`. - `mise run setup` does the whole setup (Tooling, hooks, deps, plugin check, toolchain check); `mise run ci` runs what CI runs; `mise run test-changed`; ruff in lint. `dev.py` works in jj workspaces. - **Faster dev builds.** Comment and docstring edits to the translator don't re-lift; tools/CI/docs changes build nothing; the version lives in two tiny files so a version change doesn't recompile the menu; game-only packs skip the renderer; packs keep unchanged outputs; runtimes are cloned and `rt` refreshes in place; the Windows header stamp is by content. #1792's romfs-in-place is ported onto the new devtools. - **Comments.** Code comments state the rule and its reason, without PR numbers and history (comments only; checked mechanically). ## Testing **Tested:** not run in the game, and no tests run on this PC (the maintainer asked to open it without a game run). Static checks only: every changed Python file compiles, `--help` of every changed tool, `bash -n`, `actionlint`, a link and anchor check over all docs (none broken; every `// doc:` anchor resolves), and two cold reviews by agents that did not write the code, whose findings are fixed. The first real check is CI. **Still to test:** - Linux and Windows, any GPU: the CI checks (`run_all`, the website build), by CI. - Linux and Windows: a full build and a start of the game (the version split into `host/version.c` and `gpu/vk/vk_version.cpp`, the start-up screen's run name, the comments-only sweep compiles), by hand. - `dev.py ab --env HOS_COLLECT_WIDE=0 --runs 1 --seconds 20 --scene kokiri-house` on a runtime, expecting a write-up, by hand. - `dev.py migrate --dry-run`, then `dev.py migrate` on a PC with the old layout, expecting everything moved and links left, by hand. - The Test Farm builds a base, a pack and a runtime with this branch's `dev.py packs` (@Beanstalk, after Beanstalk's change below is deployed). ## Measurements Nothing to measure: docs, dev tools and comments; the game's code is unchanged apart from the version's file and the start-up screen's name line. ## Risks - **Order:** Beanstalk's change (`dev.py packs` in place of `scripts/dev_packs.py`; `scripts/dev.py`, `scripts/devtools/`, `scripts/helpers.py` need two approvers) must be deployed first, or the Test Farm breaks. Tooling's change (forge.py for the new template, the shared `release.py`) lands together with this one, and `.tooling-rev` gets its commit (a placeholder now: release jobs fail at checkout until it is set). - **One relift** on every machine and for players' next update (the lift key now hashes translator modules by code). - **Open PRs on the old template** fail the description check until their description is moved to the new one. - Old dev commands (`job_queue.py`, `dev_build.py`, `dev_packs.py`, `dev.py build ...`) are gone; existing dev PCs run `python -I scripts/dev.py migrate` once while the queue is empty. ## Checklist ### Ready for review - [ ] Tested on my own PC where it runs (in the game, for a change the game runs), as Testing's *Tested* says. - [ ] Measured against `main`, or one line saying why there is nothing to measure. - [ ] The checks pass (CI). - [ ] Rebased on current `main`. - [ ] No game files, dumps, build output or generated files. ### When it applies - [ ] AI assistance recorded with `Co-Authored-By:` trailers. - [ ] Switches documented (their rows in gpu/vk/switches.def) and shipped at their recommended value. - [ ] A check under `tests/` that reads changed source updated in this PR, not weakened. - [ ] Docs updated. ### Ready to merge - [ ] Every *Still to test* item has run, its result in *Tested*. - [ ] Approved in review, every comment answered. - [ ] CI passes on the head commit. - [ ] On current `main` with no conflicts. 🤖 Generated with [Claude Code](https://claude.com/claude-code)
Also: the tested-configurations catalogue (docs/user/tested, its website page, validator and tests) is removed;
pull requests separate the out-of-WIP checklist from a Before merge list; claims go to the team room.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Co-Authored-By: Claude Opus 5.5 <[email protected]>
docs: pick work by the triage bot's labels (regression, priority/critical, high, low)
Some checks failed
CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Has been skipped
CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Has been skipped
CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Has been skipped
Third-party parts / The same as upstream (pull_request) Has been skipped
Website / Website on Cloudflare (pull_request) Has been skipped
CI / Conventional Commit title (pull_request) Successful in 20s
Website / Website on Cloudflare (pull_request_target) Has been skipped
Guide screenshots / Guide images and local UI source review (pull_request) Failing after 30s
CI / Workflow lint (actionlint) (pull_request) Successful in 32s
PR description / Description follows the template (pull_request_target) Successful in 32s
Third-party parts / As their notes say (pull_request) Successful in 37s
CI / Website build and checks (pull_request) Has been cancelled
d7cc51f4d7
Co-Authored-By: Claude Opus 5.5 <[email protected]>
HumanBean force-pushed refactor/dev-revamp from d7cc51f4d7
Some checks failed
CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Has been skipped
CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Has been skipped
CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Has been skipped
Third-party parts / The same as upstream (pull_request) Has been skipped
Website / Website on Cloudflare (pull_request) Has been skipped
CI / Conventional Commit title (pull_request) Successful in 20s
Website / Website on Cloudflare (pull_request_target) Has been skipped
Guide screenshots / Guide images and local UI source review (pull_request) Failing after 30s
CI / Workflow lint (actionlint) (pull_request) Successful in 32s
PR description / Description follows the template (pull_request_target) Successful in 32s
Third-party parts / As their notes say (pull_request) Successful in 37s
CI / Website build and checks (pull_request) Has been cancelled
to 75fbfb8e00
Some checks failed
CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Has been skipped
CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Has been skipped
CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Has been skipped
Third-party parts / The same as upstream (pull_request) Has been skipped
Website / Website on Cloudflare (pull_request) Has been skipped
CI / Conventional Commit title (pull_request) Successful in 18s
Guide screenshots / Guide images and local UI source review (pull_request) Failing after 24s
CI / Workflow lint (actionlint) (pull_request) Successful in 26s
Third-party parts / As their notes say (pull_request) Successful in 28s
PR description / Description follows the template (pull_request_target) Failing after 8s
CI / Website build and checks (pull_request) Failing after 3m22s
2026-11-19 22:06:32 +00:00
Compare
HumanBean changed title from WIP: refactor(dev): docs, tools and process revamp: one dev.py, dev.py ab, one dev folder, a short PR template, faster dev builds to refactor(dev): docs, tools and process revamp: one dev.py, dev.py ab, one dev folder, a short PR template, faster dev builds 2026-11-20 00:34:00 +00:00
HumanBean deleted branch refactor/dev-revamp 2026-11-20 00:34:17 +00:00
Sign in to join this conversation.
No reviewers
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
MagicBeanDev/Longshot!1802
No description provided.