feat(scripts): the PC's job queue (job_queue.py) and dev_build.py, whose builds share one translation and one set of objects #1085

Merged
HumanBean merged 1 commit from HumanBean/Longshot:feat/job-queue into main 2026-11-16 07:46:21 +00:00
Owner

Type: feat · Risk: low · Tested on: Linux, RTX 3090 · Needs: none

Why

Several agents (and the person they work for) share one PC to build and test Longshot. Builds, game runs and measurements went through a queue kept outside the repository, and under it no build ever took another build's translation from the lift store. That queue gives each job OOT_STATUS_LOCK=<job name>, and make_game.py lift_key hashes every OOT_ variable. So every job had its own lift key, and every new work folder translated the game again. The queue's rules (no build beside a timed run, the GPU shared or not, memory, nothing in /tmp) were also known only to whoever had that script. This puts the queue and a build wrapper in the repository, keeps lift keys clean, and gives every checkout the same build environment.

What changed

Two scripts. scripts/job_queue.py runs builds and game runs when they fit and keeps measurements apart. scripts/dev_build.py builds bases, packs, runtimes and whole game folders through it, all with one environment, so the second build of anything takes its translation and its objects from the shared stores. Neither changes what players build.

  • scripts/job_queue.py (run NAME --mode M -- CMD, status, history, cancel):

    • modes build, light, shared, gpu, timing, timing-cpu and profile, each a GPU use and a CPU use;
    • one rule for order (blocker): what runs holds what it uses, and every ready ticket ahead keeps its claim, so nothing jumps ahead;
    • priorities with aging, and timed runs of one group one at a time;
    • the memory gate (available memory, PSI, swap);
    • a timed run on its own L3 cache domain while everything else moves to the others; it isn't swapped out, waits while you play and while a build runs outside the queue, and records how busy the PC was;
    • a systemd scope per job, so the OOM killer stops the job and not the terminal;
    • TMPDIR moved off a RAM file system, and the NVIDIA shader cache kept apart for GPU jobs.

    A job gets LONGSHOT_JOB*, never a new OOT_, RX_ or A64_ variable, and the older queue's OOT_STATUS_* is dropped. The older queue's running tickets count as running jobs, so the two can be used side by side while sessions move over.

  • scripts/dev_build.py (env, base, pack, rt, build, vk, ls), Linux:

    • Build tools: a player's, from tools.lock through builder/linux/linux_tools.py, installed once per tools.lock under ~/.cache/longshot-dev/builder/ (no more tools borrowed from an old checkout), plus a Python with the build's packages.
    • Stores: dev_packs.py's object and translation stores, with LONGSHOT_LIFT_STORE_KEEP=6. The launcher's NGX SDK is used for DLSS.
    • Lift key: it names any A64_, OOT_ or RX_ variable you set that will enter the lift key.
    • Bases: one per commit of main in ~/.cache/longshot-dev/bases/, built by whoever asks first while the others wait for it, from a temporary workspace of that commit.
    • This checkout's files stay in its own build-linux/, so they go when the checkout goes.
    • No RAM file systems: it refuses a folder on one.
  • docs/development.md: Sharing a PC: the job queue (the modes, the rules, why no OOT_ variable, nothing on tmpfs). AGENTS.md: builds and game runs go through the queue, never on a RAM file system.

  • tests/test_job_queue.py: the rules above, the job environment, the RAM check, and two real queues (exclusive jobs one after the other, shared ones together).

Measurements

Verdict: the second build of the same main in a new work folder took 22 s instead of 6 min 48 s. It copied the first build's translation from the lift store and took 512 of 514 objects from the object store; only the two objects that carry the version were compiled. Under the older queue's variable, the lift share key changes with each job name, so that build would have translated the game again.

Hardware: Linux (CachyOS), RTX 3090, driver 620.70.06, Ryzen 9 7900X, 30 GB
Method: the build's own where the time went lines; make_game.lift_share_key called with and without the variables. One run of each: the difference is what was compiled and translated, not noise.
How to reproduce: dev_build.py base, then dev_build.py build in another checkout.

First build: dev_build.py base (main @ 02903b04, an empty store for this toolchain) Second build: dev_build.py build, this PR's checkout, a new work folder
Translation translated (74 s) copied from the lift store
Objects compiled all 2 of 514 (512 from the store)
Build (without the shader preparation) 6 min 48 s 22 s
first:  object store /home/<user>/.cache/longshot-objects/3fcd1e7be0def429: 0 objects from it
        the build took 0:06:48, compiled in 0:05:21
second: the game was translated with these tools in another work folder: its translation is copied (.../longshot-lifts/1ac5aac97ebb52f806edf3b0d906e3891580e67b)
        object store .../longshot-objects/3fcd1e7be0def429: 512 objects from it
        @@plan 2 514
        the build took 0:00:22, compiled in 0:00:11
lift share key: no queue 6a14b1158d5b | OOT_STATUS_LOCK=build-x 8668ae8cbdf5 | =build-y 17efcb610dab | job_queue (LONGSHOT_JOB*) 6a14b1158d5b

Flags and settings

No game switch. The scripts' own settings are environment variables, documented in their notes and in docs/development.md:

  • LONGSHOT_QUEUE_*: the slots, the memory limits, the CPU sets (by default the first L3 domain for timed runs), LONGSHOT_QUEUE_QUIET and the folders.
  • LONGSHOT_DEV_*: HOME, PYTHON, DEPS, ROMFS and EXEFS.

The defaults are the queue's current values; the tools are for agents' and developers' builds only.

Testing

Tested: On Linux, RTX 3090:

  • The queue: two gpu jobs ran one after the other and two shared jobs together; exit statuses pass through; the job's environment has no OOT_ variable. status named the base's shader preparation as a job's game, not someone playing.
  • dev_build.py: env installed the tools.lock build tools (5 s) and found the NGX SDK; base built and prepared the base of main 02903b04 in 12 min; build reused it as in Measurements.
  • The rules: checked by calling blocker with the cases in the new test.

Not tested:

  • pack, rt and vk were not run end to end; they call dev_packs.py pack / rt and nx2/build.py with the environment above.
  • Windows: job_queue.py has a Windows path (msvcrt lock, process start times) that has not run, and dev_build.py refuses Windows.
  • The game-free checks (tests/run_all.py, the new tests/test_job_queue.py among them) were not run on this PC, at the person's request (no test suites on this PC), and CI runs its Python source checks only by hand (workflow_dispatch), so they haven't run yet.
  • CI's Guide screenshots check fails here as it does on main at f7922126 (https://git.magicbean.dev/MagicBeanDev/Longshot/actions/runs/29145): this PR changes none of the files it fingerprints.

Must be tested:

  • Any Linux PC: python -I -B tests/run_all.py (or CI's Python source checks run by hand), expecting test_job_queue.py to pass with everything else (by hand, or the CI dispatch)

Risks

What could break: nothing a player builds or runs: the scripts are new, and nothing calls them but people and agents. A rule of the queue could hold a job longer than needed (status says what it waits for).

Turning it off: don't use it, or revert the commit.

Compatibility: Nothing changes for a player. Agents' builds move to the tools.lock toolchain, so their first build compiles cold once (a new object store namespace).

Checklist

Everyone

  • One topic.
  • The title is a Conventional Commit (type(scope): description) of the status line's type.
  • Why gives the reason plainly and links the issues that fit (the tracker searched).
  • Measurements has a verdict, or one line saying why there is nothing to measure.
  • Testing names what was tested, what wasn't, and what must still be tested, where and how.
  • Rebased on current main.
  • python -I -B tests/run_all.py passes (any failure that also fails on main is named). Not run on this PC at the person's request, and CI runs it only by hand: open.
  • No game files, dumps, build output or generated files.
  • third_party/ untouched (a new folder with a PROVENANCE.md instead).
  • The license is OK: GPL-3.0-or-later, nothing incompatible and nothing decompiled.

When it applies

  • AI assistance recorded with Assisted-by: / Co-Authored-By: trailers.
  • Switches documented (docs/development.md; the README for a menu row) and shipped at their recommended value.
  • Docs updated.

A companion Tooling PR, MagicBeanDev/Tooling#16, does the same for workspaces: making them, listing them and cleaning up the old ones.

🤖 Generated with Claude Code

**Type:** feat · **Risk:** low · **Tested on:** Linux, RTX 3090 · **Needs:** none ## Why Several agents (and the person they work for) share one PC to build and test Longshot. Builds, game runs and measurements went through a queue kept outside the repository, and under it no build ever took another build's translation from the lift store. That queue gives each job `OOT_STATUS_LOCK=<job name>`, and `make_game.py lift_key` hashes every `OOT_` variable. So every job had its own lift key, and every new work folder translated the game again. The queue's rules (no build beside a timed run, the GPU shared or not, memory, nothing in `/tmp`) were also known only to whoever had that script. This puts the queue and a build wrapper in the repository, keeps lift keys clean, and gives every checkout the same build environment. ## What changed Two scripts. `scripts/job_queue.py` runs builds and game runs when they fit and keeps measurements apart. `scripts/dev_build.py` builds bases, packs, runtimes and whole game folders through it, all with one environment, so the second build of anything takes its translation and its objects from the shared stores. Neither changes what players build. - **`scripts/job_queue.py`** (`run NAME --mode M -- CMD`, `status`, `history`, `cancel`): - modes `build`, `light`, `shared`, `gpu`, `timing`, `timing-cpu` and `profile`, each a GPU use and a CPU use; - one rule for order (`blocker`): what runs holds what it uses, and every ready ticket ahead keeps its claim, so nothing jumps ahead; - priorities with aging, and timed runs of one group one at a time; - the memory gate (available memory, PSI, swap); - a timed run on its own L3 cache domain while everything else moves to the others; it isn't swapped out, waits while you play and while a build runs outside the queue, and records how busy the PC was; - a systemd scope per job, so the OOM killer stops the job and not the terminal; - `TMPDIR` moved off a RAM file system, and the NVIDIA shader cache kept apart for GPU jobs. **A job gets `LONGSHOT_JOB*`, never a new `OOT_`, `RX_` or `A64_` variable**, and the older queue's `OOT_STATUS_*` is dropped. The older queue's running tickets count as running jobs, so the two can be used side by side while sessions move over. - **`scripts/dev_build.py`** (`env`, `base`, `pack`, `rt`, `build`, `vk`, `ls`), Linux: - **Build tools:** a player's, from `tools.lock` through `builder/linux/linux_tools.py`, installed once per `tools.lock` under `~/.cache/longshot-dev/builder/` (no more tools borrowed from an old checkout), plus a Python with the build's packages. - **Stores:** `dev_packs.py`'s object and translation stores, with `LONGSHOT_LIFT_STORE_KEEP=6`. The launcher's NGX SDK is used for DLSS. - **Lift key:** it names any `A64_`, `OOT_` or `RX_` variable you set that will enter the lift key. - **Bases:** one per commit of main in `~/.cache/longshot-dev/bases/`, built by whoever asks first while the others wait for it, from a temporary workspace of that commit. - **This checkout's files** stay in its own `build-linux/`, so they go when the checkout goes. - **No RAM file systems:** it refuses a folder on one. - **docs/development.md**: *Sharing a PC: the job queue* (the modes, the rules, why no `OOT_` variable, nothing on tmpfs). **AGENTS.md**: builds and game runs go through the queue, never on a RAM file system. - **tests/test_job_queue.py**: the rules above, the job environment, the RAM check, and two real queues (exclusive jobs one after the other, shared ones together). ## Measurements **Verdict:** the second build of the same main in a new work folder took **22 s instead of 6 min 48 s**. It copied the first build's translation from the lift store and took 512 of 514 objects from the object store; only the two objects that carry the version were compiled. Under the older queue's variable, the lift share key changes with each job name, so that build would have translated the game again. Hardware: Linux (CachyOS), RTX 3090, driver 620.70.06, Ryzen 9 7900X, 30 GB Method: the build's own `where the time went` lines; `make_game.lift_share_key` called with and without the variables. One run of each: the difference is what was compiled and translated, not noise. How to reproduce: `dev_build.py base`, then `dev_build.py build` in another checkout. | | First build: `dev_build.py base` (main @ 02903b04, an empty store for this toolchain) | Second build: `dev_build.py build`, this PR's checkout, a new work folder | |---|---|---| | Translation | translated (74 s) | copied from the lift store | | Objects compiled | all | 2 of 514 (512 from the store) | | Build (without the shader preparation) | 6 min 48 s | 22 s | ``` first: object store /home/<user>/.cache/longshot-objects/3fcd1e7be0def429: 0 objects from it the build took 0:06:48, compiled in 0:05:21 second: the game was translated with these tools in another work folder: its translation is copied (.../longshot-lifts/1ac5aac97ebb52f806edf3b0d906e3891580e67b) object store .../longshot-objects/3fcd1e7be0def429: 512 objects from it @@plan 2 514 the build took 0:00:22, compiled in 0:00:11 lift share key: no queue 6a14b1158d5b | OOT_STATUS_LOCK=build-x 8668ae8cbdf5 | =build-y 17efcb610dab | job_queue (LONGSHOT_JOB*) 6a14b1158d5b ``` ### Flags and settings No game switch. The scripts' own settings are environment variables, documented in their notes and in docs/development.md: - `LONGSHOT_QUEUE_*`: the slots, the memory limits, the CPU sets (by default the first L3 domain for timed runs), `LONGSHOT_QUEUE_QUIET` and the folders. - `LONGSHOT_DEV_*`: `HOME`, `PYTHON`, `DEPS`, `ROMFS` and `EXEFS`. The defaults are the queue's current values; the tools are for agents' and developers' builds only. ## Testing **Tested:** On Linux, RTX 3090: - **The queue:** two `gpu` jobs ran one after the other and two `shared` jobs together; exit statuses pass through; the job's environment has no `OOT_` variable. `status` named the base's shader preparation as a job's game, not someone playing. - **`dev_build.py`:** `env` installed the `tools.lock` build tools (5 s) and found the NGX SDK; `base` built and prepared the base of main 02903b04 in 12 min; `build` reused it as in Measurements. - **The rules:** checked by calling `blocker` with the cases in the new test. **Not tested:** - `pack`, `rt` and `vk` were not run end to end; they call `dev_packs.py pack` / `rt` and `nx2/build.py` with the environment above. - Windows: `job_queue.py` has a Windows path (msvcrt lock, process start times) that has not run, and `dev_build.py` refuses Windows. - The game-free checks (`tests/run_all.py`, the new `tests/test_job_queue.py` among them) were not run on this PC, at the person's request (no test suites on this PC), and CI runs its Python source checks only by hand (`workflow_dispatch`), so they haven't run yet. - CI's *Guide screenshots* check fails here as it does on main at f7922126 (https://git.magicbean.dev/MagicBeanDev/Longshot/actions/runs/29145): this PR changes none of the files it fingerprints. **Must be tested:** - Any Linux PC: `python -I -B tests/run_all.py` (or CI's Python source checks run by hand), expecting `test_job_queue.py` to pass with everything else (by hand, or the CI dispatch) ## Risks **What could break:** nothing a player builds or runs: the scripts are new, and nothing calls them but people and agents. A rule of the queue could hold a job longer than needed (`status` says what it waits for). **Turning it off:** don't use it, or revert the commit. **Compatibility:** Nothing changes for a player. Agents' builds move to the `tools.lock` toolchain, so their first build compiles cold once (a new object store namespace). ## Checklist ### Everyone - [x] One topic. - [x] The title is a Conventional Commit (`type(scope): description`) of the status line's type. - [x] Why gives the reason plainly and links the issues that fit (the tracker searched). - [x] Measurements has a verdict, or one line saying why there is nothing to measure. - [x] Testing names what was tested, what wasn't, and what must still be tested, where and how. - [x] Rebased on current `main`. - [ ] `python -I -B tests/run_all.py` passes (any failure that also fails on `main` is named). Not run on this PC at the person's request, and CI runs it only by hand: open. - [x] No game files, dumps, build output or generated files. - [x] `third_party/` untouched (a new folder with a `PROVENANCE.md` instead). - [x] The license is OK: GPL-3.0-or-later, nothing incompatible and nothing decompiled. ### When it applies - [x] AI assistance recorded with `Assisted-by:` / `Co-Authored-By:` trailers. - [x] Switches documented (docs/development.md; the README for a menu row) and shipped at their recommended value. - [x] Docs updated. A companion Tooling PR, https://git.magicbean.dev/MagicBeanDev/Tooling/pulls/16, does the same for workspaces: making them, listing them and cleaning up the old ones. 🤖 Generated with [Claude Code](https://claude.com/claude-code)
feat(scripts): the PC's job queue (job_queue.py) and dev_build.py, whose builds share one translation and one set of objects
Some checks failed
CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Has been skipped
CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Has been skipped
CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Has been skipped
Third-party parts / The same as upstream (pull_request) Has been skipped
Website / Website on Cloudflare (pull_request) Has been skipped
CI / Conventional Commit title (pull_request) Successful in 7s
Guide screenshots / Guide images and local UI source review (pull_request) Failing after 14s
CI / Workflow lint (actionlint) (pull_request) Successful in 16s
Website / Website on Cloudflare (pull_request_target) Has been skipped
Third-party parts / As their notes say (pull_request) Successful in 19s
PR description / Description follows the template (pull_request_target) Failing after 4s
20436c4e16
Several agents share one PC, and builds, game runs and measurements went through a queue outside the repository. Under it,
no build ever took another's translation from the lift store: its OOT_STATUS_LOCK=<job name> is hashed into make_game.py's
lift key. scripts/job_queue.py is that queue in the repository (build, light, shared, gpu, timing, timing-cpu, profile)
with no OOT_, RX_ or A64_ variable added to a job; scripts/dev_build.py builds bases, packs, runtimes and whole game
folders through it, with a player's build tools and the shared stores, and never on a RAM file system.

Assisted-by: Claude Code
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Author
Owner

CI: 10 passed. Guide screenshots fails as it does on main at f7922126 (https://git.magicbean.dev/MagicBeanDev/Longshot/actions/runs/29145); this PR touches none of its fingerprinted files. The Python source checks run only on workflow_dispatch, so tests/run_all.py (with the new test_job_queue.py) is still to run: the PR stays WIP until it has.

CI: 10 passed. Guide screenshots fails as it does on main at f7922126 (https://git.magicbean.dev/MagicBeanDev/Longshot/actions/runs/29145); this PR touches none of its fingerprinted files. The Python source checks run only on workflow_dispatch, so tests/run_all.py (with the new test_job_queue.py) is still to run: the PR stays WIP until it has.
HumanBean changed title from WIP: feat(scripts): the PC's job queue (job_queue.py) and dev_build.py, whose builds share one translation and one set of objects to feat(scripts): the PC's job queue (job_queue.py) and dev_build.py, whose builds share one translation and one set of objects 2026-11-16 07:45:04 +00:00
HumanBean deleted branch feat/job-queue 2026-11-16 07:46:21 +00:00
Sign in to join this conversation.
No reviewers
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
MagicBeanDev/Longshot!1085
No description provided.