fix(tests): size run_all's default checks by memory, not CPUs alone #2177

Open
dazemc wants to merge 1 commit from dazemc/Longshot:fix/run-all-memory-aware-jobs into main
Contributor

Summary

tests/run_all.py ran --jobs checks at once, and --jobs defaulted to os.cpu_count(). On this PC that is 32:
32 checks at once, each able to compile and link a test binary, on a 15.5 GiB machine. mise run test filled memory,
froze the desktop and got some checks OOM-killed.

The default now depends on the memory actually free to the runner as well as on the CPUs:

budget = min(cgroup v2 limit, MemAvailable, total RAM)
jobs   = min(cpu_count, max(2, floor((budget - 4) / 2)), 8)

one check per 2 GiB of memory after 4 GiB kept for the OS and the desktop (a compiling check peaks at a few hundred
MiB, often over a gigabyte), never more than eight (past eight the checks contend for the CPU anyway and memory is the
limit), never fewer than two. The budget is the smallest of the tightest cgroup v2 memory.max on the process (what a
systemd-run -p MemoryMax=… scope sets), the RAM available now (/proc/meminfo's MemAvailable) and the total: inside a
real 8 GiB scope this reads 8.0 GiB and picks 2, where seven checks were OOM-killed in that same scope, and 32 GiB with
5 GiB free picks 2, not 8. Windows reads GlobalMemoryStatusEx's ullTotalPhys and ullAvailPhys; any other OS, or a
read that fails, falls back to the CPU count capped at eight (a 12-core Mac picks 8). A --jobs N the caller passes
still wins, whatever the memory (argparse keeps the default None so main asks the rule).

The same one-check run on this 32-thread, 15.5 GiB PC:

job count
main 9e10f6f7 32 at once
this branch 1df1d2db, --jobs unset 2 at once as the PC stands (busy: 1.5 GiB available; 4-5 when idle)

How to see it on main: python -I -B tests/run_all.py --exact tests/test_run_all_schedule.py prints … 32 at once (linux); here it prints … 2 at once while the machine is busy (more when it is idle: the count follows the memory
free), and the same command inside systemd-run --scope -p MemoryMax=8G still prints 2 at once.

Read first: tests/run_all.py (default_jobs, check_jobs, total_memory, parser) and
tests/test_run_all_default_jobs.py.

#2171 already fixed the three scratch-repo checks (test_run_all_output, test_run_all_reap, test_run_all_mingw
now copy tests/compiler_command.py too). This PR is only the memory-aware default; it does not touch those checks.

Testing

Tested: Linux (no GPU involved), this PC (32 threads, 15.5 GiB).
python -I -B tests/run_all.py --strict test_run_all_default_jobs test_run_all_junit test_run_all_declarations test_run_all_schedule test_run_all_output test_run_all_reap test_run_all_mingw
→ 7 passed, 0 failed, 0 skipped in 22 s, in an 8 GiB scope, printed as 2 at once.

On main 9e10f6f7, the same run prints 32 at once and the three scratch-repo checks pass (they were red until #2171);
the new check fails there with AttributeError: module 'run_all' has no attribute 'default_jobs', because main has no
memory-aware rule at all. On this branch it passes.

The new check pins the rule from the numbers alone: 32 CPUs/16 GiB → 6 (the reserve), 32/64 GiB → 8 (cap), 4/64 GiB → 4,
8/8 GiB → 2, 32/2 GiB → 2 (floor), 32/10 GiB → 3, an 8 GiB cgroup limit → 2 (with 9 GiB free and 15.5 GiB total), 32 GiB
total with 5 GiB available → 2, unknown memory → the CPU count capped at 8 (12 cores → 8), one CPU → 1, a passed
--jobs (1, 3, 32, 0) winning, and --jobs unset staying None in argparse so a CPU-count default cannot come back.
This PC's own case is pinned at 5 for a 15.5 GiB budget, where the rule without the reserve gave 7; as it stands the PC
is busy and default_jobs() gives the floor of 2.

Full suite on fa21a408/7156b164 (pinned headers, one run): 934 checks, 18 skipped, 3 at once (linux) →
912 passed, 22 failed, 18 skipped in 626 s. The 22 failing names are exactly the main baseline's at 9e10f6f7
(911 passed, 22 failed, 18 skipped): set(branch) − set(main) = ∅, set(main) − set(branch) = ∅ — no added or missing
name, and the one extra pass is this PR's new check. That run does not cover this head: the branch was then rebased
onto a2dc5385 and its rule corrected to read the cgroup limit and available memory, which changed
tests/run_all.py (98 insertions / 32 deletions against 7156b164) and tests/test_run_all_default_jobs.py (67/18) — the suite ran before both
the rebase and the final rule. The unit set (this check, junit, declarations, schedule, output, reap, mingw)
was re-run on 1df1d2db in an 8 GiB scope: 7 passed, 0 failed, 0 skipped, at the default's 2 at once in 22 s (and at
--jobs 3 in 21 s). Full suite on the tree of 1df1d2db (the same tree as head 5c4051f4, below) — tests/run_all.py --strict --jobs 5 inside
systemd-run --user --scope -p MemoryMax=12G -p MemorySwapMax=0 with the pinned headers: 912 passed / 23 failed /
18 skipped in 440 s (5 at once)
, the 23 failing names identical to the a2dc5385 baseline's (compared name by name),
and this PR's tests/test_run_all_default_jobs.py passes. Head 5c4051f4 is 1df1d2db with the Assisted-by: trailer added to its message only — tree 14636770, byte-identical — so the suite and the numbers describe the head's tree.
The one earlier full run without LONGSHOT_VULKAN_HEADERS/LONGSHOT_SPIRV_HEADERS had 86 not found: … (--strict)
failures before any test ran; the reported run has the headers and none of that remains.

In-game run: N/A — test runner only. Nothing in the game loads tests/run_all.py; the change is how the game-free
checks start.

Still to test:

  • Windows: the memory read's GlobalMemoryStatusEx path, by hand or CI.

Measurements

Nothing to measure: a test-runner default, with no game or switch to A/B. The before/after job counts are in Testing.

Risks

Only how many checks run at once changes: every check still runs, in the same order, and --jobs N overrides it. The
count reads the memory actually free to the runner — the tightest cgroup v2 memory.max on the process, Linux's
MemAvailable, Windows' ullAvailPhys — and the total; the smallest wins, so a loaded desktop or a MemoryMax scope is
respected. A cgroup v1 limit or a kernel without MemAvailable is not read, falling back to the total, and the floor of
two checks applies even inside a scope smaller than 4 GiB. The Windows GlobalMemoryStatusEx path could not be run on
Windows here; if it fails, the CPU count capped at eight applies. Nothing changes for players after updating: no saves,
longshot_options.ini keys or caches.

Checklist

Ready for review

  • One topic, on current main; the title is a Conventional Commit.
  • Tested on my own PC, in the game for a change the game runs, as Testing says. (Not a game change: in-game run N/A, as Testing says.)
  • mise run test passes (any failure that also fails on main is named). Tree 14636770 at 1df1d2db (head 5c4051f4, message-only amend, same tree): 912 passed / 23 failed / 18 skipped in 440 s, --jobs 5 in a 12 GiB scope with the pinned headers; the 23 failing names are identical, name by name, to the a2dc5385 baseline's 911 / 23 / 18, and the extra pass is this PR's tests/test_run_all_default_jobs.py.
  • Measured against main, or Measurements says why there is nothing to measure.
  • No game files, build output or hand-edited generated files; third_party/ untouched; nothing GPL-incompatible.

When it applies

  • A fix: a game-free test that fails on main and passes here — tests/test_run_all_default_jobs.py: it fails on
    main with AttributeError: module 'run_all' has no attribute 'default_jobs' and passes here.
  • Docs updated.
  • AI assistance recorded with Assisted-by: / Co-Authored-By: trailers. The head carries
    Assisted-by: DeepSeek V4.1 Flash 1017 (opencode-go), added to 1df1d2db's message only (5c4051f4; same tree,
    14636770), so the suite above still measures the head's tree.

Ready to merge

  • Everything in Still to test has run, its results in Tested.
  • Approved in review, every comment answered.
  • CI passes on the head; on current main, measured again or a comment says why the numbers still hold.
## Summary `tests/run_all.py` ran `--jobs` checks at once, and `--jobs` defaulted to `os.cpu_count()`. On this PC that is **32**: 32 checks at once, each able to compile and link a test binary, on a 15.5 GiB machine. `mise run test` filled memory, froze the desktop and got some checks OOM-killed. The default now depends on the memory actually free to the runner as well as on the CPUs: budget = min(cgroup v2 limit, MemAvailable, total RAM) jobs = min(cpu_count, max(2, floor((budget - 4) / 2)), 8) one check per 2 GiB of memory **after 4 GiB kept for the OS and the desktop** (a compiling check peaks at a few hundred MiB, often over a gigabyte), never more than eight (past eight the checks contend for the CPU anyway and memory is the limit), never fewer than two. The budget is the smallest of the tightest cgroup v2 `memory.max` on the process (what a `systemd-run -p MemoryMax=…` scope sets), the RAM available now (`/proc/meminfo`'s `MemAvailable`) and the total: inside a real 8 GiB scope this reads 8.0 GiB and picks 2, where seven checks were OOM-killed in that same scope, and 32 GiB with 5 GiB free picks 2, not 8. Windows reads `GlobalMemoryStatusEx`'s `ullTotalPhys` and `ullAvailPhys`; any other OS, or a read that fails, falls back to the CPU count **capped at eight** (a 12-core Mac picks 8). A `--jobs N` the caller passes still wins, whatever the memory (`argparse` keeps the default `None` so `main` asks the rule). The same one-check run on this 32-thread, 15.5 GiB PC: | | job count | |---|---| | `main` `9e10f6f7` | `32 at once` | | this branch `1df1d2db`, `--jobs` unset | `2 at once` as the PC stands (busy: 1.5 GiB available; 4-5 when idle) | How to see it on `main`: `python -I -B tests/run_all.py --exact tests/test_run_all_schedule.py` prints `… 32 at once (linux)`; here it prints `… 2 at once` while the machine is busy (more when it is idle: the count follows the memory free), and the same command inside `systemd-run --scope -p MemoryMax=8G` still prints `2 at once`. Read first: `tests/run_all.py` (`default_jobs`, `check_jobs`, `total_memory`, `parser`) and `tests/test_run_all_default_jobs.py`. **#2171 already fixed the three scratch-repo checks** (`test_run_all_output`, `test_run_all_reap`, `test_run_all_mingw` now copy `tests/compiler_command.py` too). This PR is only the memory-aware default; it does not touch those checks. ## Testing **Tested:** Linux (no GPU involved), this PC (32 threads, 15.5 GiB). `python -I -B tests/run_all.py --strict test_run_all_default_jobs test_run_all_junit test_run_all_declarations test_run_all_schedule test_run_all_output test_run_all_reap test_run_all_mingw` → 7 passed, 0 failed, 0 skipped in 22 s, in an 8 GiB scope, printed as `2 at once`. On `main` `9e10f6f7`, the same run prints `32 at once` and the three scratch-repo checks pass (they were red until #2171); the new check fails there with `AttributeError: module 'run_all' has no attribute 'default_jobs'`, because `main` has no memory-aware rule at all. On this branch it passes. The new check pins the rule from the numbers alone: 32 CPUs/16 GiB → 6 (the reserve), 32/64 GiB → 8 (cap), 4/64 GiB → 4, 8/8 GiB → 2, 32/2 GiB → 2 (floor), 32/10 GiB → 3, an 8 GiB cgroup limit → 2 (with 9 GiB free and 15.5 GiB total), 32 GiB total with 5 GiB available → 2, unknown memory → the CPU count **capped at 8** (12 cores → 8), one CPU → 1, a passed `--jobs` (1, 3, 32, 0) winning, and `--jobs` unset staying `None` in `argparse` so a CPU-count default cannot come back. This PC's own case is pinned at 5 for a 15.5 GiB budget, where the rule without the reserve gave 7; as it stands the PC is busy and `default_jobs()` gives the floor of 2. Full suite on `fa21a408`/`7156b164` (pinned headers, one run): `934 checks, 18 skipped, 3 at once (linux)` → `912 passed, 22 failed, 18 skipped in 626 s`. The 22 failing names are **exactly** the main baseline's at `9e10f6f7` (`911 passed, 22 failed, 18 skipped`): `set(branch) − set(main) = ∅`, `set(main) − set(branch) = ∅` — no added or missing name, and the one extra pass is this PR's new check. **That run does not cover this head**: the branch was then rebased onto `a2dc5385` and its rule corrected to read the cgroup limit and available memory, which changed `tests/run_all.py` (98 insertions / 32 deletions against `7156b164`) and `tests/test_run_all_default_jobs.py` (67/18) — the suite ran before both the rebase and the final rule. The unit set (this check, `junit`, `declarations`, `schedule`, `output`, `reap`, `mingw`) was re-run on `1df1d2db` in an 8 GiB scope: 7 passed, 0 failed, 0 skipped, at the default's `2 at once` in 22 s (and at `--jobs 3` in 21 s). Full suite on the tree of `1df1d2db` (the same tree as head `5c4051f4`, below) — `tests/run_all.py --strict --jobs 5` inside `systemd-run --user --scope -p MemoryMax=12G -p MemorySwapMax=0` with the pinned headers: **912 passed / 23 failed / 18 skipped in 440 s (5 at once)**, the 23 failing names identical to the `a2dc5385` baseline's (compared name by name), and this PR's `tests/test_run_all_default_jobs.py` passes. Head `5c4051f4` is `1df1d2db` with the `Assisted-by:` trailer added to its message only — tree `14636770`, byte-identical — so the suite and the numbers describe the head's tree. The one earlier full run without `LONGSHOT_VULKAN_HEADERS`/`LONGSHOT_SPIRV_HEADERS` had 86 `not found: … (--strict)` failures before any test ran; the reported run has the headers and none of that remains. **In-game run: N/A — test runner only.** Nothing in the game loads `tests/run_all.py`; the change is how the game-free checks start. **Still to test:** - Windows: the memory read's `GlobalMemoryStatusEx` path, by hand or CI. ## Measurements Nothing to measure: a test-runner default, with no game or switch to A/B. The before/after job counts are in Testing. ## Risks Only how many checks run at once changes: every check still runs, in the same order, and `--jobs N` overrides it. The count reads the memory actually free to the runner — the tightest cgroup v2 `memory.max` on the process, Linux's `MemAvailable`, Windows' `ullAvailPhys` — and the total; the smallest wins, so a loaded desktop or a `MemoryMax` scope is respected. A cgroup v1 limit or a kernel without `MemAvailable` is not read, falling back to the total, and the floor of two checks applies even inside a scope smaller than 4 GiB. The Windows `GlobalMemoryStatusEx` path could not be run on Windows here; if it fails, the CPU count capped at eight applies. Nothing changes for players after updating: no saves, `longshot_options.ini` keys or caches. ## Checklist ### Ready for review - [x] One topic, on current `main`; the title is a Conventional Commit. - [x] Tested on my own PC, in the game for a change the game runs, as Testing says. (Not a game change: in-game run N/A, as Testing says.) - [x] `mise run test` passes (any failure that also fails on `main` is named). Tree `14636770` at `1df1d2db` (head `5c4051f4`, message-only amend, same tree): **912 passed / 23 failed / 18 skipped in 440 s**, `--jobs 5` in a 12 GiB scope with the pinned headers; the 23 failing names are identical, name by name, to the `a2dc5385` baseline's **911 / 23 / 18**, and the extra pass is this PR's `tests/test_run_all_default_jobs.py`. - [x] Measured against `main`, or Measurements says why there is nothing to measure. - [x] No game files, build output or hand-edited generated files; `third_party/` untouched; nothing GPL-incompatible. ### When it applies - [x] A fix: a game-free test that fails on `main` and passes here — `tests/test_run_all_default_jobs.py`: it fails on `main` with `AttributeError: module 'run_all' has no attribute 'default_jobs'` and passes here. - [x] Docs updated. - [x] AI assistance recorded with `Assisted-by:` / `Co-Authored-By:` trailers. The head carries `Assisted-by: DeepSeek V4.1 Flash 1017 (opencode-go)`, added to `1df1d2db`'s message only (`5c4051f4`; same tree, `14636770`), so the suite above still measures the head's tree. ### Ready to merge - [ ] Everything in Still to test has run, its results in Tested. - [ ] Approved in review, every comment answered. - [ ] CI passes on the head; on current `main`, measured again or a comment says why the numbers still hold.
fix(tests): size run_all's default checks by memory, not CPUs alone
Some checks failed
Website / Website on Cloudflare (pull_request_target) Has been skipped
PR description / Description follows the template (pull_request_target) Successful in 4s
CI / Conventional Commit title (pull_request) Has been cancelled
CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Has been cancelled
CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Has been cancelled
CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Has been cancelled
CI / Workflow lint (actionlint) (pull_request) Has been cancelled
CI / Website build and checks (pull_request) Has been cancelled
Guide screenshots / Guide images and local UI source review (pull_request) Has been cancelled
Third-party parts / As their notes say (pull_request) Has been cancelled
Third-party parts / The same as upstream (pull_request) Has been cancelled
Website / Website on Cloudflare (pull_request) Has been cancelled
7156b164eb
--jobs defaulted to os.cpu_count(): 32 checks at once on this PC, each
able to compile and link a test binary, on 15 GiB of RAM. The run swapped,
froze the desktop and some checks were OOM-killed.

The default now comes from default_jobs: the CPUs, at most one check per
2 GiB of RAM and never more than eight (two always run). Linux reads
sysconf's page size and pages, Windows ctypes' GlobalMemoryStatusEx, and
any other OS, or a read that fails, keeps the CPU count as before. A
--jobs the caller passes still wins, whatever the memory. argparse keeps
--jobs' default None so main asks the rule, which the test pins.

LONGSHOT_JOBS is left alone: it bounds nx2's build steps, which already
take one worker per 2 GiB of free memory, and not a check's own compilers,
which never read it.

tests/test_run_all_default_jobs.py pins the rule from the numbers alone.
docs/dev/developing.md and the runner's docstring say what the default is
and how to override it.
dazemc force-pushed fix/run-all-memory-aware-jobs from 7156b164eb
Some checks failed
Website / Website on Cloudflare (pull_request_target) Has been skipped
PR description / Description follows the template (pull_request_target) Successful in 4s
CI / Conventional Commit title (pull_request) Has been cancelled
CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Has been cancelled
CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Has been cancelled
CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Has been cancelled
CI / Workflow lint (actionlint) (pull_request) Has been cancelled
CI / Website build and checks (pull_request) Has been cancelled
Guide screenshots / Guide images and local UI source review (pull_request) Has been cancelled
Third-party parts / As their notes say (pull_request) Has been cancelled
Third-party parts / The same as upstream (pull_request) Has been cancelled
Website / Website on Cloudflare (pull_request) Has been cancelled
to fa21a40872
Some checks failed
PR description / Description follows the template (pull_request_target) Successful in 5s
CI / Conventional Commit title (pull_request) Has been cancelled
CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Has been cancelled
CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Has been cancelled
CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Has been cancelled
CI / Workflow lint (actionlint) (pull_request) Has been cancelled
CI / Website build and checks (pull_request) Has been cancelled
Guide screenshots / Guide images and local UI source review (pull_request) Has been cancelled
Third-party parts / As their notes say (pull_request) Has been cancelled
Third-party parts / The same as upstream (pull_request) Has been cancelled
Website / Website on Cloudflare (pull_request) Has been cancelled
2026-11-22 02:24:41 +00:00
Compare
dazemc force-pushed fix/run-all-memory-aware-jobs from fa21a40872
Some checks failed
PR description / Description follows the template (pull_request_target) Successful in 5s
CI / Conventional Commit title (pull_request) Has been cancelled
CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Has been cancelled
CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Has been cancelled
CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Has been cancelled
CI / Workflow lint (actionlint) (pull_request) Has been cancelled
CI / Website build and checks (pull_request) Has been cancelled
Guide screenshots / Guide images and local UI source review (pull_request) Has been cancelled
Third-party parts / As their notes say (pull_request) Has been cancelled
Third-party parts / The same as upstream (pull_request) Has been cancelled
Website / Website on Cloudflare (pull_request) Has been cancelled
to bfabc80f45
Some checks failed
PR description / Description follows the template (pull_request_target) Successful in 5s
CI / Conventional Commit title (pull_request) Has been cancelled
CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Has been cancelled
CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Has been cancelled
CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Has been cancelled
CI / Workflow lint (actionlint) (pull_request) Has been cancelled
CI / Website build and checks (pull_request) Has been cancelled
Guide screenshots / Guide images and local UI source review (pull_request) Has been cancelled
Third-party parts / As their notes say (pull_request) Has been cancelled
Third-party parts / The same as upstream (pull_request) Has been cancelled
Website / Website on Cloudflare (pull_request) Has been cancelled
2026-11-22 05:44:10 +00:00
Compare
dazemc force-pushed fix/run-all-memory-aware-jobs from bfabc80f45
Some checks failed
PR description / Description follows the template (pull_request_target) Successful in 5s
CI / Conventional Commit title (pull_request) Has been cancelled
CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Has been cancelled
CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Has been cancelled
CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Has been cancelled
CI / Workflow lint (actionlint) (pull_request) Has been cancelled
CI / Website build and checks (pull_request) Has been cancelled
Guide screenshots / Guide images and local UI source review (pull_request) Has been cancelled
Third-party parts / As their notes say (pull_request) Has been cancelled
Third-party parts / The same as upstream (pull_request) Has been cancelled
Website / Website on Cloudflare (pull_request) Has been cancelled
to 1df1d2dbbd
Some checks failed
PR description / Description follows the template (pull_request_target) Successful in 5s
CI / Conventional Commit title (pull_request) Has been cancelled
CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Has been cancelled
CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Has been cancelled
CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Has been cancelled
CI / Workflow lint (actionlint) (pull_request) Has been cancelled
CI / Website build and checks (pull_request) Has been cancelled
Guide screenshots / Guide images and local UI source review (pull_request) Has been cancelled
Third-party parts / As their notes say (pull_request) Has been cancelled
Third-party parts / The same as upstream (pull_request) Has been cancelled
Website / Website on Cloudflare (pull_request) Has been cancelled
2026-11-22 06:46:07 +00:00
Compare
dazemc changed title from fix(tests): size run_all's default checks by memory, not CPUs alone to WIP: fix(tests): size run_all's default checks by memory, not CPUs alone 2026-11-22 07:50:57 +00:00
Author
Contributor

The full-suite box was ticked on a run at fa21a408/7156b164, which predates the rebase onto a2dc5385 and the final rule: tests/run_all.py changed 98/32 and tests/test_run_all_default_jobs.py 67/18 since, so that result does not cover 1df1d2db. Unticked it and back to WIP. tests/run_all.py --strict --jobs 5 at 1df1d2db under MemoryMax=12G with the pinned headers is queued after the current queue; re-ready when it is green against a2dc5385's 911/23/18.

The full-suite box was ticked on a run at `fa21a408`/`7156b164`, which predates the rebase onto `a2dc5385` and the final rule: `tests/run_all.py` changed 98/32 and `tests/test_run_all_default_jobs.py` 67/18 since, so that result does not cover `1df1d2db`. Unticked it and back to WIP. `tests/run_all.py --strict --jobs 5` at `1df1d2db` under `MemoryMax=12G` with the pinned headers is queued after the current queue; re-ready when it is green against `a2dc5385`'s 911/23/18.
Author
Contributor

The suite at the head is in: tests/run_all.py --strict --jobs 5 at 1df1d2db inside systemd-run --user --scope -p MemoryMax=12G -p MemorySwapMax=0 with the pinned headers — 912 passed / 23 failed / 18 skipped in 440 s. The 23 failing names are identical to the a2dc5385 baseline's 911/23/18, compared name by name (the saved baseline list against this run's FAIL lines), and the extra pass is this PR's tests/test_run_all_default_jobs.py. Full-suite box ticked; back to ready.

The suite at the head is in: `tests/run_all.py --strict --jobs 5` at `1df1d2db` inside `systemd-run --user --scope -p MemoryMax=12G -p MemorySwapMax=0` with the pinned headers — **912 passed / 23 failed / 18 skipped in 440 s**. The 23 failing names are identical to the `a2dc5385` baseline's 911/23/18, compared name by name (the saved baseline list against this run's `FAIL` lines), and the extra pass is this PR's `tests/test_run_all_default_jobs.py`. Full-suite box ticked; back to ready.
dazemc changed title from WIP: fix(tests): size run_all's default checks by memory, not CPUs alone to fix(tests): size run_all's default checks by memory, not CPUs alone 2026-11-22 08:07:36 +00:00
dazemc left a comment

The ### When it applies section is missing the AI assistance recorded box (and the A check under tests/ box if it applies; the new tests/test_run_all_default_jobs.py is the A fix). The five Ready-for-review boxes are settled at 1df1d2db; the Windows GlobalMemoryStatusEx path is honestly in Still to test.

The `### When it applies` section is missing the `AI assistance recorded` box (and the `A check under tests/` box if it applies; the new `tests/test_run_all_default_jobs.py` is the `A fix`). The five Ready-for-review boxes are settled at `1df1d2db`; the Windows `GlobalMemoryStatusEx` path is honestly in Still to test.
Author
Contributor

Description fix from the quality sweep: the When it applies section now names the A fix's check (tests/test_run_all_default_jobs.py, the check that fails on main for the missing default_jobs and passes here) and adds the missing AI-assistance box, recorded here because a literal Assisted-by: trailer would rewrite 1df1d2db, the commit the suite ran. No code change; stays ready.

Description fix from the quality sweep: the `When it applies` section now names the A fix's check (`tests/test_run_all_default_jobs.py`, the check that fails on `main` for the missing `default_jobs` and passes here) and adds the missing AI-assistance box, recorded here because a literal `Assisted-by:` trailer would rewrite `1df1d2db`, the commit the suite ran. No code change; stays ready.
dazemc force-pushed fix/run-all-memory-aware-jobs from 1df1d2dbbd
Some checks failed
PR description / Description follows the template (pull_request_target) Successful in 5s
CI / Conventional Commit title (pull_request) Has been cancelled
CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Has been cancelled
CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Has been cancelled
CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Has been cancelled
CI / Workflow lint (actionlint) (pull_request) Has been cancelled
CI / Website build and checks (pull_request) Has been cancelled
Guide screenshots / Guide images and local UI source review (pull_request) Has been cancelled
Third-party parts / As their notes say (pull_request) Has been cancelled
Third-party parts / The same as upstream (pull_request) Has been cancelled
Website / Website on Cloudflare (pull_request) Has been cancelled
to a2dc538590
Some checks failed
Third-party parts / The same as upstream (push) Has been skipped
CI / Website build and checks (push) Failing after 25s
CI / Python source checks (ubuntu-24.04, part 2/2) (push) Has been skipped
CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (push) Has been skipped
CI / Conventional Commit title (push) Has been skipped
CI / Python source checks (ubuntu-24.04, part 1/2) (push) Has been skipped
ci/circleci: setup-main
Guide screenshots / Guide images and local UI source review (push) Failing after 31s
CI / Workflow lint (actionlint) (push) Successful in 33s
Third-party parts / As their notes say (push) Successful in 36s
ci/circleci: setup-noop-tag
ci/circleci: setup-release
Release / The version (push) Successful in 12s
Release / Linux builder (push) Successful in 0s
Release / Windows Vulkan back end (llvm-mingw) (push) Successful in 2s
Release / appimage (push) Successful in 3m25s
Release / Longshot Builder zip (push) Successful in 11m3s
Release / Windows builder (push) Successful in 0s
Release / Sign and publish (push) Successful in 55s
Guide screenshots / Guide images and local UI source review (pull_request) Has been cancelled
PR description / Description follows the template (pull_request_target) Successful in 5s
CI / Conventional Commit title (pull_request) Has been cancelled
CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Has been cancelled
CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Has been cancelled
CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Has been cancelled
CI / Workflow lint (actionlint) (pull_request) Has been cancelled
CI / Website build and checks (pull_request) Has been cancelled
Third-party parts / As their notes say (pull_request) Has been cancelled
Third-party parts / The same as upstream (pull_request) Has been cancelled
2026-11-22 10:58:48 +00:00
Compare
fix(tests): size run_all's default checks by the memory free to it
Some checks are pending
CI / Conventional Commit title (pull_request) Blocked by required conditions
CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Blocked by required conditions
CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Blocked by required conditions
CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Blocked by required conditions
CI / Workflow lint (actionlint) (pull_request) Blocked by required conditions
CI / Website build and checks (pull_request) Blocked by required conditions
Guide screenshots / Guide images and local UI source review (pull_request) Blocked by required conditions
Third-party parts / As their notes say (pull_request) Blocked by required conditions
Third-party parts / The same as upstream (pull_request) Blocked by required conditions
Website / Website on Cloudflare (pull_request) Blocked by required conditions
PR description / Description follows the template (pull_request_target) Successful in 7s
5c4051f4eb
--jobs defaulted to os.cpu_count(): 32 checks at once on this PC, each
able to compile and link a test binary, on 15.5 GiB of RAM. The run swapped,
froze the desktop and some checks were OOM-killed.

The default now comes from default_jobs: the CPUs, at most one check per
2 GiB after 4 GiB kept for the OS and the desktop, never more than eight
(two always run). The memory it counts is the smallest of the process's
cgroup v2 limit (a systemd scope's MemoryMax), the RAM available now
(MemAvailable on Linux, GlobalMemoryStatusEx's ullAvailPhys on Windows)
and the total: a scope of 8 GiB therefore picks 2, not the 7 that were
OOM-killed in that same scope, and 32 GiB with 5 GiB free picks 2, not 8.
With no memory readable it is the CPU count capped at eight, so a 12-core
Mac no longer picks 12. A --jobs the caller passes still wins; argparse
keeps --jobs' default None so main asks the rule, which the test pins.

LONGSHOT_JOBS is left alone: it bounds nx2's build steps, which already
take one worker per 2 GiB of free memory, and not a check's own compilers,
which never read it.

tests/test_run_all_default_jobs.py pins the rule from the numbers alone,
including the cgroup and available-memory cases. docs/dev/developing.md
and the runner's docstring say what the default is and how to override it.

Assisted-by: DeepSeek V4.1 Flash 1017 (opencode-go)
Author
Contributor

AI-assistance corrected to real evidence: head 5c4051f4 is 1df1d2db with a message-only Assisted-by: DeepSeek V4.1 Flash 1017 (opencode-go) trailer, force-pushed with lease. Tree 14636770 is byte-identical (verified before and after the amend: 1df1d2db^{tree} = 5c4051f4^{tree}), so the suite at 1df1d2db (912 / 23 / 18 in 440 s) describes the head's tree; the description now re-cites that tree. The AI box is ticked from the trailer.

AI-assistance corrected to real evidence: head `5c4051f4` is `1df1d2db` with a message-only `Assisted-by: DeepSeek V4.1 Flash 1017 (opencode-go)` trailer, force-pushed with lease. Tree `14636770` is byte-identical (verified before and after the amend: `1df1d2db^{tree}` = `5c4051f4^{tree}`), so the suite at `1df1d2db` (912 / 23 / 18 in 440 s) describes the head's tree; the description now re-cites that tree. The AI box is ticked from the trailer.
Some checks are pending
CI / Conventional Commit title (pull_request) Blocked by required conditions
CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Blocked by required conditions
CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Blocked by required conditions
CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Blocked by required conditions
CI / Workflow lint (actionlint) (pull_request) Blocked by required conditions
CI / Website build and checks (pull_request) Blocked by required conditions
Guide screenshots / Guide images and local UI source review (pull_request) Blocked by required conditions
Third-party parts / As their notes say (pull_request) Blocked by required conditions
Third-party parts / The same as upstream (pull_request) Blocked by required conditions
Website / Website on Cloudflare (pull_request) Blocked by required conditions
PR description / Description follows the template (pull_request_target) Successful in 7s
This pull request doesn't have enough approvals yet. 0 of 1 approvals granted.
Some workflows are waiting to be reviewed.
You are not authorized to merge this pull request.
View command line instructions

Checkout

From your project repository, check out a new branch and test the changes.
git fetch -u 'fix/run-all-memory-aware-jobs:dazemc-fix/run-all-memory-aware-jobs'
git switch 'dazemc-fix/run-all-memory-aware-jobs'
Sign in to join this conversation.
No reviewers
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
MagicBeanDev/Longshot!2177
No description provided.