feat(tests): run_all.py --changed runs the checks that read what changed #2887

Open

github-sync wants to merge 1 commit from github/pr/2887 into main

github-sync commented

2026-11-12 19:01:01 +00:00

Owner

@jordanella opened this pull request on GitHub · 2026-11-12 18:31 UTC · original

A full tests/run_all.py is about 490 checks and several minutes, and the ones an edit touches had to be picked by hand, which is how a check that cuts a function far from the edit gets missed (AGENTS.md: "Tests read the source"). This is the dev-loop proposal's D1 (#2636).

Change

tests/run_all.py --changed [BASE] takes the files changed since BASE's merge base (origin/main by default), committed or not, plus the untracked ones, and runs:

the checks whose text names one of them: by path, by file name standing alone, or as a module they import (import chunks, from chunks import);
the changed checks themselves, and the byte-compile check;
the C++ unit tests when something under gpu/ changed;
every check when the runner itself or a shared file under tests/ changed.

It prints how many files changed and how many checks read them, and that a pull request still needs the whole run. Nothing else in the runner changes.

Example

On this branch (the runner changed): --changed: 2 files changed since origin/main, every check (the runner or a shared test file changed). For a change to nx2/chunks.py alone it picks the chunk, emit and lift-cache checks (under a quarter of all), not the renderer's.

Testing

tests/test_run_all_changed.py (new): which checks a changed file picks (by path, by name and not inside a longer name, by import), that a changed check runs itself, that the runner or a shared test file runs everything, that an unnamed file runs none, and the files from git (committed, uncommitted and untracked) in a scratch repository.
run_all.py --changed origin/main on this branch, Windows: every check, as above: 423 passed, 21 failed, 79 skipped. The 21 fail the same way on main here (this PC's compilers and Linux-only checks).
The checks that read the changed files (10): pass, apart from test_spawn_emit, which fails the same way on main here.

See #2636 (D1).

🤖 Generated with Claude Code

> [@jordanella](https://github.com/jordanella) opened this pull request on GitHub · 2026-11-12 18:31 UTC · [original](https://github.com/NotRexed/Hookshot/pull/2887) A full `tests/run_all.py` is about 490 checks and several minutes, and the ones an edit touches had to be picked by hand, which is how a check that cuts a function far from the edit gets missed (AGENTS.md: "Tests read the source"). This is the dev-loop proposal's D1 (#2636). ## Change `tests/run_all.py --changed [BASE]` takes the files changed since BASE's merge base (`origin/main` by default), committed or not, plus the untracked ones, and runs: - the checks whose text names one of them: by path, by file name standing alone, or as a module they import (`import chunks`, `from chunks import`); - the changed checks themselves, and the byte-compile check; - the C++ unit tests when something under `gpu/` changed; - **every check** when the runner itself or a shared file under `tests/` changed. It prints how many files changed and how many checks read them, and that a pull request still needs the whole run. Nothing else in the runner changes. ## Example On this branch (the runner changed): `--changed: 2 files changed since origin/main, every check (the runner or a shared test file changed)`. For a change to `nx2/chunks.py` alone it picks the chunk, emit and lift-cache checks (under a quarter of all), not the renderer's. ## Testing - **`tests/test_run_all_changed.py` (new):** which checks a changed file picks (by path, by name and not inside a longer name, by import), that a changed check runs itself, that the runner or a shared test file runs everything, that an unnamed file runs none, and the files from git (committed, uncommitted and untracked) in a scratch repository. - **`run_all.py --changed origin/main` on this branch, Windows:** every check, as above: 423 passed, 21 failed, 79 skipped. The 21 fail the same way on `main` here (this PC's compilers and Linux-only checks). - **The checks that read the changed files (10):** pass, apart from `test_spawn_emit`, which fails the same way on `main` here. See #2636 (D1). 🤖 Generated with [Claude Code](https://claude.com/claude-code)

github-sync added 1 commit

2026-11-12 19:01:01 +00:00

feat(tests): run_all.py --changed runs the checks that read what changed ce0fe41e7b

A full run is ~490 checks, and the ones an edit touches had to be picked by hand. --changed [BASE] takes the files
changed since BASE's merge base (origin/main by default), committed or not, and the untracked ones, and runs the checks
whose text names one of them, by path, by name or as an imported module, the changed tests themselves, and the
byte-compile check; the C++ unit tests when gpu/ changed, and every check when the runner or a shared test file did. It
says that a pull request still needs the whole run. This is the dev-loop proposal's D1 (#2636).

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>

Commenting is not possible because the repository is archived.

No reviewers