ci: a sanitizer lane for the checks that offer --sanitize

dyst commented

2026-11-12 23:38:41 +00:00

Member

Why

Eleven checks build the code they test with the address and undefined-behaviour sanitizers when given --sanitize, but nothing ran them that way, so a use after free or an overflow in that code passes CI by luck.

What changed

tests: a check says what turns its sanitizers on with a fourth declaration, RUN_ALL_SANITIZE = ["--sanitize"], beside its RUN_ALL_ARGS (test_low_vram_texture_mips.py declares ["--sanitize", "address,undefined"], as its option takes the list). The eleven checks that declare it: test_dump_readback, test_image_view_retry, test_import_buffer_binding, test_low_vram_texture_mips, test_program_cache_cleanup, test_recording_admission, test_sampler_admission, test_texel_buffer_ownership, test_texture_staging_publication, test_vulkan_clear_mip, test_vulkan_copy_span.
tests: tests/run_all.py reads RUN_ALL_SANITIZE in read_declarations (in DECLARATIONS; checked like the others, a non-empty list of strings), and --sanitize runs only the checks that declare it, with those arguments added in python_check: no C++ unit tests, no byte-compile, and no --write-times. On Windows (without --no-skip) it skips them with the reason "the sanitizer lane runs on Linux": llvm-mingw's ASan is a DLL runtime, and the checks' Linux flags (-no-pie) stop clang there.
tests: tests/test_run_all_declarations.py covers the declaration's shapes (a list; True or [] stop the run) and a run of the lane with one declaring check (given exactly its arguments) and one that isn't (not run).
ci: .forgejo/workflows/ci.yml gets a Linux job, Sanitizer lane (ASan, UBSan), a full check: it runs when CI is started by hand (Actions > CI > Run workflow), as the Python checks do since #105. The source and the pinned Vulkan headers (v1.4.345, through scripts/fetch_commit.sh, as the Python checks get them), Python 3.12, g++ and clang, then python -I -B tests/run_all.py --sanitize --strict.

Flags and settings

None: no environment switch, menu row or ini key. The new tests/run_all.py --sanitize option and the RUN_ALL_SANITIZE declaration are the test runner's, described above.

Measurements

main: 0c74253d (2026-11-13)
this PR: ci/sanitize-lane @ c072cfce
Hardware: Ryzen 9 3900X, 64 GB, Windows 11, Docker (python:3.12, Debian's g++ and clang): the lane is Linux only
Scene: none: game-free. The checks under the sanitizers, as the CI job runs them
Method: in the container, the job's steps (the compilers, the Vulkan headers through scripts/fetch_commit.sh), then tests/run_all.py --sanitize --strict twice, timed. main has no --sanitize (nothing runs these checks under the sanitizers)
How to reproduce: docker run python:3.12 with the checkout, apt-get install g++ clang, bash scripts/fetch_commit.sh https://github.com/KhronosGroup/Vulkan-Headers 74d8a6cb930c68ef617b202c3ff3c59d919e086b _deps/vulkan-headers, LONGSHOT_VULKAN_HEADERS=_deps/vulkan-headers/include python -I -B tests/run_all.py --sanitize --strict

	main @ `0c74253d`	this PR @ `c072cfce`
checks run under ASan and UBSan	0 (no lane)	11
of them passing, both runs	-	10 alone; 11 with #202
the lane's time (s), median of 2	-	14.4 alone; 13.0 with #202

this branch alone:
lane run 1: exit 1, 13.6 s: 10 passed, 1 failed, 0 skipped in 12 s (linux, 24 at once)
lane run 2: exit 1, 15.1 s: 10 passed, 1 failed, 0 skipped in 14 s (linux, 24 at once)
  FAIL tests/test_texture_staging_publication.py: fixture.cpp: use of undeclared identifier 'log_debug' (as on main since #177)
with #202's fixture:
lane run 1: exit 0, 12.9 s: 11 passed, 0 failed, 0 skipped in 12 s (linux, 24 at once)
lane run 2: exit 0, 13.1 s: 11 passed, 0 failed, 0 skipped in 12 s (linux, 24 at once)

The failure alone is main's, not the lane's: test_texture_staging_publication fails the same way on main on Windows since #177's log levels, and #202 fixes its fixture. On Windows the 11 are skipped with the reason ("the sanitizer lane runs on Linux"). actionlint 1.7.12 with .forgejo/actionlint.yaml, as CI's lint job runs it: clean on this ci.yml.

Testing

Tested: the lane in a Linux container as the job runs it (above, twice, 11 of 11); tests/run_all.py, the whole suite, on this branch and on main 0c74253d, Windows (483 passed and 6 failed on both, no failure main doesn't have: test_check_third_party, test_display_pointer, test_fpx_sticky, test_release, test_spawn_emit and test_texture_staging_publication fail on main too (the last since #177; #202 fixes it)); tests/test_run_all_declarations.py (the declaration's shapes, the lane's run); actionlint on the workflow.

Not tested: the job on a Forgejo runner (it runs only when CI is started by hand, which is a maintainer's); the lane failing on a real sanitizer error (the checks pass clean, and no fault was put in to see it caught).

Needs other testers: None

Screenshots

Nothing on screen.

Risk and rollback

Only run_all.py --sanitize and the new job run the lane; a plain run_all.py runs the same checks with the same arguments as before. A malformed RUN_ALL_SANITIZE in a test stops the run with its file and line, as the other declarations do. The job runs only when CI is started by hand, so no pull request waits on it. Reverting the commits removes it.

Reviewer notes

Carried over from https://github.com/NotRexed/Hookshot/pull/2908 with the move to Longshot, then made a Forgejo job: the second commit starts it by hand with the full checks (#105) and takes main's action URLs and header fetch. The dev-loop proposal's D3 (https://github.com/NotRexed/Hookshot/pull/2636).

Issues

None on Longshot. See https://github.com/NotRexed/Hookshot/pull/2636 (D3).

Checklist

For everyone

One topic.
The title is a Conventional Commit (type(scope): description).
Rebased on current main.
python -I -B tests/run_all.py run and passing (any failure that also fails on main is named above).
Tested in the game, or what wasn't is named under Testing.
Testing filled in: what was tested, what wasn't, and what needs other testers (with test requests).
Measured: main and this PR at named commits, on the same PC, scene and settings, at least 2 interleaved runs each, the median reported.
No game files, dumps, build output or generated files.
third_party/ untouched (a new folder with a PROVENANCE.md instead).
The license is OK: GPL-3.0-or-later, nothing incompatible and nothing decompiled.

If applicable

AI assistance recorded with Assisted-by: / Co-Authored-By: trailers.
A check under tests/ that reads changed source updated in this PR, not weakened.

🤖 Generated with Claude Code

## Why Eleven checks build the code they test with the address and undefined-behaviour sanitizers when given `--sanitize`, but nothing ran them that way, so a use after free or an overflow in that code passes CI by luck. ## What changed - tests: a check says what turns its sanitizers on with a fourth declaration, `RUN_ALL_SANITIZE = ["--sanitize"]`, beside its `RUN_ALL_ARGS` (`test_low_vram_texture_mips.py` declares `["--sanitize", "address,undefined"]`, as its option takes the list). The eleven checks that declare it: `test_dump_readback`, `test_image_view_retry`, `test_import_buffer_binding`, `test_low_vram_texture_mips`, `test_program_cache_cleanup`, `test_recording_admission`, `test_sampler_admission`, `test_texel_buffer_ownership`, `test_texture_staging_publication`, `test_vulkan_clear_mip`, `test_vulkan_copy_span`. - tests: `tests/run_all.py` reads `RUN_ALL_SANITIZE` in `read_declarations` (in `DECLARATIONS`; checked like the others, a non-empty list of strings), and `--sanitize` runs only the checks that declare it, with those arguments added in `python_check`: no C++ unit tests, no byte-compile, and no `--write-times`. On Windows (without `--no-skip`) it skips them with the reason "the sanitizer lane runs on Linux": llvm-mingw's ASan is a DLL runtime, and the checks' Linux flags (`-no-pie`) stop clang there. - tests: `tests/test_run_all_declarations.py` covers the declaration's shapes (a list; `True` or `[]` stop the run) and a run of the lane with one declaring check (given exactly its arguments) and one that isn't (not run). - ci: `.forgejo/workflows/ci.yml` gets a Linux job, *Sanitizer lane (ASan, UBSan)*, a full check: it runs when CI is started by hand (*Actions* > *CI* > *Run workflow*), as the Python checks do since #105. The source and the pinned Vulkan headers (v1.4.345, through `scripts/fetch_commit.sh`, as the Python checks get them), Python 3.12, g++ and clang, then `python -I -B tests/run_all.py --sanitize --strict`. ## Flags and settings None: no environment switch, menu row or ini key. The new `tests/run_all.py --sanitize` option and the `RUN_ALL_SANITIZE` declaration are the test runner's, described above. ## Measurements main: 0c74253d (2026-11-13) this PR: ci/sanitize-lane @ c072cfce Hardware: Ryzen 9 3900X, 64 GB, Windows 11, Docker (python:3.12, Debian's g++ and clang): the lane is Linux only Scene: none: game-free. The checks under the sanitizers, as the CI job runs them Method: in the container, the job's steps (the compilers, the Vulkan headers through `scripts/fetch_commit.sh`), then `tests/run_all.py --sanitize --strict` twice, timed. main has no `--sanitize` (nothing runs these checks under the sanitizers) How to reproduce: `docker run` python:3.12 with the checkout, `apt-get install g++ clang`, `bash scripts/fetch_commit.sh https://github.com/KhronosGroup/Vulkan-Headers 74d8a6cb930c68ef617b202c3ff3c59d919e086b _deps/vulkan-headers`, `LONGSHOT_VULKAN_HEADERS=_deps/vulkan-headers/include python -I -B tests/run_all.py --sanitize --strict` | | main @ 0c74253d | this PR @ c072cfce | |---|---|---| | checks run under ASan and UBSan | 0 (no lane) | 11 | | of them passing, both runs | - | 10 alone; 11 with #202 | | the lane's time (s), median of 2 | - | 14.4 alone; 13.0 with #202 | ``` this branch alone: lane run 1: exit 1, 13.6 s: 10 passed, 1 failed, 0 skipped in 12 s (linux, 24 at once) lane run 2: exit 1, 15.1 s: 10 passed, 1 failed, 0 skipped in 14 s (linux, 24 at once) FAIL tests/test_texture_staging_publication.py: fixture.cpp: use of undeclared identifier 'log_debug' (as on main since #177) with #202's fixture: lane run 1: exit 0, 12.9 s: 11 passed, 0 failed, 0 skipped in 12 s (linux, 24 at once) lane run 2: exit 0, 13.1 s: 11 passed, 0 failed, 0 skipped in 12 s (linux, 24 at once) ``` The failure alone is main's, not the lane's: `test_texture_staging_publication` fails the same way on main on Windows since #177's log levels, and #202 fixes its fixture. On Windows the 11 are skipped with the reason ("the sanitizer lane runs on Linux"). actionlint 1.7.12 with `.forgejo/actionlint.yaml`, as CI's lint job runs it: clean on this `ci.yml`. ## Testing **Tested:** the lane in a Linux container as the job runs it (above, twice, 11 of 11); `tests/run_all.py`, the whole suite, on this branch and on main 0c74253d, Windows (483 passed and 6 failed on both, no failure main doesn't have: test_check_third_party, test_display_pointer, test_fpx_sticky, test_release, test_spawn_emit and test_texture_staging_publication fail on main too (the last since #177; #202 fixes it)); `tests/test_run_all_declarations.py` (the declaration's shapes, the lane's run); actionlint on the workflow. **Not tested:** the job on a Forgejo runner (it runs only when CI is started by hand, which is a maintainer's); the lane failing on a real sanitizer error (the checks pass clean, and no fault was put in to see it caught). **Needs other testers:** None ## Screenshots Nothing on screen. ## Risk and rollback Only `run_all.py --sanitize` and the new job run the lane; a plain `run_all.py` runs the same checks with the same arguments as before. A malformed `RUN_ALL_SANITIZE` in a test stops the run with its file and line, as the other declarations do. The job runs only when CI is started by hand, so no pull request waits on it. Reverting the commits removes it. ## Reviewer notes Carried over from https://github.com/NotRexed/Hookshot/pull/2908 with the move to Longshot, then made a Forgejo job: the second commit starts it by hand with the full checks (#105) and takes main's action URLs and header fetch. The dev-loop proposal's D3 (https://github.com/NotRexed/Hookshot/pull/2636). ## Issues None on Longshot. See https://github.com/NotRexed/Hookshot/pull/2636 (D3). ## Checklist ### For everyone - [x] One topic. - [x] The title is a Conventional Commit (`type(scope): description`). - [x] Rebased on current `main`. - [x] `python -I -B tests/run_all.py` run and passing (any failure that also fails on `main` is named above). - [x] Tested in the game, or what wasn't is named under Testing. - [x] Testing filled in: what was tested, what wasn't, and what needs other testers (with test requests). - [x] Measured: main and this PR at named commits, on the same PC, scene and settings, at least 2 interleaved runs each, the median reported. - [x] No game files, dumps, build output or generated files. - [x] `third_party/` untouched (a new folder with a `PROVENANCE.md` instead). - [x] The license is OK: GPL-3.0-or-later, nothing incompatible and nothing decompiled. ### If applicable - [x] AI assistance recorded with `Assisted-by:` / `Co-Authored-By:` trailers. - [x] A check under `tests/` that reads changed source updated in this PR, not weakened. 🤖 Generated with [Claude Code](https://claude.com/claude-code)

mooki referenced this pull request

2026-11-12 23:57:12 +00:00

Idea: opt-in hardware tests on self-hosted Forgejo runners #43

dyst changed title from ~~ci: a sanitizer lane for the checks that offer --sanitize~~ to WIP: ci: a sanitizer lane for the checks that offer --sanitize

2026-11-13 01:13:52 +00:00

dyst commented

2026-11-13 01:13:52 +00:00

Author

Member

Back to WIP under #48's process: this was carried over before the new template. Its description will be brought to the template and measured on Longshot (main against this PR, at named commits) when it's wanted for review; ask here if you'd like it sooner.

dyst commented

2026-11-13 01:58:18 +00:00

Author

Member

Description brought to Longshot's PR template (#48). Still WIP: its numbers are from the GitHub version, and it isn't rebased on current main; it gets measured on Longshot (main against this PR, at named commits) before it is marked ready.

dyst commented

2026-11-13 02:00:33 +00:00

Author

Member

Description moved to the template's Testing section and whole checklist (#72): what was tested, what wasn't, and what needs other testers (their test requests to open while this is WIP).

dyst commented

2026-11-13 11:45:28 +00:00

Author

Member

The description follows the template after #76 and #106: If applicable keeps only the boxes that apply (one that applies and isn't done yet stays unticked while this is WIP), the Flags table has the template's seven columns (Goal is now Recommendation), and GitHub references are full URLs. Nothing else changed.

dyst referenced this pull request

2026-11-13 13:55:19 +00:00

test(vk): the texture staging check declares LogLevel and logf_at again, failing on main since #177 #202

dyst commented

2026-11-13 13:56:38 +00:00

Author

Member

Rebased onto main 0c74253d, and a second commit makes the lane a Forgejo full check: started by hand with the others (#105), main's action URLs, the Vulkan headers through fetch_commit.sh; actionlint 1.7.12 clean. Measured in a Linux container: 10 of 11 alone, the 11th is test_texture_staging_publication, broken on main since #177 and fixed by #202; 11 of 11 with #202's fixture. Whole suite: no failure main doesn't have. Ready for review.

Rebased onto main 0c74253d, and a second commit makes the lane a Forgejo full check: started by hand with the others (#105), main's action URLs, the Vulkan headers through fetch_commit.sh; actionlint 1.7.12 clean. Measured in a Linux container: 10 of 11 alone, the 11th is test_texture_staging_publication, broken on main since #177 and fixed by #202; 11 of 11 with #202's fixture. Whole suite: no failure main doesn't have. Ready for review.

dyst force-pushed ci/sanitize-lane from 674fcdc5bd

PR description / Description follows the template (pull_request_target) Successful in 5s

Details

to c072cfce8b

CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Has been skipped

Details

CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Has been skipped

Details

CI / Sanitizer lane (ASan, UBSan) (pull_request) Has been skipped

Details

CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Has been skipped

Details

Third-party parts / The same as upstream (pull_request) Has been skipped

Details

Website / Website on Cloudflare (pull_request) Has been skipped

Details

PR description / Description follows the template (pull_request_target) Has been cancelled

Details

Third-party parts / As their notes say (pull_request) Has been cancelled

Details

CI / Conventional Commit title (pull_request) Has been cancelled

Details

CI / Workflow lint (actionlint) (pull_request) Has been cancelled

Details

2026-11-13 13:56:39 +00:00

Compare

dyst changed title from ~~WIP: ci: a sanitizer lane for the checks that offer --sanitize~~ to ci: a sanitizer lane for the checks that offer --sanitize

2026-11-13 13:57:20 +00:00

HumanBean referenced this pull request from a commit

2026-11-13 16:38:43 +00:00

test(vk): the texture staging check declares LogLevel and logf_at again, failing on main since #177 (#202)

CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Has been skipped

Details

CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Has been skipped

Details

CI / Sanitizer lane (ASan, UBSan) (pull_request) Has been skipped

Details

CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Has been skipped

Details

Third-party parts / The same as upstream (pull_request) Has been skipped

Details

Website / Website on Cloudflare (pull_request) Has been skipped

Details

PR description / Description follows the template (pull_request_target) Has been cancelled

Details

Third-party parts / As their notes say (pull_request) Has been cancelled

Details

CI / Conventional Commit title (pull_request) Has been cancelled

Details

CI / Workflow lint (actionlint) (pull_request) Has been cancelled

Details

This pull request doesn't have enough approvals yet. 0 of 1 approvals granted.

This branch is out-of-date with the base branch

You are not authorized to merge this pull request.

View command line instructions

Checkout

From your project repository, check out a new branch and test the changes.

git fetch -u 'ci/sanitize-lane:dyst-ci/sanitize-lane'

git switch 'dyst-ci/sanitize-lane'

Rows
Columns

ci: a sanitizer lane for the checks that offer --sanitize #29

Why

What changed

Flags and settings

Measurements

Testing

Screenshots

Risk and rollback

Reviewer notes

Issues

Checklist

For everyone

If applicable

Checkout