ci: add an owner-approved hardware fixture pilot #100

Closed
mooki wants to merge 2 commits from mooki/Longshot:ci/opt-in-hardware into main
Member

Why

Volunteers need a small, opt-in way to run authored Vulkan checks on real GPUs without giving an ordinary pull request access to their machines. This adds a disabled-by-default, owner-controlled pilot, not an upstream hardware service; review of this code is separate from authorization to register or activate a runner.

I offer my Arch Linux PC with an RTX 4090 and an Intel UHD 770 as a second adapter, carrying the explicit longshot-hardware-linux label. Registration would happen only after a maintainer agrees and I confirm the proposed setup.

What changed

  • .forgejo/workflows/hardware-fixtures.yml: manual dispatch only, only an owner's main, explicitly disabled in MagicBeanDev/Longshot. The full approved SHA must match the dispatch and still be the remote head before execution. A repository variable enables the job; an explicit hardware label routes it. No PR trigger, arbitrary PR checkout, release secret or required merge check.
  • scripts/hardware_fixtures.py: probe the selected GPU, then compile/run the existing five write-back cases and one copy-span case inside bounded, offline rootless Podman containers. The default --no-gpu path writes not_run without probing hardware or starting a container. Missing devices, skips, wrong-device results, repeated/missing cases, missing validation activation and command/cleanup failures are not passes.
  • Export only six tracked regular source files from the approved commit and mount them with the pinned Vulkan headers. Do not mount .git, untracked files, a home, a dump, saves, a display socket or a writable host directory. The payload receives no inherited token/proxy environment. Only fixed configuration rows leave it; raw command output and directory artifacts do not.
  • tests/test_hardware_fixtures.py: GPU-free parsing, reporting, resource-command, cleanup, source-export and workflow-contract regressions. Authored command results exercise the orchestration without pretending to prove a real GPU or container.
  • docs/forgejo-runners.md: put runner placement and security first; document the optional workflow, prerequisites, limits, safe reporting command and human fallback. Preserve main's Linux-only CI and Windows-on-Linux build instructions, its ordinary runner labels and its RELEASE_UPLOAD_API upload setting; cross-building Windows releases does not provision this GPU image.
  • .forgejo/actionlint.yaml: list the hardware runner's label longshot-hardware-linux beside quick, so main's workflow lint accepts hardware-fixtures.yml. No workflow changes.

Scope: this first slice tests an owner-reviewed default-branch snapshot, not an arbitrary PR head. Labels are routing, not isolation: another eligible workflow can select the same runner. Use only an owner's fork or dedicated control repository with trusted workflows and writers, never a personal PC in the shared upstream/organization runner pool. Game-based work, automatic request consumption and per-PR label approval remain decisions and separate work under #43.

The action and Vulkan-header pins match main. tools.lock and scripts/tools.py supply the Windows builder, not this Linux driver image, so they are unchanged. The owner must provision a reviewed image pinned by digest; the job never pulls an image, installs packages or accepts an SDK license. The aspect fixture is deliberately excluded until its explicit selector lands; the software-driver proposal in #67 remains separate.

Flags and settings

No game setting, menu option, ini key or runtime default changes. These controls are documented in the runner guide and the command's help. Recommendations apply to this opt-in command, not to enabling hardware execution for everyone.

Flag What it does Default now Recommendation Helps on Doesn't help on Untested on
--run Explicitly execute the GPU probe and fixtures Off Off by default; owner opt-in only Reviewed Linux hardware jobs Ordinary source checks Real GPU/container execution
--no-gpu Write a truthful not-run row, without GPU access Default behavior On by default; safe reporting path GPU-free checks Hardware qualification Windows execution
--commit Require a full approved SHA matching the checkout Required On by default; require explicit provenance Exact revision reporting Mutable branch approval Forgejo dispatch
--output Write one new Markdown report, never overwrite evidence Required new file On by default; preserve existing evidence Reviewed configuration rows Raw log collection Artifact upload
--image Select a preloaded digest-pinned container image Required with --run Off by default; provision and review before opting in Repeatable fixture dependencies Unprovisioned machines Real image startup
--gpu-device Expose one render node or one NVIDIA CDI GPU Required with --run Off by default; no implicit adapter Deliberate device access All-device access Intel, AMD and NVIDIA pass-through
--device-name Require one matching hardware GPU and identical fixture selection Required with --run Off by default; require explicit selection Multi-adapter PCs Ambiguous/software devices Real device inventory
--headers Select pinned Vulkan headers Required with --run; workflow uses main's v1.4.345 pin On by default; use explicitly pinned headers Fixture compilation Game builds Container compilation
HARDWARE_TESTS_ENABLED Permit manual jobs in the control repository Unset/off; only true enables Off by default; owner chooses availability Owner availability control Runner access isolation Forgejo evaluation
HARDWARE_TEST_IMAGE Supply the digest-pinned image to --image Unset Off by default; no unreviewed image Owner provisioning Dependency installation Forgejo execution
HARDWARE_TEST_DEVICE Supply the single adapter to --gpu-device Unset Off by default; no implicit device Owner provisioning Hardware inventory automation Forgejo execution
approved_sha Approve the full current owner-main revision Required On by default; require explicit revision approval Reviewed snapshots Arbitrary PR authorization Live changed-head refusal
device_name Supply the intended GPU-name substring Required Off by default; require a deliberate device choice Explicit hardware selection Software Vulkan lane Forgejo execution
VK_INSTANCE_LAYERS Enable Khronos core validation VK_LAYER_KHRONOS_validation in payload On by default; check API errors in these fixtures API error detection Speed measurements Actual layer activation
VK_LOADER_DEBUG Require layer insertion evidence layer in payload On by default; prove validation was activated Rejecting unvalidated passes Game logging Actual loader output
VK_LAYER_SETTINGS_PATH Send validation errors to bounded private output Private container temporary directory On by default; capture errors without public raw logs Fixtures without a debug callback Public raw logs Actual layer behavior

Measurements

Branch state: on main 5e072fc8c2adadba5ea610148be1c47800ba0cca; head 268bdfec27b87a1fb86a38c9bf1de041b2dc16ab; whole game-free suite on this head: 543 passed, 3 failed, 16 skipped (main 5e072fc8c2adadba5ea610148be1c47800ba0cca: 542 passed, 3 failed, 16 skipped). The 3 failing tests fail on main the same way: tests/test_framegen_resources.py, tests/test_module_ids.py, tests/test_pad_layout.py. The branch was carried onto this main by hand; what differs from a mechanical carry is named in the comment posted with this update.

main: 5eaaf7fb9c (2026-11-13; source baseline when the paired tests were run)
this PR: ci/opt-in-hardware @ 2d9962311e23b6762d1a6c511f9391749fd419b2 (paired-test head)
Rebased onto main 2a2da67f2bf909f7a49983e0499dbed1da150dae as c2a880c8331b3ee8dcadf07766e01bbd760c37a1. The three inputs to these named tests (scripts/hardware_fixtures.py, .forgejo/workflows/hardware-fixtures.yml, tests/test_hardware_fixtures.py) are byte-identical to the paired-test head, and the new main still lacks all three. The paired results below are retained at their actual tested commits, not presented as reruns. The runner-guide conflict was resolved around main's Linux-only CI and Windows-on-Linux build instructions; the new main's tests/test_windows_on_linux.py passed all four cases on the rebased head.

Hardware: the same Core i9-13900K PC, Arch Linux, Python 3.14.8. No GPU was probed.
Scene: no game; the published GPU-free regression's authored parser inputs and process-bound checks.
Method: one unmeasured warm-up on each source, then main / this PR / main / this PR. Each run uses an unchanged copy of this PR's tests/test_hardware_fixtures.py against the named source tree. Wall time includes Python startup and test discovery; medians are the middle of the two observations. The whole-suite receipts above remain at their original commits; this comparison ran only the named test.
How to reproduce: in separate copies of the two source revisions, place the published PR's unchanged tests/test_hardware_fixtures.py under tests/, without copying its implementation into main. Run python3 -I -B -m unittest discover -s tests -p test_hardware_fixtures.py, once per source to warm up, then twice each in the order above. Main's production source and workflow stay unchanged.

Check main @ 5eaaf7fb9 this PR @ 2d996231
Passed (run 1 / run 2; median) 0 / 0; 0 27 / 27; 27
Failed checks (run 1 / run 2; median) 1 / 1; 1 (import errors) 0 / 0; 0
Wall seconds (run 1 / run 2; median) 0.085150 / 0.087466; 0.086308 0.235154 / 0.233006; 0.234080
Explicitly skipped (run 1 / run 2) 0 / 0 0 / 0

What main's failure means: main does not contain scripts/hardware_fixtures.py or the hardware workflow. The identical new regression cannot import that command, so unittest reports one failed import rather than executing its 27 cases. Those cases were not run on main; they are not 27 failures or 27 passes. This is evidence of missing tooling on main and passing authored coverage on the PR, not a main GPU defect or a proof of real container isolation.

The captured outcomes were:

main run 1: 0 passed, 1 failed checks (1 import/errors), 0 skipped; wall 0.085150 s
this PR run 1: 27 passed, 0 failed checks (0 import/errors), 0 skipped; wall 0.235154 s
main run 2: 0 passed, 1 failed checks (1 import/errors), 0 skipped; wall 0.087466 s
this PR run 2: 27 passed, 0 failed checks (0 import/errors), 0 skipped; wall 0.233006 s

These times are not a speed comparison: main stops at import while the PR executes all 27 cases, including an intentional 0.1-second timeout case, under shared machine load. The claimed measurement is command availability and game-free regression outcomes. No game-speed effect is claimed.

The earlier committed --no-gpu invocation also returned hardware fixtures: not_run (no GPU access) and wrote the eight-column configuration row with the full head and not probed; it did not run hardware. YAML parsing passed. Unmodified actionlint 1.7.12 reported three unsupported full-origin Forgejo action URLs; its validation-only copy, with only those origins removed and the custom label declared, passed the remaining compatible checks. That limitation and the real Forgejo/GPU trial remain explicit below.

Testing

Tested: on Linux, the identical published regression in two interleaved runs against each named source revision, after one warm-up each. The PR passes all 27 cases in both measured runs; main reports one import error each because the new command is absent, so its behavior cases do not run. The cases cover authored GPU/driver records, wrong/missing/repeated results, validation activation, bounded output and timeout, cleanup refusal, non-overwrite behavior, exact source export and no-GPU orchestration. The earlier committed no-GPU command and its configuration row were inspected; YAML parsing and the limited actionlint check are described above. These are script/workflow correctness checks, not a hardware or speed result. After the documentation-only conflict resolution onto 2a2da67f2b, the three named-test inputs were checked byte-for-byte unchanged; their paired results are carried at the original commits, not rerun. The rebased head also passes all four tests/test_windows_on_linux.py cases, including the check that no workflow requests a Windows runner.

The earlier whole source suite at 2d9962311e23b6762d1a6c511f9391749fd419b2 passed: 510 passed, 0 failed, 19 declared skips. Its original base 77f605a25f175fbda9ad131ac6be96bbde85d9c2 had 509 passed, 0 failed, 19 skips; no new whole-suite result is claimed for the current-main source comparison.

Not tested: no runner was installed, registered or started; no real Forgejo dispatch/checkout/upload or changed-head refusal was exercised. Podman is unavailable on this PC, so no image, resource-limit enforcement, confinement/escape resistance, GPU pass-through or real cleanup/cancellation was tested. No hardware GPU, Windows execution, AMD/Deck run, game build/run, swapchain or performance measurement. Mock command results prove controller logic only. The first real isolation and publication trial is still required before activation.

Needs other testers:

  • Runner operator/security reviewer on isolated Linux (NVIDIA/Intel first; AMD/SteamOS for those lanes), plus Windows Python coverage: qualify the owner-controlled placement, enforced limits and credential/file/network isolation, stale-SHA refusal, cancellation cleanup and bounded artifact; after maintainer/owner approval, expect all six real-GPU fixture cases with active validation and correct device identity, and the portable parser checks to pass on Windows apart from the Linux-only case (test request T-ULFA)

Screenshots

Nothing on screen. No window or game screenshot was produced.

Risk and rollback

A hardware label or a manual workflow is not a security boundary. Another eligible workflow can bypass this command, and a rootless container still shares the host kernel and GPU driver. Keep the controller in a trusted owner-controlled repository and separate environment without personal/game files or release credentials; never register this PC upstream merely because this workflow is gated. A pinned image, CDI mapping and runner configuration still need review. Driver hangs/escapes and forged results remain possible.

Missing tools, unsupported driver features or changed validation output deliberately fail. An offline, skipped, refused or timed-out job is not a pass and must fall back to human testing. Disable HARDWARE_TESTS_ENABLED and stop the runner to withdraw the machine; reverting these four files removes the pilot without touching game settings or saves.

Reviewer notes

Start with container_command(), export_source(), capture() and the fixed-format parser/report functions. The workflow/controller accepts only the owner's reviewed main; it is not a secure arbitrary-PR executor. Decide runner ownership/placement, the label and image policy, who approves exact revisions and who reviews publication before activating anything. No payload signing/publishing secret is required; Forgejo's job token remains in trusted controller/action processes, not the containers.

Keep #43 open. Automatic queue consumption, revision-bound PR labels, a separate PR publisher, other OS/device qualification and any dump-using workload are outside this first slice. #69's owner-controlled rolling monitor is not a Forgejo runner and is unchanged.

Issues

See #43. Related: #67 for software Vulkan fixtures and #69 for the separate local rolling record.

Checklist

For everyone

  • One topic.
  • The title is a Conventional Commit (type(scope): description).
  • Rebased on current main.
  • python -I -B tests/run_all.py run and passing (any failure that also fails on main is named above).
  • Tested in the game, or what wasn't is named under Testing.
  • Testing filled in: what was tested, what wasn't, and what needs other testers (with test requests).
  • Measured: main and this PR at named commits, on the same PC, scene and settings, at least 2 interleaved runs each, the median reported. Named script tests on identical authored inputs, with pass/fail counts and wall-time medians above; no GPU or speed claim.
  • No game files, dumps, build output or generated files.
  • third_party/ untouched (a new folder with a PROVENANCE.md instead).
  • The license is OK: GPL-3.0-or-later, nothing incompatible and nothing decompiled.

If applicable

  • AI assistance recorded with Assisted-by: / Co-Authored-By: trailers.
  • Flags noted, and the switch reference updated (docs/development.md, the README). Test/operator controls are documented in the runner guide and command help; game-switch references are unchanged.
  • A check under tests/ that reads changed source updated in this PR, not weakened. New workflow-contract checks; existing fixture checks unchanged.
  • Docs updated.

New on this forge; not opened on GitHub.

Assisted-by: GPT-6 Astra
Co-Authored-By: GPT-6 Astra [email protected]

## Why Volunteers need a small, opt-in way to run authored Vulkan checks on real GPUs without giving an ordinary pull request access to their machines. This adds a disabled-by-default, owner-controlled pilot, not an upstream hardware service; review of this code is separate from authorization to register or activate a runner. I offer my Arch Linux PC with an RTX 4090 and an Intel UHD 770 as a second adapter, carrying the explicit `longshot-hardware-linux` label. Registration would happen only after a maintainer agrees and I confirm the proposed setup. ## What changed - `.forgejo/workflows/hardware-fixtures.yml`: manual dispatch only, only an owner's `main`, explicitly disabled in `MagicBeanDev/Longshot`. The full approved SHA must match the dispatch and still be the remote head before execution. A repository variable enables the job; an explicit hardware label routes it. No PR trigger, arbitrary PR checkout, release secret or required merge check. - `scripts/hardware_fixtures.py`: probe the selected GPU, then compile/run the existing five write-back cases and one copy-span case inside bounded, offline rootless Podman containers. The default `--no-gpu` path writes `not_run` without probing hardware or starting a container. Missing devices, skips, wrong-device results, repeated/missing cases, missing validation activation and command/cleanup failures are not passes. - Export only six tracked regular source files from the approved commit and mount them with the pinned Vulkan headers. Do not mount `.git`, untracked files, a home, a dump, saves, a display socket or a writable host directory. The payload receives no inherited token/proxy environment. Only fixed configuration rows leave it; raw command output and directory artifacts do not. - `tests/test_hardware_fixtures.py`: GPU-free parsing, reporting, resource-command, cleanup, source-export and workflow-contract regressions. Authored command results exercise the orchestration without pretending to prove a real GPU or container. - `docs/forgejo-runners.md`: put runner placement and security first; document the optional workflow, prerequisites, limits, safe reporting command and human fallback. Preserve main's Linux-only CI and Windows-on-Linux build instructions, its ordinary runner labels and its `RELEASE_UPLOAD_API` upload setting; cross-building Windows releases does not provision this GPU image. - `.forgejo/actionlint.yaml`: list the hardware runner's label `longshot-hardware-linux` beside `quick`, so main's workflow lint accepts `hardware-fixtures.yml`. No workflow changes. **Scope:** this first slice tests an owner-reviewed default-branch snapshot, not an arbitrary PR head. Labels are routing, not isolation: another eligible workflow can select the same runner. Use only an owner's fork or dedicated control repository with trusted workflows and writers, never a personal PC in the shared upstream/organization runner pool. Game-based work, automatic request consumption and per-PR label approval remain decisions and separate work under #43. The action and Vulkan-header pins match main. `tools.lock` and `scripts/tools.py` supply the Windows builder, not this Linux driver image, so they are unchanged. The owner must provision a reviewed image pinned by digest; the job never pulls an image, installs packages or accepts an SDK license. The aspect fixture is deliberately excluded until its explicit selector lands; the software-driver proposal in #67 remains separate. ## Flags and settings No game setting, menu option, ini key or runtime default changes. These controls are documented in the runner guide and the command's help. Recommendations apply to this opt-in command, not to enabling hardware execution for everyone. | Flag | What it does | Default now | Recommendation | Helps on | Doesn't help on | Untested on | |---|---|---|---|---|---|---| | `--run` | Explicitly execute the GPU probe and fixtures | Off | Off by default; owner opt-in only | Reviewed Linux hardware jobs | Ordinary source checks | Real GPU/container execution | | `--no-gpu` | Write a truthful not-run row, without GPU access | Default behavior | On by default; safe reporting path | GPU-free checks | Hardware qualification | Windows execution | | `--commit` | Require a full approved SHA matching the checkout | Required | On by default; require explicit provenance | Exact revision reporting | Mutable branch approval | Forgejo dispatch | | `--output` | Write one new Markdown report, never overwrite evidence | Required new file | On by default; preserve existing evidence | Reviewed configuration rows | Raw log collection | Artifact upload | | `--image` | Select a preloaded digest-pinned container image | Required with `--run` | Off by default; provision and review before opting in | Repeatable fixture dependencies | Unprovisioned machines | Real image startup | | `--gpu-device` | Expose one render node or one NVIDIA CDI GPU | Required with `--run` | Off by default; no implicit adapter | Deliberate device access | All-device access | Intel, AMD and NVIDIA pass-through | | `--device-name` | Require one matching hardware GPU and identical fixture selection | Required with `--run` | Off by default; require explicit selection | Multi-adapter PCs | Ambiguous/software devices | Real device inventory | | `--headers` | Select pinned Vulkan headers | Required with `--run`; workflow uses main's v1.4.345 pin | On by default; use explicitly pinned headers | Fixture compilation | Game builds | Container compilation | | `HARDWARE_TESTS_ENABLED` | Permit manual jobs in the control repository | Unset/off; only `true` enables | Off by default; owner chooses availability | Owner availability control | Runner access isolation | Forgejo evaluation | | `HARDWARE_TEST_IMAGE` | Supply the digest-pinned image to `--image` | Unset | Off by default; no unreviewed image | Owner provisioning | Dependency installation | Forgejo execution | | `HARDWARE_TEST_DEVICE` | Supply the single adapter to `--gpu-device` | Unset | Off by default; no implicit device | Owner provisioning | Hardware inventory automation | Forgejo execution | | `approved_sha` | Approve the full current owner-main revision | Required | On by default; require explicit revision approval | Reviewed snapshots | Arbitrary PR authorization | Live changed-head refusal | | `device_name` | Supply the intended GPU-name substring | Required | Off by default; require a deliberate device choice | Explicit hardware selection | Software Vulkan lane | Forgejo execution | | `VK_INSTANCE_LAYERS` | Enable Khronos core validation | `VK_LAYER_KHRONOS_validation` in payload | On by default; check API errors in these fixtures | API error detection | Speed measurements | Actual layer activation | | `VK_LOADER_DEBUG` | Require layer insertion evidence | `layer` in payload | On by default; prove validation was activated | Rejecting unvalidated passes | Game logging | Actual loader output | | `VK_LAYER_SETTINGS_PATH` | Send validation errors to bounded private output | Private container temporary directory | On by default; capture errors without public raw logs | Fixtures without a debug callback | Public raw logs | Actual layer behavior | ## Measurements <!-- branch-state --> Branch state: on main `5e072fc8c2adadba5ea610148be1c47800ba0cca`; head `268bdfec27b87a1fb86a38c9bf1de041b2dc16ab`; whole game-free suite on this head: `543 passed, 3 failed, 16 skipped` (main `5e072fc8c2adadba5ea610148be1c47800ba0cca`: `542 passed, 3 failed, 16 skipped`). The 3 failing tests fail on main the same way: `tests/test_framegen_resources.py`, `tests/test_module_ids.py`, `tests/test_pad_layout.py`. The branch was carried onto this main by hand; what differs from a mechanical carry is named in the comment posted with this update. <!-- /branch-state --> main: 5eaaf7fb9c548782a3dfabcf89ad37363ee76b1e (2026-11-13; source baseline when the paired tests were run) this PR: ci/opt-in-hardware @ 2d9962311e23b6762d1a6c511f9391749fd419b2 (paired-test head) Rebased onto main `2a2da67f2bf909f7a49983e0499dbed1da150dae` as `c2a880c8331b3ee8dcadf07766e01bbd760c37a1`. The three inputs to these named tests (`scripts/hardware_fixtures.py`, `.forgejo/workflows/hardware-fixtures.yml`, `tests/test_hardware_fixtures.py`) are byte-identical to the paired-test head, and the new main still lacks all three. The paired results below are retained at their actual tested commits, not presented as reruns. The runner-guide conflict was resolved around main's Linux-only CI and Windows-on-Linux build instructions; the new main's `tests/test_windows_on_linux.py` passed all four cases on the rebased head. Hardware: the same Core i9-13900K PC, Arch Linux, Python 3.14.8. No GPU was probed. Scene: no game; the published GPU-free regression's authored parser inputs and process-bound checks. Method: one unmeasured warm-up on each source, then main / this PR / main / this PR. Each run uses an unchanged copy of this PR's `tests/test_hardware_fixtures.py` against the named source tree. Wall time includes Python startup and test discovery; medians are the middle of the two observations. The whole-suite receipts above remain at their original commits; this comparison ran only the named test. How to reproduce: in separate copies of the two source revisions, place the published PR's unchanged `tests/test_hardware_fixtures.py` under `tests/`, without copying its implementation into main. Run `python3 -I -B -m unittest discover -s tests -p test_hardware_fixtures.py`, once per source to warm up, then twice each in the order above. Main's production source and workflow stay unchanged. | Check | main @ 5eaaf7fb9 | this PR @ 2d996231 | |---|---|---| | Passed (run 1 / run 2; median) | 0 / 0; 0 | 27 / 27; 27 | | Failed checks (run 1 / run 2; median) | 1 / 1; 1 (import errors) | 0 / 0; 0 | | Wall seconds (run 1 / run 2; median) | 0.085150 / 0.087466; 0.086308 | 0.235154 / 0.233006; 0.234080 | | Explicitly skipped (run 1 / run 2) | 0 / 0 | 0 / 0 | **What main's failure means:** main does not contain `scripts/hardware_fixtures.py` or the hardware workflow. The identical new regression cannot import that command, so unittest reports one failed import rather than executing its 27 cases. Those cases were not run on main; they are not 27 failures or 27 passes. This is evidence of missing tooling on main and passing authored coverage on the PR, not a main GPU defect or a proof of real container isolation. The captured outcomes were: ```text main run 1: 0 passed, 1 failed checks (1 import/errors), 0 skipped; wall 0.085150 s this PR run 1: 27 passed, 0 failed checks (0 import/errors), 0 skipped; wall 0.235154 s main run 2: 0 passed, 1 failed checks (1 import/errors), 0 skipped; wall 0.087466 s this PR run 2: 27 passed, 0 failed checks (0 import/errors), 0 skipped; wall 0.233006 s ``` These times are **not a speed comparison**: main stops at import while the PR executes all 27 cases, including an intentional 0.1-second timeout case, under shared machine load. The claimed measurement is command availability and game-free regression outcomes. No game-speed effect is claimed. The earlier committed `--no-gpu` invocation also returned `hardware fixtures: not_run (no GPU access)` and wrote the eight-column configuration row with the full head and `not probed`; it did not run hardware. YAML parsing passed. Unmodified actionlint 1.7.12 reported three unsupported full-origin Forgejo action URLs; its validation-only copy, with only those origins removed and the custom label declared, passed the remaining compatible checks. That limitation and the real Forgejo/GPU trial remain explicit below. ## Testing **Tested:** on Linux, the identical published regression in two interleaved runs against each named source revision, after one warm-up each. The PR passes all 27 cases in both measured runs; main reports one import error each because the new command is absent, so its behavior cases do not run. The cases cover authored GPU/driver records, wrong/missing/repeated results, validation activation, bounded output and timeout, cleanup refusal, non-overwrite behavior, exact source export and no-GPU orchestration. The earlier committed no-GPU command and its configuration row were inspected; YAML parsing and the limited actionlint check are described above. These are script/workflow correctness checks, not a hardware or speed result. After the documentation-only conflict resolution onto `2a2da67f2b`, the three named-test inputs were checked byte-for-byte unchanged; their paired results are carried at the original commits, not rerun. The rebased head also passes all four `tests/test_windows_on_linux.py` cases, including the check that no workflow requests a Windows runner. The earlier whole source suite at `2d9962311e23b6762d1a6c511f9391749fd419b2` passed: 510 passed, 0 failed, 19 declared skips. Its original base `77f605a25f175fbda9ad131ac6be96bbde85d9c2` had 509 passed, 0 failed, 19 skips; no new whole-suite result is claimed for the current-main source comparison. **Not tested:** no runner was installed, registered or started; no real Forgejo dispatch/checkout/upload or changed-head refusal was exercised. Podman is unavailable on this PC, so no image, resource-limit enforcement, confinement/escape resistance, GPU pass-through or real cleanup/cancellation was tested. No hardware GPU, Windows execution, AMD/Deck run, game build/run, swapchain or performance measurement. Mock command results prove controller logic only. The first real isolation and publication trial is still required before activation. **Needs other testers:** - Runner operator/security reviewer on isolated Linux (NVIDIA/Intel first; AMD/SteamOS for those lanes), plus Windows Python coverage: qualify the owner-controlled placement, enforced limits and credential/file/network isolation, stale-SHA refusal, cancellation cleanup and bounded artifact; after maintainer/owner approval, expect all six real-GPU fixture cases with active validation and correct device identity, and the portable parser checks to pass on Windows apart from the Linux-only case (test request T-ULFA) ## Screenshots Nothing on screen. No window or game screenshot was produced. ## Risk and rollback A hardware label or a manual workflow is not a security boundary. Another eligible workflow can bypass this command, and a rootless container still shares the host kernel and GPU driver. Keep the controller in a trusted owner-controlled repository and separate environment without personal/game files or release credentials; never register this PC upstream merely because this workflow is gated. A pinned image, CDI mapping and runner configuration still need review. Driver hangs/escapes and forged results remain possible. Missing tools, unsupported driver features or changed validation output deliberately fail. An offline, skipped, refused or timed-out job is not a pass and must fall back to human testing. Disable `HARDWARE_TESTS_ENABLED` and stop the runner to withdraw the machine; reverting these four files removes the pilot without touching game settings or saves. ## Reviewer notes Start with `container_command()`, `export_source()`, `capture()` and the fixed-format parser/report functions. The workflow/controller accepts only the owner's reviewed `main`; it is not a secure arbitrary-PR executor. Decide runner ownership/placement, the label and image policy, who approves exact revisions and who reviews publication before activating anything. No payload signing/publishing secret is required; Forgejo's job token remains in trusted controller/action processes, not the containers. Keep #43 open. Automatic queue consumption, revision-bound PR labels, a separate PR publisher, other OS/device qualification and any dump-using workload are outside this first slice. #69's owner-controlled rolling monitor is not a Forgejo runner and is unchanged. ## Issues See #43. Related: #67 for software Vulkan fixtures and #69 for the separate local rolling record. ## Checklist ### For everyone - [x] One topic. - [x] The title is a Conventional Commit (`type(scope): description`). - [x] Rebased on current `main`. - [x] `python -I -B tests/run_all.py` run and passing (any failure that also fails on `main` is named above). - [x] Tested in the game, or what wasn't is named under Testing. - [x] Testing filled in: what was tested, what wasn't, and what needs other testers (with test requests). - [x] Measured: main and this PR at named commits, on the same PC, scene and settings, at least 2 interleaved runs each, the median reported. Named script tests on identical authored inputs, with pass/fail counts and wall-time medians above; no GPU or speed claim. - [x] No game files, dumps, build output or generated files. - [x] `third_party/` untouched (a new folder with a `PROVENANCE.md` instead). - [x] The license is OK: GPL-3.0-or-later, nothing incompatible and nothing decompiled. ### If applicable - [x] AI assistance recorded with `Assisted-by:` / `Co-Authored-By:` trailers. - [x] Flags noted, and the switch reference updated (docs/development.md, the README). Test/operator controls are documented in the runner guide and command help; game-switch references are unchanged. - [x] A check under `tests/` that reads changed source updated in this PR, not weakened. New workflow-contract checks; existing fixture checks unchanged. - [x] Docs updated. --- New on this forge; not opened on GitHub. Assisted-by: GPT-6 Astra Co-Authored-By: GPT-6 Astra <[email protected]>
Author
Member

Rebased onto main 2a2da67f2b, preserving Windows-on-Linux release instructions and the hardware security sections; the two interleaved named-test pairs retain their original commits with identical input hashes, and the new workflow check passes all four cases. The real GPU, isolation and Forgejo trial remains in one test request, and registration or activation still needs maintainer agreement and my confirmation. The test request is open (T-ULFA). Rebased onto main b9cdc4326 with the change otherwise unchanged; whole suite on the new head a213814b3: 512 passed, 0 failed, 19 skipped. Ready for review.

Rebased onto main `2a2da67f2b`, preserving Windows-on-Linux release instructions and the hardware security sections; the two interleaved named-test pairs retain their original commits with identical input hashes, and the new workflow check passes all four cases. The real GPU, isolation and Forgejo trial remains in one test request, and registration or activation still needs maintainer agreement and my confirmation. The test request is open (T-ULFA). Rebased onto main b9cdc4326 with the change otherwise unchanged; whole suite on the new head a213814b3: 512 passed, 0 failed, 19 skipped. Ready for review.
mooki force-pushed ci/opt-in-hardware from 2d9962311e
Some checks failed
Linux AppImage / appimage (pull_request) Blocked by required conditions
Windows builder / Longshot Builder zip (pull_request) Blocked by required conditions
Windows builder / Windows Vulkan back end (llvm-mingw) (pull_request) Blocked by required conditions
Windows builder / dav1d for Windows (llvm-mingw) (pull_request) Blocked by required conditions
PR description / Description follows the template (pull_request_target) Successful in 2s
CI / Conventional Commit title (pull_request) Has been cancelled
CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Has been cancelled
CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Has been cancelled
CI / Python source checks (windows-2022, part 1/2) (pull_request) Has been cancelled
CI / Python source checks (windows-2022, part 2/2) (pull_request) Has been cancelled
CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Has been cancelled
CI / Workflow lint (actionlint) (pull_request) Has been cancelled
Third-party parts / As their notes say (pull_request) Has been cancelled
Third-party parts / The same as upstream (pull_request) Has been cancelled
to a213814b32
Some checks failed
PR description / Description follows the template (pull_request_target) Successful in 3s
Website / Website on Cloudflare (pull_request_target) Has been skipped
CI / Conventional Commit title (pull_request) Has been cancelled
CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Has been cancelled
CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Has been cancelled
CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Has been cancelled
CI / Workflow lint (actionlint) (pull_request) Has been cancelled
Third-party parts / As their notes say (pull_request) Has been cancelled
Third-party parts / The same as upstream (pull_request) Has been cancelled
2026-11-13 04:06:48 +00:00
Compare
mooki changed title from WIP: ci: add an owner-approved hardware fixture pilot to ci: add an owner-approved hardware fixture pilot 2026-11-13 04:06:48 +00:00
Author
Member

Carried onto main 5e072fc8c; new head 1b227ca31. This carry is mechanical: every commit replays unchanged, with no conflicts. The whole suite on this exact head: 543 passed, 3 failed, 16 skipped; its only failures are main's own tests/test_pad_layout.py, tests/test_module_ids.py and tests/test_framegen_resources.py.

Carried onto main `5e072fc8c`; new head `1b227ca31`. This carry is mechanical: every commit replays unchanged, with no conflicts. The whole suite on this exact head: 543 passed, 3 failed, 16 skipped; its only failures are main's own `tests/test_pad_layout.py`, `tests/test_module_ids.py` and `tests/test_framegen_resources.py`.
mooki force-pushed ci/opt-in-hardware from a213814b32
Some checks failed
PR description / Description follows the template (pull_request_target) Successful in 3s
Website / Website on Cloudflare (pull_request_target) Has been skipped
CI / Conventional Commit title (pull_request) Has been cancelled
CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Has been cancelled
CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Has been cancelled
CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Has been cancelled
CI / Workflow lint (actionlint) (pull_request) Has been cancelled
Third-party parts / As their notes say (pull_request) Has been cancelled
Third-party parts / The same as upstream (pull_request) Has been cancelled
to 1b227ca31c
Some checks failed
CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Has been skipped
CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Has been skipped
CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Has been skipped
Third-party parts / The same as upstream (pull_request) Has been skipped
Website / Website on Cloudflare (pull_request) Has been skipped
CI / Conventional Commit title (pull_request) Successful in 4s
PR description / Description follows the template (pull_request_target) Successful in 4s
CI / Workflow lint (actionlint) (pull_request) Failing after 5s
Third-party parts / As their notes say (pull_request) Successful in 6s
2026-11-14 13:05:21 +00:00
Compare
Author
Member

New head 268bdfec2 on main 5e072fc8c: one added commit, the earlier one unchanged. The workflow lint that main gained (actionlint 1.7.12) knows GitHub's runner labels and the ones listed in .forgejo/actionlint.yaml, so it rejected hardware-fixtures.yml line 34 for its runner label longshot-hardware-linux and this pull request's check was red. The new commit lists that label beside quick; no workflow changes. The same lint run locally on all workflows fails before the commit with that one message and passes after it. The whole suite on this exact head: 543 passed, 3 failed, 16 skipped; its only failures are main's own tests/test_pad_layout.py, tests/test_module_ids.py and tests/test_framegen_resources.py.

New head `268bdfec2` on main `5e072fc8c`: one added commit, the earlier one unchanged. The workflow lint that main gained (actionlint 1.7.12) knows GitHub's runner labels and the ones listed in `.forgejo/actionlint.yaml`, so it rejected `hardware-fixtures.yml` line 34 for its runner label `longshot-hardware-linux` and this pull request's check was red. The new commit lists that label beside `quick`; no workflow changes. The same lint run locally on all workflows fails before the commit with that one message and passes after it. The whole suite on this exact head: 543 passed, 3 failed, 16 skipped; its only failures are main's own `tests/test_pad_layout.py`, `tests/test_module_ids.py` and `tests/test_framegen_resources.py`.
ci: declare the hardware fixture runner's label for the workflow lint
All checks were successful
CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Has been skipped
CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Has been skipped
CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Has been skipped
Third-party parts / The same as upstream (pull_request) Has been skipped
CI / Conventional Commit title (pull_request) Successful in 3s
PR description / Description follows the template (pull_request_target) Successful in 4s
CI / Workflow lint (actionlint) (pull_request) Successful in 6s
Third-party parts / As their notes say (pull_request) Successful in 7s
268bdfec27
Main's workflow lint (actionlint 1.7.12, added in #146) knows GitHub's runner labels and the labels listed in .forgejo/actionlint.yaml. hardware-fixtures.yml runs on longshot-hardware-linux, a label an owner registers in a control repository of their own, so the lint rejected line 34 as an unknown label. List it beside quick. No workflow changes.

Assisted-by: Claude Fable 5
Co-Authored-By: Claude <[email protected]>
Author
Member

Closing this: the Test Farm does what this pilot set out to do. Volunteers lend their PCs through Beanstalk, a
maintainer approves each exact commit before anything runs, and runs are sandboxed on the volunteer's machine
(#542 documents it, and it is already testing pull requests such as
#838). A second, Forgejo-runner route to the same kind of test
would only split testers between two systems. #213 is the other
hardware pilot still open.

My offer stands in the farm's form: the Arch Linux PC with an RTX 4090 and an Intel UHD 770 can join Beanstalk as a
volunteer. The branch ci/opt-in-hardware stays on my fork at 268bdfec27; its isolation checks
(scripts/hardware_fixtures.py: wrong-device, missing-case and validation-layer failures are not passes) are there if
the farm wants them for the authored Vulkan fixtures.

Closing this: the Test Farm does what this pilot set out to do. Volunteers lend their PCs through Beanstalk, a maintainer approves each exact commit before anything runs, and runs are sandboxed on the volunteer's machine (https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/542 documents it, and it is already testing pull requests such as https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/838). A second, Forgejo-runner route to the same kind of test would only split testers between two systems. https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/213 is the other hardware pilot still open. My offer stands in the farm's form: the Arch Linux PC with an RTX 4090 and an Intel UHD 770 can join Beanstalk as a volunteer. The branch `ci/opt-in-hardware` stays on my fork at `268bdfec27`; its isolation checks (`scripts/hardware_fixtures.py`: wrong-device, missing-case and validation-layer failures are not passes) are there if the farm wants them for the authored Vulkan fixtures.
mooki closed this pull request 2026-11-15 18:42:24 +00:00
All checks were successful
CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Has been skipped
CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Has been skipped
CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Has been skipped
Third-party parts / The same as upstream (pull_request) Has been skipped
CI / Conventional Commit title (pull_request) Successful in 3s
PR description / Description follows the template (pull_request_target) Successful in 4s
CI / Workflow lint (actionlint) (pull_request) Successful in 6s
Third-party parts / As their notes say (pull_request) Successful in 7s

Pull request closed

Sign in to join this conversation.
No reviewers
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
MagicBeanDev/Longshot!100
No description provided.