ci: add an owner-approved hardware fixture pilot #100
No reviewers
Labels
No labels
area/accessibility
area/audio
area/input
area/install
area/menu
area/renderer
area/saves
area/tooling
area/upscaling
beanstalk: approved
beanstalk: failed
beanstalk: needs approval
beanstalk: running
beanstalk: tested
ci: approved
good first issue
gpu/amd
gpu/igpu
gpu/intel
gpu/nvidia
help wanted
kind
bug
kind
chore
kind
docs
kind
feature
kind
performance
kind
question
needs measurements
platform/linux
platform/macos
platform/steam-deck
platform/windows
preview
priority
critical
priority
high
priority
low
regression
risk
high
risk
low
risk
medium
status
blocked
status
confirmed
status
duplicate
status
invalid
status
needs-info
status
wontfix
ci: approved
No milestone
No project
No assignees
1 participant
Notifications
Due date
No due date set.
Dependencies
No dependencies set
Reference
MagicBeanDev/Longshot!100
Loading…
Reference in a new issue
No description provided.
Delete branch "mooki/Longshot:ci/opt-in-hardware"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Why
Volunteers need a small, opt-in way to run authored Vulkan checks on real GPUs without giving an ordinary pull request access to their machines. This adds a disabled-by-default, owner-controlled pilot, not an upstream hardware service; review of this code is separate from authorization to register or activate a runner.
I offer my Arch Linux PC with an RTX 4090 and an Intel UHD 770 as a second adapter, carrying the explicit
longshot-hardware-linuxlabel. Registration would happen only after a maintainer agrees and I confirm the proposed setup.What changed
.forgejo/workflows/hardware-fixtures.yml: manual dispatch only, only an owner'smain, explicitly disabled inMagicBeanDev/Longshot. The full approved SHA must match the dispatch and still be the remote head before execution. A repository variable enables the job; an explicit hardware label routes it. No PR trigger, arbitrary PR checkout, release secret or required merge check.scripts/hardware_fixtures.py: probe the selected GPU, then compile/run the existing five write-back cases and one copy-span case inside bounded, offline rootless Podman containers. The default--no-gpupath writesnot_runwithout probing hardware or starting a container. Missing devices, skips, wrong-device results, repeated/missing cases, missing validation activation and command/cleanup failures are not passes..git, untracked files, a home, a dump, saves, a display socket or a writable host directory. The payload receives no inherited token/proxy environment. Only fixed configuration rows leave it; raw command output and directory artifacts do not.tests/test_hardware_fixtures.py: GPU-free parsing, reporting, resource-command, cleanup, source-export and workflow-contract regressions. Authored command results exercise the orchestration without pretending to prove a real GPU or container.docs/forgejo-runners.md: put runner placement and security first; document the optional workflow, prerequisites, limits, safe reporting command and human fallback. Preserve main's Linux-only CI and Windows-on-Linux build instructions, its ordinary runner labels and itsRELEASE_UPLOAD_APIupload setting; cross-building Windows releases does not provision this GPU image..forgejo/actionlint.yaml: list the hardware runner's labellongshot-hardware-linuxbesidequick, so main's workflow lint acceptshardware-fixtures.yml. No workflow changes.Scope: this first slice tests an owner-reviewed default-branch snapshot, not an arbitrary PR head. Labels are routing, not isolation: another eligible workflow can select the same runner. Use only an owner's fork or dedicated control repository with trusted workflows and writers, never a personal PC in the shared upstream/organization runner pool. Game-based work, automatic request consumption and per-PR label approval remain decisions and separate work under #43.
The action and Vulkan-header pins match main.
tools.lockandscripts/tools.pysupply the Windows builder, not this Linux driver image, so they are unchanged. The owner must provision a reviewed image pinned by digest; the job never pulls an image, installs packages or accepts an SDK license. The aspect fixture is deliberately excluded until its explicit selector lands; the software-driver proposal in #67 remains separate.Flags and settings
No game setting, menu option, ini key or runtime default changes. These controls are documented in the runner guide and the command's help. Recommendations apply to this opt-in command, not to enabling hardware execution for everyone.
--run--no-gpu--commit--output--image--run--gpu-device--run--device-name--run--headers--run; workflow uses main's v1.4.345 pinHARDWARE_TESTS_ENABLEDtrueenablesHARDWARE_TEST_IMAGE--imageHARDWARE_TEST_DEVICE--gpu-deviceapproved_shadevice_nameVK_INSTANCE_LAYERSVK_LAYER_KHRONOS_validationin payloadVK_LOADER_DEBUGlayerin payloadVK_LAYER_SETTINGS_PATHMeasurements
Branch state: on main
5e072fc8c2adadba5ea610148be1c47800ba0cca; head268bdfec27b87a1fb86a38c9bf1de041b2dc16ab; whole game-free suite on this head:543 passed, 3 failed, 16 skipped(main5e072fc8c2adadba5ea610148be1c47800ba0cca:542 passed, 3 failed, 16 skipped). The 3 failing tests fail on main the same way:tests/test_framegen_resources.py,tests/test_module_ids.py,tests/test_pad_layout.py. The branch was carried onto this main by hand; what differs from a mechanical carry is named in the comment posted with this update.main:
5eaaf7fb9c(2026-11-13; source baseline when the paired tests were run)this PR: ci/opt-in-hardware @ 2d9962311e23b6762d1a6c511f9391749fd419b2 (paired-test head)
Rebased onto main
2a2da67f2bf909f7a49983e0499dbed1da150daeasc2a880c8331b3ee8dcadf07766e01bbd760c37a1. The three inputs to these named tests (scripts/hardware_fixtures.py,.forgejo/workflows/hardware-fixtures.yml,tests/test_hardware_fixtures.py) are byte-identical to the paired-test head, and the new main still lacks all three. The paired results below are retained at their actual tested commits, not presented as reruns. The runner-guide conflict was resolved around main's Linux-only CI and Windows-on-Linux build instructions; the new main'stests/test_windows_on_linux.pypassed all four cases on the rebased head.Hardware: the same Core i9-13900K PC, Arch Linux, Python 3.14.8. No GPU was probed.
Scene: no game; the published GPU-free regression's authored parser inputs and process-bound checks.
Method: one unmeasured warm-up on each source, then main / this PR / main / this PR. Each run uses an unchanged copy of this PR's
tests/test_hardware_fixtures.pyagainst the named source tree. Wall time includes Python startup and test discovery; medians are the middle of the two observations. The whole-suite receipts above remain at their original commits; this comparison ran only the named test.How to reproduce: in separate copies of the two source revisions, place the published PR's unchanged
tests/test_hardware_fixtures.pyundertests/, without copying its implementation into main. Runpython3 -I -B -m unittest discover -s tests -p test_hardware_fixtures.py, once per source to warm up, then twice each in the order above. Main's production source and workflow stay unchanged.5eaaf7fb9What main's failure means: main does not contain
scripts/hardware_fixtures.pyor the hardware workflow. The identical new regression cannot import that command, so unittest reports one failed import rather than executing its 27 cases. Those cases were not run on main; they are not 27 failures or 27 passes. This is evidence of missing tooling on main and passing authored coverage on the PR, not a main GPU defect or a proof of real container isolation.The captured outcomes were:
These times are not a speed comparison: main stops at import while the PR executes all 27 cases, including an intentional 0.1-second timeout case, under shared machine load. The claimed measurement is command availability and game-free regression outcomes. No game-speed effect is claimed.
The earlier committed
--no-gpuinvocation also returnedhardware fixtures: not_run (no GPU access)and wrote the eight-column configuration row with the full head andnot probed; it did not run hardware. YAML parsing passed. Unmodified actionlint 1.7.12 reported three unsupported full-origin Forgejo action URLs; its validation-only copy, with only those origins removed and the custom label declared, passed the remaining compatible checks. That limitation and the real Forgejo/GPU trial remain explicit below.Testing
Tested: on Linux, the identical published regression in two interleaved runs against each named source revision, after one warm-up each. The PR passes all 27 cases in both measured runs; main reports one import error each because the new command is absent, so its behavior cases do not run. The cases cover authored GPU/driver records, wrong/missing/repeated results, validation activation, bounded output and timeout, cleanup refusal, non-overwrite behavior, exact source export and no-GPU orchestration. The earlier committed no-GPU command and its configuration row were inspected; YAML parsing and the limited actionlint check are described above. These are script/workflow correctness checks, not a hardware or speed result. After the documentation-only conflict resolution onto
2a2da67f2b, the three named-test inputs were checked byte-for-byte unchanged; their paired results are carried at the original commits, not rerun. The rebased head also passes all fourtests/test_windows_on_linux.pycases, including the check that no workflow requests a Windows runner.The earlier whole source suite at
2d9962311e23b6762d1a6c511f9391749fd419b2passed: 510 passed, 0 failed, 19 declared skips. Its original base77f605a25f175fbda9ad131ac6be96bbde85d9c2had 509 passed, 0 failed, 19 skips; no new whole-suite result is claimed for the current-main source comparison.Not tested: no runner was installed, registered or started; no real Forgejo dispatch/checkout/upload or changed-head refusal was exercised. Podman is unavailable on this PC, so no image, resource-limit enforcement, confinement/escape resistance, GPU pass-through or real cleanup/cancellation was tested. No hardware GPU, Windows execution, AMD/Deck run, game build/run, swapchain or performance measurement. Mock command results prove controller logic only. The first real isolation and publication trial is still required before activation.
Needs other testers:
Screenshots
Nothing on screen. No window or game screenshot was produced.
Risk and rollback
A hardware label or a manual workflow is not a security boundary. Another eligible workflow can bypass this command, and a rootless container still shares the host kernel and GPU driver. Keep the controller in a trusted owner-controlled repository and separate environment without personal/game files or release credentials; never register this PC upstream merely because this workflow is gated. A pinned image, CDI mapping and runner configuration still need review. Driver hangs/escapes and forged results remain possible.
Missing tools, unsupported driver features or changed validation output deliberately fail. An offline, skipped, refused or timed-out job is not a pass and must fall back to human testing. Disable
HARDWARE_TESTS_ENABLEDand stop the runner to withdraw the machine; reverting these four files removes the pilot without touching game settings or saves.Reviewer notes
Start with
container_command(),export_source(),capture()and the fixed-format parser/report functions. The workflow/controller accepts only the owner's reviewedmain; it is not a secure arbitrary-PR executor. Decide runner ownership/placement, the label and image policy, who approves exact revisions and who reviews publication before activating anything. No payload signing/publishing secret is required; Forgejo's job token remains in trusted controller/action processes, not the containers.Keep #43 open. Automatic queue consumption, revision-bound PR labels, a separate PR publisher, other OS/device qualification and any dump-using workload are outside this first slice. #69's owner-controlled rolling monitor is not a Forgejo runner and is unchanged.
Issues
See #43. Related: #67 for software Vulkan fixtures and #69 for the separate local rolling record.
Checklist
For everyone
type(scope): description).main.python -I -B tests/run_all.pyrun and passing (any failure that also fails onmainis named above).third_party/untouched (a new folder with aPROVENANCE.mdinstead).If applicable
Assisted-by:/Co-Authored-By:trailers.tests/that reads changed source updated in this PR, not weakened. New workflow-contract checks; existing fixture checks unchanged.New on this forge; not opened on GitHub.
Assisted-by: GPT-6 Astra
Co-Authored-By: GPT-6 Astra [email protected]
Rebased onto main
2a2da67f2b, preserving Windows-on-Linux release instructions and the hardware security sections; the two interleaved named-test pairs retain their original commits with identical input hashes, and the new workflow check passes all four cases. The real GPU, isolation and Forgejo trial remains in one test request, and registration or activation still needs maintainer agreement and my confirmation. The test request is open (T-ULFA). Rebased onto mainb9cdc4326with the change otherwise unchanged; whole suite on the new heada213814b3: 512 passed, 0 failed, 19 skipped. Ready for review.2d9962311ea213814b32WIP: ci: add an owner-approved hardware fixture pilotto ci: add an owner-approved hardware fixture pilotCarried onto main
5e072fc8c; new head1b227ca31. This carry is mechanical: every commit replays unchanged, with no conflicts. The whole suite on this exact head: 543 passed, 3 failed, 16 skipped; its only failures are main's owntests/test_pad_layout.py,tests/test_module_ids.pyandtests/test_framegen_resources.py.a213814b321b227ca31cNew head
268bdfec2on main5e072fc8c: one added commit, the earlier one unchanged. The workflow lint that main gained (actionlint 1.7.12) knows GitHub's runner labels and the ones listed in.forgejo/actionlint.yaml, so it rejectedhardware-fixtures.ymlline 34 for its runner labellongshot-hardware-linuxand this pull request's check was red. The new commit lists that label besidequick; no workflow changes. The same lint run locally on all workflows fails before the commit with that one message and passes after it. The whole suite on this exact head: 543 passed, 3 failed, 16 skipped; its only failures are main's owntests/test_pad_layout.py,tests/test_module_ids.pyandtests/test_framegen_resources.py.Closing this: the Test Farm does what this pilot set out to do. Volunteers lend their PCs through Beanstalk, a
maintainer approves each exact commit before anything runs, and runs are sandboxed on the volunteer's machine
(#542 documents it, and it is already testing pull requests such as
#838). A second, Forgejo-runner route to the same kind of test
would only split testers between two systems. #213 is the other
hardware pilot still open.
My offer stands in the farm's form: the Arch Linux PC with an RTX 4090 and an Intel UHD 770 can join Beanstalk as a
volunteer. The branch
ci/opt-in-hardwarestays on my fork at268bdfec27; its isolation checks(
scripts/hardware_fixtures.py: wrong-device, missing-case and validation-layer failures are not passes) are there ifthe farm wants them for the authored Vulkan fixtures.
Pull request closed