WIP: ci: run authored Vulkan fixtures locally and prepare Forgejo execution #67

Draft

mooki wants to merge 6 commits from mooki/Longshot:ci/vulkan-fixtures-lavapipe into main

pull from: mooki/Longshot:ci/vulkan-fixtures-lavapipe

merge into: MagicBeanDev:main

MagicBeanDev:main

Conversation 1 Commits 6 Files changed 9 +470 -11

mooki commented 2026-11-13 01:24:41 +00:00

First-time contributor

• Copy link
• Copy Markdown

Why

Contributors can execute the existing authored Vulkan fixtures without a game dump, but the ordinary source suite leaves their device paths opt-in. This adds one strict local command and prepares a small native Forgejo workflow. Open as work in progress: the local fixtures work, but no Forgejo runner execution is proven; the earlier suite's compiler termination and its successful isolated follow-up remain recorded below.

What changed

• Keep the original explicit aspect-device selector and the strict runner for five write-back, one copy-span and seven offscreen aspect/crop cases. Every case, the selected device and validation-layer activation must be present; skips, missing/repeated cases, unavailable devices and validation errors fail.
• Move this topic's job out of the legacy .github/workflows/source-regressions.yml into .forgejo/workflows/vulkan-fixtures.yml, the documented Forgejo workflow directory. The legacy file is otherwise exactly pinned main's file; this does not migrate all CI.
• Use documented push, pull_request and manual triggers, Forgejo concurrency contexts, a full-origin commit-pinned checkout action, a five-minute job bound and the same software-ICD/package steps. The test contract follows that move without dropping any fixture/validation assertion.
• Do not treat GitHub's permissions key as a security guarantee: Forgejo does not enforce it. This job has no configured secrets/uploads, does not persist checkout credentials and removes the forge tokens from its fixture command. Disposable-runner isolation is still required.
• Document the local command and runner prerequisites; add the actual local checks to the results table. No game/renderer production code changes.

What the forge gets today: a usable local checker and a proposed workflow, not yet run on a forge runner. During the recorded checks, public repository metadata said has_actions: false; the tasks endpoint returned HTTP 404. That is not an empty task list and cannot establish that no runner has ever completed a run. No completed Forgejo task could be verified. The release/CI migration has since merged; this proposed workflow has not yet run on a forge runner.

The label ubuntu-24.04 does not buy a GitHub-hosted machine on Forgejo. This job requires Actions and an isolated runner with that label and an Ubuntu 24.04 environment containing Bash, Git, Python 3, Node.js 24 for the pinned checkout action, passwordless sudo and apt/network access. This PR does none of that. The wider release migration and this standalone job have not been tested together.

With the compiler, Vulkan loader/headers, vulkaninfo, lavapipe and Khronos validation installed, the fixtures run locally in one command. Set LVP_ICD to the single installed lavapipe manifest and VULKAN_HEADERS to its headers directory; the contribution guide also gives the Ubuntu example:

VK_DRIVER_FILES="$LVP_ICD" VK_ICD_FILENAMES="$LVP_ICD" \
  python3 -I -B tests/run_vulkan_fixtures.py --cxx g++ --headers "$VULKAN_HEADERS" --device llvmpipe

Flags and settings

No game environment switch, menu option, ini key or build default changes. These are opt-in test-driver controls and existing Vulkan-loader variables used by the new runner.

Flag	What it does	Default now	Recommendation	Helps on	Doesn't help on	Untested on
Aspect fixture --device	New: Select a case-insensitive device-name substring; refuse no match	Omitted: existing first-discrete-else-first selection	Off by default: explicit test selection	Targeted Vulkan fixture runs	Normal game play	Windows Vulkan execution (should help on: Other Vulkan drivers)
Runner --device	New: Require llvmpipe in every device result	Required, only llvmpipe accepted	On by default: for this software-only runner	Reproducible CPU Vulkan checks	Hardware GPU coverage	Forgejo runner (should help on: Configured Ubuntu runner)
Runner --headers	New: Select installed Vulkan headers	Required	Off by default: implicit search	Local/CI compilation	PCs without headers	Forgejo runner (should help on: Other header installations)
Runner --cxx	New: Select the fixture C++ compiler	g++	On by default: existing fixture convention	Linux fixture compilation	Game performance	Windows wrapper execution (should help on: Other supported compilers)
Source regression --source-root	New: Check a different tree's selector/workflow	Own checkout	On by default: own tree; override for negative checks	Historical/main controls	Device execution	Windows alternate-root execution (should help on: Windows source checks)
Source regression --cxx	New: Compile the real selector with authored device properties	g++; suite supplies its compiler	On by default: suite-selected compiler	Game-free selector coverage	Actual Vulkan dispatch	Fresh Windows execution (should help on: Windows source checks)
VK_DRIVER_FILES	Existing loader variable, required here: Pin exactly one lavapipe manifest	No default; must match the next variable	On by default: for this runner only	Prevent unintended hardware selection	Hardware testing	Forgejo runner (should help on: Configured software lane)
VK_ICD_FILENAMES	Existing loader variable, required here: Pin the same single lavapipe manifest	No default	On by default: for this runner only	Older/newer Vulkan loaders	Hardware testing	Forgejo runner (should help on: Configured software lane)
VK_INSTANCE_LAYERS	Existing loader variable, forced internally: Enable Khronos validation	VK_LAYER_KHRONOS_validation in child processes	On by default: for these checks	API-error detection	Performance measurements	Forgejo runner (should help on: Configured validation lane)
VK_LAYER_SETTINGS_PATH	Existing layer variable, set internally: Use private error-to-stdout settings	Temporary directory per runner call	On by default: for these checks	Error capture in fixtures without callbacks	Warning-count measurements	Forgejo runner (should help on: Configured validation lane)
VK_LOADER_DEBUG	Existing loader variable, forced internally: Emit actual layer insertion evidence	layer	On by default: for these checks	Proving validation activation	Game performance	Forgejo runner (should help on: Configured validation lane)

Measurements

Branch state: on main d725b3631982ed0035ea3ac9cbe1677e9fad260c; head 3969145a3fd5eabd624cab3a40b2f3c51e5b1e54; whole game-free suite on this head: 511 passed, 1 failed, 20 skipped (main d725b3631982ed0035ea3ac9cbe1677e9fad260c: 510 passed, 1 failed, 20 skipped); the commits are unchanged by rebases apart from the results table.

The only failure on both heads is tests/test_updater_asset_digest.py: local .NET NU1100, unable to resolve the missing Microsoft.AspNetCore.App.Ref (= 10.0.13) reference pack for net10.0.

main: d4a69456c9 (2026-11-12)
this PR: ci/vulkan-fixtures-lavapipe @ 668e1913508f3896c7797843150eb5863da9450e (measured fixture content)

Main d725b363 differs from measured d4a69456 only by scripts, docs and tests, all six topic patches are unchanged, and the fixture measurements above remain valid at their original commits.
Hardware: Core i9-13900K, 94 GiB RAM, Arch Linux kernel 7.2.15-1-omarchy; CPU-only llvmpipe, Mesa 26.2.5-arch1.1 / LLVM 23.1.3, Vulkan 1.4.354; Khronos validation 1.4.363, Vulkan headers 1.4.345. Device fixtures: GCC 16.2.1. Source suite: default Clang 23.1.3, Python 3.12.16.
Scene: authored buffers, four texels and synthetic offscreen images; no dump, save, window, swapchain or game.
Method: unmeasured warm-up, then main/branch/main/branch on the same PC and sole software ICD, fresh HOME each command, ordinary shared conditions. Main uses its three original fixture commands; the branch uses the new one-command runner. Main's aspect fixture lacks a selector, but the sole pinned ICD exposes only llvmpipe. Counts and medians below are correctness evidence, not speed measurements. Separately run the new source regression twice each with --source-root selecting main/branch/main/branch. The whole source suite uses --jobs 6 --strict, unsharded. The base suite was run once for all moved pull requests of this day, on this commit; the final-head suite ran here once.
How to reproduce: with only lavapipe exposed in both ICD variables and Khronos validation installed, run the local command above on the branch; on main run test_wb_diff_observations_gpu.py --run --device llvmpipe, test_vulkan_copy_span.py --gpu --gpu-device llvmpipe and test_present_aspect_vulkan.py in turn, supplying the same compiler and header arguments. Repeat main/branch/main/branch. For the source negative control use tests/test_vulkan_fixtures_ci.py --source-root with the pinned main checkout; run the whole suite with python -I -B tests/run_all.py --jobs 6 --strict.

	main @ d4a69456	this PR @ 668e1913
Authored Vulkan cases passed, run 1 / run 2; median	13 / 13; 13	13 / 13; 13
Write-back / copy-span / aspect cases in each arm	5 / 1 / 7	5 / 1 / 7
Reported core-validation errors, run 1 / run 2; median	0 / 0; 0	0 / 0; 0
New source-regression errors, run 1 / run 2; median	2 / 2; 2	0 / 0; 0

The whole suite in the branch-state block ran after the clean rebase, including the final documentation rows. The shared d725b363 base suite's printed duration was 234 s. The depth fixture that lost a compiler in the earlier a283e04a run passes in this complete rebased-head suite too. The previous failure and its unknown cause remain in the historical receipts below; the remaining reason for work-in-progress status is the unrun Forgejo workflow.

The source negative control reports missing tooling, not a failing GPU: main lacks the explicit aspect selector and the new Forgejo workflow. Its other nine output/validation guard tests run against the new runner; they are not baseline GPU coverage. Actual excerpts:

ERROR: test_aspect_selector_never_falls_back (__main__.Selection.test_aspect_selector_never_falls_back)
ValueError: substring not found
ERROR: test_hosted_job_rules (__main__.Workflow.test_hosted_job_rules)
Ran 11 tests in 0.004s
FAILED (errors=2)

Ran 11 tests in 0.242s
OK

Both interleaved Vulkan pairs contain these actual result lines (the first seven lines are the five write-back cases and their completion/device identity):

Device: llvmpipe (LLVM 23.1.3, 256 bits)
PASS: off, guest/shadow parity and feedback guards
PASS: mixed chunks, guest/shadow parity and feedback guards
PASS: reused clean result, guest/shadow parity and feedback guards
PASS: zero count, guest/shadow parity and feedback guards
PASS: 65535-wide 2D grid, guest/shadow parity and feedback guards
PASS: authored GPU write-back observations
Khronos validation: enabled
PASS: 24-byte production copy layout, four GPU texels correct and padding untouched
PASS physical16:9: 1600000 GPU readback pixels
PASS wide matching: 4953600 GPU readback pixels
PASS maximize geometry: 1937280 GPU readback pixels
PASS wide into16:9: 921600 GPU readback pixels
PASS restore geometry: 1238400 GPU readback pixels
PASS padded active crop: 887164 GPU readback pixels
PASS invalid override: 1600000 GPU readback pixels
PASS: seven offscreen GPU cases; no swapchain/window, no game assets

Every fixture's own output also contains both loader records Insert instance layer "VK_LAYER_KHRONOS_validation" and Inserted device layer "VK_LAYER_KHRONOS_validation"; merely enumerating the layer is not counted as activation. Branch summaries:

3 passed, 0 failed, 0 skipped in 2.35 s (lavapipe, validation on)
3 passed, 0 failed, 0 skipped in 1.91 s (lavapipe, validation on)

The printed durations include compilation under shared load, not a performance comparison. The bounded run's kernel interval was readable with no NVIDIA fault signatures; no coredump was recorded. Ordinary core validation is covered; warning counts, extra synchronization validation and GPU-assisted validation are not claimed.

Whole-suite failures are retained, not called green:

shared main:
507 passed, 2 failed, 20 skipped in 504 s (linux, 6 at once)
initial branch at 668e1913:
509 passed, 1 failed, 20 skipped in 258 s (linux, 6 at once)
exact final head a283e04a:
508 passed, 2 failed, 20 skipped in 209 s (linux, 6 at once)

All have the same local prerequisite failure (temporary path omitted):

error NU1100: Unable to resolve 'Microsoft.AspNetCore.App.Ref (= 10.0.13)' for 'net10.0'.

The shared main run also had two timing-sensitive test_appimage_package_retry.py assertions under heavy load; its separate isolated follow-up passed all nine cases. Both branch suites pass that file. The final branch run instead lost a compiler in nx2/test_depth_sample_types.py: clang++ compiling emit_spirv_barriers.cpp “died with <Signals.SIGKILL: 9>”. Its complete nx2/gpu inputs and depth fixture are identical to main, and the earlier branch suite passed it. One requested isolated follow-up on the unchanged final head passed:

pass       4.3s  nx2/test_depth_sample_types.py
1 passed, 0 failed, 0 skipped in 4 s (linux, 1 at once)

The kernel log covering the failed compile's minute is readable but has no out-of-memory kill or memory-pressure explanation. Cause unknown: the isolated pass does not prove a memory kill or erase the full-suite failure. It is not labelled an inherited baseline failure. No further rerun was made; the PR stays a draft.

Windows-target syntax: three translation units (selector, write-back and aspect) compile with warnings as errors; not run on Windows.

Historical GitHub evidence, not new Forgejo evidence: the source workflow on original PR head 6ea3f6a1f9b4a793259f2a5f89dc50cd9f1e78e2 reports both Ubuntu parts, both Windows parts, Vulkan and actionlint successful. Across that head the public API lists 12 successful checks and two release-only skips. The original Hookshot report also records the earlier validated fork job completing in 23 seconds including setup; that duration and its Mesa 25.2.8 runner were not reproduced on Forgejo. Earlier NVIDIA/Intel fixture trials remain labelled Hookshot and were not repeated here. The new runner itself is byte-identical to the green GitHub head; the C++ aspect fixture has only the application-name rename. Workflow placement/contracts and documentation are the named migration adaptations, not a claim of unchanged hosted behavior.

Testing

Tested: On this PC: Arch Linux, RTX 4090, NVIDIA 615.78.08, Core i9-13900K. The RTX 4090 was not selected: device checks used CPU-only llvmpipe, Mesa 26.2.5-arch1.1 / LLVM 23.1.3 with Khronos validation. Two interleaved main/branch pairs at the measured commits above passed all 13 authored write-back/copy-span/aspect cases in each arm with validation active. The source regression checked main/branch/main/branch; main lacked the selector/workflow, while the branch passed all 11 tests. The whole suite ran on the current head after the final documentation commit. Three Windows-target fixture units compiled; no Windows executable was run. Historical GitHub checks remain historical, not Forgejo execution.

Tested but not measured: none beyond the explicitly recorded correctness checks; no speed measurement is claimed from their printed durations.

Not tested: Not yet run on a forge runner; the requested workflow trial is below, and the earlier public tasks-endpoint response cannot establish historical runner completions. No new test request was posted by this migration.

• The earlier a283e04a suite's depth-fixture compiler SIGKILL remains unexplained despite its one successful isolated follow-up; that failure is retained separately from the rebased-head suite.
• No fresh Windows execution of the selector, write-back or aspect C++ fixtures: only cross-compilation and old GitHub execution are available, not a new Windows run.
• No game build/run, full renderer startup/replay, hardware GPU, AMD/Deck, swapchain, HDR, DLSS, frame generation or hardware-performance claim. This is a game-free software-driver lane; software Vulkan is not a substitute for those.
• No combined integration trial with the wider Forgejo release/CI migration. A matching runner and a completed trial must be supplied before calling this ready.

Needs other testers:

• Configured Ubuntu 24.04 Forgejo runner: run the proposed workflow, expecting checkout and package setup to complete and all 13 fixture cases to pass with validation active (request to open)

Screenshots

Nothing on screen. These are offscreen pixel-readback assertions; no window or game screenshot was produced.

Risk and rollback

Mesa, the validation layer or loader messages can change and make this strict lane fail; runner images may lack Node.js 24 or the required packages. A failure must stay a failure, not become a skip. Disable/remove this standalone workflow or revert this PR to roll back; the ordinary source suite remains GPU-independent, and no game settings or saves need migration.

Reviewer notes

Start with check_fixture() in tests/run_vulkan_fixtures.py: every case and both layer-insertion markers must be present in each fixture's own output. The aspect selector keeps the old preference when no filter is passed and refuses an unmatched filter. The outstanding work is an operator-provided, isolated Forgejo runner trial, not a missing local implementation; the wider release migration remains separate.

Issues

See #43 — the forge's hardware-test-runner proposal; this supplies only its game-free software-driver part, not a hardware service. Original report: https://github.com/NotRexed/Hookshot/issues/2797. Original PR: https://github.com/NotRexed/Hookshot/issues/2866. The corresponding proposal now exists at #43; it is not closed by this PR.

Checklist

For everyone

• One topic.
• The title is a Conventional Commit (type(scope): description).
• Rebased on current main.
• python -I -B tests/run_all.py run and passing (any failure that also fails on main is named above). — The earlier compiler kill is retained as history, not this run's result.
• Tested in the game, or what wasn't is named under Testing. — No game/runtime change; all unrun paths are explicit.
• Testing filled in: what was tested, what wasn't, and what needs other testers (with test requests).
• Measured: main and this PR at named commits, on the same PC, scene and settings, at least 2 interleaved runs each, the median reported.
• No game files, dumps, build output or generated files.
• third_party/ untouched (a new folder with a PROVENANCE.md instead).
• The license is OK: GPL-3.0-or-later, nothing incompatible and nothing decompiled.

If applicable

• AI assistance recorded with Assisted-by: / Co-Authored-By: trailers.
• A row in docs/tested-configurations.md for each measured run.
• A check under tests/ that reads changed source updated in this PR, not weakened.
• Docs updated.

---

Moved from NotRexed/Hookshot#2866.

Assisted-by: GPT-6 Astra
Co-Authored-By: GPT-6 Astra [email protected]

## Why Contributors can execute the existing authored Vulkan fixtures without a game dump, but the ordinary source suite leaves their device paths opt-in. This adds one strict local command and prepares a small native Forgejo workflow. **Open as work in progress:** the local fixtures work, but no Forgejo runner execution is proven; the earlier suite's compiler termination and its successful isolated follow-up remain recorded below. ## What changed - Keep the original explicit aspect-device selector and the strict runner for five write-back, one copy-span and seven offscreen aspect/crop cases. Every case, the selected device and validation-layer activation must be present; skips, missing/repeated cases, unavailable devices and validation errors fail. - Move this topic's job out of the legacy `.github/workflows/source-regressions.yml` into **`.forgejo/workflows/vulkan-fixtures.yml`**, the [documented Forgejo workflow directory](https://forgejo.org/docs/v17.0/user/actions/basic-concepts/). The legacy file is otherwise exactly pinned main's file; this does not migrate all CI. - Use documented `push`, `pull_request` and manual triggers, Forgejo concurrency contexts, a full-origin commit-pinned checkout action, a five-minute job bound and the same software-ICD/package steps. The test contract follows that move without dropping any fixture/validation assertion. - Do not treat GitHub's `permissions` key as a security guarantee: [Forgejo does not enforce it](https://forgejo.org/docs/v17.0/user/actions/github-actions/). This job has no configured secrets/uploads, does not persist checkout credentials and removes the forge tokens from its fixture command. Disposable-runner isolation is still required. - Document the local command and runner prerequisites; add the actual local checks to the results table. No game/renderer production code changes. **What the forge gets today:** a usable local checker and a proposed workflow, not yet run on a forge runner. During the recorded checks, public repository metadata said `has_actions: false`; the [tasks endpoint](https://git.magicbean.dev/api/v1/repos/MagicBeanDev/Longshot/actions/tasks) returned **HTTP 404**. That is not an empty task list and cannot establish that no runner has *ever* completed a run. No completed Forgejo task could be verified. The [release/CI migration](https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/13) has since merged; this proposed workflow has not yet run on a forge runner. The label **`ubuntu-24.04`** does not buy a GitHub-hosted machine on Forgejo. This job requires Actions and an isolated runner with that label and an Ubuntu 24.04 environment containing Bash, Git, Python 3, Node.js 24 for the pinned checkout action, passwordless sudo and apt/network access. This PR does none of that. The wider release migration and this standalone job have not been tested together. With the compiler, Vulkan loader/headers, `vulkaninfo`, lavapipe and Khronos validation installed, the fixtures run locally in one command. Set `LVP_ICD` to the single installed lavapipe manifest and `VULKAN_HEADERS` to its headers directory; the contribution guide also gives the Ubuntu example: ```sh VK_DRIVER_FILES="$LVP_ICD" VK_ICD_FILENAMES="$LVP_ICD" \ python3 -I -B tests/run_vulkan_fixtures.py --cxx g++ --headers "$VULKAN_HEADERS" --device llvmpipe ``` ## Flags and settings No game environment switch, menu option, ini key or build default changes. These are opt-in test-driver controls and existing Vulkan-loader variables used by the new runner. | Flag | What it does | Default now | Recommendation | Helps on | Doesn't help on | Untested on | |---|---|---|---|---|---|---| | Aspect fixture `--device` | New: Select a case-insensitive device-name substring; refuse no match | Omitted: existing first-discrete-else-first selection | Off by default: explicit test selection | Targeted Vulkan fixture runs | Normal game play | Windows Vulkan execution (should help on: Other Vulkan drivers) | | Runner `--device` | New: Require `llvmpipe` in every device result | Required, only `llvmpipe` accepted | On by default: for this software-only runner | Reproducible CPU Vulkan checks | Hardware GPU coverage | Forgejo runner (should help on: Configured Ubuntu runner) | | Runner `--headers` | New: Select installed Vulkan headers | Required | Off by default: implicit search | Local/CI compilation | PCs without headers | Forgejo runner (should help on: Other header installations) | | Runner `--cxx` | New: Select the fixture C++ compiler | `g++` | On by default: existing fixture convention | Linux fixture compilation | Game performance | Windows wrapper execution (should help on: Other supported compilers) | | Source regression `--source-root` | New: Check a different tree's selector/workflow | Own checkout | On by default: own tree; override for negative checks | Historical/main controls | Device execution | Windows alternate-root execution (should help on: Windows source checks) | | Source regression `--cxx` | New: Compile the real selector with authored device properties | `g++`; suite supplies its compiler | On by default: suite-selected compiler | Game-free selector coverage | Actual Vulkan dispatch | Fresh Windows execution (should help on: Windows source checks) | | `VK_DRIVER_FILES` | Existing loader variable, required here: Pin exactly one lavapipe manifest | No default; must match the next variable | On by default: for this runner only | Prevent unintended hardware selection | Hardware testing | Forgejo runner (should help on: Configured software lane) | | `VK_ICD_FILENAMES` | Existing loader variable, required here: Pin the same single lavapipe manifest | No default | On by default: for this runner only | Older/newer Vulkan loaders | Hardware testing | Forgejo runner (should help on: Configured software lane) | | `VK_INSTANCE_LAYERS` | Existing loader variable, forced internally: Enable Khronos validation | `VK_LAYER_KHRONOS_validation` in child processes | On by default: for these checks | API-error detection | Performance measurements | Forgejo runner (should help on: Configured validation lane) | | `VK_LAYER_SETTINGS_PATH` | Existing layer variable, set internally: Use private error-to-stdout settings | Temporary directory per runner call | On by default: for these checks | Error capture in fixtures without callbacks | Warning-count measurements | Forgejo runner (should help on: Configured validation lane) | | `VK_LOADER_DEBUG` | Existing loader variable, forced internally: Emit actual layer insertion evidence | `layer` | On by default: for these checks | Proving validation activation | Game performance | Forgejo runner (should help on: Configured validation lane) | ## Measurements <!-- branch-state --> Branch state: on main `d725b3631982ed0035ea3ac9cbe1677e9fad260c`; head `3969145a3fd5eabd624cab3a40b2f3c51e5b1e54`; whole game-free suite on this head: `511 passed, 1 failed, 20 skipped` (main `d725b3631982ed0035ea3ac9cbe1677e9fad260c`: `510 passed, 1 failed, 20 skipped`); the commits are unchanged by rebases apart from the results table. <!-- /branch-state --> The only failure on both heads is `tests/test_updater_asset_digest.py`: local .NET `NU1100`, unable to resolve the missing `Microsoft.AspNetCore.App.Ref (= 10.0.13)` reference pack for net10.0. main: d4a69456c9de5131589af1fcf9a09adba4c403d1 (2026-11-12) this PR: ci/vulkan-fixtures-lavapipe @ 668e1913508f3896c7797843150eb5863da9450e (measured fixture content) Main `d725b363` differs from measured `d4a69456` only by scripts, docs and tests, all six topic patches are unchanged, and the fixture measurements above remain valid at their original commits. Hardware: Core i9-13900K, 94 GiB RAM, Arch Linux kernel 7.2.15-1-omarchy; **CPU-only llvmpipe**, Mesa 26.2.5-arch1.1 / LLVM 23.1.3, Vulkan 1.4.354; Khronos validation 1.4.363, Vulkan headers 1.4.345. Device fixtures: GCC 16.2.1. Source suite: default Clang 23.1.3, Python 3.12.16. Scene: authored buffers, four texels and synthetic offscreen images; no dump, save, window, swapchain or game. Method: unmeasured warm-up, then main/branch/main/branch on the same PC and sole software ICD, fresh HOME each command, ordinary shared conditions. Main uses its three original fixture commands; the branch uses the new one-command runner. Main's aspect fixture lacks a selector, but the sole pinned ICD exposes only llvmpipe. Counts and medians below are correctness evidence, **not speed measurements**. Separately run the new source regression twice each with `--source-root` selecting main/branch/main/branch. The whole source suite uses `--jobs 6 --strict`, unsharded. **The base suite was run once for all moved pull requests of this day, on this commit**; the final-head suite ran here once. How to reproduce: with only lavapipe exposed in both ICD variables and Khronos validation installed, run the local command above on the branch; on main run `test_wb_diff_observations_gpu.py --run --device llvmpipe`, `test_vulkan_copy_span.py --gpu --gpu-device llvmpipe` and `test_present_aspect_vulkan.py` in turn, supplying the same compiler and header arguments. Repeat main/branch/main/branch. For the source negative control use `tests/test_vulkan_fixtures_ci.py --source-root` with the pinned main checkout; run the whole suite with `python -I -B tests/run_all.py --jobs 6 --strict`. | | main @ d4a69456 | this PR @ 668e1913 | |---|---|---| | Authored Vulkan cases passed, run 1 / run 2; median | 13 / 13; 13 | 13 / 13; 13 | | Write-back / copy-span / aspect cases in each arm | 5 / 1 / 7 | 5 / 1 / 7 | | Reported core-validation errors, run 1 / run 2; median | 0 / 0; 0 | 0 / 0; 0 | | New source-regression errors, run 1 / run 2; median | 2 / 2; 2 | 0 / 0; 0 | The whole suite in the branch-state block ran after the clean rebase, including the final documentation rows. The shared `d725b363` base suite's printed duration was 234 s. The depth fixture that lost a compiler in the earlier a283e04a run passes in this complete rebased-head suite too. The previous failure and its unknown cause remain in the historical receipts below; the remaining reason for work-in-progress status is the unrun Forgejo workflow. The source negative control reports **missing tooling**, not a failing GPU: main lacks the explicit aspect selector and the new Forgejo workflow. Its other nine output/validation guard tests run against the new runner; they are not baseline GPU coverage. Actual excerpts: ```text ERROR: test_aspect_selector_never_falls_back (__main__.Selection.test_aspect_selector_never_falls_back) ValueError: substring not found ERROR: test_hosted_job_rules (__main__.Workflow.test_hosted_job_rules) Ran 11 tests in 0.004s FAILED (errors=2) Ran 11 tests in 0.242s OK ``` Both interleaved Vulkan pairs contain these actual result lines (the first seven lines are the five write-back cases and their completion/device identity): ```text Device: llvmpipe (LLVM 23.1.3, 256 bits) PASS: off, guest/shadow parity and feedback guards PASS: mixed chunks, guest/shadow parity and feedback guards PASS: reused clean result, guest/shadow parity and feedback guards PASS: zero count, guest/shadow parity and feedback guards PASS: 65535-wide 2D grid, guest/shadow parity and feedback guards PASS: authored GPU write-back observations Khronos validation: enabled PASS: 24-byte production copy layout, four GPU texels correct and padding untouched PASS physical16:9: 1600000 GPU readback pixels PASS wide matching: 4953600 GPU readback pixels PASS maximize geometry: 1937280 GPU readback pixels PASS wide into16:9: 921600 GPU readback pixels PASS restore geometry: 1238400 GPU readback pixels PASS padded active crop: 887164 GPU readback pixels PASS invalid override: 1600000 GPU readback pixels PASS: seven offscreen GPU cases; no swapchain/window, no game assets ``` Every fixture's own output also contains both loader records `Insert instance layer "VK_LAYER_KHRONOS_validation"` and `Inserted device layer "VK_LAYER_KHRONOS_validation"`; merely enumerating the layer is not counted as activation. Branch summaries: ```text 3 passed, 0 failed, 0 skipped in 2.35 s (lavapipe, validation on) 3 passed, 0 failed, 0 skipped in 1.91 s (lavapipe, validation on) ``` The printed durations include compilation under shared load, not a performance comparison. The bounded run's kernel interval was readable with no NVIDIA fault signatures; no coredump was recorded. Ordinary core validation is covered; warning counts, extra synchronization validation and GPU-assisted validation are not claimed. **Whole-suite failures are retained, not called green:** ```text shared main: 507 passed, 2 failed, 20 skipped in 504 s (linux, 6 at once) initial branch at 668e1913: 509 passed, 1 failed, 20 skipped in 258 s (linux, 6 at once) exact final head a283e04a: 508 passed, 2 failed, 20 skipped in 209 s (linux, 6 at once) ``` All have the same local prerequisite failure (temporary path omitted): ```text error NU1100: Unable to resolve 'Microsoft.AspNetCore.App.Ref (= 10.0.13)' for 'net10.0'. ``` The shared main run also had two timing-sensitive `test_appimage_package_retry.py` assertions under heavy load; its separate isolated follow-up passed all nine cases. Both branch suites pass that file. **The final branch run instead lost a compiler in `nx2/test_depth_sample_types.py`:** clang++ compiling `emit_spirv_barriers.cpp` “died with `<Signals.SIGKILL: 9>`”. Its complete `nx2`/`gpu` inputs and depth fixture are identical to main, and the earlier branch suite passed it. **One requested isolated follow-up on the unchanged final head passed:** ```text pass 4.3s nx2/test_depth_sample_types.py 1 passed, 0 failed, 0 skipped in 4 s (linux, 1 at once) ``` The kernel log covering the failed compile's minute is readable but has no out-of-memory kill or memory-pressure explanation. **Cause unknown:** the isolated pass does not prove a memory kill or erase the full-suite failure. It is not labelled an inherited baseline failure. No further rerun was made; the PR stays a draft. Windows-target syntax: three translation units (selector, write-back and aspect) compile with warnings as errors; not run on Windows. **Historical GitHub evidence, not new Forgejo evidence:** [the source workflow on original PR head `6ea3f6a1f9b4a793259f2a5f89dc50cd9f1e78e2`](https://github.com/NotRexed/Hookshot/actions/runs/43282642302) reports both Ubuntu parts, both Windows parts, Vulkan and actionlint successful. Across that head the public API lists 12 successful checks and two release-only skips. The original Hookshot report also records the earlier validated fork job completing in 23 seconds including setup; that duration and its Mesa 25.2.8 runner were not reproduced on Forgejo. Earlier NVIDIA/Intel fixture trials remain labelled Hookshot and were not repeated here. The new runner itself is byte-identical to the green GitHub head; the C++ aspect fixture has only the application-name rename. Workflow placement/contracts and documentation are the named migration adaptations, not a claim of unchanged hosted behavior. ## Testing **Tested:** On this PC: Arch Linux, RTX 4090, NVIDIA 615.78.08, Core i9-13900K. The RTX 4090 was not selected: device checks used CPU-only llvmpipe, Mesa 26.2.5-arch1.1 / LLVM 23.1.3 with Khronos validation. Two interleaved main/branch pairs at the measured commits above passed all 13 authored write-back/copy-span/aspect cases in each arm with validation active. The source regression checked main/branch/main/branch; main lacked the selector/workflow, while the branch passed all 11 tests. The whole suite ran on the current head after the final documentation commit. Three Windows-target fixture units compiled; no Windows executable was run. Historical GitHub checks remain historical, not Forgejo execution. Tested but not measured: none beyond the explicitly recorded correctness checks; no speed measurement is claimed from their printed durations. **Not tested:** Not yet run on a forge runner; the requested workflow trial is below, and the earlier public tasks-endpoint response cannot establish historical runner completions. No new test request was posted by this migration. - The earlier a283e04a suite's depth-fixture compiler SIGKILL remains unexplained despite its one successful isolated follow-up; that failure is retained separately from the rebased-head suite. - No fresh Windows execution of the selector, write-back or aspect C++ fixtures: only cross-compilation and old GitHub execution are available, not a new Windows run. - No game build/run, full renderer startup/replay, hardware GPU, AMD/Deck, swapchain, HDR, DLSS, frame generation or hardware-performance claim. This is a game-free software-driver lane; software Vulkan is not a substitute for those. - No combined integration trial with the wider Forgejo release/CI migration. A matching runner and a completed trial must be supplied before calling this ready. **Needs other testers:** - Configured Ubuntu 24.04 Forgejo runner: run the proposed workflow, expecting checkout and package setup to complete and all 13 fixture cases to pass with validation active (request to open) ## Screenshots Nothing on screen. These are offscreen pixel-readback assertions; no window or game screenshot was produced. ## Risk and rollback Mesa, the validation layer or loader messages can change and make this strict lane fail; runner images may lack Node.js 24 or the required packages. A failure must stay a failure, not become a skip. Disable/remove this standalone workflow or revert this PR to roll back; the ordinary source suite remains GPU-independent, and no game settings or saves need migration. ## Reviewer notes Start with `check_fixture()` in `tests/run_vulkan_fixtures.py`: every case and both layer-insertion markers must be present in each fixture's own output. The aspect selector keeps the old preference when no filter is passed and refuses an unmatched filter. The outstanding work is an operator-provided, isolated Forgejo runner trial, not a missing local implementation; the wider release migration remains separate. ## Issues See #43 — the forge's hardware-test-runner proposal; this supplies only its game-free software-driver part, not a hardware service. Original report: https://github.com/NotRexed/Hookshot/issues/2797. Original PR: https://github.com/NotRexed/Hookshot/issues/2866. The corresponding proposal now exists at https://git.magicbean.dev/MagicBeanDev/Longshot/issues/43; it is not closed by this PR. ## Checklist ### For everyone - [x] One topic. - [x] The title is a Conventional Commit (`type(scope): description`). - [x] Rebased on current `main`. - [x] `python -I -B tests/run_all.py` run and passing (any failure that also fails on `main` is named above). — The earlier compiler kill is retained as history, not this run's result. - [x] Tested in the game, or what wasn't is named under Testing. — No game/runtime change; all unrun paths are explicit. - [x] Testing filled in: what was tested, what wasn't, and what needs other testers (with test requests). - [x] Measured: main and this PR at named commits, on the same PC, scene and settings, at least 2 interleaved runs each, the median reported. - [x] No game files, dumps, build output or generated files. - [x] `third_party/` untouched (a new folder with a `PROVENANCE.md` instead). - [x] The license is OK: GPL-3.0-or-later, nothing incompatible and nothing decompiled. ### If applicable - [x] AI assistance recorded with `Assisted-by:` / `Co-Authored-By:` trailers. - [x] A row in `docs/tested-configurations.md` for each measured run. - [x] A check under `tests/` that reads changed source updated in this PR, not weakened. - [x] Docs updated. --- Moved from NotRexed/Hookshot#2866. Assisted-by: GPT-6 Astra Co-Authored-By: GPT-6 Astra <[email protected]>

mooki added 6 commits 2026-11-13 01:24:41 +00:00

test(vk): select the offscreen fixture device explicitly ... 61aaa630b6

Keep the existing discrete-device preference without --device. An explicit substring selects only matching devices and prints the enumerated names on failure.

Assisted-by: GPT-6 Astra
Co-Authored-By: GPT-6 Astra <[email protected]>

ci: execute authored Vulkan fixtures on lavapipe ... 17f5295a15

Run the three game-free Vulkan fixtures with the Khronos validation layer on Ubuntu 24.04. Restrict the loader to lavapipe, record versions and identity, and reject skipped, incomplete, wrong-device or validation-error results. Keep the source suite GPU-independent and test the strict result contract without a device.

Refs #2797

Assisted-by: GPT-6 Astra
Co-Authored-By: GPT-6 Astra <[email protected]>

test(ci): require active validation in every Vulkan fixture ... f498637ee8

Request layer diagnostics and require both instance and device validation-layer insertion records from each process. Merely enumerating the installed layer or setting its environment variable no longer satisfies the validation claim. Keep rejecting errors even when every fixture case reports success.

Assisted-by: GPT-6 Astra
Co-Authored-By: GPT-6 Astra <[email protected]>

docs(test): distinguish source checks from Vulkan execution ... 439a591756

Replace the old normal-CI-skips-GPU wording with the source-suite and dedicated lavapipe-job distinction. Include maintenance4 in the write-back fixture requirements. No executable code changes.

Assisted-by: GPT-6 Astra
Co-Authored-By: GPT-6 Astra <[email protected]>

ci: prepare the lavapipe fixture job for Forgejo and document local execution ... 73ad5c5376

Move this job from the legacy GitHub workflow to .forgejo/workflows, pin the action host explicitly, and state the required runner image. Preserve all fixture output and validation guards. Forgejo does not enforce GitHub permissions; do not promise confinement from that key. Keep local execution usable without Actions and require a real Forgejo runner trial before claiming hosted coverage.

Assisted-by: GPT-6 Astra
Co-Authored-By: GPT-6 Astra <[email protected]>

docs(test): record paired Longshot lavapipe fixture execution ... 3969145a3f

Record both manual-main and strict-wrapper arms, the source regression checks, whole suite and Windows-target syntax results. These are ordinary software Vulkan correctness runs, not game, hardware performance or Forgejo runner evidence.

Assisted-by: GPT-6 Astra
Co-Authored-By: GPT-6 Astra <[email protected]>

mooki commented 2026-11-13 02:25:06 +00:00

Author

First-time contributor

• Copy link
• Copy Markdown

Added the three-part Testing section and updated the full checklist. Added one branch base/head/suite block under Measurements; apart from consolidating duplicate results and minor wording cleanup, nothing else changed.

Added the three-part Testing section and updated the full checklist. Added one branch base/head/suite block under Measurements; apart from consolidating duplicate results and minor wording cleanup, nothing else changed.

mooki referenced this pull request 2026-11-13 02:36:25 +00:00

WIP: fix(docs): give tested runs independent result files #96

mooki referenced this pull request 2026-11-13 02:54:25 +00:00

WIP: ci: add an owner-approved hardware fixture pilot #100

mooki referenced this pull request 2026-11-13 02:54:41 +00:00

Idea: opt-in hardware tests on self-hosted Forgejo runners #43

All checks were successful

Hide checks

PR description / Description follows the template (pull_request_target) Successful in 2s

Details

This pull request has changes conflicting with the target branch.

• .forgejo/workflows/ci.yml
• docs/tested-configurations.md

View command line instructions

Manual merge helper

Use this merge commit message when completing the merge manually.

Merge commit title WIP: ci: run authored Vulkan fixtures locally and prepare Forgejo execution (#67)

Merge commit body Reviewed-on: https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/67

WIP: ci: run authored Vulkan fixtures locally and prepare Forgejo execution (#67) Reviewed-on: https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/67 Copy merge commit message

Checkout

From your project repository, check out a new branch and test the changes.

git fetch -u 'ci/vulkan-fixtures-lavapipe:mooki-ci/vulkan-fixtures-lavapipe'

git switch 'mooki-ci/vulkan-fixtures-lavapipe'

Sign in to join this conversation.

Reviewers

No reviewers

Labels

Clear labels   

accessibility

Barrier affecting people with disabilities

  

autorelease: pending

  

autorelease: tagged

  

bug

Something isn't working

  

dependencies

Pull requests that update a dependency file

  

documentation

Improvements or additions to documentation

  

duplicate

This issue or pull request already exists

  

enhancement

New feature or request

  

github_actions

Pull requests that update GitHub Actions code

  

good first issue

Good for newcomers

  

help wanted

Extra attention is needed

  

invalid

This doesn't seem right

  

needs measurements

Waiting for before/after measurements

  

question

Further information is requested

  

testing wanted

Wanted: a test on other hardware (posts a test request in #test-requests)

  

wontfix

This will not be worked on

No labels

accessibility

autorelease: pending

autorelease: tagged

bug

dependencies

documentation

duplicate

enhancement

github_actions

good first issue

help wanted

invalid

needs measurements

question

testing wanted

wontfix

Milestone

Clear milestone

No items

No milestone

Projects

Clear projects

No items

No project

Assignees

Clear assignees

No assignees

1 participant

Notifications

Subscribe

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference

MagicBeanDev/Longshot!67

No description provided.