ci: execute authored Vulkan fixtures on lavapipe #2866
Open
github-sync
wants to merge 4 commits from
github/pr/2866 into main
pull from: github/pr/2866
merge into: MagicBeanDev:main
MagicBeanDev:main
MagicBeanDev:github/pr/2893
MagicBeanDev:github/pr/2890
MagicBeanDev:github/pr/2887
MagicBeanDev:github/pr/2878
MagicBeanDev:github/pr/2875
MagicBeanDev:github/pr/2874
MagicBeanDev:github/pr/2873
MagicBeanDev:github/pr/2871
MagicBeanDev:github/pr/2756
MagicBeanDev:github/pr/2897
MagicBeanDev:github/pr/2896
MagicBeanDev:github/pr/2718
MagicBeanDev:github/pr/2842
MagicBeanDev:github/pr/2691
MagicBeanDev:github/pr/2895
MagicBeanDev:github/pr/2743
MagicBeanDev:github/pr/2765
MagicBeanDev:github/pr/2708
MagicBeanDev:github/pr/2732
MagicBeanDev:github/pr/2755
MagicBeanDev:github/pr/2817
MagicBeanDev:github/pr/2876
MagicBeanDev:release-please--branches--main
MagicBeanDev:release-please--branches--main--release-notes
MagicBeanDev:github/pr/2778
MagicBeanDev:github/pr/2888
MagicBeanDev:github/pr/2877
MagicBeanDev:github/pr/2800
MagicBeanDev:github/pr/2829
MagicBeanDev:github/pr/2850
MagicBeanDev:github/pr/2889
MagicBeanDev:github/pr/2882
MagicBeanDev:github/pr/2860
MagicBeanDev:github/pr/2849
MagicBeanDev:github/pr/2886
MagicBeanDev:github/pr/2794
MagicBeanDev:github/pr/2840
MagicBeanDev:github/pr/2830
MagicBeanDev:github/pr/2834
MagicBeanDev:github/pr/2719
MagicBeanDev:github/pr/2742
MagicBeanDev:github/pr/2741
MagicBeanDev:github/pr/2820
MagicBeanDev:github/pr/2771
MagicBeanDev:github/pr/2859
MagicBeanDev:github/pr/2858
MagicBeanDev:github/pr/2823
MagicBeanDev:github/pr/2757
MagicBeanDev:github/pr/2769
MagicBeanDev:github/pr/2844
MagicBeanDev:github/pr/2843
MagicBeanDev:github/pr/2673
MagicBeanDev:github/pr/2855
MagicBeanDev:github/pr/2739
MagicBeanDev:github/pr/2807
MagicBeanDev:github/pr/2851
MagicBeanDev:github/pr/2789
MagicBeanDev:github/pr/2814
MagicBeanDev:github/pr/2818
MagicBeanDev:github/pr/2774
MagicBeanDev:github/pr/2772
MagicBeanDev:github/pr/2750
MagicBeanDev:github/pr/2786
MagicBeanDev:github/pr/2785
MagicBeanDev:github/pr/2782
MagicBeanDev:github/pr/2779
MagicBeanDev:github/pr/2775
MagicBeanDev:github/pr/2706
MagicBeanDev:github/pr/2766
MagicBeanDev:github/pr/2747
MagicBeanDev:github/pr/2759
MagicBeanDev:github/pr/2745
MagicBeanDev:github/pr/2740
MagicBeanDev:github/pr/2733
MagicBeanDev:github/pr/2725
MagicBeanDev:github/pr/2727
MagicBeanDev:github/pr/2728
MagicBeanDev:github/pr/2729
MagicBeanDev:github/pr/2721
MagicBeanDev:github/pr/2722
MagicBeanDev:github/pr/2715
MagicBeanDev:github/pr/2714
MagicBeanDev:github/pr/2707
MagicBeanDev:github/pr/2703
MagicBeanDev:github/pr/2705
MagicBeanDev:github/pr/2697
MagicBeanDev:github/pr/2679
MagicBeanDev:github/pr/2676
MagicBeanDev:github/pr/2669
MagicBeanDev:docs/what-hookshot-connects-to
MagicBeanDev:fix/hdr-encode-test
MagicBeanDev:fix/main-tests
MagicBeanDev:fix/play-cache-files
MagicBeanDev:perf/game-barrier-order
MagicBeanDev:feat/heap-lazy
MagicBeanDev:feat/texture-detail
MagicBeanDev:perf/astc-opaque-bc1
MagicBeanDev:fix/fs-dir-lock
MagicBeanDev:fix/framegen-native-kept
MagicBeanDev:fix/update-no-downgrade
MagicBeanDev:port/k10-gcn-f16-mirrors
MagicBeanDev:fix/framegen-mult-cap
MagicBeanDev:chore/release-as-2.1.1
MagicBeanDev:perf/pcache-load-switch
MagicBeanDev:fix/prepare-progress-log
MagicBeanDev:fix/update-closes-stale-game
MagicBeanDev:fix/hdr-scaled-sampled
MagicBeanDev:perf/first-start-preset
MagicBeanDev:perf/zero-upload
MagicBeanDev:fix/vcore-escape-2s
MagicBeanDev:test/fake-gh-atomic
MagicBeanDev:perf/batch-idle-flush
MagicBeanDev:test/finish-stub-a
MagicBeanDev:chore/record-2.1.0
MagicBeanDev:fix/main-red-tests
MagicBeanDev:fix/test-display-startup-driver-cache
MagicBeanDev:feat/mesh-shader-mode
MagicBeanDev:fix/menu-probes-at-startup
MagicBeanDev:fix/fps-counter-classic
MagicBeanDev:fix/test-log-batch-flake
MagicBeanDev:fix/narrow-screen-hud
MagicBeanDev:ci/prereleases
MagicBeanDev:release/v2.0.15
MagicBeanDev:hotfix/v2.0.15
MagicBeanDev:fix/present-ahead-replay
MagicBeanDev:release/v2.0.13
MagicBeanDev:fix/bracket-paths-2013
MagicBeanDev:fix/hdr-overlays-2013
MagicBeanDev:fix/fps-counter-scale-2013
MagicBeanDev:fix/stick-seen-at-rest
MagicBeanDev:hotfix/menu-fits-screen
MagicBeanDev:fix/f1-always-opens-menu
MagicBeanDev:fix/ini-keep-lines
MagicBeanDev:claude/cranky-pike-7cf247
MagicBeanDev:perf/texture-answer-flat
MagicBeanDev:fix/stamp-test-modules
MagicBeanDev:fix/appimage-fonts
MagicBeanDev:fix/display-awake
MagicBeanDev:release/v2.0.12
MagicBeanDev:fix/discord-windows
MagicBeanDev:rc/v2.0.12-tests
MagicBeanDev:fix/dlss-keep-feature
MagicBeanDev:rc/v2.0.12
MagicBeanDev:button-icons
No reviewers
Labels
Clear labels
accessibility
Barrier affecting people with disabilities
autorelease: pending
autorelease: tagged
bug
Something isn't working
dependencies
Pull requests that update a dependency file
documentation
Improvements or additions to documentation
duplicate
This issue or pull request already exists
enhancement
New feature or request
github_actions
Pull requests that update GitHub Actions code
good first issue
Good for newcomers
help wanted
Extra attention is needed
invalid
This doesn't seem right
question
Further information is requested
testing wanted
wontfix
This will not be worked on
No labels
accessibility
autorelease: pending
autorelease: tagged
bug
dependencies
documentation
duplicate
enhancement
github_actions
good first issue
help wanted
invalid
question
testing wanted
wontfix
Milestone
Clear milestone
No items
No milestone
Assignees
Clear assignees
No assignees
1 participant
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".
No due date set.
Dependencies
No dependencies set
Reference
MagicBeanDev/Hookshot!2866
Loading…
Reference in a new issue
No description provided.
Delete branch "github/pr/2866"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Result: all 13 authored cases pass on
NVIDIA GeForce RTX 4090(615.78.08),Intel(R) Graphics (RPL-S)(Mesa ANV 26.2.5), and Ubuntu's llvmpipe (Mesa 25.2.8). Validation is directly confirmed for every fixture on lavapipe in CI; it was not installed for the local NVIDIA/Intel runs. The validated fork job took 23 seconds from start to finish, including setup.What this is for: give contributors real Vulkan execution checks without a game dump, following #2797's idea of game-free GPU tests. The existing fixtures use the production write-back shader and copy/presentation helpers, but ordinary source checks did not execute their device paths. Not measured: the game, a swapchain, HDR, DLSS, full renderer behavior or performance. Lavapipe is CPU software Vulkan, not a hardware GPU.
Refs #2797. This is only the hosted, dump-free part of that proposal. It needs no exception to “No game in CI,” no self-hosted runner and no access to a tester's machine.
Device selection is explicit in both places: CI pins the lavapipe ICD and matches the reported name
llvmpipe; local runs pin the NVIDIA/Intel ICD and matchRTX 4090/RPL-Sfromvulkaninfo, not a marketing label.Problem
tests/run_all.py --strictfails missing build tools, but retains each test's declared OS skips. Its shards do not add GPU opt-ins.test_wb_diff_observations_gpu.pytherefore only compiles.test_vulkan_copy_span.pyruns its CPU fixture unless asked for--gpu. The offscreen aspect fixture is skipped on both CI systems.SKIP. Write-back and aspect also do not turn validation messages into process failures themselves.What changed
--device SUBSTRING, case-insensitive. No match fails and lists the devices seen. Without the flag its existing first-discrete-else-first preference and output stay unchanged.source-regressions.yml:Vulkan fixtures (lavapipe, validation), onubuntu-24.04. It installs only the C++ compiler, Mesa Vulkan driver, loader/headers, validation layer andvulkaninfo; records their versions; and requires exactly one lavapipe ICD manifest. Both Vulkan loader variables point to that one file. Checkout remains commit-pinned, without persisted credentials. Permissions remain read-only; existing concurrency applies.tests/run_vulkan_fixtures.py: recordsvulkaninfo --summary, requires one llvmpipe device and the validation layer, then explicitly selects llvmpipe in all three fixtures. Nonzero exits, exit 77, skips, missing/wrong device names, missing/repeated cases, zero readback counts and validation errors fail the job. It keeps running the other fixtures after a failure and writes the usual result/time/check/note summary.These capabilities are present in Mesa 25.2.8's lavapipe declarations and format handling, the Mesa release currently listed for Ubuntu 24.04 updates. The actual runner's package and device versions are logged. There is no reduced “pass” set: all 13 cases are required; a missing capability fails.
When it runs: every PR to main, main push and manual source-check run, matching the existing unfiltered source workflow. Packaging jobs ignore documentation-only changes and the large DLSS link job has narrow filters; this small authored execution check stays with source regressions. It also notices changes in the runner's Mesa on otherwise unrelated PRs.
How it was tested
Original runtime base
51da0464b4cf0c7f56b81ebd6bb309325835e175; final rebased basef5c9acc15bbadc88e32bdfb7c4e1eed782c1c0e4, headd5d04b20a36433e9ff1de18acdf95ae0581eb879. All four commits are patch-identical across the rebase. The fixture drivers/templates, shader and copy inputs are unchanged; the exact generated aspect program is byte-identical despite unrelated changes in its source file. Hardware results therefore carry without repeating the same programs. The final docstring-only commit changes no executable code.Linux kernel
7.2.15-1-omarchy, i9-13900K, Clang 23.1.3, GCC 16.2.1, Python 3.12.16, Vulkan headers 1.4.345; 2026-11-12. All fixture execution is offscreen, without a window or swapchain. Final author/trailer/title/no-game-output checks pass on the rebased head.Ran 11 tests in 0.213s,OK. The added case rejects either missing activation marker from every fixture.d7760411:476 passed, 0 failed, 20 skipped in 213 s (linux, 4 at once), with--strict --jobs 4; no shard.d5d04b20:501 passed, 2 failed, 20 skipped in 294 s (linux, 4 at once). The new wrapper test passes. The two failures are main's own at this base, not this change's:test_program_cache_one_variant.py(fixed by #2861) andtest_readme_menu_rows.py(fixed by #2857). On Windows main has a third,test_wb_shadow_remake.py(a fix is on its way as its own pull request). The fork's run on this rebased head fails in exactly those three tests and nowhere else; itsVulkan fixtures (lavapipe, validation)job, both Windows builds, the AppImage and actionlint pass.5857491d:test_vulkan_fixtures_ci.pypasses in part 2/2 (1.3 s), including the selector helper. Windows totals: part 1/2227 passed, 0 failed, 17 skipped in 209 s; part 2/2236 passed, 0 failed, 16 skipped in 165 s. Full Windows Vulkan execution remains untested.RPL-S, all commands exit 0; write-back 0.669 s, copy 0.901 s, aspect 0.963 s, including compilation. Runs at 11:32:02–11:32:05 -06:00; load 121.24/115.20/76.38, unchanged during the three commands. These are correctness results on a busy PC, not a speed comparison. Validation layer not installed here; kernel interval empty.UHD Graphics 770, which is not in the name the driver reports, so the write-back fixture returned 77 with zero cases run. That is the wrapper refusing a wrong device, as intended; the Intel cases were then run withRPL-S.6085c658, on the base from before main's three failures: every job passes, both Windows parts included, and the Vulkan job confirms the Khronos layer in each fixture's instance and device chains. Whole job: 23 to 26 seconds in the earlier runs, 11 of them package setup.The first lavapipe job selected llvmpipe (LLVM 20.1.2, 256 bits), API 1.4.318, Mesa 25.2.8-0ubuntu0.24.04.4, with validation package 1.3.275.0-1, loader/headers 1.3.275.0-1build1, Vulkan tools 1.3.275.0+dfsg1-1, GCC 13.3.0 and Python 3.12.3.
The validated result is
3 passed, 0 failed, 0 skipped in 6.30 s (lavapipe, validation on).No local lavapipe execution or validation-layer result is claimed for these runs: the software ICD was absent when checked, and the Khronos layer was unavailable to both local driver runs. No system packages were installed for this work. The fork job supplies the lavapipe/validation proof.
Not tested
Risks
Overlaps
docs/dev/testing.mdwhen the two changes meet. Other contributor-guide hunks read: #2820's license list, #2317's data/cache paths, #2201's version-policy guidance; #2274 also touches only the license list here.Assisted-by: GPT-6 Astra
Co-Authored-By: GPT-6 Astra [email protected]
d5d04b20a3to6ea3f6a1f9