WIP: perf(vk): the fog's lighting pass in 256-invocation workgroups on the Steam Deck (RX_VK_WG_SPLIT=1, off by default) #1377

Draft
dukenheimer wants to merge 3 commits from dukenheimer/Longshot:brett-d/fog-wg-split into main
Contributor

Type: perf · Risk: low · Tested on: Steam Deck OLED, handheld on AC (SteamOS, RADV VANGOGH, Mesa 26.2.0-devel git-5c65d4fc31); Windows, RTX 5090 (pictures only, debug switch) · Needs: a second Deck (LCD), and Deck runs with the switch on by more people

Why

On the Steam Deck the ground fog costs about 5 ms of GPU time a frame, 13–16% of the frame. One of its four programs is the lighting pass (27ee9883), and that pass runs well below the GPU's occupancy.

The cause is in Mesa 26.x's shader compiler (ACO), which the Deck's current SteamOS ships. It aims for about 10 waves per SIMD for this shader. It then rounds down to whole workgroups: one 512-invocation workgroup is 4 waves per SIMD, so 10 becomes 8, and the compiler hands out 128 registers to each thread. The same shader on Mesa 25.2 got 80 registers and 12 waves.

Splitting the workgroup in half (8x8x4, 256 invocations) lets the Deck's own compiler choose 80 registers and 12 waves again, with no spills.

See #373, which asks for an option to reduce the fog. This PR keeps the fog as it is and only makes it cheaper.

What changed

An opt-in renderer switch, RX_VK_WG_SPLIT=1. The gate is a RADV driver whose device name says VANGOGH, with compute pinned to 32 lanes; it is not tied to a Mesa version. There, the renderer rewrites the fog lighting pass's SPIR-V to half-height workgroups. It builds that as one extra shader module and pipeline of its own, and binds it for the pass's direct dispatches, with the grid's z doubled.

The translator, the translated modules, and the program cache's format and stamp are unchanged: nothing is translated or prepared again. The driver's own pipeline cache can gain the split pipeline as an entry of its own. With the switch off, nothing changes.

  • The rewrite: gpu/vk/wg_split.{h,cpp}, in wgsplit::rewrite().
    • It sets LocalSize z to z/k and WorkgroupId.z to WorkgroupId.z / k, and adds (WorkgroupId.z % k)·(z/k) to LocalInvocationId.z.
    • It refuses the program unless the module shows that no invocation reaches outside its new group:
      • no barrier, memory barrier or group operation wider than a subgroup;
      • workgroup memory is touched only through Load/Store, and only at each invocation's own slot. The index is followed as an affine expression with checked arithmetic (every coefficient below 2^31; X, Y, Z ≤ 1024; slot ≤ 2^20), and every access width is checked against the slot.
      • no other ID built-in, no LocalSizeId, and no pointer that escapes.
    • Phis whose incoming values are all the same index, a loop's back edge to the phi itself included, are followed. A phi whose incoming values differ, or change around a loop, is refused.
  • The renderer: gpu/vk/vk_draw.cpp, in wg_split_on, wg_split_build, wg_split_for and the bind and dispatch in dispatch().
    • The split pipeline lives in a side table keyed by (key, variant, layout): a wgsplit::Slot.
      • Its k is fixed when the slot is made.
      • A compare-exchange from 0 to 1 claims the build.
      • The pipeline is published before the slot is marked ready.
      • A dispatch reads the slot only once it is ready.
    • Indirect dispatches, and any dispatch whose z·k would pass the device limit, run today's pipeline. The product is widened, so it can't wrap.
    • The temporary shader module is destroyed once the pipeline is made, or when the build is declined.
  • The program cache: gpu/vk/vk_progcache.{h,cpp}.
    • program_cache_code() reads a program's SPIR-V back from the cache file.
    • A wg_split_code_seen hook keeps the code a listed program was read with.
    • Limit: a program translated during this run has no code at hand, so it declines with "translated this run: a later run". The split is used from the next start.
  • Switches: gpu/vk/switches.def gains two rows, RX_VK_WG_SPLIT (attempt) and RX_VK_WG_SPLIT_ANY_GPU (diag).
  • Lane assumption: each wave32 sits inside one 64-invocation z plane, which the 8x8x8 pipeline already relies on. The pass uses subgroupAny, a ballot and a shuffle of LocalInvocationId.z, the NVN hint that z is the same across a warp.
    • A game-free Vulkan probe confirmed this on the Deck for both 8x8x8 and 8x8x4 (see Testing).
    • So the switch is gated to that driver and device, and says so in its log line and in the code comment.

Measurements

Verdict: an opt-in, measured GPU saving, on this driver, this program and this shape.

  • The fog lighting pass (27ee9883): 27–28% less GPU time in Kokiri Forest, about 0.5 ms a frame, at both settings. In Hyrule Field, H's clean timing run shows 29–35% (0.25–0.30 ms). D's Field timing (−31%) comes from a window in which pipelines were built, and is flagged †.
  • Kokiri Forest:
    • Setup H (the Deck's Auto choice): fps +1.5% (24.15 → 24.52) and GPU time a frame −1.9%, both with the ranges apart.
    • Setup D (Brett's settings, GPU-bound): GPU time a frame −1.0%, ranges apart. fps +0.9% (18.25 → 18.42) is within the spread: the ranges overlap by 0.01 fps. H's gain doesn't show that D gained.
  • Hyrule Field: fps within the spread at both settings, so inconclusive. GPU time a frame −1.8% in D (ranges apart) and −0.8% in H (ranges overlap).
  • Power limits: the on runs were flagged at the Deck's power limits (FPPT/SPPT) more often in D's Field and in H's Kokiri. In D's Kokiri and H's Field the counts are similar.
  • The picture: no visible fog change. Off and on show comparable, overlapping run-to-run pixel variability (see Testing). This is not a deterministic picture-equality test.

Hardware: Steam Deck OLED. AMD Custom APU 0932 (4 cores, 8 threads), 16 GB; AMD Custom GPU 0932, RADV VANGOGH, Mesa 26.2.0-devel git-5c65d4fc31, compute pinned to 32 lanes. SteamOS 3.9.5 (BUILD_ID 20261023.100), kernel 7.2.7-valve2-2-neptune-72-g31504582a073.
Display and power, checked in every run counted:

  • handheld, the built-in panel at 1280x800 and 90 Hz (the log's "screen 1280x800 at 90 Hz" and "vk: swapchain 1280x800"), DP disconnected;
  • on AC (D's runs charging; H's not charging at 80%);
  • 15 W cap, GPU at its peak clock level (profile_peak), frame generation off.

Scene: the repo save kokiri-start. Kokiri Forest standing still, 45 s; then a teleport to Hyrule Field (child), turning, 30 s.

Settings: each run used its own copy of the setup's options file. Within a setup, all copies came back identical after the game's exit-time rewrite: D 7757e809…, H 4f7df9fa…. They were made from the templates hisfg0.ini (552eb6cc…) and hh.ini (872edeea…). No switch overrides a row.

  • Setup D (Brett's options): the game's own upscaler at 100% render resolution (the 3D at 1280x800; the scene pass and the game's network read and write 1280x800). Output 1280x800, docked profile, quality preset Custom, sharpness 0, V-Sync Auto (mailbox), frame rate cap 60, HDR Native (HDR10 swapchain), memory budget row 0.
  • Setup H (the Deck's Auto choice): quality preset Minimum, the game's own upscaler at 50% render resolution (the 3D at 640x400, upscaled to 1280x800 by the renderer's helper; the game's network isn't dispatched). Memory budget row 1. The other rows as in D: output 1280x800, docked profile, sharpness 0, V-Sync Auto (mailbox), cap 60, HDR Native.
  • Common to every run: RX_VK_COLD_CHECK=0, a Mesa shader cache of 5 GB per game copy, and Fossilize disabled. The arms differ only in RX_VK_WG_SPLIT.

Method: one build (bda836f4) and two arms, RX_VK_WG_SPLIT unset (O) and =1 (N). A warm-up run of each arm, then 3 interleaved runs each; medians and observed ranges.

  • fps is frames over the window's summed frame times. The 1% low is 1000 ÷ the mean of the slowest floor(n/100) frame times of the window's n frames: 8 of 840, 7 of 720. p50 and p99 are the frame-time percentiles of the Deck's report, by floor rank. (A recomputation from the raw frame times, which checked these numbers, interpolates its percentiles, so its p99 differs slightly from the report's; its fps, p50 and 1% low agree.) "GPU time a frame" is the renderer's own log line, averaged over the window.
  • RX_VK_PASS_TIMING=2 in a separate run per arm, for the pass's span (instrumented runs, not the fps windows).
  • Each N window starts after the log's "workgroup split of 27ee9883e2a20323 (variant 130c8d4842e475e5) … bound for its direct dispatches from now on" line. Every N run had that line before its window, and none had a "declined" line.
  • A window in which the game built pipelines is left out of the fps and GPU medians: D-2N's and H-1O's Field windows, so those arms have n=2.

How to reproduce: the repo save kokiri-start, the switch unset and set, in turn.

  • Start each run from a fresh copy of the repo save kokiri-start, the game folder and the options file.
  • Warm the program cache with one run per arm: the split is made only for a program read from the cache.
  • Then run O and N in turn, 3 each. Measure 45 s standing in Kokiri Forest, after the ready line on N; then teleport to Hyrule Field (child) and measure 30 s turning.
  • Take RX_VK_PASS_TIMING=2 from a separate run per arm.
  • Check each log for screen 1280x800 at 90 Hz and vk: swapchain 1280x800, and on N for the bound for its direct dispatches line before the window.

Baseline: this PR @ bda836f4 with the switch unset, which runs main's pipeline and dispatch for every program.

  • The runs were measured at bda836f4, on main 6d72ca93.
  • The branch is now rebased onto main 544229a2. The only difference is main's #1332 (which Vulkan driver the loader picks on Windows); the split's sources are byte-identical. So the numbers are labelled bda836f4 and were not re-measured.

Setup D (Brett's options, 100% render), handheld, on AC, 1280x800 at 90 Hz

Before, RX_VK_WG_SPLIT unset: this PR @ bda836f4 After, RX_VK_WG_SPLIT=1: this PR @ bda836f4
Kokiri: 27ee9883 GPU time (PASS_TIMING=2) 1.871 ms 1.360 ms (−27%)
Kokiri: fps, median (range) 18.25 (18.24–18.37) 18.42 (18.36–18.45)
Kokiri: 1% low, median (range) 6.96 (6.34–7.30) 6.74 (6.52–7.10)
Kokiri: p50 frame time, median (range) 53.6 ms (53.5–53.8) 53.2 ms (53.1–53.4)
Kokiri: p99 frame time, median (range) 68.3 ms (64.9–80.4) 69.7 ms (67.5–85.2)
Kokiri: GPU time a frame (log), median (range) 43.96 ms (43.96–44.11) 43.53 ms (43.49–43.60)
Kokiri: process CPU ms per bridge game frame, median (a sub-percent difference, not a change) 152.76 152.33
Field: 27ee9883 GPU time (PASS_TIMING=2) 0.871 ms 0.604 ms (−31%) †
Field: fps, median (range) 24.26 (23.84–24.53), n=3 24.03 (23.82–24.24), n=2
Field: 1% low, median (range) 8.24 (7.99–8.48) 8.75 (8.70–8.81)
Field: p50 frame time, median (range) 38.5 ms (38.4–39.0) 38.9 ms (38.7–39.1)
Field: p99 frame time, median (range) 86.6 ms (83.0–89.1) 81.6 ms (78.1–85.1)
Field: GPU time a frame (log), median (range) 30.00 ms (29.90–30.07) 29.46 ms (29.42–29.50)
Field: power-limit flags (FPPT / SPPT, of 60 samples) 45–50 / 29–33 54–55 / 47–49

† D-TN's Field timing window also had 4 pipelines built in it. It is shown for completeness, not as a clean measurement. H's clean Field pair is the evidence for the Field.

Setup H (the Deck's Auto choice: Minimum, 50% render), handheld, on AC, 1280x800 at 90 Hz

Before, RX_VK_WG_SPLIT unset: this PR @ bda836f4 After, RX_VK_WG_SPLIT=1: this PR @ bda836f4
Kokiri: 27ee9883 GPU time (PASS_TIMING=2) 1.836 ms 1.317 ms (−28%)
Kokiri: fps, median (range) 24.15 (24.14–24.21) 24.52 (24.50–24.59)
Kokiri: 1% low, median (range) 9.33 (9.14–9.46) 9.91 (9.78–10.14)
Kokiri: p50 frame time, median (range) 40.5 ms (40.5–40.6) 39.9 ms (39.8–40.0)
Kokiri: p99 frame time, median (range) 52.6 ms (50.3–54.4) 55.9 ms (51.5–56.3)
Kokiri: GPU time a frame (log), median (range) 31.23 ms (31.16–31.24) 30.63 ms (30.63–30.66)
Kokiri: power-limit flags (FPPT / SPPT, of 90 samples) 61–71 / 24–36 78–86 / 65–72
Field: 27ee9883 GPU time (PASS_TIMING=2) 0.867 ms 0.612 / 0.565 ms (two lines; −29% / −35%)
Field: fps, median (range) 29.44 (29.40–29.48), n=2 29.60 (27.85–30.41), n=3
Field: 1% low, median (range) 9.59 (9.13–10.06) 9.17 (9.13–9.55)
Field: p50 frame time, median (range) 31.9 ms (31.7–32.1) 32.1 ms (30.9–34.0)
Field: p99 frame time, median (range) 70.3 ms (65.2–75.3) 65.5 ms (57.6–69.5)
Field: GPU time a frame (log), median (range) 18.11 ms (17.73–18.49) 17.96 ms (17.90–18.19)

Reading:

  • Kokiri: the pass's saving and the frame's drop are of similar size. This is consistent, not a full accounting: the pass times come from separate instrumented runs with few span records, under varying clocks and power.
    • D: the pass saves 0.51 ms, GPU time a frame drops 0.43 ms, and frame time drops 0.51 ms (at the fps medians).
    • H: the pass saves 0.52 ms, GPU time a frame drops 0.60 ms, and frame time drops 0.62 ms.
  • The pass's time was nearly the same at 50% and 100% render resolution (Kokiri, off: 1.836 and 1.871 ms). How its grid scales isn't measured here.
  • The power-limit flags are sampled flags, not durations, and don't show which resource limited the frame. That the extra throttling takes back part of the saving is a hypothesis, not something measured.
  • p99 and 1% low are noisy in both arms and both setups. Two 1% lows have disjoint observed ranges: H's Kokiri (9.14–9.46 against 9.78–10.14) and D's Field (7.99–8.48 against 8.70–8.81, n=3/2). From so few runs, neither shows a general improvement in the slowest frames.
vk: workgroup split (RX_VK_WG_SPLIT=1): on (RADV Van Gogh, compute at 32 lanes)
vk: workgroup split of 27ee9883e2a20323 (variant 130c8d4842e475e5): 8x8x8 as 8x8x4, 15 loads given the IDs, 24 shared accesses of 9-word slots checked; bound for its direct dispatches from now on
D-TO  vk:   m   1.871 ms     1.0 x   1870.7 us       1.0 items    0.00 MB  dispatch  prog 27ee9883e2a20323   (Kokiri)
D-TN  vk:   m   1.360 ms     1.0 x   1360.1 us       1.0 items    0.00 MB  dispatch  prog 27ee9883e2a20323   (Kokiri)
D-TO  vk:   m   0.871 ms     1.0 x    871.4 us       1.0 items    0.00 MB  dispatch  prog 27ee9883e2a20323   (Field)
D-TN  vk:   m   0.604 ms     1.0 x    604.1 us       1.0 items    0.00 MB  dispatch  prog 27ee9883e2a20323   (Field; 4 pipelines built in this window)
H-TO  vk:   m   1.836 ms     1.0 x   1835.9 us       1.0 items    0.00 MB  dispatch  prog 27ee9883e2a20323   (Kokiri)
H-TN  vk:   m   1.317 ms     1.0 x   1316.6 us       1.0 items    0.00 MB  dispatch  prog 27ee9883e2a20323   (Kokiri)
H-TO  vk:   m   0.867 ms     1.0 x    865.4 us       1.0 items    0.00 MB  dispatch  prog 27ee9883e2a20323   (Field)
H-TN  vk:   m   0.612 ms     1.0 x    611.3 us       1.0 items    0.00 MB  dispatch  prog 27ee9883e2a20323   (Field)
H-TN  vk:   m   0.565 ms     1.0 x    565.3 us       1.0 items    0.00 MB  dispatch  prog 27ee9883e2a20323   (Field)
D-1O  frames=840 fps=18.25 p50=53.6 p95=58.8 p99=80.4 max=172.9 over100=8                (Kokiri; each run's summary line is in docs/tested)

Side note: docked, on AC, 1280x720 at 60 Hz, setup H's options. These are earlier runs, not the measurement above. They were taken before the Deck was undocked at 21:38:30; H-3N, taken after the undock, is left out.

  • Kokiri, off n=3 (H-1O, H-2O, H-3O) and on n=2 (H-1N, H-2N).
  • fps 24.46 (24.41–24.56) → 24.86 (24.84–24.88), and GPU time a frame 30.68 (30.62–30.70) → 30.12 ms (30.10–30.13). Both have their ranges apart.

Offline, the Deck's Mesa: RADV 26.2.0-devel git-5c65d4fc31 (steamos-26.05), VANGOGH through amdgpu drm-shim, with RADV_DEBUG=shaderstats. The split module was made at bda836f4; the split's sources are byte-identical at this head.

27ee9883 VGPRs waves / SIMD spills LDS code size
original (8x8x8) 128 8 0 18432 15928
split (8x8x4) 80 12 0 18432 16044

Flags and settings

Two new switches, both off by default, plus a debug value of the first.

Flag What it is Before this PR After this PR Effect Helps / doesn't help / untested Recommendation
RX_VK_WG_SPLIT attempt none unset: off 1: the fog lighting pass in 8x8x4 workgroups for its direct dispatches, on a RADV driver whose device is VANGOGH, with compute at 32 lanes (no Mesa version gate) Saves GPU time in the pass on a Deck OLED with Mesa 26.2.0-devel (above). Untested on the Deck LCD, on other Mesa versions and on other RADV GPUs Off until more Deck runs agree
RX_VK_WG_SPLIT=<hex,…> debug value none — Splits the programs whose keys start with those prefixes, under the same refusal rules. This widens the risk beyond the fog pass Debug only: nothing beyond 27ee9883 was checked in the game Debug only
RX_VK_WG_SPLIT_ANY_GPU diag none unset: off 1 allows the split on any GPU whose compute runs 32 lanes, to look at the picture there; logged as "a debug run, not a measurement". This widens the risk beyond RADV Van Gogh Debug only: the lane mapping was shown only on RADV Van Gogh Off: debug only

Testing

Tested:

  • The game-free check: tests/test_wg_split.py, with tests/wg_split/wg_split_test.cpp and fixture.spvasm, a module shaped like the fog's pass.
    • It interprets the rewritten entry function's ID and shared-index arithmetic for every invocation of an asymmetric 3x2x5 grid at k = 2, 4 and 8, and checks that each physical invocation gets the same logical (group, local) tuple, address and count.
    • Three wrong remaps are each caught.
    • Refusals:
      • wider barriers and group operations;
      • workgroup atomics, and a pointer that escapes;
      • overlapping or overflowing slots, and a stride that would wrap;
      • a shift by 31, and a coefficient past 2^31 cancelled later;
      • differing and changing phis.
    • The slot, choice and bind in each state (pending, ready, declined), across direct, indirect and direct; and the limit cases (an even limit hit exactly, z·k past 32 bits).
    • spirv-val on the original and the k = 2, 4 and 8 modules.
    • What "fails on main" means: the test fails to build on main, because main has no gpu/vk/wg_split.cpp. The correctness evidence is the emitted-op mutations above.
    • Not covered: the slot tests are sequential helper tests, not threaded or TSan. The interpreter runs the index arithmetic only: not subgroup ops, loads and stores, the full control flow or called functions. That is why the pictures and the A/B below were run.
  • At bda836f4, recorded in one run (its output is in the evidence comment below; the split's sources are byte-identical at this head):
    • the test with spirv-val (SPIRV-Tools v2025.1) on Linux, g++ 13;
    • the test under ASan/UBSan;
    • the rewrite of the real 27ee9883 and 0a55781f under ASan/UBSan;
    • spirv-val of the real 27ee9883, original and split (Vulkan 1.3, scalar block layout);
    • the Mesa 26.2 stats above.
  • At this head (rebased on 544229a2; tested at 4d619558, which differs from it only in the docs/tested rows): tests/test_wg_split.py passes on Windows and on Linux (with spirv-val), and the renderer builds (dev_build.py vk, 0 failed, exports checked).
  • The full suite at this head: python -I -B tests/run_all.py on Windows; the receipts are in the evidence comment below.
    • Branch: 577 passed, 48 failed. Main 544229a2: 577 passed, 47 failed.
    • The 47 failures are common to both, listed in the evidence comment below. One of them is test_longshot_ctl, which is flaky: run alone, it passes on the branch and fails on main.
    • Branch only: test_gpu_hang_watch.py, a 2 s timing test that failed under a 24-way parallel run. Run alone, it passes on the branch and on main.
    • test_wg_split is the branch's extra test, and it passes.
    • The earlier run at bda836f4 against main 6d72ca93 matched this: 46 common failures, plus test_longshot_ctl on the branch only.
  • The lane probe: a game-free C and Vulkan program, run on the Deck (19:36:42, RADV VANGOGH, Mesa 26.2.0-devel git-5c65d4fc31, compute pinned to 32).
    • For 8x8x8 on a 3x2x5 grid and 8x8x4 on a 3x2x10 grid: 480 subgroups of 32 each, 0 bad records, 0 subgroups spanning z planes, 0 shuffle-z mismatches, 0 partial subgroups, lane = index%32 everywhere; rc=0.
    • After review, the probe's PASS also requires a lane offset of 0 and a size of 32. That rebuilt binary (sha256 9068d6fa…) was rerun on the PC only (RTX 5090, PASS). The Deck run above, from the earlier binary, already showed both.
    • Its source, lane_probe.c and probe.comp, and its Deck output are in the evidence comment below.
  • Pictures:
    • Windows, RTX 5090, driver 621.14: headless, RX_VK_WG_SPLIT=1 + RX_VK_WG_SPLIT_ANY_GPU=1 (debug).
      • Off, on, off at the same spots: Hyrule Field and Kokiri Forest, at arrival +600 and +900 frames.
      • The on run's ready line came before every shot.
      • Off vs on shows comparable, overlapping run-to-run variability; some off-vs-on pairs differ more than the off-vs-off pair (Field +600: 4.19 against 3.50). The differences are animation, and no fog brightness shift shows.
    • The Deck, handheld on AC, setup H (bda836f4, RX_VK_WG_SPLIT unset vs =1): no obvious fog change by eye. The side-by-side of H-2O (unset) and H-2N (=1), Kokiri and the Field, is in the evidence comment below, with the PC pair.
      • Over 3 runs each, the off-vs-on pixel differences overlap the within-arm differences without being contained in them. Mean absolute difference: Kokiri 3.78–5.18 against 3.51–5.02 within an arm; Field 3.68–5.47 against 3.57–5.13.
      • Mean brightness overlaps between the arms.
      • Pose, camera and animation differ between runs (the fairy, leaves, Link's idle), so this is not a deterministic picture-equality test.

Not tested: the Deck LCD; other Mesa versions (the gate is the device, not the version); other RADV GPUs (the switch declines there); cold translations (they decline until the next start, by design).

Must be tested:

  • Steam Deck LCD, SteamOS 3.9, Mesa 26.x (Test Farm tag steamdeck): before merge, so the PR stays WIP until then (or until a maintainer narrows this). Start from the repo save kokiri-start, warm up once, then run Kokiri Forest standing and Hyrule Field turning with RX_VK_WG_SPLIT unset and =1 in turn, 3 runs each, plus one RX_VK_PASS_TIMING=2 run per arm. Expecting the log's "workgroup split of 27ee9883… bound for its direct dispatches from now on" line before each =1 window, no "declined" line, the fog looking the same, and a lower 27ee9883 time under RX_VK_PASS_TIMING=2 (Test Farm: @Beanstalk steamdeck, the fog split off vs on in Kokiri Forest and Hyrule Field on an LCD Deck; or by hand: the same, by anyone with an LCD Deck)

Risks

What could break: only with the switch on, on a RADV Van Gogh driver: the fog's lighting pass. A wrong remap would show as blocky or shifted fog. The rewrite refuses the program, and today's pipeline stays, when it can't show that no invocation reaches outside its new group. The lane assumption was shown on one Mesa (26.2.0-devel); another Mesa version is untested. The debug values (hex prefixes, RX_VK_WG_SPLIT_ANY_GPU) widen the risk to other programs and GPUs.

Turning it off: unset RX_VK_WG_SPLIT, which is the default.

Compatibility: nothing changes for a player who doesn't set the switch.

  • The translated modules, the program cache's format and stamp and the vk ABI are unchanged, and nothing is translated or prepared again.
  • With the switch on, one extra shader module and pipeline is made at run time. The driver's pipeline cache can keep it as an entry of its own.

Issues

See #373 (an option to reduce the fog; this PR makes it cheaper instead).

Checklist

Everyone

  • One topic.
  • The title is a Conventional Commit (type(scope): description) of the status line's type.
  • Why gives the reason plainly and links the issues that fit (the tracker searched).
  • Measurements has a verdict, or one line saying why there is nothing to measure.
  • Testing names what was tested, what wasn't, and what must still be tested, where and how.
  • Rebased on current main. (544229a2; re-check before opening)
  • python -I -B tests/run_all.py passes (any failure that also fails on main is named). (The 47 common failures listed; the two timing-flaky tests explained.)
  • No game files, dumps, build output or generated files. (tests/wg_split/fixture_spv.h is generated from the hand-written fixture.spvasm, with its SHA-256 checked; it is not game-derived.)
  • third_party/ untouched (a new folder with a PROVENANCE.md instead).
  • The license is OK: GPL-3.0-or-later, nothing incompatible and nothing decompiled.

When it applies

  • AI assistance recorded with Assisted-by: / Co-Authored-By: trailers.
  • Numbers from the same PC, scene and settings, at least 2 interleaved runs each, the median reported.
  • Switches documented (docs/development.md; the README for a menu row) and shipped at their recommended value. (rows in gpu/vk/switches.def)
  • Screenshots of the same spot, captioned like the Measurements columns, for a change that shows on screen. (In the evidence comment: the Deck pair, captioned "Before: bda836f4, RX_VK_WG_SPLIT unset (H-2O)" and "After: bda836f4, RX_VK_WG_SPLIT=1 (H-2N)".)
  • A file in docs/tested/ for each new measured run (format in docs/tested-configurations.md). (12 files: D and H, off and on, runs 1–3; commit e4a1471d.)
  • A game-free test added for the fix, failing on main and passing here. (It fails to build on main.)
  • A check under tests/ that reads changed source updated in this PR, not weakened. (test_intel_quick_compute.py, plus stubs in the two program-cache tests)
  • Host code builds on both Windows (llvm-mingw) and Linux, with UTF-8 paths.
  • Docs updated.
**Type:** perf · **Risk:** low · **Tested on:** Steam Deck OLED, handheld on AC (SteamOS, RADV VANGOGH, Mesa 26.2.0-devel git-5c65d4fc31); Windows, RTX 5090 (pictures only, debug switch) · **Needs:** a second Deck (LCD), and Deck runs with the switch on by more people ## Why On the Steam Deck the ground fog costs about 5 ms of GPU time a frame, 13–16% of the frame. One of its four programs is the lighting pass (27ee9883), and that pass runs well below the GPU's occupancy. The cause is in Mesa 26.x's shader compiler (ACO), which the Deck's current SteamOS ships. It aims for about 10 waves per SIMD for this shader. It then rounds down to whole workgroups: one 512-invocation workgroup is 4 waves per SIMD, so 10 becomes 8, and the compiler hands out 128 registers to each thread. The same shader on Mesa 25.2 got 80 registers and 12 waves. Splitting the workgroup in half (8x8x4, 256 invocations) lets the Deck's own compiler choose 80 registers and 12 waves again, with no spills. See https://git.magicbean.dev/MagicBeanDev/Longshot/issues/373, which asks for an option to reduce the fog. This PR keeps the fog as it is and only makes it cheaper. ## What changed An opt-in renderer switch, `RX_VK_WG_SPLIT=1`. The gate is a RADV driver whose device name says VANGOGH, with compute pinned to 32 lanes; it is not tied to a Mesa version. There, the renderer rewrites the fog lighting pass's SPIR-V to half-height workgroups. It builds that as one extra shader module and pipeline of its own, and binds it for the pass's direct dispatches, with the grid's z doubled. The translator, the translated modules, and the program cache's format and stamp are unchanged: nothing is translated or prepared again. The driver's own pipeline cache can gain the split pipeline as an entry of its own. With the switch off, nothing changes. - **The rewrite:** `gpu/vk/wg_split.{h,cpp}`, in `wgsplit::rewrite()`. - It sets LocalSize z to z/k and WorkgroupId.z to WorkgroupId.z / k, and adds (WorkgroupId.z % k)·(z/k) to LocalInvocationId.z. - It refuses the program unless the module shows that no invocation reaches outside its new group: - no barrier, memory barrier or group operation wider than a subgroup; - workgroup memory is touched only through Load/Store, and only at each invocation's own slot. The index is followed as an affine expression with checked arithmetic (every coefficient below 2^31; X, Y, Z ≤ 1024; slot ≤ 2^20), and every access width is checked against the slot. - no other ID built-in, no LocalSizeId, and no pointer that escapes. - Phis whose incoming values are all the same index, a loop's back edge to the phi itself included, are followed. A phi whose incoming values differ, or change around a loop, is refused. - **The renderer:** `gpu/vk/vk_draw.cpp`, in `wg_split_on`, `wg_split_build`, `wg_split_for` and the bind and dispatch in `dispatch()`. - The split pipeline lives in a side table keyed by (key, variant, layout): a `wgsplit::Slot`. - Its k is fixed when the slot is made. - A compare-exchange from 0 to 1 claims the build. - The pipeline is published before the slot is marked ready. - A dispatch reads the slot only once it is ready. - Indirect dispatches, and any dispatch whose z·k would pass the device limit, run today's pipeline. The product is widened, so it can't wrap. - The temporary shader module is destroyed once the pipeline is made, or when the build is declined. - **The program cache:** `gpu/vk/vk_progcache.{h,cpp}`. - `program_cache_code()` reads a program's SPIR-V back from the cache file. - A `wg_split_code_seen` hook keeps the code a listed program was read with. - **Limit:** a program translated during this run has no code at hand, so it declines with "translated this run: a later run". The split is used from the next start. - **Switches:** `gpu/vk/switches.def` gains two rows, `RX_VK_WG_SPLIT` (attempt) and `RX_VK_WG_SPLIT_ANY_GPU` (diag). - **Lane assumption:** each wave32 sits inside one 64-invocation z plane, which the 8x8x8 pipeline already relies on. The pass uses subgroupAny, a ballot and a shuffle of LocalInvocationId.z, the NVN hint that z is the same across a warp. - A game-free Vulkan probe confirmed this on the Deck for both 8x8x8 and 8x8x4 (see Testing). - So the switch is gated to that driver and device, and says so in its log line and in the code comment. ## Measurements **Verdict:** an opt-in, measured GPU saving, on this driver, this program and this shape. - **The fog lighting pass (27ee9883):** 27–28% less GPU time in Kokiri Forest, about 0.5 ms a frame, at both settings. In Hyrule Field, H's clean timing run shows 29–35% (0.25–0.30 ms). D's Field timing (−31%) comes from a window in which pipelines were built, and is flagged †. - **Kokiri Forest:** - Setup H (the Deck's Auto choice): fps +1.5% (24.15 → 24.52) and GPU time a frame −1.9%, both with the ranges apart. - Setup D (Brett's settings, GPU-bound): GPU time a frame −1.0%, ranges apart. fps +0.9% (18.25 → 18.42) is within the spread: the ranges overlap by 0.01 fps. H's gain doesn't show that D gained. - **Hyrule Field:** fps within the spread at both settings, so inconclusive. GPU time a frame −1.8% in D (ranges apart) and −0.8% in H (ranges overlap). - **Power limits:** the on runs were flagged at the Deck's power limits (FPPT/SPPT) more often in D's Field and in H's Kokiri. In D's Kokiri and H's Field the counts are similar. - **The picture:** no visible fog change. Off and on show comparable, overlapping run-to-run pixel variability (see Testing). This is not a deterministic picture-equality test. Hardware: Steam Deck OLED. AMD Custom APU 0932 (4 cores, 8 threads), 16 GB; AMD Custom GPU 0932, RADV VANGOGH, Mesa 26.2.0-devel git-5c65d4fc31, compute pinned to 32 lanes. SteamOS 3.9.5 (BUILD_ID 20261023.100), kernel 7.2.7-valve2-2-neptune-72-g31504582a073. Display and power, checked in every run counted: - handheld, the built-in panel at 1280x800 and 90 Hz (the log's "screen 1280x800 at 90 Hz" and "vk: swapchain 1280x800"), DP disconnected; - on AC (D's runs charging; H's not charging at 80%); - 15 W cap, GPU at its peak clock level (`profile_peak`), frame generation off. Scene: the repo save kokiri-start. Kokiri Forest standing still, 45 s; then a teleport to Hyrule Field (child), turning, 30 s. Settings: each run used its own copy of the setup's options file. Within a setup, all copies came back identical after the game's exit-time rewrite: D 7757e809…, H 4f7df9fa…. They were made from the templates hisfg0.ini (552eb6cc…) and hh.ini (872edeea…). No switch overrides a row. - **Setup D (Brett's options):** the game's own upscaler at 100% render resolution (the 3D at 1280x800; the scene pass and the game's network read and write 1280x800). Output 1280x800, docked profile, quality preset Custom, sharpness 0, V-Sync Auto (mailbox), frame rate cap 60, HDR Native (HDR10 swapchain), memory budget row 0. - **Setup H (the Deck's Auto choice):** quality preset Minimum, the game's own upscaler at 50% render resolution (the 3D at 640x400, upscaled to 1280x800 by the renderer's helper; the game's network isn't dispatched). Memory budget row 1. The other rows as in D: output 1280x800, docked profile, sharpness 0, V-Sync Auto (mailbox), cap 60, HDR Native. - **Common to every run:** `RX_VK_COLD_CHECK=0`, a Mesa shader cache of 5 GB per game copy, and Fossilize disabled. The arms differ only in `RX_VK_WG_SPLIT`. Method: one build (bda836f4) and two arms, `RX_VK_WG_SPLIT` unset (O) and `=1` (N). A warm-up run of each arm, then 3 interleaved runs each; medians and observed ranges. - fps is frames over the window's summed frame times. The 1% low is 1000 ÷ the mean of the slowest floor(n/100) frame times of the window's n frames: 8 of 840, 7 of 720. p50 and p99 are the frame-time percentiles of the Deck's report, by floor rank. (A recomputation from the raw frame times, which checked these numbers, interpolates its percentiles, so its p99 differs slightly from the report's; its fps, p50 and 1% low agree.) "GPU time a frame" is the renderer's own log line, averaged over the window. - `RX_VK_PASS_TIMING=2` in a separate run per arm, for the pass's span (instrumented runs, not the fps windows). - Each N window starts after the log's "workgroup split of 27ee9883e2a20323 (variant 130c8d4842e475e5) … bound for its direct dispatches from now on" line. Every N run had that line before its window, and none had a "declined" line. - A window in which the game built pipelines is left out of the fps and GPU medians: D-2N's and H-1O's Field windows, so those arms have n=2. How to reproduce: the repo save kokiri-start, the switch unset and set, in turn. - Start each run from a fresh copy of the repo save kokiri-start, the game folder and the options file. - Warm the program cache with one run per arm: the split is made only for a program read from the cache. - Then run O and N in turn, 3 each. Measure 45 s standing in Kokiri Forest, after the ready line on N; then teleport to Hyrule Field (child) and measure 30 s turning. - Take `RX_VK_PASS_TIMING=2` from a separate run per arm. - Check each log for `screen 1280x800 at 90 Hz` and `vk: swapchain 1280x800`, and on N for the `bound for its direct dispatches` line before the window. **Baseline:** this PR @ bda836f4 with the switch unset, which runs main's pipeline and dispatch for every program. - The runs were measured at bda836f4, on main 6d72ca93. - The branch is now rebased onto main 544229a2. The only difference is main's #1332 (which Vulkan driver the loader picks on Windows); the split's sources are byte-identical. So the numbers are labelled bda836f4 and were not re-measured. **Setup D (Brett's options, 100% render), handheld, on AC, 1280x800 at 90 Hz** | | Before, RX_VK_WG_SPLIT unset: this PR @ bda836f4 | After, RX_VK_WG_SPLIT=1: this PR @ bda836f4 | |---|---|---| | Kokiri: 27ee9883 GPU time (PASS_TIMING=2) | 1.871 ms | 1.360 ms (−27%) | | Kokiri: fps, median (range) | 18.25 (18.24–18.37) | 18.42 (18.36–18.45) | | Kokiri: 1% low, median (range) | 6.96 (6.34–7.30) | 6.74 (6.52–7.10) | | Kokiri: p50 frame time, median (range) | 53.6 ms (53.5–53.8) | 53.2 ms (53.1–53.4) | | Kokiri: p99 frame time, median (range) | 68.3 ms (64.9–80.4) | 69.7 ms (67.5–85.2) | | Kokiri: GPU time a frame (log), median (range) | 43.96 ms (43.96–44.11) | 43.53 ms (43.49–43.60) | | Kokiri: process CPU ms per bridge game frame, median (a sub-percent difference, not a change) | 152.76 | 152.33 | | Field: 27ee9883 GPU time (PASS_TIMING=2) | 0.871 ms | 0.604 ms (−31%) † | | Field: fps, median (range) | 24.26 (23.84–24.53), n=3 | 24.03 (23.82–24.24), n=2 | | Field: 1% low, median (range) | 8.24 (7.99–8.48) | 8.75 (8.70–8.81) | | Field: p50 frame time, median (range) | 38.5 ms (38.4–39.0) | 38.9 ms (38.7–39.1) | | Field: p99 frame time, median (range) | 86.6 ms (83.0–89.1) | 81.6 ms (78.1–85.1) | | Field: GPU time a frame (log), median (range) | 30.00 ms (29.90–30.07) | 29.46 ms (29.42–29.50) | | Field: power-limit flags (FPPT / SPPT, of 60 samples) | 45–50 / 29–33 | 54–55 / 47–49 | † D-TN's Field timing window also had 4 pipelines built in it. It is shown for completeness, not as a clean measurement. H's clean Field pair is the evidence for the Field. **Setup H (the Deck's Auto choice: Minimum, 50% render), handheld, on AC, 1280x800 at 90 Hz** | | Before, RX_VK_WG_SPLIT unset: this PR @ bda836f4 | After, RX_VK_WG_SPLIT=1: this PR @ bda836f4 | |---|---|---| | Kokiri: 27ee9883 GPU time (PASS_TIMING=2) | 1.836 ms | 1.317 ms (−28%) | | Kokiri: fps, median (range) | 24.15 (24.14–24.21) | 24.52 (24.50–24.59) | | Kokiri: 1% low, median (range) | 9.33 (9.14–9.46) | 9.91 (9.78–10.14) | | Kokiri: p50 frame time, median (range) | 40.5 ms (40.5–40.6) | 39.9 ms (39.8–40.0) | | Kokiri: p99 frame time, median (range) | 52.6 ms (50.3–54.4) | 55.9 ms (51.5–56.3) | | Kokiri: GPU time a frame (log), median (range) | 31.23 ms (31.16–31.24) | 30.63 ms (30.63–30.66) | | Kokiri: power-limit flags (FPPT / SPPT, of 90 samples) | 61–71 / 24–36 | 78–86 / 65–72 | | Field: 27ee9883 GPU time (PASS_TIMING=2) | 0.867 ms | 0.612 / 0.565 ms (two lines; −29% / −35%) | | Field: fps, median (range) | 29.44 (29.40–29.48), n=2 | 29.60 (27.85–30.41), n=3 | | Field: 1% low, median (range) | 9.59 (9.13–10.06) | 9.17 (9.13–9.55) | | Field: p50 frame time, median (range) | 31.9 ms (31.7–32.1) | 32.1 ms (30.9–34.0) | | Field: p99 frame time, median (range) | 70.3 ms (65.2–75.3) | 65.5 ms (57.6–69.5) | | Field: GPU time a frame (log), median (range) | 18.11 ms (17.73–18.49) | 17.96 ms (17.90–18.19) | Reading: - **Kokiri: the pass's saving and the frame's drop are of similar size.** This is consistent, not a full accounting: the pass times come from separate instrumented runs with few span records, under varying clocks and power. - D: the pass saves 0.51 ms, GPU time a frame drops 0.43 ms, and frame time drops 0.51 ms (at the fps medians). - H: the pass saves 0.52 ms, GPU time a frame drops 0.60 ms, and frame time drops 0.62 ms. - **The pass's time was nearly the same at 50% and 100% render resolution** (Kokiri, off: 1.836 and 1.871 ms). How its grid scales isn't measured here. - **The power-limit flags** are sampled flags, not durations, and don't show which resource limited the frame. That the extra throttling takes back part of the saving is a hypothesis, not something measured. - **p99 and 1% low are noisy** in both arms and both setups. Two 1% lows have disjoint observed ranges: H's Kokiri (9.14–9.46 against 9.78–10.14) and D's Field (7.99–8.48 against 8.70–8.81, n=3/2). From so few runs, neither shows a general improvement in the slowest frames. ``` vk: workgroup split (RX_VK_WG_SPLIT=1): on (RADV Van Gogh, compute at 32 lanes) vk: workgroup split of 27ee9883e2a20323 (variant 130c8d4842e475e5): 8x8x8 as 8x8x4, 15 loads given the IDs, 24 shared accesses of 9-word slots checked; bound for its direct dispatches from now on D-TO vk: m 1.871 ms 1.0 x 1870.7 us 1.0 items 0.00 MB dispatch prog 27ee9883e2a20323 (Kokiri) D-TN vk: m 1.360 ms 1.0 x 1360.1 us 1.0 items 0.00 MB dispatch prog 27ee9883e2a20323 (Kokiri) D-TO vk: m 0.871 ms 1.0 x 871.4 us 1.0 items 0.00 MB dispatch prog 27ee9883e2a20323 (Field) D-TN vk: m 0.604 ms 1.0 x 604.1 us 1.0 items 0.00 MB dispatch prog 27ee9883e2a20323 (Field; 4 pipelines built in this window) H-TO vk: m 1.836 ms 1.0 x 1835.9 us 1.0 items 0.00 MB dispatch prog 27ee9883e2a20323 (Kokiri) H-TN vk: m 1.317 ms 1.0 x 1316.6 us 1.0 items 0.00 MB dispatch prog 27ee9883e2a20323 (Kokiri) H-TO vk: m 0.867 ms 1.0 x 865.4 us 1.0 items 0.00 MB dispatch prog 27ee9883e2a20323 (Field) H-TN vk: m 0.612 ms 1.0 x 611.3 us 1.0 items 0.00 MB dispatch prog 27ee9883e2a20323 (Field) H-TN vk: m 0.565 ms 1.0 x 565.3 us 1.0 items 0.00 MB dispatch prog 27ee9883e2a20323 (Field) D-1O frames=840 fps=18.25 p50=53.6 p95=58.8 p99=80.4 max=172.9 over100=8 (Kokiri; each run's summary line is in docs/tested) ``` **Side note: docked, on AC, 1280x720 at 60 Hz, setup H's options.** These are earlier runs, not the measurement above. They were taken before the Deck was undocked at 21:38:30; H-3N, taken after the undock, is left out. - Kokiri, off n=3 (H-1O, H-2O, H-3O) and on n=2 (H-1N, H-2N). - fps 24.46 (24.41–24.56) → 24.86 (24.84–24.88), and GPU time a frame 30.68 (30.62–30.70) → 30.12 ms (30.10–30.13). Both have their ranges apart. **Offline, the Deck's Mesa:** RADV 26.2.0-devel git-5c65d4fc31 (steamos-26.05), VANGOGH through amdgpu drm-shim, with RADV_DEBUG=shaderstats. The split module was made at bda836f4; the split's sources are byte-identical at this head. | 27ee9883 | VGPRs | waves / SIMD | spills | LDS | code size | |---|---|---|---|---|---| | original (8x8x8) | 128 | 8 | 0 | 18432 | 15928 | | split (8x8x4) | 80 | 12 | 0 | 18432 | 16044 | ### Flags and settings Two new switches, both off by default, plus a debug value of the first. | Flag | What it is | Before this PR | After this PR | Effect | Helps / doesn't help / untested | Recommendation | |---|---|---|---|---|---|---| | `RX_VK_WG_SPLIT` | attempt | none | unset: off | `1`: the fog lighting pass in 8x8x4 workgroups for its direct dispatches, on a RADV driver whose device is VANGOGH, with compute at 32 lanes (no Mesa version gate) | Saves GPU time in the pass on a Deck OLED with Mesa 26.2.0-devel (above). Untested on the Deck LCD, on other Mesa versions and on other RADV GPUs | Off until more Deck runs agree | | `RX_VK_WG_SPLIT=<hex,…>` | debug value | none | — | Splits the programs whose keys start with those prefixes, under the same refusal rules. This widens the risk beyond the fog pass | Debug only: nothing beyond 27ee9883 was checked in the game | Debug only | | `RX_VK_WG_SPLIT_ANY_GPU` | diag | none | unset: off | `1` allows the split on any GPU whose compute runs 32 lanes, to look at the picture there; logged as "a debug run, not a measurement". This widens the risk beyond RADV Van Gogh | Debug only: the lane mapping was shown only on RADV Van Gogh | Off: debug only | ## Testing **Tested:** - **The game-free check:** `tests/test_wg_split.py`, with `tests/wg_split/wg_split_test.cpp` and `fixture.spvasm`, a module shaped like the fog's pass. - It interprets the rewritten entry function's ID and shared-index arithmetic for every invocation of an asymmetric 3x2x5 grid at k = 2, 4 and 8, and checks that each physical invocation gets the same logical (group, local) tuple, address and count. - Three wrong remaps are each caught. - **Refusals:** - wider barriers and group operations; - workgroup atomics, and a pointer that escapes; - overlapping or overflowing slots, and a stride that would wrap; - a shift by 31, and a coefficient past 2^31 cancelled later; - differing and changing phis. - The slot, choice and bind in each state (pending, ready, declined), across direct, indirect and direct; and the limit cases (an even limit hit exactly, z·k past 32 bits). - spirv-val on the original and the k = 2, 4 and 8 modules. - **What "fails on main" means:** the test fails to build on main, because main has no `gpu/vk/wg_split.cpp`. The correctness evidence is the emitted-op mutations above. - **Not covered:** the slot tests are sequential helper tests, not threaded or TSan. The interpreter runs the index arithmetic only: not subgroup ops, loads and stores, the full control flow or called functions. That is why the pictures and the A/B below were run. - **At bda836f4, recorded in one run** (its output is in the evidence comment below; the split's sources are byte-identical at this head): - the test with spirv-val (SPIRV-Tools v2025.1) on Linux, g++ 13; - the test under ASan/UBSan; - the rewrite of the real 27ee9883 and 0a55781f under ASan/UBSan; - spirv-val of the real 27ee9883, original and split (Vulkan 1.3, scalar block layout); - the Mesa 26.2 stats above. - **At this head (rebased on 544229a2; tested at 4d619558, which differs from it only in the docs/tested rows):** `tests/test_wg_split.py` passes on Windows and on Linux (with spirv-val), and the renderer builds (`dev_build.py vk`, 0 failed, exports checked). - **The full suite at this head:** `python -I -B tests/run_all.py` on Windows; the receipts are in the evidence comment below. - Branch: 577 passed, 48 failed. Main 544229a2: 577 passed, 47 failed. - The 47 failures are common to both, listed in the evidence comment below. One of them is `test_longshot_ctl`, which is flaky: run alone, it passes on the branch and fails on main. - Branch only: `test_gpu_hang_watch.py`, a 2 s timing test that failed under a 24-way parallel run. Run alone, it passes on the branch and on main. - `test_wg_split` is the branch's extra test, and it passes. - The earlier run at bda836f4 against main 6d72ca93 matched this: 46 common failures, plus `test_longshot_ctl` on the branch only. - **The lane probe:** a game-free C and Vulkan program, run on the Deck (19:36:42, RADV VANGOGH, Mesa 26.2.0-devel git-5c65d4fc31, compute pinned to 32). - For 8x8x8 on a 3x2x5 grid and 8x8x4 on a 3x2x10 grid: 480 subgroups of 32 each, 0 bad records, 0 subgroups spanning z planes, 0 shuffle-z mismatches, 0 partial subgroups, lane = index%32 everywhere; rc=0. - After review, the probe's PASS also requires a lane offset of 0 and a size of 32. That rebuilt binary (sha256 9068d6fa…) was rerun on the PC only (RTX 5090, PASS). The Deck run above, from the earlier binary, already showed both. - Its source, `lane_probe.c` and `probe.comp`, and its Deck output are in the evidence comment below. - **Pictures:** - **Windows, RTX 5090, driver 621.14:** headless, `RX_VK_WG_SPLIT=1` + `RX_VK_WG_SPLIT_ANY_GPU=1` (debug). - Off, on, off at the same spots: Hyrule Field and Kokiri Forest, at arrival +600 and +900 frames. - The on run's ready line came before every shot. - Off vs on shows comparable, overlapping run-to-run variability; some off-vs-on pairs differ more than the off-vs-off pair (Field +600: 4.19 against 3.50). The differences are animation, and no fog brightness shift shows. - **The Deck, handheld on AC, setup H** (bda836f4, `RX_VK_WG_SPLIT` unset vs `=1`): no obvious fog change by eye. The side-by-side of H-2O (unset) and H-2N (`=1`), Kokiri and the Field, is in the evidence comment below, with the PC pair. - Over 3 runs each, the off-vs-on pixel differences overlap the within-arm differences without being contained in them. Mean absolute difference: Kokiri 3.78–5.18 against 3.51–5.02 within an arm; Field 3.68–5.47 against 3.57–5.13. - Mean brightness overlaps between the arms. - Pose, camera and animation differ between runs (the fairy, leaves, Link's idle), so this is not a deterministic picture-equality test. **Not tested:** the Deck LCD; other Mesa versions (the gate is the device, not the version); other RADV GPUs (the switch declines there); cold translations (they decline until the next start, by design). **Must be tested:** - Steam Deck LCD, SteamOS 3.9, Mesa 26.x (Test Farm tag steamdeck): before merge, so the PR stays WIP until then (or until a maintainer narrows this). Start from the repo save kokiri-start, warm up once, then run Kokiri Forest standing and Hyrule Field turning with `RX_VK_WG_SPLIT` unset and `=1` in turn, 3 runs each, plus one `RX_VK_PASS_TIMING=2` run per arm. Expecting the log's "workgroup split of 27ee9883… bound for its direct dispatches from now on" line before each `=1` window, no "declined" line, the fog looking the same, and a lower 27ee9883 time under `RX_VK_PASS_TIMING=2` (Test Farm: @Beanstalk steamdeck, the fog split off vs on in Kokiri Forest and Hyrule Field on an LCD Deck; or by hand: the same, by anyone with an LCD Deck) ## Risks **What could break:** only with the switch on, on a RADV Van Gogh driver: the fog's lighting pass. A wrong remap would show as blocky or shifted fog. The rewrite refuses the program, and today's pipeline stays, when it can't show that no invocation reaches outside its new group. The lane assumption was shown on one Mesa (26.2.0-devel); another Mesa version is untested. The debug values (hex prefixes, `RX_VK_WG_SPLIT_ANY_GPU`) widen the risk to other programs and GPUs. **Turning it off:** unset `RX_VK_WG_SPLIT`, which is the default. **Compatibility:** nothing changes for a player who doesn't set the switch. - The translated modules, the program cache's format and stamp and the vk ABI are unchanged, and nothing is translated or prepared again. - With the switch on, one extra shader module and pipeline is made at run time. The driver's pipeline cache can keep it as an entry of its own. ## Issues See https://git.magicbean.dev/MagicBeanDev/Longshot/issues/373 (an option to reduce the fog; this PR makes it cheaper instead). ## Checklist ### Everyone - [x] One topic. - [x] The title is a Conventional Commit (`type(scope): description`) of the status line's type. - [x] Why gives the reason plainly and links the issues that fit (the tracker searched). - [x] Measurements has a verdict, or one line saying why there is nothing to measure. - [x] Testing names what was tested, what wasn't, and what must still be tested, where and how. - [x] Rebased on current `main`. (544229a2; re-check before opening) - [x] `python -I -B tests/run_all.py` passes (any failure that also fails on `main` is named). (The 47 common failures listed; the two timing-flaky tests explained.) - [x] No game files, dumps, build output or generated files. (`tests/wg_split/fixture_spv.h` is generated from the hand-written `fixture.spvasm`, with its SHA-256 checked; it is not game-derived.) - [x] `third_party/` untouched (a new folder with a `PROVENANCE.md` instead). - [x] The license is OK: GPL-3.0-or-later, nothing incompatible and nothing decompiled. ### When it applies - [x] AI assistance recorded with `Assisted-by:` / `Co-Authored-By:` trailers. - [x] Numbers from the same PC, scene and settings, at least 2 interleaved runs each, the median reported. - [x] Switches documented (docs/development.md; the README for a menu row) and shipped at their recommended value. (rows in `gpu/vk/switches.def`) - [x] Screenshots of the same spot, captioned like the Measurements columns, for a change that shows on screen. (In the evidence comment: the Deck pair, captioned "Before: bda836f4, RX_VK_WG_SPLIT unset (H-2O)" and "After: bda836f4, RX_VK_WG_SPLIT=1 (H-2N)".) - [x] A file in `docs/tested/` for each new measured run (format in `docs/tested-configurations.md`). (12 files: D and H, off and on, runs 1–3; commit e4a1471d.) - [x] A game-free test added for the fix, failing on `main` and passing here. (It fails to build on main.) - [x] A check under `tests/` that reads changed source updated in this PR, not weakened. (`test_intel_quick_compute.py`, plus stubs in the two program-cache tests) - [x] Host code builds on both Windows (llvm-mingw) and Linux, with UTF-8 paths. - [x] Docs updated.
tests/test_wg_split.py builds tests/wg_split/wg_split_test.cpp with gpu/vk/wg_split.cpp (RX_VK_WG_SPLIT, next commit) and runs
the split module's ID and shared-index arithmetic for every invocation of a 3x2x5 grid at k = 2, 4 and 8, three wrong remaps,
the refusals (wider barriers and group operations, workgroup atomics and escaping pointers, overlapping or overflowing slots,
differing or changing phis, coefficients past 2^31), the renderer's slot, choice and bind in each state, and the dispatch's
choice at the limits; spirv-val on the modules where the PC has it. Fails here: there is no split.

Co-authored-by: Copilot App <[email protected]>
Mesa 26.x's ACO trades waves for registers and rounds the waves to whole workgroups: on the Deck's RADV (steamos-26.05) the
ground fog's lighting pass 27ee9883 (8x8x8, 512 invocations) gets 128 VGPRs and 8 waves. As 8x8x4 groups, twice as many in z,
it compiles to 80 VGPRs and 12 waves, no spills (offline, the Deck's Mesa through drm-shim).

gpu/vk/wg_split.cpp rewrites a listed compute program's SPIR-V: LocalSize z / k, WorkgroupId.z / k and LocalInvocationId.z +
(WorkgroupId.z % k) * (z / k); it refuses unless the module shows no invocation meets one outside its new group (no barrier,
memory barrier or group operation wider than a subgroup; workgroup memory only each invocation's own slot, its index followed
with checked arithmetic; no other ID built-in). vk_draw.cpp makes its pipeline once, its own (wgsplit::Slot: k fixed when made,
the pipeline published before ready, claimed 0 -> 1 only), from the SPIR-V the program cache read, and binds it for direct
dispatches only (vkCmdDispatch(x, y, z * k)); indirect dispatches and everything else run as before. Only on RADV Van Gogh
with compute pinned to 32 lanes: the split relies on each wave32 sitting inside one 64-invocation z plane, as the 8x8x8
pipeline already relies on, which a game-free probe showed there for both shapes. No translator change: the program cache
and every module are as before; with the switch off the dispatch is unchanged.

Co-authored-by: Copilot App <[email protected]>
docs(tested): the fog lighting pass's workgroup split on a Steam Deck OLED, off vs on, at the user's settings and at Auto
Some checks are pending
CI / Conventional Commit title (pull_request) Blocked by required conditions
CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Blocked by required conditions
CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Blocked by required conditions
CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Blocked by required conditions
CI / Workflow lint (actionlint) (pull_request) Blocked by required conditions
Guide screenshots / Guide images and local UI source review (pull_request) Blocked by required conditions
Third-party parts / As their notes say (pull_request) Blocked by required conditions
Third-party parts / The same as upstream (pull_request) Blocked by required conditions
Website / Website on Cloudflare (pull_request) Blocked by required conditions
Website / Website on Cloudflare (pull_request_target) Has been skipped
PR description / Description follows the template (pull_request_target) Successful in 4s
e4a1471dfa
Twelve runs of brett-d/fog-wg-split @ bda836f4 (RX_VK_WG_SPLIT unset vs =1), handheld on AC at 1280x800 and 90 Hz:
3 interleaved runs each of each arm, setups D (the user's options, 100% render) and H (Auto: Minimum, 50%), Kokiri Forest
standing and Hyrule Field turning. Two Field windows in which the game built pipelines are marked as left out.

Co-authored-by: Copilot App <[email protected]>
Author
Contributor

Tested in game: Steam Deck OLED, SteamOS 3.9.5, RADV VANGOGH (Mesa 26.2.0-devel git-5c65d4fc31), handheld on AC at 1280x800 and 90 Hz, at the user's settings and at Auto (Minimum): Kokiri Forest and Hyrule Field, RX_VK_WG_SPLIT unset vs =1, 3 interleaved runs each at bda836f4. The split was bound before every window and never declined, the fog looked the same, and the lighting pass 27ee9883 took 27-28% less GPU time in Kokiri. Also Windows, RTX 5090, driver 621.14, with the debug RX_VK_WG_SPLIT_ANY_GPU=1: the same spots, no visible change.

Tested in game: Steam Deck OLED, SteamOS 3.9.5, RADV VANGOGH (Mesa 26.2.0-devel git-5c65d4fc31), handheld on AC at 1280x800 and 90 Hz, at the user's settings and at Auto (Minimum): Kokiri Forest and Hyrule Field, RX_VK_WG_SPLIT unset vs =1, 3 interleaved runs each at bda836f4. The split was bound before every window and never declined, the fog looked the same, and the lighting pass 27ee9883 took 27-28% less GPU time in Kokiri. Also Windows, RTX 5090, driver 621.14, with the debug RX_VK_WG_SPLIT_ANY_GPU=1: the same spots, no visible change.
Author
Contributor

Evidence for the description: screenshots, the lane probe, the receipts at bda836f4, and the suite's failure list.

Pictures, Steam Deck OLED, handheld on AC, setup H. Left: Before, RX_VK_WG_SPLIT unset: this PR @ bda836f4 (H-2O). Right: After, RX_VK_WG_SPLIT=1: this PR @ bda836f4 (H-2N). Top: Kokiri Forest; bottom: Hyrule Field.

Deck: before (H-2O, unset) left, after (H-2N, =1) right, bda836f4

Pictures, Windows, RTX 5090, driver 621.14. This is a debug run with RX_VK_WG_SPLIT_ANY_GPU=1, not a measurement. Taken headless at arrival +900 frames, at bda836f4.

PC Field, Before, RX_VK_WG_SPLIT unset: this PR @ bda836f4
PC Field, After, RX_VK_WG_SPLIT=1 (+ANY_GPU): this PR @ bda836f4
PC Kokiri, Before, RX_VK_WG_SPLIT unset: this PR @ bda836f4
PC Kokiri, After, RX_VK_WG_SPLIT=1 (+ANY_GPU): this PR @ bda836f4

The lane probe's output. The Deck run used the first binary. The PC rerun used the revised binary, 9068d6fa…, whose PASS also requires lane = index%32 and size 32.

lane_probe on the Steam Deck (relayed by the coordinator from the Deck agent), 2026-11-16 19:36:42:
device: AMD Custom GPU 0932 (RADV VANGOGH); Mesa 26.2.0-devel (git-5c65d4fc31); compute pinned to 32
8x8x8, 3x2x5 groups: subgroup size 32, 480 subgroups; records bad 0, subgroups spanning z planes 0, shuffle z != own z 0, partial subgroups 0, lane != index%32 0: PASS
8x8x4, 3x2x10 groups: subgroup size 32, 480 subgroups; records bad 0, subgroups spanning z planes 0, shuffle z != own z 0, partial subgroups 0, lane != index%32 0: PASS
PASS (rc=0)

PC reference (RTX 5090, NVIDIA 621.14), 2026-11-16 ~19:30: the same two PASS lines, rc=0.
binary sha256 dc6af28202f7045ecf8b596603ade2f959ec0d4bb07f49e31e2549b0d5fa738e (lane_probe.c, probe.comp, probe_spv.h beside it)

After GPT-6.1's review (2026-11-16 ~20:20): PASS now also requires lane == index%32 and subgroup size 32 (both already 0 / 32 in
the Deck's run above, so its verdict stands). New binary sha256 9068d6fa7b3c9fabbdb413d4c9c0a955078ab9ccff4c660aafeb95574bbba944;
PC rerun with it: both shapes PASS, rc 0.
probe.comp
#version 450
// Lane-mapping probe for the 27ee9883 workgroup split: which invocations share a wave32, and what the ballot/shuffle z
// idiom of 27ee9883 returns. Game-free. Built with -DLZ=8 and -DLZ=4.
#extension GL_KHR_shader_subgroup_basic : require
#extension GL_KHR_shader_subgroup_ballot : require
#extension GL_KHR_shader_subgroup_shuffle : require
layout(local_size_x = 8, local_size_y = 8, local_size_z = LZ) in;
layout(std430, set = 0, binding = 0) buffer Out { uint d[]; };
void main() {
    uint idx = gl_LocalInvocationIndex;
    uint g = gl_WorkGroupID.x + gl_NumWorkGroups.x * (gl_WorkGroupID.y + gl_NumWorkGroups.y * gl_WorkGroupID.z);
    uint base = (g * (8u * 8u * uint(LZ)) + idx) * 8u;
    // (27ee9883's idiom: the first active lane's local z, read by a shuffle)
    uvec4 b = subgroupBallot(true);
    uint first = uint(findMSB(b.x & (-b.x))) & 31u;
    uint zf = subgroupShuffle(gl_LocalInvocationID.z, first);
    d[base + 0] = idx;
    d[base + 1] = gl_SubgroupID;
    d[base + 2] = gl_SubgroupInvocationID;
    d[base + 3] = gl_LocalInvocationID.z;
    d[base + 4] = zf;
    d[base + 5] = subgroupBallotBitCount(b);
    d[base + 6] = gl_SubgroupSize;
    d[base + 7] = 0xC0DE0000u | g;
}
lane_probe.c
// Lane-mapping probe for the 27ee9883 workgroup split (Longshot fog work). Game-free: no game files, no game.
// Dispatches an 8x8x8 and an 8x8x4 compute shader pinned to 32 lanes as the renderer pins compute (pin_subgroup: required
// subgroup size 32, no full-subgroups flag since X = 8), and checks, for every subgroup of every workgroup:
//   - all its invocations have the same LocalInvocationID.z (each wave32 sits inside one 64-invocation z plane),
//   - 27ee9883's ballot/shuffle idiom returns each invocation's own z,
//   - the subgroup is full (32 active lanes), its size is 32, and lane == LocalInvocationIndex % 32.
// usage: lane_probe [device-name-substring]    exit 0: PASS on both shapes; 1: FAIL; 2: no usable device
#define VK_NO_PROTOTYPES
#include <vulkan/vulkan.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#ifdef _WIN32
#include <windows.h>
#else
#include <dlfcn.h>
#endif
#include "probe_spv.h"

#define CK(x) do { VkResult r_ = (x); if (r_ != VK_SUCCESS) { fprintf(stderr, "%s failed: %d\n", #x, (int)r_); exit(2); } } while (0)
#define L(name) PFN_##name name = (PFN_##name)gipa(inst, #name)
#define LD(name) PFN_##name name = (PFN_##name)gdpa(dev, #name)

int main(int argc, char** argv) {
#ifdef _WIN32
    HMODULE lib = LoadLibraryA("vulkan-1.dll");
    PFN_vkGetInstanceProcAddr gipa = lib ? (PFN_vkGetInstanceProcAddr)GetProcAddress(lib, "vkGetInstanceProcAddr") : NULL;
#else
    void* lib = dlopen("libvulkan.so.1", RTLD_NOW);
    PFN_vkGetInstanceProcAddr gipa = lib ? (PFN_vkGetInstanceProcAddr)dlsym(lib, "vkGetInstanceProcAddr") : NULL;
#endif
    if (!gipa) { fprintf(stderr, "no Vulkan loader\n"); return 2; }
    VkInstance inst = VK_NULL_HANDLE;
    L(vkCreateInstance);
    VkApplicationInfo app = {VK_STRUCTURE_TYPE_APPLICATION_INFO, 0, "lane_probe", 1, 0, 0, VK_API_VERSION_1_3};
    VkInstanceCreateInfo ici = {VK_STRUCTURE_TYPE_INSTANCE_CREATE_INFO, 0, 0, &app};
    CK(vkCreateInstance(&ici, 0, &inst));
    L(vkEnumeratePhysicalDevices); L(vkGetPhysicalDeviceProperties2); L(vkGetPhysicalDeviceFeatures2);
    L(vkGetPhysicalDeviceMemoryProperties); L(vkCreateDevice); L(vkGetDeviceProcAddr);
    PFN_vkGetDeviceProcAddr gdpa = vkGetDeviceProcAddr;
    uint32_t np = 16; VkPhysicalDevice pds[16]; CK(vkEnumeratePhysicalDevices(inst, &np, pds));
    VkPhysicalDevice pd = VK_NULL_HANDLE;
    VkPhysicalDeviceVulkan13Properties p13 = {VK_STRUCTURE_TYPE_PHYSICAL_DEVICE_VULKAN_1_3_PROPERTIES};
    VkPhysicalDeviceDriverProperties drv = {VK_STRUCTURE_TYPE_PHYSICAL_DEVICE_DRIVER_PROPERTIES, &p13};
    VkPhysicalDeviceProperties2 props = {VK_STRUCTURE_TYPE_PHYSICAL_DEVICE_PROPERTIES_2, &drv};
    for (uint32_t i = 0; i < np && !pd; i++) {
        vkGetPhysicalDeviceProperties2(pds[i], &props);
        if (props.properties.deviceType == VK_PHYSICAL_DEVICE_TYPE_CPU) continue;
        if (argc > 1 && !strstr(props.properties.deviceName, argv[1])) continue;
        pd = pds[i];
    }
    if (!pd) { fprintf(stderr, "no device%s%s\n", argc > 1 ? " matching " : "", argc > 1 ? argv[1] : ""); return 2; }
    printf("device: %s; driver: %s %s\n", props.properties.deviceName, drv.driverName, drv.driverInfo);
    VkPhysicalDeviceVulkan13Features f13 = {VK_STRUCTURE_TYPE_PHYSICAL_DEVICE_VULKAN_1_3_FEATURES};
    VkPhysicalDeviceFeatures2 f2 = {VK_STRUCTURE_TYPE_PHYSICAL_DEVICE_FEATURES_2, &f13};
    vkGetPhysicalDeviceFeatures2(pd, &f2);
    const int pin = f13.subgroupSizeControl && (p13.requiredSubgroupSizeStages & VK_SHADER_STAGE_COMPUTE_BIT) &&
                    p13.minSubgroupSize <= 32 && p13.maxSubgroupSize >= 32;
    printf("subgroup sizes %u-%u, compute pinned to 32: %s\n", p13.minSubgroupSize, p13.maxSubgroupSize, pin ? "yes" : "no");
    VkPhysicalDeviceVulkan13Features en13 = {VK_STRUCTURE_TYPE_PHYSICAL_DEVICE_VULKAN_1_3_FEATURES};
    en13.subgroupSizeControl = pin ? VK_TRUE : VK_FALSE;
    float prio = 1;
    VkDeviceQueueCreateInfo qci = {VK_STRUCTURE_TYPE_DEVICE_QUEUE_CREATE_INFO, 0, 0, 0, 1, &prio};   // (family 0 has compute on RADV/NVIDIA)
    VkDeviceCreateInfo dci = {VK_STRUCTURE_TYPE_DEVICE_CREATE_INFO, &en13, 0, 1, &qci};
    VkDevice dev; CK(vkCreateDevice(pd, &dci, 0, &dev));
    LD(vkGetDeviceQueue); LD(vkCreateBuffer); LD(vkGetBufferMemoryRequirements); LD(vkAllocateMemory); LD(vkBindBufferMemory);
    LD(vkMapMemory); LD(vkCreateShaderModule); LD(vkCreateDescriptorSetLayout); LD(vkCreatePipelineLayout);
    LD(vkCreateComputePipelines); LD(vkCreateDescriptorPool); LD(vkAllocateDescriptorSets); LD(vkUpdateDescriptorSets);
    LD(vkCreateCommandPool); LD(vkAllocateCommandBuffers); LD(vkBeginCommandBuffer); LD(vkCmdBindPipeline);
    LD(vkCmdBindDescriptorSets); LD(vkCmdDispatch); LD(vkEndCommandBuffer); LD(vkQueueSubmit); LD(vkQueueWaitIdle);
    LD(vkCmdFillBuffer); LD(vkCmdPipelineBarrier);
    VkQueue q; vkGetDeviceQueue(dev, 0, 0, &q);
    VkPhysicalDeviceMemoryProperties mp; vkGetPhysicalDeviceMemoryProperties(pd, &mp);

    const uint32_t GX = 3, GY = 2, GZ8 = 5;   // (asymmetric grid; the 8x8x4 shape runs twice the groups in z)
    const VkDeviceSize words = (VkDeviceSize)GX * GY * GZ8 * 512 * 8;
    VkBufferCreateInfo bci = {VK_STRUCTURE_TYPE_BUFFER_CREATE_INFO, 0, 0, words * 4,
                              VK_BUFFER_USAGE_STORAGE_BUFFER_BIT | VK_BUFFER_USAGE_TRANSFER_DST_BIT};
    VkBuffer buf; CK(vkCreateBuffer(dev, &bci, 0, &buf));
    VkMemoryRequirements mr; vkGetBufferMemoryRequirements(dev, buf, &mr);
    uint32_t mt = UINT32_MAX;
    const VkMemoryPropertyFlags want = VK_MEMORY_PROPERTY_HOST_VISIBLE_BIT | VK_MEMORY_PROPERTY_HOST_COHERENT_BIT;
    for (uint32_t i = 0; i < mp.memoryTypeCount; i++)
        if ((mr.memoryTypeBits & (1u << i)) && (mp.memoryTypes[i].propertyFlags & want) == want) { mt = i; break; }
    if (mt == UINT32_MAX) { fprintf(stderr, "no host-visible memory\n"); return 2; }
    VkMemoryAllocateInfo mai = {VK_STRUCTURE_TYPE_MEMORY_ALLOCATE_INFO, 0, mr.size, mt};
    VkDeviceMemory mem; CK(vkAllocateMemory(dev, &mai, 0, &mem)); CK(vkBindBufferMemory(dev, buf, mem, 0));
    uint32_t* out; CK(vkMapMemory(dev, mem, 0, VK_WHOLE_SIZE, 0, (void**)&out));

    VkDescriptorSetLayoutBinding lb = {0, VK_DESCRIPTOR_TYPE_STORAGE_BUFFER, 1, VK_SHADER_STAGE_COMPUTE_BIT, 0};
    VkDescriptorSetLayoutCreateInfo dlci = {VK_STRUCTURE_TYPE_DESCRIPTOR_SET_LAYOUT_CREATE_INFO, 0, 0, 1, &lb};
    VkDescriptorSetLayout dsl; CK(vkCreateDescriptorSetLayout(dev, &dlci, 0, &dsl));
    VkPipelineLayoutCreateInfo plci = {VK_STRUCTURE_TYPE_PIPELINE_LAYOUT_CREATE_INFO, 0, 0, 1, &dsl};
    VkPipelineLayout pl; CK(vkCreatePipelineLayout(dev, &plci, 0, &pl));
    VkDescriptorPoolSize ps = {VK_DESCRIPTOR_TYPE_STORAGE_BUFFER, 1};
    VkDescriptorPoolCreateInfo dpci = {VK_STRUCTURE_TYPE_DESCRIPTOR_POOL_CREATE_INFO, 0, 0, 1, 1, &ps};
    VkDescriptorPool dp; CK(vkCreateDescriptorPool(dev, &dpci, 0, &dp));
    VkDescriptorSetAllocateInfo dsai = {VK_STRUCTURE_TYPE_DESCRIPTOR_SET_ALLOCATE_INFO, 0, dp, 1, &dsl};
    VkDescriptorSet ds; CK(vkAllocateDescriptorSets(dev, &dsai, &ds));
    VkDescriptorBufferInfo dbi = {buf, 0, VK_WHOLE_SIZE};
    VkWriteDescriptorSet wds = {VK_STRUCTURE_TYPE_WRITE_DESCRIPTOR_SET, 0, ds, 0, 0, 1, VK_DESCRIPTOR_TYPE_STORAGE_BUFFER, 0, &dbi};
    vkUpdateDescriptorSets(dev, 1, &wds, 0, 0);
    VkCommandPoolCreateInfo cpci = {VK_STRUCTURE_TYPE_COMMAND_POOL_CREATE_INFO, 0, 0, 0};
    VkCommandPool cp; CK(vkCreateCommandPool(dev, &cpci, 0, &cp));

    int fail = 0;
    for (int shape = 0; shape < 2; shape++) {
        const uint32_t lz = shape ? 4 : 8, gz = shape ? GZ8 * 2 : GZ8, inv = 64 * lz;
        const uint32_t* code = shape ? probe_lz4 : probe_lz8;
        const size_t code_size = shape ? sizeof probe_lz4 : sizeof probe_lz8;
        VkShaderModuleCreateInfo smci = {VK_STRUCTURE_TYPE_SHADER_MODULE_CREATE_INFO, 0, 0, code_size, code};
        VkShaderModule sm; CK(vkCreateShaderModule(dev, &smci, 0, &sm));
        VkPipelineShaderStageRequiredSubgroupSizeCreateInfo rss = {
            VK_STRUCTURE_TYPE_PIPELINE_SHADER_STAGE_REQUIRED_SUBGROUP_SIZE_CREATE_INFO, 0, 32};
        VkComputePipelineCreateInfo cci = {VK_STRUCTURE_TYPE_COMPUTE_PIPELINE_CREATE_INFO, 0, 0,
                                           {VK_STRUCTURE_TYPE_PIPELINE_SHADER_STAGE_CREATE_INFO, pin ? &rss : 0, 0,
                                            VK_SHADER_STAGE_COMPUTE_BIT, sm, "main", 0},
                                           pl, 0, 0};
        VkPipeline pipe; CK(vkCreateComputePipelines(dev, 0, 1, &cci, 0, &pipe));
        VkCommandBufferAllocateInfo cbai = {VK_STRUCTURE_TYPE_COMMAND_BUFFER_ALLOCATE_INFO, 0, cp, VK_COMMAND_BUFFER_LEVEL_PRIMARY, 1};
        VkCommandBuffer cb; CK(vkAllocateCommandBuffers(dev, &cbai, &cb));
        VkCommandBufferBeginInfo cbbi = {VK_STRUCTURE_TYPE_COMMAND_BUFFER_BEGIN_INFO};
        CK(vkBeginCommandBuffer(cb, &cbbi));
        vkCmdFillBuffer(cb, buf, 0, VK_WHOLE_SIZE, 0xFFFFFFFFu);
        VkMemoryBarrier mb = {VK_STRUCTURE_TYPE_MEMORY_BARRIER, 0, VK_ACCESS_TRANSFER_WRITE_BIT, VK_ACCESS_SHADER_WRITE_BIT};
        vkCmdPipelineBarrier(cb, VK_PIPELINE_STAGE_TRANSFER_BIT, VK_PIPELINE_STAGE_COMPUTE_SHADER_BIT, 0, 1, &mb, 0, 0, 0, 0);
        vkCmdBindPipeline(cb, VK_PIPELINE_BIND_POINT_COMPUTE, pipe);
        vkCmdBindDescriptorSets(cb, VK_PIPELINE_BIND_POINT_COMPUTE, pl, 0, 1, &ds, 0, 0);
        vkCmdDispatch(cb, GX, GY, gz);
        VkMemoryBarrier hb = {VK_STRUCTURE_TYPE_MEMORY_BARRIER, 0, VK_ACCESS_SHADER_WRITE_BIT, VK_ACCESS_HOST_READ_BIT};
        vkCmdPipelineBarrier(cb, VK_PIPELINE_STAGE_COMPUTE_SHADER_BIT, VK_PIPELINE_STAGE_HOST_BIT, 0, 1, &hb, 0, 0, 0, 0);
        CK(vkEndCommandBuffer(cb));
        VkSubmitInfo si = {VK_STRUCTURE_TYPE_SUBMIT_INFO, 0, 0, 0, 0, 1, &cb};
        CK(vkQueueSubmit(q, 1, &si, VK_NULL_HANDLE)); CK(vkQueueWaitIdle(q));

        const uint32_t groups = GX * GY * gz;
        long bad_tag = 0, mixed_z = 0, shuf = 0, partial = 0, lane_off = 0, subgroups = 0, sgsize = 0;
        for (uint32_t g = 0; g < groups; g++) {
            const uint32_t nsg = inv / 32 + 1;
            int sg_z[64]; int sg_n[64];
            for (uint32_t s = 0; s < nsg && s < 64; s++) sg_z[s] = -1, sg_n[s] = 0;
            for (uint32_t i = 0; i < inv; i++) {
                const uint32_t* r = out + ((size_t)g * inv + i) * 8;
                if (r[7] != (0xC0DE0000u | g) || r[0] != i) { bad_tag++; continue; }
                sgsize = r[6];
                const uint32_t s = r[1];
                if (s >= 64) { bad_tag++; continue; }
                if (sg_z[s] < 0) sg_z[s] = (int)r[3], subgroups++;
                else if (sg_z[s] != (int)r[3]) mixed_z++;
                sg_n[s]++;
                if (r[4] != r[3]) shuf++;
                if (r[5] != r[6]) partial++;
                if (r[6] == 32 && (r[2] != i % 32 || s != i / 32)) lane_off++;
            }
        }
        const int ok = !bad_tag && !mixed_z && !shuf && !partial && !lane_off && sgsize == 32;
        printf("8x8x%u, %ux%ux%u groups: subgroup size %ld, %ld subgroups; records bad %ld, subgroups spanning z planes %ld, "
               "shuffle z != own z %ld, partial subgroups %ld, lane != index%%32 %ld: %s\n",
               lz, GX, GY, gz, sgsize, subgroups, bad_tag, mixed_z, shuf, partial, lane_off, ok ? "PASS" : "FAIL");
        fail |= !ok;
    }
    printf("%s\n", fail ? "FAIL" : "PASS");
    return fail ? 1 : 0;
}
Receipts at bda836f4: source hashes, the test with spirv-val, ASan/UBSan, spirv-val of the real module, Mesa 26.2 stats
head: bda836f4f4dc77b8050460af51da985868eae7e4
6b4623c31556c62a16ea2e3480308b3764a1b910f1b42e75e3919cc48aa441a9  gpu/vk/wg_split.cpp
b163318d0f535566cd2462a03e3a19bc65852a4f3d8361ae68d51ad1fd543bac  gpu/vk/wg_split.h
7b8a87aef00dee6cfe52a4f210a84c5d05c152633a67e712bf4e5aad65e11d59  tests/wg_split/wg_split_test.cpp
0491762e006db16830ae0bbc9f9419935519ac5e9ff1586bbd3763e65748cbb8  tests/wg_split/fixture_spv.h
8613c5ef4a82ba29b6ec02443e93fec72668d230159a72f4ea11936eb9561b88  tests/test_wg_split.py

== spirv-val: SPIRV-Tools v2025.1 unknown hash, 2026-05-13T06:03:38+00:00
== tests/test_wg_split.py (Linux, g++ 13)
ok: listed: unset/empty/0 none, 1 the fog's pass alone, hex prefixes in any case
PASS
spirv-val: the 8x8x8 fixture module is valid
spirv-val: the 2-way module is valid
spirv-val: the 4-way module is valid
spirv-val: the 8-way module is valid

== ASan/UBSan: the game-free test
ok: listed: unset/empty/0 none, 1 the fog's pass alone, hex prefixes in any case
PASS
rc=0

== ASan/UBSan: the rewrite of the real 27ee9883 and 0a55781f at k=2 (game-derived input, read in place)
27ee9883e2a20323.5.spv: SPLIT  (local 8x8x8, slot 9 words, 24 shared accesses, 15 loads remapped)
0a55781f46174010.5.spv: SPLIT  (local 8x8x8, slot 9 words, 24 shared accesses, 13 loads remapped)
rc=0
spirv-val 27ee9883e2a20323.5.spv (split): ok
spirv-val 27ee9883 (original): ok
split module identical to the one measured offline earlier
937d28cb87b0cea0

== RADV (Mesa 26.2.0-devel git-5c65d4fc31, VANGOGH via drm-shim), RADV_DEBUG=shaderstats
original: STAT VGPRs = 128 STAT Spilled SGPRs = 0 STAT Spilled VGPRs = 0 STAT Code size = 15928 STAT LDS size = 18432 STAT Subgroups per SIMD = 8 STAT Pre-Sched VGPRs = 71 
split: STAT VGPRs = 80 STAT Spilled SGPRs = 0 STAT Spilled VGPRs = 0 STAT Code size = 16044 STAT LDS size = 18432 STAT Subgroups per SIMD = 12 STAT Pre-Sched VGPRs = 71
The 47 failures common to this branch and main 544229a2 (Windows run_all)
tests/test_auto_handheld.py
tests/test_bug_report_defaults.py
tests/test_bug_report.py
tests/test_cpu_report.py
tests/test_crash_dialog.py
tests/test_crash_groups.py
tests/test_crt_options.py
tests/test_damage_debug_menu.py
tests/test_discover.py
tests/test_display_startup_failure.py
tests/test_dlss_feature_size.py
tests/test_failed_compile_keeps_objects.py
tests/test_fastfp_fma.py
tests/test_fpx_sticky.py
tests/test_framegen_preset.py
tests/test_guest_imported.py
tests/test_hardware_report.py
tests/test_hud_layout_row.py
tests/test_invalidate_vcore.py
tests/test_job_queue.py
tests/test_lift_lookups.py
tests/test_linux_preflight.py
tests/test_linux_pressure.py
tests/test_longshot_ctl.py
tests/test_mirror_budget.py
tests/test_missing_nup.py
tests/test_pads_after_menu.py
tests/test_pending_guest_import.py
tests/test_perf_report.py
tests/test_pipeline_play_cache.py
tests/test_present_thread.py
tests/test_profiling_mode.py
tests/test_readback_timeout_source.py
tests/test_rumble_menu.py
tests/test_run_all_changed.py
tests/test_slow_frames.py
tests/test_spawn_emit.py
tests/test_switch_table.py
tests/test_target_image_pool.py
tests/test_tested_configurations.py
tests/test_upscaler_round_trip.py
tests/test_upscaler_rows.py
tests/test_vcore_escape.py
tests/test_vk_occlusion_queries.py
tests/test_vk_swapchain_compat.py
tests/test_walk_key.py
tests/test_walk.py
Evidence for the description: screenshots, the lane probe, the receipts at bda836f4, and the suite's failure list. **Pictures, Steam Deck OLED, handheld on AC, setup H.** Left: Before, RX_VK_WG_SPLIT unset: this PR @ bda836f4 (H-2O). Right: After, RX_VK_WG_SPLIT=1: this PR @ bda836f4 (H-2N). Top: Kokiri Forest; bottom: Hyrule Field. ![Deck: before (H-2O, unset) left, after (H-2N, =1) right, bda836f4](https://git.magicbean.dev/attachments/bd1cb391-5f52-4ed1-a4e7-201f2717f8aa) **Pictures, Windows, RTX 5090, driver 621.14.** This is a debug run with `RX_VK_WG_SPLIT_ANY_GPU=1`, not a measurement. Taken headless at arrival +900 frames, at bda836f4. ![PC Field, Before, RX_VK_WG_SPLIT unset: this PR @ bda836f4](https://git.magicbean.dev/attachments/a117106e-05a5-4e83-831d-6af521a669ae) ![PC Field, After, RX_VK_WG_SPLIT=1 (+ANY_GPU): this PR @ bda836f4](https://git.magicbean.dev/attachments/40e33311-a6e3-4e21-8ace-5450fbb091ab) ![PC Kokiri, Before, RX_VK_WG_SPLIT unset: this PR @ bda836f4](https://git.magicbean.dev/attachments/8c274848-ee07-4590-8ecb-eedfb1f44bde) ![PC Kokiri, After, RX_VK_WG_SPLIT=1 (+ANY_GPU): this PR @ bda836f4](https://git.magicbean.dev/attachments/31a7b38d-f426-4cb8-9fac-f8ce65f86e0b) **The lane probe's output.** The Deck run used the first binary. The PC rerun used the revised binary, 9068d6fa…, whose PASS also requires lane = index%32 and size 32. ``` lane_probe on the Steam Deck (relayed by the coordinator from the Deck agent), 2026-11-16 19:36:42: device: AMD Custom GPU 0932 (RADV VANGOGH); Mesa 26.2.0-devel (git-5c65d4fc31); compute pinned to 32 8x8x8, 3x2x5 groups: subgroup size 32, 480 subgroups; records bad 0, subgroups spanning z planes 0, shuffle z != own z 0, partial subgroups 0, lane != index%32 0: PASS 8x8x4, 3x2x10 groups: subgroup size 32, 480 subgroups; records bad 0, subgroups spanning z planes 0, shuffle z != own z 0, partial subgroups 0, lane != index%32 0: PASS PASS (rc=0) PC reference (RTX 5090, NVIDIA 621.14), 2026-11-16 ~19:30: the same two PASS lines, rc=0. binary sha256 dc6af28202f7045ecf8b596603ade2f959ec0d4bb07f49e31e2549b0d5fa738e (lane_probe.c, probe.comp, probe_spv.h beside it) After GPT-6.1's review (2026-11-16 ~20:20): PASS now also requires lane == index%32 and subgroup size 32 (both already 0 / 32 in the Deck's run above, so its verdict stands). New binary sha256 9068d6fa7b3c9fabbdb413d4c9c0a955078ab9ccff4c660aafeb95574bbba944; PC rerun with it: both shapes PASS, rc 0. ``` <details><summary>probe.comp</summary> ```glsl #version 450 // Lane-mapping probe for the 27ee9883 workgroup split: which invocations share a wave32, and what the ballot/shuffle z // idiom of 27ee9883 returns. Game-free. Built with -DLZ=8 and -DLZ=4. #extension GL_KHR_shader_subgroup_basic : require #extension GL_KHR_shader_subgroup_ballot : require #extension GL_KHR_shader_subgroup_shuffle : require layout(local_size_x = 8, local_size_y = 8, local_size_z = LZ) in; layout(std430, set = 0, binding = 0) buffer Out { uint d[]; }; void main() { uint idx = gl_LocalInvocationIndex; uint g = gl_WorkGroupID.x + gl_NumWorkGroups.x * (gl_WorkGroupID.y + gl_NumWorkGroups.y * gl_WorkGroupID.z); uint base = (g * (8u * 8u * uint(LZ)) + idx) * 8u; // (27ee9883's idiom: the first active lane's local z, read by a shuffle) uvec4 b = subgroupBallot(true); uint first = uint(findMSB(b.x & (-b.x))) & 31u; uint zf = subgroupShuffle(gl_LocalInvocationID.z, first); d[base + 0] = idx; d[base + 1] = gl_SubgroupID; d[base + 2] = gl_SubgroupInvocationID; d[base + 3] = gl_LocalInvocationID.z; d[base + 4] = zf; d[base + 5] = subgroupBallotBitCount(b); d[base + 6] = gl_SubgroupSize; d[base + 7] = 0xC0DE0000u | g; } ``` </details> <details><summary>lane_probe.c</summary> ```c // Lane-mapping probe for the 27ee9883 workgroup split (Longshot fog work). Game-free: no game files, no game. // Dispatches an 8x8x8 and an 8x8x4 compute shader pinned to 32 lanes as the renderer pins compute (pin_subgroup: required // subgroup size 32, no full-subgroups flag since X = 8), and checks, for every subgroup of every workgroup: // - all its invocations have the same LocalInvocationID.z (each wave32 sits inside one 64-invocation z plane), // - 27ee9883's ballot/shuffle idiom returns each invocation's own z, // - the subgroup is full (32 active lanes), its size is 32, and lane == LocalInvocationIndex % 32. // usage: lane_probe [device-name-substring] exit 0: PASS on both shapes; 1: FAIL; 2: no usable device #define VK_NO_PROTOTYPES #include <vulkan/vulkan.h> #include <stdio.h> #include <stdlib.h> #include <string.h> #ifdef _WIN32 #include <windows.h> #else #include <dlfcn.h> #endif #include "probe_spv.h" #define CK(x) do { VkResult r_ = (x); if (r_ != VK_SUCCESS) { fprintf(stderr, "%s failed: %d\n", #x, (int)r_); exit(2); } } while (0) #define L(name) PFN_##name name = (PFN_##name)gipa(inst, #name) #define LD(name) PFN_##name name = (PFN_##name)gdpa(dev, #name) int main(int argc, char** argv) { #ifdef _WIN32 HMODULE lib = LoadLibraryA("vulkan-1.dll"); PFN_vkGetInstanceProcAddr gipa = lib ? (PFN_vkGetInstanceProcAddr)GetProcAddress(lib, "vkGetInstanceProcAddr") : NULL; #else void* lib = dlopen("libvulkan.so.1", RTLD_NOW); PFN_vkGetInstanceProcAddr gipa = lib ? (PFN_vkGetInstanceProcAddr)dlsym(lib, "vkGetInstanceProcAddr") : NULL; #endif if (!gipa) { fprintf(stderr, "no Vulkan loader\n"); return 2; } VkInstance inst = VK_NULL_HANDLE; L(vkCreateInstance); VkApplicationInfo app = {VK_STRUCTURE_TYPE_APPLICATION_INFO, 0, "lane_probe", 1, 0, 0, VK_API_VERSION_1_3}; VkInstanceCreateInfo ici = {VK_STRUCTURE_TYPE_INSTANCE_CREATE_INFO, 0, 0, &app}; CK(vkCreateInstance(&ici, 0, &inst)); L(vkEnumeratePhysicalDevices); L(vkGetPhysicalDeviceProperties2); L(vkGetPhysicalDeviceFeatures2); L(vkGetPhysicalDeviceMemoryProperties); L(vkCreateDevice); L(vkGetDeviceProcAddr); PFN_vkGetDeviceProcAddr gdpa = vkGetDeviceProcAddr; uint32_t np = 16; VkPhysicalDevice pds[16]; CK(vkEnumeratePhysicalDevices(inst, &np, pds)); VkPhysicalDevice pd = VK_NULL_HANDLE; VkPhysicalDeviceVulkan13Properties p13 = {VK_STRUCTURE_TYPE_PHYSICAL_DEVICE_VULKAN_1_3_PROPERTIES}; VkPhysicalDeviceDriverProperties drv = {VK_STRUCTURE_TYPE_PHYSICAL_DEVICE_DRIVER_PROPERTIES, &p13}; VkPhysicalDeviceProperties2 props = {VK_STRUCTURE_TYPE_PHYSICAL_DEVICE_PROPERTIES_2, &drv}; for (uint32_t i = 0; i < np && !pd; i++) { vkGetPhysicalDeviceProperties2(pds[i], &props); if (props.properties.deviceType == VK_PHYSICAL_DEVICE_TYPE_CPU) continue; if (argc > 1 && !strstr(props.properties.deviceName, argv[1])) continue; pd = pds[i]; } if (!pd) { fprintf(stderr, "no device%s%s\n", argc > 1 ? " matching " : "", argc > 1 ? argv[1] : ""); return 2; } printf("device: %s; driver: %s %s\n", props.properties.deviceName, drv.driverName, drv.driverInfo); VkPhysicalDeviceVulkan13Features f13 = {VK_STRUCTURE_TYPE_PHYSICAL_DEVICE_VULKAN_1_3_FEATURES}; VkPhysicalDeviceFeatures2 f2 = {VK_STRUCTURE_TYPE_PHYSICAL_DEVICE_FEATURES_2, &f13}; vkGetPhysicalDeviceFeatures2(pd, &f2); const int pin = f13.subgroupSizeControl && (p13.requiredSubgroupSizeStages & VK_SHADER_STAGE_COMPUTE_BIT) && p13.minSubgroupSize <= 32 && p13.maxSubgroupSize >= 32; printf("subgroup sizes %u-%u, compute pinned to 32: %s\n", p13.minSubgroupSize, p13.maxSubgroupSize, pin ? "yes" : "no"); VkPhysicalDeviceVulkan13Features en13 = {VK_STRUCTURE_TYPE_PHYSICAL_DEVICE_VULKAN_1_3_FEATURES}; en13.subgroupSizeControl = pin ? VK_TRUE : VK_FALSE; float prio = 1; VkDeviceQueueCreateInfo qci = {VK_STRUCTURE_TYPE_DEVICE_QUEUE_CREATE_INFO, 0, 0, 0, 1, &prio}; // (family 0 has compute on RADV/NVIDIA) VkDeviceCreateInfo dci = {VK_STRUCTURE_TYPE_DEVICE_CREATE_INFO, &en13, 0, 1, &qci}; VkDevice dev; CK(vkCreateDevice(pd, &dci, 0, &dev)); LD(vkGetDeviceQueue); LD(vkCreateBuffer); LD(vkGetBufferMemoryRequirements); LD(vkAllocateMemory); LD(vkBindBufferMemory); LD(vkMapMemory); LD(vkCreateShaderModule); LD(vkCreateDescriptorSetLayout); LD(vkCreatePipelineLayout); LD(vkCreateComputePipelines); LD(vkCreateDescriptorPool); LD(vkAllocateDescriptorSets); LD(vkUpdateDescriptorSets); LD(vkCreateCommandPool); LD(vkAllocateCommandBuffers); LD(vkBeginCommandBuffer); LD(vkCmdBindPipeline); LD(vkCmdBindDescriptorSets); LD(vkCmdDispatch); LD(vkEndCommandBuffer); LD(vkQueueSubmit); LD(vkQueueWaitIdle); LD(vkCmdFillBuffer); LD(vkCmdPipelineBarrier); VkQueue q; vkGetDeviceQueue(dev, 0, 0, &q); VkPhysicalDeviceMemoryProperties mp; vkGetPhysicalDeviceMemoryProperties(pd, &mp); const uint32_t GX = 3, GY = 2, GZ8 = 5; // (asymmetric grid; the 8x8x4 shape runs twice the groups in z) const VkDeviceSize words = (VkDeviceSize)GX * GY * GZ8 * 512 * 8; VkBufferCreateInfo bci = {VK_STRUCTURE_TYPE_BUFFER_CREATE_INFO, 0, 0, words * 4, VK_BUFFER_USAGE_STORAGE_BUFFER_BIT | VK_BUFFER_USAGE_TRANSFER_DST_BIT}; VkBuffer buf; CK(vkCreateBuffer(dev, &bci, 0, &buf)); VkMemoryRequirements mr; vkGetBufferMemoryRequirements(dev, buf, &mr); uint32_t mt = UINT32_MAX; const VkMemoryPropertyFlags want = VK_MEMORY_PROPERTY_HOST_VISIBLE_BIT | VK_MEMORY_PROPERTY_HOST_COHERENT_BIT; for (uint32_t i = 0; i < mp.memoryTypeCount; i++) if ((mr.memoryTypeBits & (1u << i)) && (mp.memoryTypes[i].propertyFlags & want) == want) { mt = i; break; } if (mt == UINT32_MAX) { fprintf(stderr, "no host-visible memory\n"); return 2; } VkMemoryAllocateInfo mai = {VK_STRUCTURE_TYPE_MEMORY_ALLOCATE_INFO, 0, mr.size, mt}; VkDeviceMemory mem; CK(vkAllocateMemory(dev, &mai, 0, &mem)); CK(vkBindBufferMemory(dev, buf, mem, 0)); uint32_t* out; CK(vkMapMemory(dev, mem, 0, VK_WHOLE_SIZE, 0, (void**)&out)); VkDescriptorSetLayoutBinding lb = {0, VK_DESCRIPTOR_TYPE_STORAGE_BUFFER, 1, VK_SHADER_STAGE_COMPUTE_BIT, 0}; VkDescriptorSetLayoutCreateInfo dlci = {VK_STRUCTURE_TYPE_DESCRIPTOR_SET_LAYOUT_CREATE_INFO, 0, 0, 1, &lb}; VkDescriptorSetLayout dsl; CK(vkCreateDescriptorSetLayout(dev, &dlci, 0, &dsl)); VkPipelineLayoutCreateInfo plci = {VK_STRUCTURE_TYPE_PIPELINE_LAYOUT_CREATE_INFO, 0, 0, 1, &dsl}; VkPipelineLayout pl; CK(vkCreatePipelineLayout(dev, &plci, 0, &pl)); VkDescriptorPoolSize ps = {VK_DESCRIPTOR_TYPE_STORAGE_BUFFER, 1}; VkDescriptorPoolCreateInfo dpci = {VK_STRUCTURE_TYPE_DESCRIPTOR_POOL_CREATE_INFO, 0, 0, 1, 1, &ps}; VkDescriptorPool dp; CK(vkCreateDescriptorPool(dev, &dpci, 0, &dp)); VkDescriptorSetAllocateInfo dsai = {VK_STRUCTURE_TYPE_DESCRIPTOR_SET_ALLOCATE_INFO, 0, dp, 1, &dsl}; VkDescriptorSet ds; CK(vkAllocateDescriptorSets(dev, &dsai, &ds)); VkDescriptorBufferInfo dbi = {buf, 0, VK_WHOLE_SIZE}; VkWriteDescriptorSet wds = {VK_STRUCTURE_TYPE_WRITE_DESCRIPTOR_SET, 0, ds, 0, 0, 1, VK_DESCRIPTOR_TYPE_STORAGE_BUFFER, 0, &dbi}; vkUpdateDescriptorSets(dev, 1, &wds, 0, 0); VkCommandPoolCreateInfo cpci = {VK_STRUCTURE_TYPE_COMMAND_POOL_CREATE_INFO, 0, 0, 0}; VkCommandPool cp; CK(vkCreateCommandPool(dev, &cpci, 0, &cp)); int fail = 0; for (int shape = 0; shape < 2; shape++) { const uint32_t lz = shape ? 4 : 8, gz = shape ? GZ8 * 2 : GZ8, inv = 64 * lz; const uint32_t* code = shape ? probe_lz4 : probe_lz8; const size_t code_size = shape ? sizeof probe_lz4 : sizeof probe_lz8; VkShaderModuleCreateInfo smci = {VK_STRUCTURE_TYPE_SHADER_MODULE_CREATE_INFO, 0, 0, code_size, code}; VkShaderModule sm; CK(vkCreateShaderModule(dev, &smci, 0, &sm)); VkPipelineShaderStageRequiredSubgroupSizeCreateInfo rss = { VK_STRUCTURE_TYPE_PIPELINE_SHADER_STAGE_REQUIRED_SUBGROUP_SIZE_CREATE_INFO, 0, 32}; VkComputePipelineCreateInfo cci = {VK_STRUCTURE_TYPE_COMPUTE_PIPELINE_CREATE_INFO, 0, 0, {VK_STRUCTURE_TYPE_PIPELINE_SHADER_STAGE_CREATE_INFO, pin ? &rss : 0, 0, VK_SHADER_STAGE_COMPUTE_BIT, sm, "main", 0}, pl, 0, 0}; VkPipeline pipe; CK(vkCreateComputePipelines(dev, 0, 1, &cci, 0, &pipe)); VkCommandBufferAllocateInfo cbai = {VK_STRUCTURE_TYPE_COMMAND_BUFFER_ALLOCATE_INFO, 0, cp, VK_COMMAND_BUFFER_LEVEL_PRIMARY, 1}; VkCommandBuffer cb; CK(vkAllocateCommandBuffers(dev, &cbai, &cb)); VkCommandBufferBeginInfo cbbi = {VK_STRUCTURE_TYPE_COMMAND_BUFFER_BEGIN_INFO}; CK(vkBeginCommandBuffer(cb, &cbbi)); vkCmdFillBuffer(cb, buf, 0, VK_WHOLE_SIZE, 0xFFFFFFFFu); VkMemoryBarrier mb = {VK_STRUCTURE_TYPE_MEMORY_BARRIER, 0, VK_ACCESS_TRANSFER_WRITE_BIT, VK_ACCESS_SHADER_WRITE_BIT}; vkCmdPipelineBarrier(cb, VK_PIPELINE_STAGE_TRANSFER_BIT, VK_PIPELINE_STAGE_COMPUTE_SHADER_BIT, 0, 1, &mb, 0, 0, 0, 0); vkCmdBindPipeline(cb, VK_PIPELINE_BIND_POINT_COMPUTE, pipe); vkCmdBindDescriptorSets(cb, VK_PIPELINE_BIND_POINT_COMPUTE, pl, 0, 1, &ds, 0, 0); vkCmdDispatch(cb, GX, GY, gz); VkMemoryBarrier hb = {VK_STRUCTURE_TYPE_MEMORY_BARRIER, 0, VK_ACCESS_SHADER_WRITE_BIT, VK_ACCESS_HOST_READ_BIT}; vkCmdPipelineBarrier(cb, VK_PIPELINE_STAGE_COMPUTE_SHADER_BIT, VK_PIPELINE_STAGE_HOST_BIT, 0, 1, &hb, 0, 0, 0, 0); CK(vkEndCommandBuffer(cb)); VkSubmitInfo si = {VK_STRUCTURE_TYPE_SUBMIT_INFO, 0, 0, 0, 0, 1, &cb}; CK(vkQueueSubmit(q, 1, &si, VK_NULL_HANDLE)); CK(vkQueueWaitIdle(q)); const uint32_t groups = GX * GY * gz; long bad_tag = 0, mixed_z = 0, shuf = 0, partial = 0, lane_off = 0, subgroups = 0, sgsize = 0; for (uint32_t g = 0; g < groups; g++) { const uint32_t nsg = inv / 32 + 1; int sg_z[64]; int sg_n[64]; for (uint32_t s = 0; s < nsg && s < 64; s++) sg_z[s] = -1, sg_n[s] = 0; for (uint32_t i = 0; i < inv; i++) { const uint32_t* r = out + ((size_t)g * inv + i) * 8; if (r[7] != (0xC0DE0000u | g) || r[0] != i) { bad_tag++; continue; } sgsize = r[6]; const uint32_t s = r[1]; if (s >= 64) { bad_tag++; continue; } if (sg_z[s] < 0) sg_z[s] = (int)r[3], subgroups++; else if (sg_z[s] != (int)r[3]) mixed_z++; sg_n[s]++; if (r[4] != r[3]) shuf++; if (r[5] != r[6]) partial++; if (r[6] == 32 && (r[2] != i % 32 || s != i / 32)) lane_off++; } } const int ok = !bad_tag && !mixed_z && !shuf && !partial && !lane_off && sgsize == 32; printf("8x8x%u, %ux%ux%u groups: subgroup size %ld, %ld subgroups; records bad %ld, subgroups spanning z planes %ld, " "shuffle z != own z %ld, partial subgroups %ld, lane != index%%32 %ld: %s\n", lz, GX, GY, gz, sgsize, subgroups, bad_tag, mixed_z, shuf, partial, lane_off, ok ? "PASS" : "FAIL"); fail |= !ok; } printf("%s\n", fail ? "FAIL" : "PASS"); return fail ? 1 : 0; } ``` </details> <details><summary>Receipts at bda836f4: source hashes, the test with spirv-val, ASan/UBSan, spirv-val of the real module, Mesa 26.2 stats</summary> ``` head: bda836f4f4dc77b8050460af51da985868eae7e4 6b4623c31556c62a16ea2e3480308b3764a1b910f1b42e75e3919cc48aa441a9 gpu/vk/wg_split.cpp b163318d0f535566cd2462a03e3a19bc65852a4f3d8361ae68d51ad1fd543bac gpu/vk/wg_split.h 7b8a87aef00dee6cfe52a4f210a84c5d05c152633a67e712bf4e5aad65e11d59 tests/wg_split/wg_split_test.cpp 0491762e006db16830ae0bbc9f9419935519ac5e9ff1586bbd3763e65748cbb8 tests/wg_split/fixture_spv.h 8613c5ef4a82ba29b6ec02443e93fec72668d230159a72f4ea11936eb9561b88 tests/test_wg_split.py == spirv-val: SPIRV-Tools v2025.1 unknown hash, 2026-05-13T06:03:38+00:00 == tests/test_wg_split.py (Linux, g++ 13) ok: listed: unset/empty/0 none, 1 the fog's pass alone, hex prefixes in any case PASS spirv-val: the 8x8x8 fixture module is valid spirv-val: the 2-way module is valid spirv-val: the 4-way module is valid spirv-val: the 8-way module is valid == ASan/UBSan: the game-free test ok: listed: unset/empty/0 none, 1 the fog's pass alone, hex prefixes in any case PASS rc=0 == ASan/UBSan: the rewrite of the real 27ee9883 and 0a55781f at k=2 (game-derived input, read in place) 27ee9883e2a20323.5.spv: SPLIT (local 8x8x8, slot 9 words, 24 shared accesses, 15 loads remapped) 0a55781f46174010.5.spv: SPLIT (local 8x8x8, slot 9 words, 24 shared accesses, 13 loads remapped) rc=0 spirv-val 27ee9883e2a20323.5.spv (split): ok spirv-val 27ee9883 (original): ok split module identical to the one measured offline earlier 937d28cb87b0cea0 == RADV (Mesa 26.2.0-devel git-5c65d4fc31, VANGOGH via drm-shim), RADV_DEBUG=shaderstats original: STAT VGPRs = 128 STAT Spilled SGPRs = 0 STAT Spilled VGPRs = 0 STAT Code size = 15928 STAT LDS size = 18432 STAT Subgroups per SIMD = 8 STAT Pre-Sched VGPRs = 71 split: STAT VGPRs = 80 STAT Spilled SGPRs = 0 STAT Spilled VGPRs = 0 STAT Code size = 16044 STAT LDS size = 18432 STAT Subgroups per SIMD = 12 STAT Pre-Sched VGPRs = 71 ``` </details> <details><summary>The 47 failures common to this branch and main 544229a2 (Windows run_all)</summary> ``` tests/test_auto_handheld.py tests/test_bug_report_defaults.py tests/test_bug_report.py tests/test_cpu_report.py tests/test_crash_dialog.py tests/test_crash_groups.py tests/test_crt_options.py tests/test_damage_debug_menu.py tests/test_discover.py tests/test_display_startup_failure.py tests/test_dlss_feature_size.py tests/test_failed_compile_keeps_objects.py tests/test_fastfp_fma.py tests/test_fpx_sticky.py tests/test_framegen_preset.py tests/test_guest_imported.py tests/test_hardware_report.py tests/test_hud_layout_row.py tests/test_invalidate_vcore.py tests/test_job_queue.py tests/test_lift_lookups.py tests/test_linux_preflight.py tests/test_linux_pressure.py tests/test_longshot_ctl.py tests/test_mirror_budget.py tests/test_missing_nup.py tests/test_pads_after_menu.py tests/test_pending_guest_import.py tests/test_perf_report.py tests/test_pipeline_play_cache.py tests/test_present_thread.py tests/test_profiling_mode.py tests/test_readback_timeout_source.py tests/test_rumble_menu.py tests/test_run_all_changed.py tests/test_slow_frames.py tests/test_spawn_emit.py tests/test_switch_table.py tests/test_target_image_pool.py tests/test_tested_configurations.py tests/test_upscaler_round_trip.py tests/test_upscaler_rows.py tests/test_vcore_escape.py tests/test_vk_occlusion_queries.py tests/test_vk_swapchain_compat.py tests/test_walk_key.py tests/test_walk.py ``` </details>
Author
Contributor

Description: one cell corrected. The setup-D Kokiri CPU row was the process's CPU divided by the control bridge's game-frame counter, under a vaguer name and from an earlier reading. It is now labelled 'process CPU ms per bridge game frame', with the corrected medians 152.76 -> 152.33 ms: a sub-percent difference, not a change. The GPU, fps and 27ee9883 numbers are unchanged.

Description: one cell corrected. The setup-D Kokiri CPU row was the process's CPU divided by the control bridge's game-frame counter, under a vaguer name and from an earlier reading. It is now labelled 'process CPU ms per bridge game frame', with the corrected medians 152.76 -> 152.33 ms: a sub-percent difference, not a change. The GPU, fps and 27ee9883 numbers are unchanged.
Contributor

On this Steam Deck OLED, at the player's settings, the split binds and the frame's GPU time does not move. Not a no: it does not show here.

unset RX_VK_WG_SPLIT=1
Hyrule Field, standing: GPU ms a frame, fps 20.50, 32.36 20.52, 32.33
Kokiri Forest, standing: GPU ms a frame, fps 16.65, 25.09 17.03, 25.35
  • The workgroup split of ... line is in both =1 runs, and no declined line.
  • One run each; standing, not turning; no RX_VK_PASS_TIMING=2 run, so this is the whole frame's GPU time, not the pass's own.
  • What differs from your runs and may explain it: the GPU clock set by hand at 1600 MHz, and a low internal resolution (768x432 passes), where the fog's pass is a smaller part of the frame.

Hardware and method: Steam Deck OLED (AMD Custom APU 0932, 4 cores / 8 threads, RADV VANGOGH), SteamOS 3.9.5, 16 GB, on the charger, the GPU at a manual 1600 MHz. One game build on the Deck by the Linux builder (--jobs 4), no hot list: main 544229a2 with this PR and three others merged locally (#1209 00ad3e5b, #1158 4e889b09, #1377 e4a1471d, #875 d51401bc), each measured by its own switch in that one binary. Headless runs from a save, 720 presents measured once the scene has settled, the player's settings (a low internal resolution, 768x432 passes; a texture budget of 2,800 MB) with the frame cap lifted (HOS_FPS_MAX=1000), numbers from the game's own log lines. No picture check (no display). 0 fatal lines in the 18 starts and 2 tours of the session.

**On this Steam Deck OLED, at the player's settings, the split binds and the frame's GPU time does not move.** Not a no: it does not show here. | | unset | `RX_VK_WG_SPLIT=1` | |---|---|---| | Hyrule Field, standing: GPU ms a frame, fps | 20.50, 32.36 | 20.52, 32.33 | | Kokiri Forest, standing: GPU ms a frame, fps | 16.65, 25.09 | 17.03, 25.35 | - The `workgroup split of ...` line is in both `=1` runs, and no `declined` line. - One run each; standing, not turning; no `RX_VK_PASS_TIMING=2` run, so this is the whole frame's GPU time, not the pass's own. - What differs from your runs and may explain it: the GPU clock set by hand at 1600 MHz, and a low internal resolution (768x432 passes), where the fog's pass is a smaller part of the frame. **Hardware and method:** Steam Deck OLED (AMD Custom APU 0932, 4 cores / 8 threads, RADV VANGOGH), SteamOS 3.9.5, 16 GB, on the charger, the GPU at a manual 1600 MHz. One game build on the Deck by the Linux builder (`--jobs 4`), no hot list: main 544229a2 with this PR and three others merged locally (#1209 00ad3e5b, #1158 4e889b09, #1377 e4a1471d, #875 d51401bc), each measured by its own switch in that one binary. Headless runs from a save, 720 presents measured once the scene has settled, the player's settings (a low internal resolution, 768x432 passes; a texture budget of 2,800 MB) with the frame cap lifted (`HOS_FPS_MAX=1000`), numbers from the game's own log lines. No picture check (no display). 0 fatal lines in the 18 starts and 2 tours of the session.
Some checks are pending
CI / Conventional Commit title (pull_request) Blocked by required conditions
CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Blocked by required conditions
CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Blocked by required conditions
CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Blocked by required conditions
CI / Workflow lint (actionlint) (pull_request) Blocked by required conditions
Guide screenshots / Guide images and local UI source review (pull_request) Blocked by required conditions
Third-party parts / As their notes say (pull_request) Blocked by required conditions
Third-party parts / The same as upstream (pull_request) Blocked by required conditions
Website / Website on Cloudflare (pull_request) Blocked by required conditions
Website / Website on Cloudflare (pull_request_target) Has been skipped
PR description / Description follows the template (pull_request_target) Successful in 4s
This pull request is marked as a work in progress.
Some workflows are waiting to be reviewed.
This branch is out-of-date with the base branch
View command line instructions

Checkout

From your project repository, check out a new branch and test the changes.
git fetch -u 'brett-d/fog-wg-split:dukenheimer-brett-d/fog-wg-split'
git switch 'dukenheimer-brett-d/fog-wg-split'
Sign in to join this conversation.
No reviewers
No milestone
No project
No assignees
2 participants
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
MagicBeanDev/Longshot!1377
No description provided.