MagicBeanDev/Hookshot

Archived

Watch

Fork 0

perf(vk): build Intel compute programs that cannot see their lanes without the 32-lane pin #2564

Open

thomas-montano wants to merge 1 commit from thomas-montano/perf/intel-compute-pin into main

thomas-montano commented

2026-11-11 23:21:35 +00:00

(Migrated from github.com)

Part of #1001

What changed

spirv_uses_subgroups reads the SPIR-V's declared capabilities: GroupNonUniform*, the KHR ballot and vote, Groups and cooperative matrices mean the lane count matters. Unknown or malformed SPIR-V is treated as subgroup-using.
On Intel, pin_compute leaves a compute program without those capabilities unpinned, so the driver chooses its width. A program with shared memory and at most 32 invocations stays pinned: a compiler may drop barriers a single-warp workgroup does not need.
The property is recorded on translation and recalculated when a cached program is read; the cache format does not change. Every game compute pipeline creation goes through pin_compute.
The networks and the mesh fallback's programs use subgroup operations and stay pinned. Other GPUs keep the old behavior by default. RX_VK_PIN_ALL=1 pins every program as before; =0 enables the selective rule on any GPU.

This complements #2513: that change reduces the SPIR-V for table-heavy programs; this one removes the SIMD32 requirement only from programs that cannot observe their subgroup.

Checks

tests/test_compute_pin.py checks the capability scan, the pin decision, the shared-memory exception and every game compute pipeline's use of pin_compute.
tests/test_program_cache_cleanup.py compiles the scan with the cache loader.
tests/test_precompile_compute_guard.py compiles the precompile's compute arm with a pin_compute stand-in; which programs are pinned is left to tests/test_compute_pin.py.

Tested in the game

Iris Xe (Raptor Lake-P, i9-13900H, 32 GB), Mesa ANV 26.2.4, Bazzite. Headless at the Deku Tree, with the serial queue's escape:

9790303532c0c309, 76b54f5bb0d7ce97, 37d9fd3069f0e5e4 and 0fe7f6c474c57be5 each built in under 0.2 s without the pin, against 5.5, 26.4, 14.7 and 14.5 s pinned.
The picture and frame rate were the same as with RX_VK_PIN_ALL=1 (9.2 against 9.1 fps).

Not tested: other Intel GPUs, Windows.

On Intel/Mesa, game compute programs are required to run at 32 lanes even when they cannot see their subgroup. On an Iris Xe that makes some first builds take 5-26 seconds, leaving the effects they draw missing while the player waits. Part of #1001 ## What changed - `spirv_uses_subgroups` reads the SPIR-V's declared capabilities: GroupNonUniform*, the KHR ballot and vote, Groups and cooperative matrices mean the lane count matters. Unknown or malformed SPIR-V is treated as subgroup-using. - On Intel, `pin_compute` leaves a compute program without those capabilities unpinned, so the driver chooses its width. A program with shared memory and at most 32 invocations stays pinned: a compiler may drop barriers a single-warp workgroup does not need. - The property is recorded on translation and recalculated when a cached program is read; the cache format does not change. Every game compute pipeline creation goes through `pin_compute`. - The networks and the mesh fallback's programs use subgroup operations and stay pinned. Other GPUs keep the old behavior by default. `RX_VK_PIN_ALL=1` pins every program as before; `=0` enables the selective rule on any GPU. This complements #2513: that change reduces the SPIR-V for table-heavy programs; this one removes the SIMD32 requirement only from programs that cannot observe their subgroup. ## Checks - `tests/test_compute_pin.py` checks the capability scan, the pin decision, the shared-memory exception and every game compute pipeline's use of `pin_compute`. - `tests/test_program_cache_cleanup.py` compiles the scan with the cache loader. - `tests/test_precompile_compute_guard.py` compiles the precompile's compute arm with a `pin_compute` stand-in; which programs are pinned is left to `tests/test_compute_pin.py`. ## Tested in the game Iris Xe (Raptor Lake-P, i9-13900H, 32 GB), Mesa ANV 26.2.4, Bazzite. Headless at the Deku Tree, with the serial queue's escape: - `9790303532c0c309`, `76b54f5bb0d7ce97`, `37d9fd3069f0e5e4` and `0fe7f6c474c57be5` each built in under 0.2 s without the pin, against 5.5, 26.4, 14.7 and 14.5 s pinned. - The picture and frame rate were the same as with `RX_VK_PIN_ALL=1` (9.2 against 9.1 fps). Not tested: other Intel GPUs, Windows.

github-sync commented

2026-11-12 14:43:00 +00:00

Owner

@Joey-Magnus reviewed on GitHub · 2026-11-12 14:33 UTC · original

I merged this into current main. tests/test_compute_pin.py, test_program_cache_cleanup.py and test_env_on.py pass. test_precompile_compute_guard.py needs Windows SEH, so it skipped here, but CI ran it green. The capability scan is correct, and anything it can't parse counts as subgroup-using. Other vendors keep the pin by default. The 5.5–26 s first builds against under 0.2 s on the Iris Xe are a measurement worth having.

One risk I'd like you to address or rule out. vk_draw.cpp:198 keeps the pin for shared-memory programs only when they have at most 32 invocations. Guest code written for Maxwell can also be warp-synchronous inside a larger workgroup. The classic case is a reduction tail through shared memory without BAR for the last 32 lanes. That needs no subgroup capability in the SPIR-V. It was safe at SIMD32, and on ANV's SIMD8/16 it can read stale values. Could you check whether the four slow programs use shared memory at all? If they don't, keeping every shared-memory program pinned costs nothing and removes the question. The picture matching RX_VK_PIN_ALL=1 at the Deku Tree is good, but it is a single scene.

> [@Joey-Magnus](https://github.com/Joey-Magnus) reviewed on GitHub · 2026-11-12 14:33 UTC · [original](https://github.com/NotRexed/Hookshot/pull/2564#pullrequestreview-5876215845) I merged this into current main. tests/test_compute_pin.py, test_program_cache_cleanup.py and test_env_on.py pass. test_precompile_compute_guard.py needs Windows SEH, so it skipped here, but CI ran it green. The capability scan is correct, and anything it can't parse counts as subgroup-using. Other vendors keep the pin by default. The 5.5–26 s first builds against under 0.2 s on the Iris Xe are a measurement worth having. One risk I'd like you to address or rule out. vk_draw.cpp:198 keeps the pin for shared-memory programs only when they have at most 32 invocations. Guest code written for Maxwell can also be warp-synchronous inside a larger workgroup. The classic case is a reduction tail through shared memory without BAR for the last 32 lanes. That needs no subgroup capability in the SPIR-V. It was safe at SIMD32, and on ANV's SIMD8/16 it can read stale values. Could you check whether the four slow programs use shared memory at all? If they don't, keeping every shared-memory program pinned costs nothing and removes the question. The picture matching RX_VK_PIN_ALL=1 at the Deku Tree is good, but it is a single scene.

github-sync referenced this pull request

2026-11-12 16:54:36 +00:00

fix(video): keep a slow movie's pictures and survive Intel's compiler #2849

github-sync commented

2026-11-12 17:58:05 +00:00

Owner

@thomas-montano commented on GitHub · 2026-11-12 16:54 UTC · original

You’re right @Joey-Magnus that the capability scan cannot rule out implicit warp-synchronous shared-memory access. I checked all four measured programs, including all 16 cached variants of 76b54f5bb0d7ce97: every one has shared_memory == 0, and their SPIR-V contains no Workgroup variables or control barriers. I’ll remove the invocations <= 32 exception and keep every shared-memory program pinned. The four measured programs will still be unpinned, so this removes the risk without giving up their compile-time improvement.

Do you want me to push those changes up to this PR or will you make them outside of this PR?

> [@thomas-montano](https://github.com/thomas-montano) commented on GitHub · 2026-11-12 16:54 UTC · [original](https://github.com/NotRexed/Hookshot/pull/2564#issuecomment-6830106216) You’re right @Joey-Magnus that the capability scan cannot rule out implicit warp-synchronous shared-memory access. I checked all four measured programs, including all 16 cached variants of 76b54f5bb0d7ce97: every one has shared_memory == 0, and their SPIR-V contains no Workgroup variables or control barriers. I’ll remove the invocations <= 32 exception and keep every shared-memory program pinned. The four measured programs will still be unpinned, so this removes the risk without giving up their compile-time improvement. Do you want me to push those changes up to this PR or will you make them outside of this PR?

github-sync commented

2026-11-12 19:01:38 +00:00

Owner

@mooki000 commented on GitHub · 2026-11-12 18:03 UTC · original

A test on a second Intel GPU, the gap your description names ("other Intel GPUs").

Hardware: Intel UHD 770 (Raptor Lake-S), Mesa ANV 26.2.5-arch1.1, Arch Linux 7.2.15-1-omarchy, i9-13900K. Built: 1ef2a295e43c85a287c29086e2ed256a8cdb3a8b. Your announced change to keep every shared-memory program pinned was not in this build.

The selective rule did not clear this GPU's known false colours in these two bounded starts. Same Kokiri Forest save and fixed camera, windowless, 1920×1080 output, 75% rendering, game's upscaler, frame generation off. Startup precompile was off. Both starts used new private application and Mesa cache directories; no existing user cache was changed.

Rule on the same binary	Length	Retained captures	Colour result
`RX_VK_PIN_ALL=1`	470.7 s	214	Still false-coloured at the end
Unset, selective	470.9 s	187	Still false-coloured at the end

I inspected 166 and 138 gameplay stills respectively. Matched captures at presents 2700 and 5610 show the same black/neon-coloured failure, with the HUD readable. Neither run reported a crash, device loss or kernel driver fault. This was ordinary shared-machine testing: one-minute load 34.89→23.03 and 121.24→51.89. No speed comparison is implied.

The writer still waits. Both runs completed 56 ordinary/fallback build jobs, then ended with fallback program 3b8c07f2eb4bb7d9 still compiling. RGB9E5 writer 6637ac1d052d22d4 was requested but its build never started, and no dispatch was recorded. The selective rule correctly marked that writer unpinned (shared=0, workgroup 128×1×1), but it still waited behind the serial queue. The typed upscale continued running. This is consistent with the missing-producer explanation already reported in #1001; it is not a new driver-colour diagnosis.

The shared-memory question: both runs observed the same 93 compute key/variant records. The selective rule chose 65 to be unpinned; all 65 had zero shared memory. All 10 observed shared-memory variants declared subgroup use and remained pinned. Thus the requested list of observed unpinned, shared-memory programs with more than 32 invocations is empty. Your announced restriction would change none of these observed decisions; that is a classification of this inventory, not a test of the coming commit.

Your measured program	Observed shared bytes / invocations	Coverage here
`9790303532c0c309`	Not observed	Not reached in this view
`76b54f5bb0d7ce97`	0 / 64	One variant requested; build not reached
`37d9fd3069f0e5e4`	0 / 64	Requested; build not reached
`0fe7f6c474c57be5`	0 / 64	Requested; build not reached

So I cannot supply first-build times for those four, or an all-pipelines-ready image comparison. The cold timing comparison is held for the announced rule change. Similar broken pictures while compilation is incomplete do not prove equal compute results, and this scene did not exercise the risky shared-memory class.

The per-program records and retained durations come from an identical local logging-only patch: 31 added, 1 removed source lines on both arms. It logs pin decisions, compiler entry/completion and the first writer dispatch. The replaced line computes the same ms from one clock read, reused for the higher-resolution log. No scheduling, shaders, pin policy, waits or barriers were changed. Logging has some overhead; durations use the measured call interval, not log arrival. I can provide the patch.

Checks: test_compute_pin.py and all eight cache-cleanup cases pass with GCC 16.2.1 and Clang 23.1.3. On main-base source, the revised tests stop at the missing new scan function rather than demonstrating a compiled behavioral failure. The Windows SEH test cannot execute on Linux; its generated fixture passed Windows-target syntax checking after resolving the compiler-identification probe.

The local Windows-target preflight also found a portability omission in the new compute-pin fixture: it calls getenv without _CRT_SECURE_NO_WARNINGS before its includes. The existing, unchanged cache-cleanup fixture has the same opt-out omission for tmpfile/CRT calls. These are local preflight findings, not a claim about current upstream CI. No full-suite rerun or Windows execution was performed.

Not tested: the coming shared-memory restriction, the four advertised compile-time gains, correctness after all pipelines finish, other scenes or GPUs, real windows, sound or long play.

> [@mooki000](https://github.com/mooki000) commented on GitHub · 2026-11-12 18:03 UTC · [original](https://github.com/NotRexed/Hookshot/pull/2564#issuecomment-6831774347) A test on a second Intel GPU, the gap your description names ("other Intel GPUs"). **Hardware:** Intel UHD 770 (Raptor Lake-S), Mesa ANV 26.2.5-arch1.1, Arch Linux 7.2.15-1-omarchy, i9-13900K. **Built:** `1ef2a295e43c85a287c29086e2ed256a8cdb3a8b`. Your announced change to keep every shared-memory program pinned was **not** in this build. **The selective rule did not clear this GPU's known false colours in these two bounded starts.** Same Kokiri Forest save and fixed camera, windowless, 1920×1080 output, 75% rendering, game's upscaler, frame generation off. Startup precompile was off. Both starts used new private application and Mesa cache directories; no existing user cache was changed. | Rule on the same binary | Length | Retained captures | Colour result | |---|---:|---:|---| | `RX_VK_PIN_ALL=1` | 470.7 s | 214 | Still false-coloured at the end | | Unset, selective | 470.9 s | 187 | Still false-coloured at the end | ![Kokiri Forest on the UHD 770 at presents 2700 and 5610: all pinned on the left, the selective rule on the right, the same false colours in both](https://gist.githubusercontent.com/mooki000/b86bd3f35969803fb032f71f035844f8/raw/be1e914931f21c7e1931e6f14dd30dbba4b8a989/hookshot-pr2564-intel-colours.jpg) I inspected 166 and 138 gameplay stills respectively. Matched captures at presents 2700 and 5610 show the same black/neon-coloured failure, with the HUD readable. Neither run reported a crash, device loss or kernel driver fault. This was ordinary shared-machine testing: one-minute load 34.89→23.03 and 121.24→51.89. No speed comparison is implied. **The writer still waits.** Both runs completed 56 ordinary/fallback build jobs, then ended with fallback program `3b8c07f2eb4bb7d9` still compiling. RGB9E5 writer `6637ac1d052d22d4` was requested but its build never started, and no dispatch was recorded. The selective rule correctly marked that writer unpinned (`shared=0`, workgroup `128×1×1`), but it still waited behind the serial queue. The typed upscale continued running. This is consistent with the missing-producer explanation already reported in #1001; it is not a new driver-colour diagnosis. **The shared-memory question:** both runs observed the same 93 compute key/variant records. The selective rule chose 65 to be unpinned; all 65 had zero shared memory. All 10 observed shared-memory variants declared subgroup use and remained pinned. Thus the requested list of observed unpinned, shared-memory programs with more than 32 invocations is empty. Your announced restriction would change none of these observed decisions; that is a classification of this inventory, not a test of the coming commit. | Your measured program | Observed shared bytes / invocations | Coverage here | |---|---|---| | `9790303532c0c309` | Not observed | Not reached in this view | | `76b54f5bb0d7ce97` | 0 / 64 | One variant requested; build not reached | | `37d9fd3069f0e5e4` | 0 / 64 | Requested; build not reached | | `0fe7f6c474c57be5` | 0 / 64 | Requested; build not reached | So I cannot supply first-build times for those four, or an all-pipelines-ready image comparison. The cold timing comparison is held for the announced rule change. Similar broken pictures while compilation is incomplete do not prove equal compute results, and this scene did not exercise the risky shared-memory class. The per-program records and retained durations come from an identical local **logging-only patch: 31 added, 1 removed source lines** on both arms. It logs pin decisions, compiler entry/completion and the first writer dispatch. The replaced line computes the same `ms` from one clock read, reused for the higher-resolution log. No scheduling, shaders, pin policy, waits or barriers were changed. Logging has some overhead; durations use the measured call interval, not log arrival. I can provide the patch. **Checks:** `test_compute_pin.py` and all eight cache-cleanup cases pass with GCC 16.2.1 and Clang 23.1.3. On main-base source, the revised tests stop at the missing new scan function rather than demonstrating a compiled behavioral failure. The Windows SEH test cannot execute on Linux; its generated fixture passed Windows-target syntax checking after resolving the compiler-identification probe. The local Windows-target preflight also found a portability omission in the new compute-pin fixture: it calls `getenv` without `_CRT_SECURE_NO_WARNINGS` before its includes. The existing, unchanged cache-cleanup fixture has the same opt-out omission for `tmpfile`/CRT calls. These are local preflight findings, not a claim about current upstream CI. No full-suite rerun or Windows execution was performed. **Not tested:** the coming shared-memory restriction, the four advertised compile-time gains, correctness after all pipelines finish, other scenes or GPUs, real windows, sound or long play.

Commenting is not possible because the repository is archived.

No reviewers