MagicBeanDev/Longshot

Watch

Fork 99

fix(vk): storage image and texel buffer arrays are indexed nonuniformly where the GPU can #502

Open

mooki wants to merge 3 commits from mooki/Longshot:fix/nonuniform-image-arrays into main

mooki commented

2026-11-14 16:53:52 +00:00

Member

Type: fix · Risk: low · Tested on: macOS 26.7.1, Apple M4 Max (game-free checks, no game run) · Needs: Windows or Linux, any GPU (AMD preferred)

Why

A shader that picks a storage image, an image buffer or a texture buffer from an array, with an index that can differ
between the pixels or threads of one draw or dispatch, was translated as if they all picked the same one. Vulkan leaves
that undefined, so on a GPU that relies on it some of them read or write the wrong image. Texture arrays were already
translated right.

How to reproduce

Seen on: macOS 26.7.1, Apple M4 Max, in the game-free test (tests/test_nonuniform_image_arrays.py --tree main and
--tree #450's head). make_profile is the same code on every OS; what it decides depends on the GPU's Vulkan 1.2
features. Not seen in the game: whether it has such arrays is not known (its programs were not read).
Settings and switches: defaults (no switch: the device's features decide).
Steps: python -I -B tests/test_nonuniform_image_arrays.py --headers <SPIRV-Headers/include> --spirv-val spirv-val --cxx clang++ --tree <a tree of a66f16b0> (#450's head; main's tree fails earlier, on #450's gap): 13 synthetic
programs translated with the profiles of five stand-in GPUs.
Expected: on a GPU with a kind's feature, NonUniform on every access chain and atomic texel pointer that reaches an
array of that kind, with the kind's capability; a stamp that keeps GPUs with and without the features apart.
Actual: 37 failures on #450's head: every array of the three kinds left undecorated on a GPU with the feature, and
no stamp word (45 failures on main, where #450's gap comes first).

What changed

The renderer's shader profile now tells the translator to index storage image, image buffer and texture buffer arrays
nonuniformly wherever the GPU reports the Vulkan 1.2 feature for that kind, as it already did for texture arrays. On a
GPU that lacks one of the features the translator stamp gets a word of its own, so programs made with and without the
decorations are kept apart; no back-end file changes.

Read first: this PR is stacked on #450 (fix/descriptor-image-arrays), at its head a66f16b0, because #450 is what
declares these arrays as arrays. Until #450 merges this PR shows its two commits too; this PR's own change is the
last commit, f8f7695c. In it, read the two hunks in vk_shader.cpp, then MarkNonUniform and its callers in
emit_spirv_image.cpp to see that the three flags are all the back end needs.
vk: make_profile in gpu/vk/vk_shader.cpp sets support_storage_image_array_nonuniform_indexing,
support_uniform_texel_buffer_array_nonuniform_indexing and support_storage_texel_buffer_array_nonuniform_indexing
from the device's Vulkan 1.2 features of the same names (dev.vk12). Before, they were left false and only the
sampled image flag was set (unconditionally, as it still is).
- The device already enables every feature it reports: create_device in vk_main.cpp hands the queried feature
  chain to vkCreateDevice, so nothing changes there.
- The back end already decorates each kind of array where its flag is set, exactly as it does texture arrays
  (MarkNonUniform in emit_spirv_image.cpp, and Image() in emit_spirv_image_atomic.cpp from #450). For any
  index but an immediate it puts NonUniform on the index, the access chain, the loaded image and an atomic's texel
  pointer, and declares the kind's capability. No back-end file changes in this PR.
- A GPU without one of the features keeps today's translation of that kind: the index is taken as the same in every
  invocation, and an index that differs stays undefined there, as it is on main for every GPU.
vk: translator_stamp adds a word (nnua, and which of the three features are missing) on a GPU that lacks one.
Programs translated with the decorations are then not used on a GPU without the feature, nor the other way round (a
laptop's two GPUs share one cache folder). A GPU with all three adds nothing.
tests: tests/test_nonuniform_image_arrays.py (new), with its fixture tests/nonuniform_image_arrays_ir.cpp (new).
- It cuts make_profile, storage_binding_align (vk_internal.h) and the stamp's new word out of the tree, and
  takes the profile of five stand-in GPUs: none of the three features, each one alone, and all three.
- With each profile it translates 13 synthetic compute programs (no game data) through the real texture pass, dead
  code elimination, CollectShaderInfoPass and the back end. They have a 2-element image buffer array and a
  3-element storage image array, each read and written and each added to atomically; a 2-element texture buffer
  array and a 2-element texture array fetched; one program with all four kinds; and the same programs with single
  descriptors. That makes 65 modules.
- Every module must translate and pass spirv-val --target-env vulkan1.3. An array of a kind the GPU indexes
  nonuniformly (textures always) must have NonUniform on every access chain and atomic texel pointer that reaches
  it, and the kind's capability. Any other array must have neither, and a module without such an array no
  NonUniform at all.
- The stamp's word must be absent for the GPU with all three features and different for each of the other four.
  create_device must hand the queried Vulkan 1.2 features to vkCreateDevice without switching these off.
- --tree runs it on another checkout; --base-tree also requires every module whose program has no array of a kind
  the GPU's features turn on to be byte for byte that checkout's. On the GPU with none of the features that is every
  module.
tests: tests/test_amd_subgroup_profiles.py compiles make_profile with a stand-in device, which now has the three
features make_profile reads.
Design: the sampled image flag stays set without checking shaderSampledImageArrayNonUniformIndexing, as before (the
handle tables check it themselves, handle_tables_on). Gating it too would change texture arrays on GPUs without
that feature, which is outside this PR.
Left out: nx2/bake_ds/bake_ds.cpp and gpu/shaderbatch.cpp keep copies of an older make_profile (no float add,
gather offset or fragment shuffle settings either) and are left as they are. Touching bake_ds.cpp would also change
the lift key (nx2/make_game.py lift_sources), so every player's next build would lift the game again.

Measurements

Verdict: fixed, game-free: the new test fails on main (45 failures) and on #450's head (37) and passes here, with
the 51 modules that have no array of a kind the GPU's features turn on byte for byte #450's; the suite adds no failure
(533 passed against main's 531, the same 32 failures and 17 skips). A translation fix: nothing to time.

Commits: main 6c9edffe5d (2026-11-14); this PR fix/nonuniform-image-arrays @
f8f7695c99, on #450 @ a66f16b0.
Hardware: Apple M4 Max (16 CPUs), macOS 26.7.1, Apple clang 21, spirv-val of SPIRV-Tools v2026.4 (Homebrew spirv-tools 1.4.363.0). Game-free: no GPU, no game
Scene: none (a translation fix; its proof is the test, as CONTRIBUTING's fix row allows)
Method: tests/test_nonuniform_image_arrays.py on this PR with --base-tree #450's head, and with --tree main and
--tree #450's head (the fixture and checks of this PR, the profile and back end of that tree); the full game-free suite
How to reproduce: python -I -B tests/test_nonuniform_image_arrays.py --headers <SPIRV-Headers/include> --spirv-val spirv-val --cxx clang++ --base-tree <a tree of a66f16b0>; add --tree <a tree of main or of a66f16b0> for their side
Baseline: main @ 6c9edffe is the first column. The second, #450 @ a66f16b0, is the PR this one is stacked on: it declares
these arrays as arrays, so against it only this PR's decorations and stamp word differ, while main fails earlier on
#450's gap.

	Before: main @ `6c9edffe`	Before: #450 @ `a66f16b0`	After: this PR @ `f8f7695c`
`tests/test_nonuniform_image_arrays.py`	fails: 45 failures	fails: 37 failures	passes
modules that translate	45 of 65: the 20 with a texture buffer array or an atomic on an array are refused (#450's gap)	65 of 65	65 of 65
modules that pass spirv-val	35 of the 45: the 10 image buffer and storage image array modules fail (#450's gap)	65 of 65	65 of 65
modules where the GPU's feature calls for `NonUniform` on a storage image, image buffer or texture buffer array	0 of 14 decorated (10 refused, 4 invalid)	0 of 14 decorated	14 of 14 decorated, with the kind's capability
other modules: decorations only on texture arrays	yes	yes	yes
modules without such an array, compared with #450 @ `a66f16b0`	-	-	51 of 51 byte for byte the same (all 13 on the GPU without the features)
`translator_stamp` for the five GPUs	the same for all five	the same for all five	nothing added with all three features; four different words for the others
full game-free suite on macOS (`tests/run_all.py --jobs 6`)	531 passed, 32 failed, 17 skipped	(its stack top is this PR)	533 passed, 32 failed, 17 skipped

main @ 6c9edffe (tests/test_nonuniform_image_arrays.py --tree <main>), first lines of each kind:
stamp none: missing
FAIL: all.img-atomic-3: translation refused: Image indexing is not implemented (ImageAtomicIAdd32) is not implemented
FAIL: all.mixed: translation refused: Array of texture buffers is not implemented
FAIL: all.img-3: spirv-val: error: line 77: OpAccessChain reached non-composite type while indexes still remain to be traversed.
FAIL: all.img-3: arrays of [] declared, not of ['img']
FAIL: all.img-3: no StorageImageArrayNonUniformIndexing
FAIL: translator_stamp has no nonuniform word ('// (and which descriptor arrays the GPU indexes nonuniformly'): a program made for a GPU with the features is kept for one without them

#450 @ a66f16b0 (tests/test_nonuniform_image_arrays.py --tree <a66f16b0>), first lines:
stamp none: missing
FAIL: all.img-3: 2 of 2 access chains of the img array without NonUniform
FAIL: all.img-3: no StorageImageArrayNonUniformIndexing
FAIL: all.img-atomic-3: 1 of 1 access chains of the img array without NonUniform
FAIL: all.img-atomic-3: 1 of 1 atomic texel pointers of the img array without NonUniform
FAIL: all.img-atomic-3: no StorageImageArrayNonUniformIndexing
FAIL: all.imgbuf-2: 2 of 2 access chains of the imgbuf array without NonUniform

this PR @ f8f7695c (tests/test_nonuniform_image_arrays.py --base-tree <a66f16b0>):
stamp none: 6e6e7561,00000007
stamp storage-image: 6e6e7561,00000006
stamp uniform-texel-buffer: 6e6e7561,00000005
stamp storage-texel-buffer: 6e6e7561,00000003
stamp all: -
PASS: storage image, image buffer and texture buffer arrays decorated NonUniform exactly where the GPU indexes them nonuniformly; 65 modules validate; the stamp keeps the GPUs apart; 51 modules without such an array byte for byte the base tree's

Screenshots: none; nothing on screen (no game run).

Flags and settings

None. No switch: the device's features decide.

Testing

Tested:

On an Apple M4 Max with macOS 26.7.1, game-free only.
tests/test_nonuniform_image_arrays.py passes here, with all 51 modules that have no array of a kind the GPU's
features turn on byte for byte #450's. It fails on main (45 failures) and on #450's head (37 failures: every array
of the three kinds left undecorated on a GPU with the feature, and no stamp word).
All 65 of this PR's modules also pass spirv-val --target-env vulkan1.3 --uniform-buffer-standard-layout, as CI's
SPIR-V fixture job validates them.
Before the rebase, spirv-dis of the all-four-kinds program on the GPU with only the storage image feature:
NonUniform on the storage image and texture arrays' indices, access chains and loads, with
StorageImageArrayNonUniformIndexing and SampledImageArrayNonUniformIndexing; none on the texture buffer and image
buffer arrays, and neither of their capabilities.
vk_shader.cpp compiled as an object with the renderer's flags (nx2/build.py), Apple clang 21, arm64 and x86_64:
exit 0 on both here and on main 6c9edffe (whose vk_shader.cpp is #450's). Main's renderer does not build on macOS as
it is (environ), so these compiles used a compile-only prelude declaring it, on both sides.
The checks that read make_profile or translator_stamp pass: test_amd_subgroup_profiles (updated),
test_amd_fragment_warp, test_native_float_atomic_add, test_vendor_fixes, test_handle_table_arrays,
test_program_stamp_knobs, test_prepared_stamps, test_program_cache_format, test_global_table_loop,
test_shader_stamp_inputs and test_shader_stamp_cache.
The full suite on both sides, compared by test name with main's at 6c9edffe.
- By test name, against main 6c9edffe's run on the same Mac: 2 statuses changed: test_image_arrays, absent on main,
  passes here; test_nonuniform_image_arrays, absent on main, passes here.
- These 32 checks fail here and on main on this Mac, by name (not triaged here: some need the macOS port's other pull
  requests, some are checks main added or changed since 5e072fc8): test_appimage_package_retry,
  test_appimage_player_build, test_collect_wide, test_control, test_cpm_discovery, test_crash_dialog,
  test_crash_record, test_env_on, test_game_layout, test_lift_cleanup_error, test_linux_preflight,
  test_longshot_ctl, test_low_memory_notice, test_make_game_cache, test_mem_gate, test_mem_gate_children,
  test_mirror_trim, test_oot_state_tap, test_pcache_header_read, test_pipeline_cache_file,
  test_pipeline_cache_save, test_pipeline_play_cache, test_pipeline_tour, test_prof_symbols,
  test_program_cache_cleanup, test_program_cache_one_variant, test_readback_late, test_report_run,
  test_split_giants, test_vk_pgo, test_vram_budget_row, test_walk_key.
- The same 17 checks skipped as on main.
- Crash reports during the run: recorder_sync_test.exe-2026-11-14-191118.ips (main's run:
  fault-arm64-2026-11-14-180704.ips, recorder_sync_test.exe-2026-11-14-180744.ips).
This Mac's drivers report all three features: MoltenVK (the release and the SDK's) and KosmicKrisp, vulkaninfo on
the Apple M4 Max.
After the rebase onto main aa95d35f, at 6547c9bf (f8f7695c before the rebases onto e090cc48 and aa95d35f):
Sweep3's checks of this head (the C/C++ it changes compiled, and its own tests run, on arm64 and x86_64 where they
apply) gave the same results as before; python -I -B tests/run_all.py on wave S1's merge 11a5b4e7, which carries
this head: 568/22/17 (passed/failed/skipped) against main aa95d35f's 543/33/17 on this Mac, no new failure
(reports/2026-11-15-rebase-sweep-3.md).

Not tested:

A GPU run. Whether the game has such arrays is not known: its programs were not read. Where it has one and the GPU has
the feature, the program now carries NonUniform and the kind's capability; nothing else changes.
Other GPUs' drivers were not checked for the three features: where one reports a feature, that kind of array now gets
the decorations; where it doesn't, nothing changes.
Windows and Linux: not run. run_all.py skips the new test on Windows, as it skips the other spirv-val checks; on
Linux it runs like them. The change is in what the translator is told, which the test checks on any OS with spirv-val.
Not built on Windows or Linux, so the Windows and Linux build box is not in the checklist.

Must be tested:

Windows or Linux, any GPU (AMD preferred), any driver: build this PR (head 6547c9bf, on #450) and main aa95d35f the
same way; note whether vulkaninfo reports shaderStorageImageArrayNonUniformIndexing,
shaderUniformTexelBufferArrayNonUniformIndexing and shaderStorageTexelBufferArrayNonUniformIndexing; on Linux run
python -I -B tests/run_all.py image_arrays (it needs spirv-val); start each build to the title and play a minute,
expecting both tests to pass, this PR's first start to translate every program again (the translator changed), its
vk: prepared N shader programs (... M failed ...) line to count no more failed programs than main's, and no new
error in longshot_log.txt (by hand: test request T-AUG6)

Risks

What could break: low. Only programs with a storage image, image buffer or texture buffer array change, and only on
a GPU with that kind's feature; every other module is byte for byte the same.

A driver may compile a nonuniform access as a loop over the distinct descriptors the invocations pick; when they all
pick the same one it runs once [INFERENCE: not measured]. Texture arrays have had the same decorations all along.
A GPU without a feature keeps today's translation, which is undefined there for an index that differs. Nothing better
is possible on such a GPU without the feature.
Landing order: stacked on #450, so it lands after #450 (until then this PR shows #450's two commits too). Rebased
with #450 on main 6c9edffe (2026-11-14): #450's two commits first, then this one onto them, all without a conflict.
Main has not changed vk_shader.cpp, the back end or this PR's tests since 5e072fc8. Landing it in the same release
as #450 (which changes the stamp too) makes the cache rebuild below one rebuild, not two. No conflict with the
author's other open PRs as rebased on 6c9edffe (git merge-tree of the heads, 2026-11-14).

Turning it off: revert the commit (no switch: the device's features decide).

Compatibility: the caches start over once. vk_shader.cpp is one of the files of the translator stamp
(LONGSHOT_SHADER_STAMP, listed in nx2/translator_files.py), so after updating, every player's program cache,
Vulkan pipeline cache and prepare record are dropped at the first start and every program is translated again, as for
any change to the translator. On a GPU without one of the three features the new word gives the cache a stamp of its
own. The program cache's format and Shader::Info are unchanged; saves, the options file and the vk ABI are unchanged.

Checklist

Everyone

One topic.
The title is a Conventional Commit (type(scope): description) of the status line's type.
Why gives the reason plainly and links the issues that fit (the tracker searched).
Measurements has a verdict, or one line saying why there is nothing to measure.
Testing names what was tested, what wasn't, and what must still be tested, where and how.
Rebased on current main: on aa95d35f (#761, 2026-11-15), head 6547c9bf, on #450 38ba67dd, without a
conflict; this PR's own commits, old → new: f8f7695c → 6547c9bf; the commit names elsewhere in this description
are from before the rebase.
python -I -B tests/run_all.py passes (any failure that also fails on main is named). On macOS: 533 passed, 17
skipped, and 32 failures that also fail on main 6c9edffe on this Mac, named under Testing. After the rebase onto main
aa95d35f (head 6547c9bf): wave S1's merge 11a5b4e7, 568/22/17 against main aa95d35f's 543/33/17, no new failure.
No game files, dumps, build output or generated files.
third_party/ untouched (a new folder with a PROVENANCE.md instead).
The license is OK: GPL-3.0-or-later, nothing incompatible and nothing decompiled.

When it applies

AI assistance recorded with Assisted-by: / Co-Authored-By: trailers.
A game-free test added for the fix, failing on main and passing here.
A check under tests/ that reads changed source updated in this PR, not weakened.

**Type:** fix · **Risk:** low · **Tested on:** macOS 26.7.1, Apple M4 Max (game-free checks, no game run) · **Needs:** Windows or Linux, any GPU (AMD preferred) ## Why A shader that picks a storage image, an image buffer or a texture buffer from an array, with an index that can differ between the pixels or threads of one draw or dispatch, was translated as if they all picked the same one. Vulkan leaves that undefined, so on a GPU that relies on it some of them read or write the wrong image. Texture arrays were already translated right. ## How to reproduce **Seen on:** macOS 26.7.1, Apple M4 Max, in the game-free test (`tests/test_nonuniform_image_arrays.py --tree` main and `--tree` #450's head). `make_profile` is the same code on every OS; what it decides depends on the GPU's Vulkan 1.2 features. Not seen in the game: whether it has such arrays is not known (its programs were not read). **Settings and switches:** defaults (no switch: the device's features decide). **Steps:** `python -I -B tests/test_nonuniform_image_arrays.py --headers <SPIRV-Headers/include> --spirv-val spirv-val --cxx clang++ --tree <a tree of a66f16b0>` (#450's head; main's tree fails earlier, on #450's gap): 13 synthetic programs translated with the profiles of five stand-in GPUs. **Expected:** on a GPU with a kind's feature, `NonUniform` on every access chain and atomic texel pointer that reaches an array of that kind, with the kind's capability; a stamp that keeps GPUs with and without the features apart. **Actual:** 37 failures on #450's head: every array of the three kinds left undecorated on a GPU with the feature, and no stamp word (45 failures on main, where #450's gap comes first). ## What changed The renderer's shader profile now tells the translator to index storage image, image buffer and texture buffer arrays nonuniformly wherever the GPU reports the Vulkan 1.2 feature for that kind, as it already did for texture arrays. On a GPU that lacks one of the features the translator stamp gets a word of its own, so programs made with and without the decorations are kept apart; no back-end file changes. - Read first: this PR is stacked on #450 (`fix/descriptor-image-arrays`), at its head a66f16b0, because #450 is what declares these arrays as arrays. Until #450 merges this PR shows its two commits too; this PR's own change is the last commit, f8f7695c. In it, read the two hunks in `vk_shader.cpp`, then `MarkNonUniform` and its callers in `emit_spirv_image.cpp` to see that the three flags are all the back end needs. - vk: `make_profile` in `gpu/vk/vk_shader.cpp` sets `support_storage_image_array_nonuniform_indexing`, `support_uniform_texel_buffer_array_nonuniform_indexing` and `support_storage_texel_buffer_array_nonuniform_indexing` from the device's Vulkan 1.2 features of the same names (`dev.vk12`). Before, they were left false and only the sampled image flag was set (unconditionally, as it still is). - The device already enables every feature it reports: `create_device` in `vk_main.cpp` hands the queried feature chain to `vkCreateDevice`, so nothing changes there. - The back end already decorates each kind of array where its flag is set, exactly as it does texture arrays (`MarkNonUniform` in `emit_spirv_image.cpp`, and `Image()` in `emit_spirv_image_atomic.cpp` from #450). For any index but an immediate it puts `NonUniform` on the index, the access chain, the loaded image and an atomic's texel pointer, and declares the kind's capability. No back-end file changes in this PR. - A GPU without one of the features keeps today's translation of that kind: the index is taken as the same in every invocation, and an index that differs stays undefined there, as it is on main for every GPU. - vk: `translator_stamp` adds a word (`nnua`, and which of the three features are missing) on a GPU that lacks one. Programs translated with the decorations are then not used on a GPU without the feature, nor the other way round (a laptop's two GPUs share one cache folder). A GPU with all three adds nothing. - tests: `tests/test_nonuniform_image_arrays.py` (new), with its fixture `tests/nonuniform_image_arrays_ir.cpp` (new). - It cuts `make_profile`, `storage_binding_align` (`vk_internal.h`) and the stamp's new word out of the tree, and takes the profile of five stand-in GPUs: none of the three features, each one alone, and all three. - With each profile it translates 13 synthetic compute programs (no game data) through the real texture pass, dead code elimination, `CollectShaderInfoPass` and the back end. They have a 2-element image buffer array and a 3-element storage image array, each read and written and each added to atomically; a 2-element texture buffer array and a 2-element texture array fetched; one program with all four kinds; and the same programs with single descriptors. That makes 65 modules. - Every module must translate and pass `spirv-val --target-env vulkan1.3`. An array of a kind the GPU indexes nonuniformly (textures always) must have `NonUniform` on every access chain and atomic texel pointer that reaches it, and the kind's capability. Any other array must have neither, and a module without such an array no `NonUniform` at all. - The stamp's word must be absent for the GPU with all three features and different for each of the other four. `create_device` must hand the queried Vulkan 1.2 features to `vkCreateDevice` without switching these off. - `--tree` runs it on another checkout; `--base-tree` also requires every module whose program has no array of a kind the GPU's features turn on to be byte for byte that checkout's. On the GPU with none of the features that is every module. - tests: `tests/test_amd_subgroup_profiles.py` compiles `make_profile` with a stand-in device, which now has the three features `make_profile` reads. - Design: the sampled image flag stays set without checking `shaderSampledImageArrayNonUniformIndexing`, as before (the handle tables check it themselves, `handle_tables_on`). Gating it too would change texture arrays on GPUs without that feature, which is outside this PR. - Left out: `nx2/bake_ds/bake_ds.cpp` and `gpu/shaderbatch.cpp` keep copies of an older `make_profile` (no float add, gather offset or fragment shuffle settings either) and are left as they are. Touching `bake_ds.cpp` would also change the lift key (`nx2/make_game.py lift_sources`), so every player's next build would lift the game again. ## Measurements **Verdict:** fixed, game-free: the new test fails on main (45 failures) and on #450's head (37) and passes here, with the 51 modules that have no array of a kind the GPU's features turn on byte for byte #450's; the suite adds no failure (533 passed against main's 531, the same 32 failures and 17 skips). A translation fix: nothing to time. Commits: main 6c9edffe5da1f5b5383977aeab3f544c0de25d4c (2026-11-14); this PR fix/nonuniform-image-arrays @ f8f7695c99b55cac415d7a03a2262e541b49bb82, on #450 @ a66f16b0. Hardware: Apple M4 Max (16 CPUs), macOS 26.7.1, Apple clang 21, spirv-val of SPIRV-Tools v2026.4 (Homebrew spirv-tools 1.4.363.0). Game-free: no GPU, no game Scene: none (a translation fix; its proof is the test, as CONTRIBUTING's `fix` row allows) Method: `tests/test_nonuniform_image_arrays.py` on this PR with `--base-tree` #450's head, and with `--tree` main and `--tree` #450's head (the fixture and checks of this PR, the profile and back end of that tree); the full game-free suite How to reproduce: `python -I -B tests/test_nonuniform_image_arrays.py --headers <SPIRV-Headers/include> --spirv-val spirv-val --cxx clang++ --base-tree <a tree of a66f16b0>`; add `--tree <a tree of main or of a66f16b0>` for their side Baseline: main @ 6c9edffe is the first column. The second, #450 @ a66f16b0, is the PR this one is stacked on: it declares these arrays as arrays, so against it only this PR's decorations and stamp word differ, while main fails earlier on #450's gap. | | Before: main @ 6c9edffe | Before: #450 @ a66f16b0 | After: this PR @ f8f7695c | |---|---|---|---| | `tests/test_nonuniform_image_arrays.py` | fails: 45 failures | fails: 37 failures | passes | | modules that translate | 45 of 65: the 20 with a texture buffer array or an atomic on an array are refused (#450's gap) | 65 of 65 | 65 of 65 | | modules that pass spirv-val | 35 of the 45: the 10 image buffer and storage image array modules fail (#450's gap) | 65 of 65 | 65 of 65 | | modules where the GPU's feature calls for `NonUniform` on a storage image, image buffer or texture buffer array | 0 of 14 decorated (10 refused, 4 invalid) | 0 of 14 decorated | 14 of 14 decorated, with the kind's capability | | other modules: decorations only on texture arrays | yes | yes | yes | | modules without such an array, compared with #450 @ a66f16b0 | - | - | 51 of 51 byte for byte the same (all 13 on the GPU without the features) | | `translator_stamp` for the five GPUs | the same for all five | the same for all five | nothing added with all three features; four different words for the others | | full game-free suite on macOS (`tests/run_all.py --jobs 6`) | 531 passed, 32 failed, 17 skipped | (its stack top is this PR) | 533 passed, 32 failed, 17 skipped | ``` main @ 6c9edffe (tests/test_nonuniform_image_arrays.py --tree <main>), first lines of each kind: stamp none: missing FAIL: all.img-atomic-3: translation refused: Image indexing is not implemented (ImageAtomicIAdd32) is not implemented FAIL: all.mixed: translation refused: Array of texture buffers is not implemented FAIL: all.img-3: spirv-val: error: line 77: OpAccessChain reached non-composite type while indexes still remain to be traversed. FAIL: all.img-3: arrays of [] declared, not of ['img'] FAIL: all.img-3: no StorageImageArrayNonUniformIndexing FAIL: translator_stamp has no nonuniform word ('// (and which descriptor arrays the GPU indexes nonuniformly'): a program made for a GPU with the features is kept for one without them #450 @ a66f16b0 (tests/test_nonuniform_image_arrays.py --tree <a66f16b0>), first lines: stamp none: missing FAIL: all.img-3: 2 of 2 access chains of the img array without NonUniform FAIL: all.img-3: no StorageImageArrayNonUniformIndexing FAIL: all.img-atomic-3: 1 of 1 access chains of the img array without NonUniform FAIL: all.img-atomic-3: 1 of 1 atomic texel pointers of the img array without NonUniform FAIL: all.img-atomic-3: no StorageImageArrayNonUniformIndexing FAIL: all.imgbuf-2: 2 of 2 access chains of the imgbuf array without NonUniform this PR @ f8f7695c (tests/test_nonuniform_image_arrays.py --base-tree <a66f16b0>): stamp none: 6e6e7561,00000007 stamp storage-image: 6e6e7561,00000006 stamp uniform-texel-buffer: 6e6e7561,00000005 stamp storage-texel-buffer: 6e6e7561,00000003 stamp all: - PASS: storage image, image buffer and texture buffer arrays decorated NonUniform exactly where the GPU indexes them nonuniformly; 65 modules validate; the stamp keeps the GPUs apart; 51 modules without such an array byte for byte the base tree's ``` Screenshots: none; nothing on screen (no game run). ### Flags and settings None. No switch: the device's features decide. ## Testing **Tested:** - On an Apple M4 Max with macOS 26.7.1, game-free only. - `tests/test_nonuniform_image_arrays.py` passes here, with all 51 modules that have no array of a kind the GPU's features turn on byte for byte #450's. It fails on main (45 failures) and on #450's head (37 failures: every array of the three kinds left undecorated on a GPU with the feature, and no stamp word). - All 65 of this PR's modules also pass `spirv-val --target-env vulkan1.3 --uniform-buffer-standard-layout`, as CI's SPIR-V fixture job validates them. - Before the rebase, `spirv-dis` of the all-four-kinds program on the GPU with only the storage image feature: `NonUniform` on the storage image and texture arrays' indices, access chains and loads, with `StorageImageArrayNonUniformIndexing` and `SampledImageArrayNonUniformIndexing`; none on the texture buffer and image buffer arrays, and neither of their capabilities. - `vk_shader.cpp` compiled as an object with the renderer's flags (`nx2/build.py`), Apple clang 21, arm64 and x86_64: exit 0 on both here and on main 6c9edffe (whose `vk_shader.cpp` is #450's). Main's renderer does not build on macOS as it is (`environ`), so these compiles used a compile-only prelude declaring it, on both sides. - The checks that read `make_profile` or `translator_stamp` pass: `test_amd_subgroup_profiles` (updated), `test_amd_fragment_warp`, `test_native_float_atomic_add`, `test_vendor_fixes`, `test_handle_table_arrays`, `test_program_stamp_knobs`, `test_prepared_stamps`, `test_program_cache_format`, `test_global_table_loop`, `test_shader_stamp_inputs` and `test_shader_stamp_cache`. - The full suite on both sides, compared by test name with main's at 6c9edffe. - By test name, against main 6c9edffe's run on the same Mac: 2 statuses changed: `test_image_arrays`, absent on main, passes here; `test_nonuniform_image_arrays`, absent on main, passes here. - These 32 checks fail here and on main on this Mac, by name (not triaged here: some need the macOS port's other pull requests, some are checks main added or changed since 5e072fc8): `test_appimage_package_retry`, `test_appimage_player_build`, `test_collect_wide`, `test_control`, `test_cpm_discovery`, `test_crash_dialog`, `test_crash_record`, `test_env_on`, `test_game_layout`, `test_lift_cleanup_error`, `test_linux_preflight`, `test_longshot_ctl`, `test_low_memory_notice`, `test_make_game_cache`, `test_mem_gate`, `test_mem_gate_children`, `test_mirror_trim`, `test_oot_state_tap`, `test_pcache_header_read`, `test_pipeline_cache_file`, `test_pipeline_cache_save`, `test_pipeline_play_cache`, `test_pipeline_tour`, `test_prof_symbols`, `test_program_cache_cleanup`, `test_program_cache_one_variant`, `test_readback_late`, `test_report_run`, `test_split_giants`, `test_vk_pgo`, `test_vram_budget_row`, `test_walk_key`. - The same 17 checks skipped as on main. - Crash reports during the run: recorder_sync_test.exe-2026-11-14-191118.ips (main's run: fault-arm64-2026-11-14-180704.ips, recorder_sync_test.exe-2026-11-14-180744.ips). - This Mac's drivers report all three features: MoltenVK (the release and the SDK's) and KosmicKrisp, `vulkaninfo` on the Apple M4 Max. - **After the rebase onto main aa95d35f**, at `6547c9bf` (`f8f7695c` before the rebases onto e090cc48 and aa95d35f): Sweep3's checks of this head (the C/C++ it changes compiled, and its own tests run, on arm64 and x86_64 where they apply) gave the same results as before; `python -I -B tests/run_all.py` on wave S1's merge `11a5b4e7`, which carries this head: 568/22/17 (passed/failed/skipped) against main aa95d35f's 543/33/17 on this Mac, no new failure (`reports/2026-11-15-rebase-sweep-3.md`). **Not tested:** - A GPU run. Whether the game has such arrays is not known: its programs were not read. Where it has one and the GPU has the feature, the program now carries `NonUniform` and the kind's capability; nothing else changes. - Other GPUs' drivers were not checked for the three features: where one reports a feature, that kind of array now gets the decorations; where it doesn't, nothing changes. - Windows and Linux: not run. `run_all.py` skips the new test on Windows, as it skips the other spirv-val checks; on Linux it runs like them. The change is in what the translator is told, which the test checks on any OS with spirv-val. - Not built on Windows or Linux, so the Windows and Linux build box is not in the checklist. **Must be tested:** - Windows or Linux, any GPU (AMD preferred), any driver: build this PR (head 6547c9bf, on #450) and main aa95d35f the same way; note whether `vulkaninfo` reports `shaderStorageImageArrayNonUniformIndexing`, `shaderUniformTexelBufferArrayNonUniformIndexing` and `shaderStorageTexelBufferArrayNonUniformIndexing`; on Linux run `python -I -B tests/run_all.py image_arrays` (it needs spirv-val); start each build to the title and play a minute, expecting both tests to pass, this PR's first start to translate every program again (the translator changed), its `vk: prepared N shader programs (... M failed ...)` line to count no more failed programs than main's, and no new error in `longshot_log.txt` (by hand: test request T-AUG6) ## Risks **What could break:** low. Only programs with a storage image, image buffer or texture buffer array change, and only on a GPU with that kind's feature; every other module is byte for byte the same. - A driver may compile a nonuniform access as a loop over the distinct descriptors the invocations pick; when they all pick the same one it runs once [INFERENCE: not measured]. Texture arrays have had the same decorations all along. - A GPU without a feature keeps today's translation, which is undefined there for an index that differs. Nothing better is possible on such a GPU without the feature. - Landing order: stacked on #450, so it lands after #450 (until then this PR shows #450's two commits too). Rebased with #450 on main 6c9edffe (2026-11-14): #450's two commits first, then this one onto them, all without a conflict. Main has not changed `vk_shader.cpp`, the back end or this PR's tests since 5e072fc8. Landing it in the same release as #450 (which changes the stamp too) makes the cache rebuild below one rebuild, not two. No conflict with the author's other open PRs as rebased on 6c9edffe (`git merge-tree` of the heads, 2026-11-14). **Turning it off:** revert the commit (no switch: the device's features decide). **Compatibility:** the caches start over once. `vk_shader.cpp` is one of the files of the translator stamp (`LONGSHOT_SHADER_STAMP`, listed in `nx2/translator_files.py`), so after updating, every player's program cache, Vulkan pipeline cache and prepare record are dropped at the first start and every program is translated again, as for any change to the translator. On a GPU without one of the three features the new word gives the cache a stamp of its own. The program cache's format and `Shader::Info` are unchanged; saves, the options file and the vk ABI are unchanged. ## Checklist ### Everyone - [x] One topic. - [x] The title is a Conventional Commit (`type(scope): description`) of the status line's type. - [x] Why gives the reason plainly and links the issues that fit (the tracker searched). - [x] Measurements has a verdict, or one line saying why there is nothing to measure. - [x] Testing names what was tested, what wasn't, and what must still be tested, where and how. - [x] Rebased on current `main`: on aa95d35f (#761, 2026-11-15), head `6547c9bf`, on #450 `38ba67dd`, without a conflict; this PR's own commits, old → new: `f8f7695c` → `6547c9bf`; the commit names elsewhere in this description are from before the rebase. - [x] `python -I -B tests/run_all.py` passes (any failure that also fails on `main` is named). On macOS: 533 passed, 17 skipped, and 32 failures that also fail on main 6c9edffe on this Mac, named under Testing. After the rebase onto main aa95d35f (head `6547c9bf`): wave S1's merge `11a5b4e7`, 568/22/17 against main aa95d35f's 543/33/17, no new failure. - [x] No game files, dumps, build output or generated files. - [x] `third_party/` untouched (a new folder with a `PROVENANCE.md` instead). - [x] The license is OK: GPL-3.0-or-later, nothing incompatible and nothing decompiled. ### When it applies - [x] AI assistance recorded with `Assisted-by:` / `Co-Authored-By:` trailers. - [x] A game-free test added for the fix, failing on `main` and passing here. - [x] A check under `tests/` that reads changed source updated in this PR, not weakened.

mooki added 3 commits

2026-11-14 16:53:52 +00:00

fix(vk): image, image-buffer and texture-buffer arrays are arrays in the SPIR-V 9305a148c4

A descriptor of count above 1 (a handle read at a dynamic constant buffer offset) is an array in Shader::Info, and the
renderer writes count descriptors to its binding. The SPIR-V back end declared image buffer and storage image arrays as
one image and indexed it with an OpAccessChain, which is invalid SPIR-V, and refused texture buffer arrays outright
("Array of texture buffers"). DefineTextureBuffers, DefineImageBuffers and DefineImages now declare DescType's array,
as DefineTextures does, and a texture buffer's element is reached through its own pointer type. A count of 1 declares
what it did before, so those modules are byte for byte the same.

nx2/test_image_arrays.py (tests/image_arrays_ir.cpp): synthetic compute programs with 2-element image buffer and
texture buffer arrays and a 3-element storage image array, plain, indexed nonuniformly and with bounded fetches, and
the same programs with single descriptors, through the real texture pass and back end. Every module must validate and
declare the counts Shader::Info has; --base-tree compares the single-descriptor modules with another tree's.

Assisted-by: Claude Opus 5.5
Co-Authored-By: Claude Opus 5.5 <[email protected]>

fix(vk): an atomic on an image or image buffer array takes its element

CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Has been skipped

Details

CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Has been skipped

Details

CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Has been skipped

Details

Third-party parts / The same as upstream (pull_request) Has been skipped

Details

CI / Conventional Commit title (pull_request) Successful in 7s

Details

PR description / Description follows the template (pull_request_target) Successful in 9s

Details

CI / Workflow lint (actionlint) (pull_request) Successful in 10s

Details

Third-party parts / As their notes say (pull_request) Successful in 10s

Details

Website / Website on Cloudflare (pull_request_target) Has been skipped

Details

8c9cee0df8

The arrays DefineImages and DefineImageBuffers now declare were still refused by the image atomics: ImageAtomicU32 threw
"Image indexing" for any index but 0, and the texture pass's index of an array is the element (UMin(element, count - 1)).
An atomic on an array now takes the element's pointer with an OpAccessChain, as Image() does for loads and stores, and
decorates it and the texel pointer NonUniform where the profile indexes that kind of array so. A single image takes no
index and is refused for one as before; its module is byte for byte the same.

nx2/test_image_arrays.py: a 2-element image buffer array and a 3-element storage image array added to atomically, with
their single-descriptor programs; a nonuniform atomic's texel pointer must be decorated NonUniform.

Assisted-by: Claude Opus 5.5
Co-Authored-By: Claude Opus 5.5 <[email protected]>

fix(vk): storage image and texel buffer arrays are indexed nonuniformly where the GPU can

CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Has been skipped

Details

CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Has been skipped

Details

CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Has been skipped

Details

Third-party parts / The same as upstream (pull_request) Has been skipped

Details

CI / Conventional Commit title (pull_request) Successful in 5s

Details

CI / Workflow lint (actionlint) (pull_request) Successful in 10s

Details

Third-party parts / As their notes say (pull_request) Successful in 15s

Details

PR description / Description follows the template (pull_request_target) Successful in 6s

Details

Website / Website on Cloudflare (pull_request_target) Has been skipped

Details

9dbdb36482

make_profile told the translator that only sampled image arrays may be indexed nonuniformly. A storage image, image buffer
or texture buffer array that each invocation indexes with its own element was translated as if every invocation of a draw
or dispatch picked the same descriptor, without NonUniform, which is undefined when they differ. The profile now takes
those three kinds from the device's Vulkan 1.2 features (shaderStorageImageArrayNonUniformIndexing,
shaderUniformTexelBufferArrayNonUniformIndexing, shaderStorageTexelBufferArrayNonUniformIndexing); the device is already
created with every feature it reports. The back end then decorates them as it does texture arrays (the index, the element
and an atomic's texel pointer, for any index but an immediate) and declares the kind's capability. A GPU without a feature
keeps today's translation of that kind. translator_stamp adds a word only on a GPU that lacks one of the three, so a cache
made there is not used on a GPU that has them, nor the other way round.

tests/test_nonuniform_image_arrays.py (tests/nonuniform_image_arrays_ir.cpp): make_profile and the stamp's new word cut
out of the tree, for five GPUs (none of the features, each alone, all three), and 13 synthetic compute programs through
the real texture pass and back end. Decorations and capabilities must be exactly where the GPU's features put them, every
module must pass spirv-val, and the five stamps must differ. --base-tree requires every module without such an array (on
a GPU with none of the features, every module) to be byte for byte the other tree's. tests/test_amd_subgroup_profiles.py:
its stand-in device has the three features make_profile now reads.

Assisted-by: Claude Opus 5.5
Co-Authored-By: Claude Opus 5.5 <[email protected]>

mooki added the

labels

2026-11-14 17:12:33 +00:00

mooki added the

platform/linux

platform/windows

labels

2026-11-14 18:24:47 +00:00

mooki force-pushed fix/nonuniform-image-arrays from 9dbdb36482

CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Has been skipped

Details

CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Has been skipped

Details

CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Has been skipped

Details

Third-party parts / The same as upstream (pull_request) Has been skipped

Details

CI / Conventional Commit title (pull_request) Successful in 5s

Details

CI / Workflow lint (actionlint) (pull_request) Successful in 10s

Details

Third-party parts / As their notes say (pull_request) Successful in 15s

Details

PR description / Description follows the template (pull_request_target) Successful in 6s

Details

Website / Website on Cloudflare (pull_request_target) Has been skipped

Details

to f8f7695c99

CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Has been skipped

Details

CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Has been skipped

Details

CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Has been skipped

Details

Third-party parts / The same as upstream (pull_request) Has been skipped

Details

Website / Website on Cloudflare (pull_request) Has been skipped

Details

CI / Conventional Commit title (pull_request) Successful in 4s

Details

CI / Workflow lint (actionlint) (pull_request) Successful in 6s

Details

Third-party parts / As their notes say (pull_request) Successful in 10s

Details

PR description / Description follows the template (pull_request_target) Successful in 14s

Details

2026-11-15 01:51:38 +00:00

Compare

mooki referenced this pull request

2026-11-15 01:52:39 +00:00

fix(vk): image, image-buffer and texture-buffer arrays are arrays in the SPIR-V #450

mooki changed title from ~~WIP: fix(vk): storage image and texel buffer arrays are indexed nonuniformly where the GPU can~~ to fix(vk): storage image and texel buffer arrays are indexed nonuniformly where the GPU can

2026-11-15 03:34:03 +00:00

mooki force-pushed fix/nonuniform-image-arrays from f8f7695c99

CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Has been skipped

Details

CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Has been skipped

Details

CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Has been skipped

Details

Third-party parts / The same as upstream (pull_request) Has been skipped

Details

Website / Website on Cloudflare (pull_request) Has been skipped

Details

CI / Conventional Commit title (pull_request) Successful in 4s

Details

CI / Workflow lint (actionlint) (pull_request) Successful in 6s

Details

Third-party parts / As their notes say (pull_request) Successful in 10s

Details

PR description / Description follows the template (pull_request_target) Successful in 14s

Details

to 7c0ad5988d

PR description / Description follows the template (pull_request_target) Successful in 4s

Details

2026-11-15 07:16:47 +00:00

Compare

mooki force-pushed fix/nonuniform-image-arrays from 7c0ad5988d

PR description / Description follows the template (pull_request_target) Successful in 4s

Details

to 6547c9bfb8

CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Has been skipped

Details

CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Has been skipped

Details

CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Has been skipped

Details

Third-party parts / The same as upstream (pull_request) Has been skipped

Details

Website / Website on Cloudflare (pull_request) Has been skipped

Details

CI / Conventional Commit title (pull_request) Successful in 4s

Details

CI / Workflow lint (actionlint) (pull_request) Successful in 10s

Details

Guide screenshots / Guide images and local UI source review (pull_request) Failing after 7s

Details

Third-party parts / As their notes say (pull_request) Successful in 8s

Details

PR description / Description follows the template (pull_request_target) Successful in 4s

Details

2026-11-15 13:09:38 +00:00

Compare

HumanBean approved these changes

2026-11-15 19:04:16 +00:00

HumanBean left a comment

This is stacked on #450 (its diff contains #450's commits); the PR's own change is small and right. make_profile sets the three flags from dev.vk12's matching features (create_device hands the queried chain to vkCreateDevice, so a reported feature is enabled), and #450's Image()/MarkNonUniform plus emit_spirv.cpp's capability block turn them into NonUniform decorations and capabilities. So the two PRs fit: #450 declares and indexes the arrays, this one decides when they are indexed nonuniformly. The stamp word is only added when a feature is missing, so GPUs with all three keep their cache, and a GPU without one keeps apart from one with it. The stand-in device in test_amd_subgroup_profiles.py is updated for the new fields. Leaving the sampled-image flag unconditional is explained. I did not run the tests. Land #450 first.

This is stacked on #450 (its diff contains #450's commits); the PR's own change is small and right. `make_profile` sets the three flags from `dev.vk12`'s matching features (create_device hands the queried chain to vkCreateDevice, so a reported feature is enabled), and #450's `Image()`/`MarkNonUniform` plus emit_spirv.cpp's capability block turn them into NonUniform decorations and capabilities. So the two PRs fit: #450 declares and indexes the arrays, this one decides when they are indexed nonuniformly. The stamp word is only added when a feature is missing, so GPUs with all three keep their cache, and a GPU without one keeps apart from one with it. The stand-in device in test_amd_subgroup_profiles.py is updated for the new fields. Leaving the sampled-image flag unconditional is explained. I did not run the tests. Land #450 first.