Use float sampled types for depth comparisons #834

Closed

klimovich008 wants to merge 2 commits from codex/depth-float-sampling into main

klimovich008 commented

2026-11-08 22:48:27 +00:00

(Migrated from github.com)

What this fixes

Compute warm-up can create a depth-comparison shader with an integer sampled type. Vulkan reports VUID-VkShaderModuleCreateInfo-pCode-08737; reproduced on an RX 9070 XT.

Why merge

One-line change. Depth comparisons emit float results, so their sampled type must be float.
Ordinary float depth shaders stay byte-identical; non-depth sampling, defaults and environment fallbacks stay unchanged.
The existing translator stamp invalidates old translated caches.

Change

gpu/vk/eden/shader_recompiler/ir_opt/texture_pass.cpp: const bool is_integer{!flags.is_depth && is_texture_pixel_format_integer(cbuf)};.
tests/depth_sample_types_ir.cpp: exercise real texture-pass and emitter paths for 2D and array depth samples.
nx2/test_depth_sample_types.py: validate four modules and compare float/integer-answer outputs.
tests/run_all.py: register the regression with existing compiler/header/validator bindings.

Tested

python -I -B tests/run_all.py --jobs 1 --strict depth_sample_types; checks actual pass/emitter output and integer-query counts.
Fails on the old code, passes now. All four fixed modules validate; float controls stay byte-identical.
Tested on AMD only: RX 9070 XT builds four fresh compute variants with validation enabled and no VUIDs; all four emitted modules validate. All seven CI checks pass, including the Linux regression.

Not covered

GPU check used #772 with #848's separate storage-image fix; full-game validation remains pending.

No game data is included.

## What this fixes Compute warm-up can create a depth-comparison shader with an integer sampled type. Vulkan reports `VUID-VkShaderModuleCreateInfo-pCode-08737`; reproduced on an RX 9070 XT. ## Why merge - One-line change. Depth comparisons emit float results, so their sampled type must be float. - Ordinary float depth shaders stay byte-identical; non-depth sampling, defaults and environment fallbacks stay unchanged. - The existing translator stamp invalidates old translated caches. ## Change - `gpu/vk/eden/shader_recompiler/ir_opt/texture_pass.cpp`: `const bool is_integer{!flags.is_depth && is_texture_pixel_format_integer(cbuf)};`. - `tests/depth_sample_types_ir.cpp`: exercise real texture-pass and emitter paths for 2D and array depth samples. - `nx2/test_depth_sample_types.py`: validate four modules and compare float/integer-answer outputs. - `tests/run_all.py`: register the regression with existing compiler/header/validator bindings. ## Tested - `python -I -B tests/run_all.py --jobs 1 --strict depth_sample_types`; checks actual pass/emitter output and integer-query counts. - Fails on the old code, passes now. All four fixed modules validate; float controls stay byte-identical. - Tested on AMD only: RX 9070 XT builds four fresh compute variants with validation enabled and no VUIDs; all four emitted modules validate. All seven CI checks pass, including the Linux regression. ## Not covered - GPU check used #772 with #848's separate storage-image fix; full-game validation remains pending. No game data is included.

tickernelz commented

2026-11-09 04:30:21 +00:00

(Migrated from github.com)

Same hardware check as I put on #894, on an RX 6800 XT (RADV NAVI21, Mesa 26.2.6, Fedora 44) with the v2.0.5 game and this branch's hookshot_vk.so from the Linux AppImage build of 9b438fa.

Fresh cache: prepared 38809 shader programs (46502 new, 165 failed, 0 stage libraries; 10083 compute variants, 752 programs with texture variants, 65 at the limit) in 76.1 s, then precompile: 10083 programs, 0 graphics and 10083 compute pipelines in 861 ms, then 59-60 fps with no CRASH. The unpatched v2.0.5 hookshot_vk.so on the same folder crashes in under a minute at libvulkan_radeon.so+0x14a7d2.

spirv-val --target-env vulkan1.3 over every unique module in the rebuilt cache: 50,800 modules, 6 rejected, 0 of them depth-compare with an integer sampled type (the shipped 2.0.5 cache has 216 of those out of 222 rejected). The six left are four R32ui storage-image modules with a float sampled type (#848's class) and two fragment modules with a dominance violation, which no open PR I can find covers.

Both this and #894 remove the crash on this card; this one is the narrower fix, since it corrects the sampled type instead of discarding the variant.

Same hardware check as I put on #894, on an **RX 6800 XT** (RADV NAVI21, Mesa 26.2.6, Fedora 44) with the v2.0.5 game and this branch's `hookshot_vk.so` from the Linux AppImage build of `9b438fa`. Fresh cache: `prepared 38809 shader programs (46502 new, 165 failed, 0 stage libraries; 10083 compute variants, 752 programs with texture variants, 65 at the limit) in 76.1 s`, then `precompile: 10083 programs, 0 graphics and 10083 compute pipelines in 861 ms`, then 59-60 fps with no `CRASH`. The unpatched v2.0.5 `hookshot_vk.so` on the same folder crashes in under a minute at `libvulkan_radeon.so+0x14a7d2`. `spirv-val --target-env vulkan1.3` over every unique module in the rebuilt cache: 50,800 modules, 6 rejected, 0 of them depth-compare with an integer sampled type (the shipped 2.0.5 cache has 216 of those out of 222 rejected). The six left are four `R32ui` storage-image modules with a float sampled type (#848's class) and two fragment modules with a dominance violation, which no open PR I can find covers. Both this and #894 remove the crash on this card; this one is the narrower fix, since it corrects the sampled type instead of discarding the variant.

NotRexed commented

2026-11-09 05:17:03 +00:00

(Migrated from github.com)