fix(vk): take the shader float modes, 64-bit atomics and subgroup stages from the device #1448
No reviewers
Labels
No labels
area/accessibility
area/audio
area/input
area/install
area/menu
area/renderer
area/saves
area/tooling
area/upscaling
beanstalk: approved
beanstalk: failed
beanstalk: needs approval
beanstalk: running
beanstalk: tested
ci: approved
good first issue
gpu/amd
gpu/igpu
gpu/intel
gpu/nvidia
help wanted
kind
bug
kind
chore
kind
docs
kind
feature
kind
performance
kind
question
needs measurements
platform/linux
platform/macos
platform/steam-deck
platform/windows
preview
priority
critical
priority
high
priority
low
regression
risk
high
risk
low
risk
medium
status
blocked
status
confirmed
status
duplicate
status
invalid
status
needs-info
status
wontfix
ci: approved
No milestone
No project
No assignees
2 participants
Notifications
Due date
No due date set.
Dependencies
No dependencies set
Reference
MagicBeanDev/Longshot!1448
Loading…
Reference in a new issue
No description provided.
Delete branch "jcabarca/Longshot:fix/vk-shader-profile-caps"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Type: fix · Risk: low · Tested on: Windows 11 IoT Enterprise LTSC 24H2 (build 26100.9711), AMD Radeon RX 9070 XT, AMD Software: Adrenalin Edition 26.10.1 · Needs: none
TL;DR
The shader translator assumed every graphics card keeps special float values, handles denormals, has 64-bit buffer atomics and runs subgroup operations in every stage. A card lacking one still got shaders using it, which Vulkan forbids. This PR reads those abilities from the card. A new test fails on main and passes here.
Breakdown
Why
The shader translator's profile says every GPU keeps signed zeros, infinities and NaNs, preserves and flushes denormals, has 64-bit buffer atomics and runs subgroup operations in every stage. A GPU that lacks one of these still gets shaders that declare the mode or use the feature, which Vulkan forbids (VUID-RuntimeSpirv-shaderSignedZeroInfNanPreserveFloat32-06294, VUID-RuntimeSpirv-None-06343, VUID-RuntimeSpirv-None-06278), and the driver may refuse such a pipeline or run it wrongly.
How to reproduce
Seen on: not seen in the game: found by reading
make_profile()ingpu/vk/vk_shader.cpp. It applies to any GPU or driver that reports one of these capabilities as false:VkPhysicalDeviceFloatControlsProperties(signed zero/Inf/NaN preserve, denorm preserve and flush, denorm independence),VkPhysicalDeviceSubgroupProperties::supportedStages, orshaderBufferInt64AtomicsSettings and switches: defaults
Steps: game-free:
python -I -B tests/test_shader_profile_caps.py --cxx clang++(the test from this PR) on a main checkout; it compilesmake_profile()andprofile_pinned()against a stand-in device and checks the profile for six devicesExpected: each float mode, 64-bit atomics and each stage's subgroup support in the profile is on only when the device reports it
Actual: on main they are on for every device: the test fails on its first case, a device with none of these capabilities (
p.support_fp32_signed_zero_nan_preserve == onfails)What changed
The profile now follows what the device reports.
create_device()readsVkPhysicalDeviceFloatControlsPropertiesandsupportedStagesin the properties query it already makes, andmake_profile()takes each float mode, the subgroup stages and 64-bit atomics from them and fromshaderBufferInt64Atomics. A GPU that has everything gets the same profile as before.gpu/vk/vk_internal.h:dev.float_controls(VkPhysicalDeviceFloatControlsProperties) anddev.subgroup_stages(VkShaderStageFlags).create_device()ingpu/vk/vk_main.cpp:VkPhysicalDeviceFloatControlsPropertiesis chained into the subgroup properties query (afterVkPhysicalDeviceDriverProperties);dev.subgroup_stages = sg.supportedStages,dev.float_controls = fc(itspNextcleared).make_profile()ingpu/vk/vk_shader.cpp:VK_SHADER_FLOAT_CONTROLS_INDEPENDENCE_ALL;VK_SHADER_FLOAT_CONTROLS_INDEPENDENCE_NONE(every float type would need the same one, 06290);supported_subgroup_stagesmapped fromdev.subgroup_stages(VertexA and VertexB from the vertex bit, then tessellation, geometry, fragment, compute, task and mesh);support_int64_atomicsfromdev.vk12.shaderBufferInt64Atomics.profile_pinned(): the task and mesh stages follow the compute bit, since the mesh fallback runs those programs as compute shaders.translator_stamp(): a word "ncap" (with which capabilities are missing and the stage mask) only when something is missing, so a GPU with every capability keeps its shader and pipeline caches.tests/test_shader_profile_caps.py(new); the fixtures oftests/test_amd_subgroup_profiles.pyandtests/nonuniform_image_arrays_ir.cppget the newdevfields, every capability on, so their results are those of main.The bug, step by step:
make_profile()fillsShader::Profile, which tells the translator what the GPU can run.support_fp32_signed_zero_nan_preserve,support_fp16_signed_zero_nan_preserve,support_fp32_denorm_preserve,support_fp32_denorm_flush,support_separate_denorm_behaviorandsupport_separate_rounding_modetotrue,supported_subgroup_stagestoVK_SHADER_STAGE_ALLandsupport_int64_atomicstotrue, without looking at the device.SignedZeroInfNanPreserve,DenormPreserveorDenormFlushToZeroexecution modes, subgroup operations in any stage and 64-bit atomics on storage buffers whenever a program asks for them.The rules (validusage.json):
shaderBufferInt64Atomicsis queried intof12and that chain is whatvkCreateDevicetakes increate_device(), so the feature is enabled exactly when the device reports it;dev.vk12 = f12is whatmake_profile()reads.Before (main):
After:
What the test checks: it cuts
storage_binding_align,make_profile()andprofile()/profile_pinned()out of the tree and compiles them against a stand-indevfor six devices (every capability on or off, with subgroup stage masks 0xff, 0x30, 0x20, 0x10 and 0). For each it checks fp32 and fp16 signed zero/Inf/NaN preserve, fp32 denorm preserve and flush, separate denorm behaviour, 64-bit atomics,SupportsSubgroupStage()for every stage, and thatprofile_pinned()'s task and mesh stages follow compute.Start reading at the comment "the float execution modes the GPU takes" in
make_profile().Measurements
Verdict: it fixes the bug:
test_shader_profile_capsfails on main and passes here. Both interleaved runs gave the same result. No speed change is expected.Hardware: Windows 11 IoT Enterprise LTSC 24H2 (build 26100.9711), AMD Radeon RX 9070 XT, AMD Software: Adrenalin Edition 26.10.1; the test is game-free (it compiles a cut-out of
make_profile()with llvm-mingw, it does not use the GPU); Python 3.12Scene: none, game-free
Method: the test from this PR runs against each side's checkout, 2 runs each, interleaved (main, PR, main, PR); the result is pass/fail, so the median is the result both runs gave
How to reproduce:
python -I -B tests/test_shader_profile_caps.py --cxx clang++from each side's checkout6d949cf92f5babd1tests/test_shader_profile_caps.py, run 1tests/test_shader_profile_caps.py, run 2tests/test_shader_profile_caps.py, medianFlags and settings
None.
Testing
Tested: on this branch, Windows, Python 3.12, llvm-mingw clang++:
test_shader_profile_capspasses (fails on main), and the other checks that readmake_profile()pass:test_amd_subgroup_profiles,test_amd_fragment_warp,test_native_float_atomic_add,test_vendor_fixes,test_predict_types.test_nonuniform_image_arrays's modules all translate, pass spirv-val and are byte for byte main's (--base-tree), but the test fails on its device check (create_device: no 'ci.pNext = &f;' before vkCreateDevice) on main and here alike.Built and played: this commit builds on Windows (llvm-mingw back end) and on Linux (
longshot_vk.so, without and with AMD FSR 4). Played on Windows 11 IoT Enterprise LTSC 24H2 (build 26100.9711), AMD Radeon RX 9070 XT, AMD Software: Adrenalin Edition 26.10.1: the game runs normally.Not tested: a GPU that lacks one of these capabilities: the RX 9070 XT's driver reports them, so the new paths (a mode or stage left out) did not run on a GPU; the test covers them with stand-in devices. Linux was built, not run.
Must be tested: None.
Risks
What could break: on a GPU that reports a capability as false, shaders that used it are translated without it: no signed zero/Inf/NaN or denorm execution mode (the driver's default float behaviour), the translator's fallbacks without subgroup operations in stages that lack them, and, for a 64-bit storage atomic, its fallback in
emit_spirv_atomic.cpp: a plain load and store, which is not atomic if two invocations hit the same address. Those programs may give different results there, but the pipelines no longer break the rules above. A GPU that reports everything gets the same profile and the same modules as before.Turning it off: revert the commit.
Compatibility: On GPUs that report every capability nothing changes, the shader and pipeline caches included. On a GPU that lacks one,
translator_stamp()changes, so its shaders are translated again and its pipeline cache rebuilt once.Checklist
Everyone
type(scope): description) of the status line's type.main.python -I -B tests/run_all.pypasses (any failure that also fails onmainis named). (the checks that read the changed code, on this PR, Windows: all pass excepttest_nonuniform_image_arrays's device check, which fails onmaintoo)third_party/untouched (a new folder with aPROVENANCE.mdinstead).When it applies
Assisted-by:/Co-Authored-By:trailers.mainand passing here.tests/that reads changed source updated in this PR, not weakened. (tests/test_amd_subgroup_profiles.pyandtests/nonuniform_image_arrays_ir.cppget the new device fields, every capability on)Tested in game: Windows 11 IoT Enterprise LTSC 24H2 (build 26100.9711), AMD Radeon RX 9070 XT, AMD Software: Adrenalin Edition 26.10.1: played with this change; the game runs normally, no rendering errors or crashes seen.
Brought up to the ready-for-review standard (#1454): the checklist's Everyone list has two new boxes, added here from what this PR's description and comments show (please check they match what you know).
7ea4c0b804a69aeea6c1Rebased onto main @
c14641a6(no conflicts in this change). The test was run again, interleaved main, PR, main, PR: it fails on main (the profile does not follow the device) and passes ona69aeea6both times. The measurements in the description are updated.Description: added the two Everyone boxes (tested on my own PC, nothing gets worse that I know of): the change was played on an AMD Radeon RX 9070 XT on Windows with no regression seen, and it only changes which capabilities the profile turns on, so it has no runtime cost.
WIP: fix(vk): take the shader float modes, 64-bit atomics and subgroup stages from the deviceto fix(vk): take the shader float modes, 64-bit atomics and subgroup stages from the devicec3872ae3692f5babd1caRebased onto main @
6d949cf9; the branch head is now2f5babd1. New since the last update: the test now also covers the pins_32 data and dev.policy. Measured again on the new base: tests/test_shader_profile_caps.py fails on main and passes on this PR, in both interleaved runs (Measurements updated). The description also gains a TL;DR and a Breakdown.Rebased onto main @
7daafa7a; the related tests pass on it.Added cost (read from the code, not measured in a game): make_profile reads float-control and atomic features; stamp changes only if one is missing; how often: at start and shader translation. On an AMD Radeon RX 9070 XT: yes; no per-frame cost. The box for nothing getting worse stays unticked until this is measured.
8d1a50fe098ddfcae7f4View command line instructions
Checkout
From your project repository, check out a new branch and test the changes.