perf(vk): enable storage table loop and mesh optimizer on AMD RDNA 4 #1308
No reviewers
Labels
No labels
area/accessibility
area/audio
area/input
area/install
area/menu
area/renderer
area/saves
area/tooling
area/upscaling
beanstalk: approved
beanstalk: failed
beanstalk: needs approval
beanstalk: running
beanstalk: tested
ci: approved
good first issue
gpu/amd
gpu/igpu
gpu/intel
gpu/nvidia
help wanted
kind
bug
kind
chore
kind
docs
kind
feature
kind
performance
kind
question
needs measurements
platform/linux
platform/macos
platform/steam-deck
platform/windows
preview
priority
critical
priority
high
priority
low
regression
risk
high
risk
low
risk
medium
status
blocked
status
confirmed
status
duplicate
status
invalid
status
needs-info
status
wontfix
ci: approved
No milestone
No project
No assignees
4 participants
Notifications
Due date
No due date set.
Dependencies
No dependencies set
Reference
MagicBeanDev/Longshot!1308
Loading…
Reference in a new issue
No description provided.
Delete branch "gmourao/Longshot:perf/vk-rdna4-table-loop-mesh-opt"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Type: perf · Risk: medium · Tested on: Windows 11, AMD Radeon RX 9060 XT (Adrenalin 26.10.1 LLPC,
2+2interleavedmain @ 79d4a143vs4173f4b7; rebased cleanly ontomain @ 430eef4fasbb95fcd9) · Needs: Windows, RDNA 2 / Linux, RDNA 4 (RADV)TL;DR
On AMD RX 9000 series (RDNA 4) GPUs on Windows, unrolled global-memory lookup tables bloat SPIR-V shaders and stall on-the-fly pipeline compilation (
135.1draws left out/frame,259queued jobs onmain), while Task/Mesh shaders build without driver optimization. This PR enablesglobal_table_loop(matching Intel Mesa's default) and the driver's mesh optimizer on RDNA 4 (RX 9/GFX12). On an RX 9060 XT inkokiri-start, cold compilation finished747pipelines in317.4 s(424.9 ms/pipe,0draws left out) vs569in357.8 s(628.8 ms/pipe,-32.4%), trading+0.2 ms(9.6 -> 9.8 ms,-0.8%FPS) on warm frames.Breakdown
259queued jobs,135.1skipped draws/frame), while Task/Mesh pipelines built withVK_PIPELINE_CREATE_DISABLE_OPTIMIZATION_BITon RDNA 4.RX 9/GFX12) defaultsglobal_table_loop(on AMD proprietary) andmesh_optto enabled, withRX_VK_GM_LOOPandRX_VK_MESH_OPToverrides intact.origin/main @ 430eef4f(bb95fcd9), targeted checks passed (5passed inrun_all.py, plustest_doc_anchors.pyandcheck_third_party.py), earlier--changed origin/mainon4173f4b7(153passed,9skipped), standalonelongshot_vk.dllbuild, cold/on-the-fly warmup comparison, and isolated2+2interleaved warm runs on RX 9060 XT.RX 6600/ Kokiri Forest) and Linuxclang/gccbuild + fulltests/run_all.pyon RADV (requested from Test Farm).DontUnrollSSBO lookup loops trade+0.2 ms(~+0.124 msin compute dispatches,-0.8%FPS) on warm frames to eliminate the-32.4%/pipeline on-the-fly compile backlog (0draws left out vs135.1), and either path can be disabled withRX_VK_GM_LOOP=0andRX_VK_MESH_OPT=0.Why
On AMD RDNA 4 GPUs running AMD's proprietary Windows driver (LLPC), unrolled storage buffer lookup tables in global-memory access helpers generate huge SPIR-V shaders (>100k instructions) that stall driver compilation and starve render worker queues when new shader variants arrive on the fly (leaving
259jobs queued and135.1draws skipped per frame in our coldkokiri-starttest, causing visible objects/surfaces to pop in late or vanish), while Task/Mesh pipelines default toVK_PIPELINE_CREATE_DISABLE_OPTIMIZATION_BIT. Enablingglobal_table_loop(the same trade-off already shipped by default for Intel Mesa) cuts per-pipeline LLPC compile time by32.4%and empties the compile queue completely (0queued jobs,0.0draws left out) for a minor+0.2 ms(-0.8%) warm-frame dispatch cost, whilemesh_no_optimizer=0stops crippling RDNA 4 Task/Mesh codegen—leaving GCN, RDNA 1-3, RADV, Intel, NVIDIA, and compute fallback behavior unchanged.What changed
Added an RDNA 4 detection helper, scoped both
global_table_loop(on AMD proprietary) and the defaultmesh_optflag inbuild_graphics()to RDNA 4 GPUs, and syncedgpu/vk/device_policy.h(mesh_no_optimizerandglobal_table_loop) along with its tests and documentation, preserving environment variable overrides, compute fallback policy, and existing test-extraction boundaries.gpu/vk/vk_internal.h,gpu/vk/device_policy.h):is_rdna4()andDeviceFacts::rdna4()match AMD (0x1002) devices whosedeviceNamecontainsRX 9orGFX12;device_policy()clearsmesh_no_optimizeron RDNA 4 and enablesglobal_table_loopon RDNA 4 withVK_DRIVER_ID_AMD_PROPRIETARYand dynamic SSBO array indexing.gpu/vk/vk_coopmat.cpp):table_loop_wanted()takesbool rdna4and defaults to enabled onVK_DRIVER_ID_INTEL_OPEN_SOURCE_MESAorrdna4 && driver == VK_DRIVER_ID_AMD_PROPRIETARY(requiringdynamic_indexing, withRX_VK_GM_LOOPoverride intact).gpu/vk/vk_draw.cpp):build_graphics()computesmesh_optinline (self-contained fortests/test_fallback_pipeline_state.py), defaulting totrueon RDNA 4 whenRX_VK_MESH_OPTis unset or empty while preservingVK_PIPELINE_CREATE_DISABLE_OPTIMIZATION_BITon GCN and RDNA 1-3 and leavingfallback_compute_pipelines()unchanged.tests/test_global_table_loop.py,tests/test_radv_mesh_optimizer.py,tests/test_device_policy.py,docs/development/device-policy.md): verifiedis_rdna4(), RDNA 4 defaults, GCN/RDNA 1-3/RADV/NVIDIA/Intel preservation,RX_VK_GM_LOOP/RX_VK_MESH_OPToverrides, and updated theRX 9070 XT, AMD's Windows driver (LLPC)profile.Measurements
Verdict: intentional compile-time vs warm-GPU trade-off (matching Intel Mesa's
global_table_looprationale) — cold/on-the-fly pipeline compilation is32.4%faster per pipeline, completing747pipelines in317.4 s(424.9 ms/pipe,0jobs queued,0.0draws left out) vs569in357.8 s(628.8 ms/pipe,259queued,135.1draws left out) onmain, eliminating missing draws when shaders compile on the fly; in warm2+2steady state (compiled 0),DontUnrollSSBO table loops add+0.2 msGPU time (9.6 -> 9.8 ms,+0.124 msinm dispatchand+0.049 msinm pass) and-0.46 fps(55.99 -> 55.53 fps,-0.8%). After rebasing cleanly ontomain @ 430eef4f(bb95fcd9), these numbers still hold because79d4a143..430eef4fdid not altervk_coopmat.cpp,device_policy.h,vk_internal.h, orbuild_graphics()'smesh_optlogic.Hardware: Windows 11 (build 26300), AMD Radeon RX 9060 XT (16 GB, driver 26.10.1 LLPC,
1002:7590), AMD Ryzen 7 5700X (16 logical CPUs), 32 GB RAMScene:
kokiri-start(KokiriForest_Child);output_res=2560x1440,dlss=0(Upscaling = Game),resolution=1(100%),fullscreen=0,vsync=0,frame_rate_cap=0,HOS_FPS_MAX=1000,RX_VK_PASS_TIMING=2Method: separate per-arm cache folders (
RX_VK_PRECOMPILE=0cold warmup recorded once per arm, then 4 interleaved warm runsmain,PR,main,PRwithcompiled 0verified inprogram census (close));15 ssettle +45 smeasurement window vialongshot_ctl.pyandlongshot_log.txtfiltered to>= s0_frame; midpoint of each arm's two warm runs reported alongside both individual runsHow to reproduce:
python -I scripts/longshot_ctl.py --json launch <arm_env> --repo-save kokiri-start --ready play --env HOS_OPTIONS_FILE=<ini_2560x1440_dlss0_res1> --env HOS_FPS_MAX=1000 --env RX_VK_PASS_TIMING=279d4a1434173f4b7warm_*at 15480 presents)569pipelines in357,804 ms(628.8 ms/pipe;13,096 mstr,259queued)747pipelines in317,423 ms(424.9 ms/pipe;11,043 mstr,0queued)-32.4%ms/pipeline (0draws left out)ctl_fps, 45 s, A1/A2 vs B1/B2)56.21/55.77->55.99 fps55.61/55.45->55.53 fps-0.46 fps(-0.8%)56.2/55.75->55.98 fps55.7/55.5->55.60 fps-0.38 fps(-0.7%)9.6/9.6->9.6 ms9.8/9.8->9.8 ms+0.2 ms(+2.1%)m dispatch/m pass(block A1 vs B1)4.883 ms/4.038 ms5.007 ms/4.087 ms+0.124 ms/+0.049 msFlags and settings
No new flags added; changes the default on AMD RDNA 4 for two existing environment switches.
RX_VK_GM_LOOPDontUnrollloop628.8 -> 424.9 ms/pipe,-32.4%;0draws left out vs135.1) at+0.2 ms(9.6 -> 9.8 ms) warm GPU cost+0.124 msdispatch time in warmkokiri-startRX_VK_MESH_OPTDISABLE_OPTIMIZATION_BITon RDNA 4 Task/Mesh shadersRX 6600hangs)Testing
Tested: on Windows 11 (
AMD Radeon RX 9060 XT, Adrenalin26.10.1LLPC,llvm-mingwclang++.exe) on rebased4173f4b7:2+2runs (main @ 79d4a143vsthis PR @ 4173f4b7) inkokiri-start(2560x1440,Upscaling = Game100%): verifiedglobal_table_loop=1,mesh_no_optimizer=0,rdna4=1,vk: storage buffer tables searched in loops,0unoptimized task/mesh fallback logs,-32.4%cold pipeline compile time per pipeline, and+0.2 ms(9.6->9.8 ms) /-0.46 fpswarm steady-state delta.longshot_vk.dll(RX_VK_ONLY=1,llvm-mingw) compiled in31 s(0errors, linked14 MBin4 s).bb95fcd9(origin/main @ 430eef4f):tests/run_all.pyfortest_device_policy.py,test_global_table_loop.py,test_radv_mesh_optimizer.py,test_fallback_pipeline_state.py, andtest_utf8_paths.pypassed (5 passed, 0 failed), alongsidescripts/check_third_party.py(14parts) andtests/test_doc_anchors.py(15anchors). Earlier on4173f4b7,tests/run_all.py --changed origin/mainpassed153checks (with9skipped for Linux/spirv-val, and10pre-existing unrelated Windows failures onorigin/main:test_draw_stream_visibility,test_pipeline_play_cache,test_compute_warm_apart,test_upscaler_rows,test_recorder_sync,test_fsr_uniform_ownership,test_dev_packs,test_linux_pressure,test_replay_write_order,test_vk_occlusion_queries).Not tested: Linux (
clang/gccbuild, fulltests/run_all.py), AMD RDNA 4 on Linux RADV, AMD RDNA 2 regression check, andRX_VK_MESH_SHADERS=nativein gameplay.Must be tested:
VK_PIPELINE_CREATE_DISABLE_OPTIMIZATION_BITto remain active (task and mesh programs built without the driver's optimizer) and no device lost (Test Farm: @Beanstalk Please test on Windows with an AMD RX 6000 / RDNA 2 GPU on proprietary drivers in Kokiri Forest, confirming the log still printstask and mesh programs built without the driver's optimizerand no device lost occurs)radv): build host code and runpython -I -B tests/run_all.pyplus a gameplay smoke test, expecting unchanged behavior and0failures (Test Farm: @Beanstalk Please test on Linux with RADV by runningpython -I -B tests/run_all.pyand a gameplay smoke test)Risks
What could break:
DontUnrollSSBO table loops trade+0.2 ms(9.6 ms -> 9.8 ms,-0.46 fps/-0.8%) per warm frame inkokiri-starton RX 9060 XT in order to cut cold/on-the-fly pipeline compile time by32.4%(628.8 -> 424.9 ms/pipeline) and prevent missing draws (135.1 -> 0.0skipped draws/frame) during streaming/cutscenes; separately, an RDNA 4 driver regression with optimized Task/Mesh pipelines (RX_VK_MESH_SHADERS=native) or indexed SSBO tables could affect rendering or stability.Turning it off: set
RX_VK_GM_LOOP=0andRX_VK_MESH_OPT=0, or revert the commit.Compatibility:
global_table_loopis part oftranslator_stamp(), so on AMD RDNA 4 proprietary Windows drivers the first launch after updating invalidates and rebuildslongshot_programs.binandlongshot_vk_pipelines.binonce (all other GPUs keep existing caches untouched); saves,longshot_options.ini, andMKWVK_API_VERSION(33) are unchanged.Checklist
Everyone
type(scope): description) of the status line's type.can't be run by hand is in Testing's Not testable by hand, with why.
in Risks.
main.python -I -B tests/run_all.pypasses (--changed origin/mainon4173f4b7:153passed,9skipped for Linux/spirv-val,10pre-existing unrelated Windows failures onorigin/mainnamed in Testing).third_party/untouched (a new folder with aPROVENANCE.mdinstead).When it applies
tests/that reads changed source updated in this PR, not weakened.Tested in game: Windows 11, AMD Radeon RX 9060 XT (26.10.1): default env (no overrides), storage buffer tables searched in loops active, 0 unoptimized mesh logs, 185.2 avg / 230.3 med FPS, 1.82 ms avg / 0.40 ms med GPU
@Beanstalk Please compare this PR against main
e41de662on Windows with an AMD Radeon RX 9000-series GPU and proprietary driver using default settings without RX_VK_GM_LOOP or RX_VK_MESH_OPT overrides, warm up then run main/PR/main/PR in the same scene and report frame-time, FPS and preparation results@gmourao only members of
BigBeanscan start the farm.One of them can approve this request by reacting 👍 to it or to this comment.This request was ford79a058, and the PR has changed since (e7c73a5), so a 👍 no longer approves it. Comment@Beanstalkagain to ask for the new commit.Brought up to the ready-for-review standard (#1454): the checklist's Everyone list has two new boxes, added here from what this PR's description and comments show (please check they match what you know).
d79a0580bae7c73a55cfRebased onto main @
310e7c9d(e7c73a55), added TL;DR and Breakdown, and added the measured one-time shader cache rebuild time in Risks.@Beanstalk Please compare this PR against main
310e7c9don Windows with an AMD Radeon RX 9000-series GPU and proprietary driver using default settings without RX_VK_GM_LOOP or RX_VK_MESH_OPT overrides, warm up then run main/PR/main/PR in the same scene and report frame-time, FPS and preparation results@gmourao only members of
BigBeanscan start the farm.One of them can approve this request by reacting 👍 to it or to this comment.This request was fore7c73a5, and the PR has changed since (5e8d920), so a 👍 no longer approves it. Comment@Beanstalkagain to ask for the new commit.e7c73a55cf5e8d920181Synced RDNA 4 mesh_no_optimizer and global_table_loop rules in gpu/vk/device_policy.h, tests/test_device_policy.py, and docs/development/device-policy.md.
@Beanstalk Please compare this PR against main
310e7c9don Windows with an AMD Radeon RX 9000-series GPU and proprietary driver using default settings without RX_VK_GM_LOOP or RX_VK_MESH_OPT overrides, warm up then run main/PR/main/PR in the same scene and report frame-time, FPS and preparation results@gmourao only members of
BigBeanscan start the farm.One of them can approve this request by reacting 👍 to it or to this comment.This request was for5e8d920, and the PR has changed since (4173f4b), so a 👍 no longer approves it. Comment@Beanstalkagain to ask for the new commit.Farm readiness review for
5e8d9201: the PR changes the default for RDNA 4 and still lists its isolated comparison and full suite as outstanding. Please publish the isolated main/PR runs and full-suite result first. For volunteer coverage, name the Windows RDNA 4 scene and how the logs prove both new defaults engaged. State separately how shader preparation will be measured if it is part of the claim.5e8d9201814173f4b778Rebased onto main @
79d4a143(4173f4b7) and completed isolated 2+2 measurements on Windows 11 / AMD RX 9060 XT (kokiri-start, 2560x1440, Upscaling = Game 100%). Cold pipeline compilation improved (-32.4% per pipeline: 747 pipelines in 317,423 ms with 0 draws left out vs 569 in 357,804 ms with 135.1 draws left out on main), while steady-state warm 2+2 runs showed a +0.2 ms GPU trade-off (9.6 -> 9.8 ms, +0.124 ms in m dispatch, 55.99 -> 55.53 fps) from DontUnroll SSBO table loops. Keeping WIP until deciding whether global_table_loop stays default-on alongside mesh_no_optimizer=0. Local checks: 153 passed, 9 skipped (Linux/spirv-val), 10 pre-existing origin/main Windows failures.@Beanstalk Please compare this PR (
4173f4b7) against main79d4a143on Windows with an AMD Radeon RX 9000-series GPU (RX 9060 XT / RX 9070 XT) and RX 6000 / RDNA 2 GPU on proprietary drivers in Kokiri Forest, plus Linux RADV with python -I -B tests/run_all.py.@gmourao only members of
BigBeanscan start the farm.One of them can approve this request by reacting 👍 to it or to this comment.This request was for4173f4b, and the PR has changed since (bb95fcd), so a 👍 no longer approves it. Comment@Beanstalkagain to ask for the new commit.Clarified the compile-time vs warm-GPU trade-off and marked ready for review: just as global_table_loop is already enabled on Intel Mesa to prevent giant unrolled SSBO lookup shaders from choking driver compilation, on AMD RDNA 4 Windows LLPC it cuts cold/on-the-fly compilation by 32.4% per pipeline (747 pipelines built in 317.4 s with 0 queued jobs and 0.0 draws left out vs 569 pipelines in 357.8 s with 259 queued jobs and 135.1 skipped draws/frame on main), eliminating late pop-in/missing geometry during cutscenes or area transitions at a +0.2 ms (9.6 -> 9.8 ms, -0.8% FPS) warm dispatch cost (with RX_VK_GM_LOOP=0 available as override).
WIP: perf(vk): enable storage table loop and mesh optimizer on AMD RDNA 4to perf(vk): enable storage table loop and mesh optimizer on AMD RDNA 44173f4b778bb95fcd9efRebased onto main @
430eef4f(bb95fcd9). Confirmed 79d4a143..430eef4f did not modify vk_coopmat.cpp, device_policy.h, vk_internal.h, or build_graphics()'s mesh_opt logic, so the79d4a143vs4173f4b7measurements still hold. Re-ran test_device_policy.py, test_global_table_loop.py, test_radv_mesh_optimizer.py, test_fallback_pipeline_state.py, test_utf8_paths.py (5 passed, 0 failed), test_doc_anchors.py, and check_third_party.py.@Beanstalk Please compare this PR (
bb95fcd9) against main430eef4fon Windows with an AMD Radeon RX 9000-series GPU (RX 9060 XT / RX 9070 XT) and RX 6000 / RDNA 2 GPU on proprietary drivers in Kokiri Forest, plus Linux RADV with python -I -B tests/run_all.py.@gmourao only members of
BigBeanscan start the farm. One of them can approve this request by reacting 👍 to it or to this comment.View command line instructions
Checkout
From your project repository, check out a new branch and test the changes.