fix(vk): build task and mesh programs with the optimizer on RADV #1965
No reviewers
Labels
No labels
accessibility
autorelease: pending
autorelease: tagged
bug
dependencies
documentation
duplicate
enhancement
github_actions
good first issue
help wanted
invalid
question
testing wanted
wontfix
No milestone
No assignees
1 participant
Due date
No due date set.
Dependencies
No dependencies set
Reference
MagicBeanDev/Hookshot!1965
Loading…
Reference in a new issue
No description provided.
Delete branch "fix/radv-mesh-optimizer"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
On Linux with Mesa's RADV driver, the GPU resets while the save loads at the Great Deku Tree on the Steam Deck (and RDNA2 desktop cards) unless players find
RX_VK_MESH_OPT=1in an issue thread. The cause is the PR 430 rule that builds task and mesh programs without the driver's optimizer on every AMD GPU. That rule was made and tested on AMD's Windows driver; on RADV, the unoptimized foliage task program runs long enough to trip the GPU timeout.What changed
build_graphics(gpu/vk/vk_draw.cpp) now applies the no-optimizer rule to AMD's own drivers only, not toVK_DRIVER_ID_MESA_RADV. On RADV, task and mesh programs are built with the optimizer, like every other pipeline. AMD on Windows is unchanged, andRX_VK_MESH_OPT=1still forces the optimizer on any driver. The check readsdev.driver_id, whichcreate_devicealready stores.tests/test_radv_mesh_optimizer.pychecks that RADV stays out of the rule and the override stays.What it doesn't fix
This fixes the reset while the save loads. Other RADV hangs remain, and the optimizer doesn't cure them: rere225 (RX 6900 XT) still hangs entering the Great Deku Tree with
RX_VK_MESH_OPT=1, and KewinBR (Deck LCD) still hangs in Lon Lon Ranch's open view with it (RX_VK_FORCE_MESH_FALLBACK=1played). Both runs had the optimizer on already, so neither is a regression from this change.Testing
Steam Deck OLED: RADV VANGOGH, Mesa 26.2.0-devel, SteamOS 3.9.5. v2.0.14 with only this change, Kokiri Forest, loading the same save each run,
RX_VK_MESH_OPTnot set:ring gfx_0.0.0 timeoutand MODE2 reset while the save loads. The log says "task and mesh programs built without the driver's optimizer"RX_VK_MESH_OPT=1on the release)RX_VK_MESH_OPT=1on the release has also run about 30 minutes across five launches with no reset on this Deck. @manianac reports the same on an RX 6800 XT, RADV 26.2.6 (#1000).tests/run_all.py(Ubuntu 22.04 with g++-13, as in CI): 321 passed, 13 failed, 11 skipped, both onmainand with this change: the same 13 fail without it (tools missing in my container), none new.Related: #1000, #817, #430.
🤖 Generated with Claude Code
Thanks. The change does what the description says, and it's narrow. It's the same as setting
RX_VK_MESH_OPT=1by default on RADV, which is what Deck and RADV desktop users already do by hand.Detection.
vendorID == 0x1002 && driverID != VK_DRIVER_ID_MESA_RADVis the right axis: the hang depends on the driver, not the chip. It covers every RADV GPU with mesh shaders: RDNA2 desktop (6800 XT, 6900 XT), the Deck (VanGogh, LCD and OLED), RDNA3/4 and the APUs. RDNA1 has no mesh shaders, so it runs the issue 640 fallback and never builds task/mesh graphics pipelines. Unchanged: AMD's Windows driver, and AMDVLK on Linux (AMD_OPEN_SOURCE, the workaround NotRexed posted in #1000). Both keep the PR 430 rule.Correctness and the cloth hang. The cloth guard (
p.key == 0x5fa56485a6b07619, #461/#1540) is a separate block:_WIN32plusVK_DRIVER_ID_AMD_PROPRIETARY. This PR doesn't touch it, and it never applied on RADV. Windows AMD still builds task/mesh programs without the optimizer, so the cloth hang and the RX 6600 hang that PR 430 was made for both stay avoided there. Nothing comes back on that side.Compile time. Task/mesh pipelines on RADV now go through ACO's full optimizer, so a cold cache spends a bit longer on them. The driver cache is keyed on the create flags, so the first start after this change rebuilds those pipelines once. Main has no optimizer time budget any more (the old
build_raced), so a slow build just takes longer. If you have it, the shader-prep time on the Deck before and after would be worth adding.Evidence, and what it doesn't cover. Your Deck OLED A/B plus about 30 min with
RX_VK_MESH_OPT=1, and manianac's RX 6800 XT RADV report in #1000, both support it. But two newer reports in #1000 show the optimizer is not the whole RADV story:1002:73af): still hangs every time entering the Great Deku Tree withRX_VK_MESH_OPT=1, a real hang, still stuck at a 30 s lockup timeout.RX_VK_MESH_OPT=1.RX_VK_FORCE_MESH_FALLBACK=1played 15 min.Both also had
RX_VK_MESH_NOCULLset, so the setup differs a little. Neither is a regression from this PR, since they already had the optimizer on. Still, I'd soften "and built with it they play" in the code comment and the description: this fixes the reset while the save loads, and some RADV hangs remain. On main,RX_VK_TASK_MESH_FALLBACKis read outside the#ifdef _WIN32, so rere225's request to try it on RADV Navi 21 works with a main build already.Fit with main. The PR is 48 commits behind, but it merges cleanly. It is independent of the Mesh shaders row (#1971): under Native or Automatic, task/mesh pipelines are built with the optimizer on RADV, and under Compute none get built. It is also independent of the mesh-query gating (
dev.mesh_queries, RADV hasmeshShaderQueriesoff on RDNA2).Relation to #1992 (Navi 21 on the task + mesh compute fallback on Windows): no shared code. #1992 only edits the
#ifdef _WIN32default in vk_main.cpp, and this one only edits theamdcheck in vk_draw.cpp, so they can land in either order.Before merge:
action_required).tests/that cuts the block out by text. A smalltests/test_radv_mesh_optimizer.py, asserting thatVK_DRIVER_ID_MESA_RADVis excluded and theRX_VK_MESH_OPToverride stays, would keep this from being undone by accident. Optional, but in keeping with the repo.The code itself looks right to me. Once CI is green, I have no concerns about the change.
Nit:
create_devicealready stores this asdev.driver_id(vk_main.cpp, the subgroup block), andvk_shader.cppreads it the same way (dev.driver_id == VK_DRIVER_ID_AMD_PROPRIETARY). So this could bedev.props.vendorID == 0x1002 && dev.driver_id != VK_DRIVER_ID_MESA_RADVwith no second properties query.Additional Steam Deck LCD measurements:
3601726and pinned mainHardware in the test logs: Steam Deck LCD, RADV VANGOGH, driver
0x6801063.Legacy
3601726baseline: enablingRX_VK_MESH_OPT=1on the original renderer let the opening sequence reach player control, where earlier trials had failed. It did not resolve the later color/particle defects or camera hitches. Our subsequently tested legacy branch combines the RADV optimizer policy with upstream-derived DMA-buf imports, allocator-aware texture squeeze, and smaller/growing recording streams. It is not an optimizer-only performance result.In a matched legacy host/renderer/save/settings DMA A/B, settled room samples rose from roughly 20 to 25–26 fps and queue-submit CPU timing fell from roughly 21 to 2 ms. That gain belongs to the DMA change, not this PR. The combined legacy build was subjectively playable during Kokiri traversal. A separate 30-minute mixed play/menu/view observation had median 20.0 fps and range 10.2–27.7 in five-second VI reports, with no captured kernel timeout. This is not locked-25/30-fps or continuous-route acceptance.
Pinned main
ebbba2ea868458cde654a008e481f474841f0dff: official nightly toolchain and matching packaged renderer, FSR 3.1 Performance, frame generation off, isolated copies of the same Kokiri save:gfx_0.0.0 timeout, a GPU reset, andVK_ERROR_DEVICE_LOST.RX_VK_MESH_OPT=1added: Kokiri loaded and rendered terrain/characters; no kernel timeout during the captured gameplay window. The sampled docked Saria view was about 16 fps.The reset and cache-stamped option change prevented a cache-matched performance A/B, so I am treating this as additional hardware corroboration for the RADV policy, not proof of a universal speedup or a complete correctness fix.
Public legacy source milestone (not current-main tested source patches):
github.com/omgitsgela/Hookshot-SteamDeck@8d1347f977. No game files, saves, cache files, or raw game-data logs attached.Note from the duplicate #2311 (closed): on the Steam Deck (Van Gogh), the foliage culling mesh program 89ca750f9852b7ca built without the optimizer wedged the GPU about 30 s after a save loaded in Kokiri Forest, every run (ring gfx timeout, then a reset); built with it, the forest plays.
Thanks. All three are in c81cf977:
dev.driver_id. The second properties query is gone.tests/test_radv_mesh_optimizer.pychecks that RADV stays excluded andRX_VK_MESH_OPTstays. I checked that it fails if the RADV exclusion is removed.The renderer builds clean. I don't have before/after shader-prep times on the Deck yet.
🤖 Generated with Claude Code
This is still needed: #2311 was closed as a duplicate of this PR, not merged, and main still builds task/mesh programs without the optimizer on every AMD driver (
gpu/vk/vk_draw.cpp:2187). Your original commit merges cleanly and run_all passes on it (376 passed, Linux).The new commit c81cf97 conflicts with main, though. Since #2042, main reads the override as
env_on("RX_VK_MESH_OPT")(vk_draw.cpp:2192), and the branch still hasgetenv(...) != nullptr. Either way of resolving it fails a test. If you keepgetenv,tests/test_env_on.pyflagsRX_VK_MESH_OPT (gpu/vk/vk_draw.cpp:2191). If you keepenv_on, your newtests/test_radv_mesh_optimizer.pyfails with "RX_VK_MESH_OPT=1 no longer builds them with the optimizer", because it looks for thegetenvtext. I tried both. The fix is to rebase, keep youramdline withdev.driver_id(create_device always sets it), keep main'senv_on, and change the test's needle toenv_on("RX_VK_MESH_OPT"). Usingdev.driver_idand the softer comment both look right to me. Since the RC is today, this is worth doing quickly.Done in
55c48f7: the rule readsdev.props.vendorIDanddev.driver_idnow, no second properties query. Thanks.Thanks for the rebase. Everything from my last review and from Joey-Magnus's is addressed at
55c48f7:env_on("RX_VK_MESH_OPT"), and the test's needle is now"RX_VK_MESH_OPT", so it matchesenv_onandtests/test_env_on.pyhas nothing to flag. Against main, the only change invk_draw.cppis theamdline and its comment. It's 8 commits behind but merges cleanly, and none of those commits touch this block.dev.driver_id: done.create_devicesetsdev.props(vkGetPhysicalDeviceProperties) anddev.driver_id(the driver-properties query) before any pipeline gets built, so thestaticis safe to initialize on the first build.tests/test_radv_mesh_optimizer.pychecks the window from theamdline totold++for the AMD check, the RADV exclusion, the override, theif (amd && !mesh_opt)gate and the flag. It needs no arguments, so it's fine with noRUN_ALL_declarations.The other evidence agrees with this: NotRexed's note from the closed duplicate #2311 (Van Gogh, the foliage program built without the optimizer wedges the GPU about 30 s after the save loads, plays with it), and omgitsgela's Deck LCD run on pinned main (the device is lost on the default policy, Kokiri loads with only
RX_VK_MESH_OPT=1added).With #2362 (Automatic picks Compute on Linux AMD, issue 2245): the two don't share code. #2362 edits
vk_main.cpp/vk_menu.cppand this one editsvk_draw.cpp, so they merge in either order. Both are worth having. Once #2362 is in, Automatic on RADV builds no task/mesh pipelines at all, so this PR then matters for anyone who picks Native by hand (and for every RADV user until #2362 lands). It fixes the load reset there, and #2362's default covers the hangs this one leaves open. I'd merge this one first: it's narrower, it has been tested on the Deck by several people, and it doesn't depend on the #2362 decision.The only thing left before merge is CI. The three workflows at this head are still
action_requiredand need a maintainer to approve them. Approving on the code.github-sync referenced this pull request2026-11-12 06:22:54 +00:00