fix(vk): a load-at-create start whose mesh fallback builds stall in the driver goes on to the title, with progress on the way #1450
No reviewers
Labels
No labels
area/accessibility
area/audio
area/input
area/install
area/menu
area/renderer
area/saves
area/tooling
area/upscaling
beanstalk: approved
beanstalk: failed
beanstalk: needs approval
beanstalk: running
beanstalk: tested
ci: approved
good first issue
gpu/amd
gpu/igpu
gpu/intel
gpu/nvidia
help wanted
kind
bug
kind
chore
kind
docs
kind
feature
kind
performance
kind
question
needs measurements
platform/linux
platform/macos
platform/steam-deck
platform/windows
preview
priority
critical
priority
high
priority
low
regression
risk
high
risk
low
risk
medium
status
blocked
status
confirmed
status
duplicate
status
invalid
status
needs-info
status
wontfix
ci: approved
No milestone
No project
No assignees
2 participants
Notifications
Due date
No due date set.
Dependencies
No dependencies set
Reference
MagicBeanDev/Longshot!1450
Loading…
Reference in a new issue
No description provided.
Delete branch "HumanBean/Longshot:fix/load-at-create-stall"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Type: fix · Risk: medium · Tested on: Linux, RTX 3090 (the start through the changed function; game-free checks left to CI) · Needs: Windows, AMD without mesh shaders (RX 5600 XT / 5700 XT, RDNA1; Vega)
Why
On RX 5600 XT and RX 5700 XT cards (AMD's Windows driver 26.5.2 and 26.10.1), rc.7 never gets past "Loading pipelines". After the update the first start reaches 100% and then freezes. Every start after that stays at 0% on "Loading the shader cache" until the window is closed. Setting
RX_VK_LOAD=startreaches the title on the same install (zimoroque's tests on #1373). An rc.5 report on a Vega iGPU (7730U, #1262) ends at the same point.The cause is
precompile_at_createingpu/vk/vk_draw.cpp. UnderRX_VK_LOAD=create, the default since #975 on the compute fallback too, the start calls it before the title. On a GPU without mesh shaders it builds every mesh fallback program's compute pipelines in the driver there. It also asks the caches for those programs' noted pipelines. It never tells the start-up screen anything, so the screen shows!total, which reads "Loading the shader cache" at 0% (vk_progress.h). It then joins its threads with no bound.precompile_run, the pathRX_VK_LOAD=starttakes, does the same builds. Since 2026-11-08 ("stuck at 99%" on an RX 5700 XT) it goes on to the game afterRX_VK_PRECOMPILE_STALL_S(30 s) with nothing finished. rc.7 made it much more likely that this pass runs long: its translator changed (#906's soa fallback layout, #1086), so every fallback pipeline is compiled again. A close in this pass did not end within the close's 10 s ("the preparation did not stop within 10 s: closing anyway" in both logs). So the close's save of the pipeline cache never ran, and the next start began again from nothing.The logs agree. The last line before the close is
pcache_init'svk: pipeline cache ... (1405978 KB).precompile_at_create's ownvk: precompile: load at create ...line never comes, and the close census shows zero programs created.Fixes #1373, #1405. See #1404 (the same PC, its close recorded as a crash), #1430 (its launcher half, "Prepare shaders" at 0%, is the builder's, not this) and #1262.
How to reproduce
Seen on: Windows 10/11, RX 5600 XT (26.5.2, 26.10.1) and RX 5700 XT (26.10.1), v3.0.0-rc.7
Settings and switches: defaults (Auto: the task + mesh compute fallback,
RX_VK_LOAD=create)Steps: update to rc.7 and start the game
Expected: the screen counts the work and reaches the title
Actual: 100% then a freeze at the first start, then 0% "Loading the shader cache" at every start, ending with
the preparation did not stop within 10 s: closing anywayon closeWhat changed
precompile_at_createnow works likeprecompile_run. It reports progress, waits with a bound, and keeps what it built.gpu/vk/bounded_wait.h(new, standard library only).bounded_wait::waitticks the progress and ends in one of three ways: every item is done, the window is closed, or no item has finished for the stall's seconds.precompile_at_create(progress)ingpu/vk/vk_draw.cpp, given its progress byprecompile_run:NoteRefcopies, not pointers intorefs.RX_VK_PRECOMPILE_STALL_S, the same switch and default (30 s) asprecompile_run. When it stalls or the window closes, no new item is taken, the builds still in the driver are detached and noted (prepare_left_workers_note), and their state is never freed. A close therefore returns at once, and the close's own pipeline cache save runs.RX_VK_PRECOMPILE_SAVE_S(30 s) or more writes the pipeline cache before the title, asprecompile_run's does (precompile_save::due).gpu/vk/switches.def:RX_VK_PRECOMPILE_STALL_S's row names this pass.tests/test_bounded_wait.pywithtests/bounded_wait_test.cpp(the wait against a fake clock, and the wiring).tests/test_fallback_loaded.pyfollows the new signature.A program whose fallback build is still in the driver after the start has gone on keeps
compute_state1. Its draws are left out until that build ends, which is howprecompile_run's stall already behaves.Measurements
Verdict: a hang fix, so there are no numbers to compare. This PC can't show the stall (an RTX 3090 has mesh shaders, so the pass has nothing to build). The game-free test shows the bound, and the in-game run here shows the default start still reaching the title through the changed function. The hardware run is asked of the Test Farm below.
Flags and settings
No new switch.
RX_VK_PRECOMPILE_STALL_SandRX_VK_PRECOMPILE_SAVE_Snow also apply to the load-at-create start.RX_VK_PRECOMPILE_STALL_SRX_VK_LOAD=startonlyRX_VK_PRECOMPILE_SAVE_STesting
Tested:
8e9cfc3a), and the game started through it. See the--tested-in-gamecomment for what the log showed.tests/run_all.py, including the newtests/test_bounded_wait.pyand the changedtests/test_fallback_loaded.py.Not tested: a real stall, or a long fallback pass on AMD's Windows driver. I don't have that hardware.
Must be tested:
N / Mand reach the title both times, the log showingvk: precompile: load at create: ... stepsand then theload at create (RX_VK_LOAD=create)summary ornothing finished for 30 s: the game starts(Test Farm: @Beanstalk, asked in the comments)Risks
What could break: on a fallback GPU a start whose builds stall now plays with those programs' draws left out until the driver finishes them, rather than never starting. Every load-at-create start with something to build waits at least one 100 ms round.
Turning it off:
RX_VK_PRECOMPILE_STALL_S=0waits for every build as before (progress and the save stay); revert for the rest.Compatibility: Nothing changes for caches, options or
MKWVK_API_VERSION.Issues
Fixes #1373
Fixes #1405
See #1404, #1430, #1262
Checklist
Everyone
type(scope): description) of the status line's type.main.python -I -B tests/run_all.pypasses (any failure that also fails onmainis named). Not run on this PC; Forgejo CI runs it.third_party/untouched (a new folder with aPROVENANCE.mdinstead).When it applies
Assisted-by:/Co-Authored-By:trailers.mainand passing here. Written as the first commit; not run here, CI runs it.tests/that reads changed source updated in this PR, not weakened.🤖 Generated with Claude Code
Tested in game: Linux, RTX 3090 (NVIDIA, native mesh shaders), headless start on a dev runtime over base
8e9cfc3awith this change's renderer: the start ran through the new precompile_at_create (after the warm pass: 32655 noted + 9927 compute pipelines into the cache in 37.7 s), logged 'vk: precompile: load at create (RX_VK_LOAD=create): 32390 noted pipelines found ... 0 mesh fallback programs kept ready ... in 920 ms' and went on to play frames (167 fps, 0 draws left out). This GPU has nothing for the pass to build, so the stall bound and the progress were NOT exercised here: they need an AMD card without mesh shaders on Windows (Test Farm asked). The pack predates the last small edit (prepare_left_workers_note on a close too).@Beanstalk On Windows with an AMD GPU that has no mesh shaders (an RX 5600 XT or 5700 XT, RDNA1, or a Vega), with the AMD proprietary driver and default settings: build this branch, start the game once to the title, quit, then start it again. Expected: the loading screen counts 'N / M pipelines' instead of sitting at 0% on 'Loading the shader cache', and both starts reach the title. The log should have 'vk: precompile: load at create' lines, ending with either the 'load at create (RX_VK_LOAD=create)' summary or 'nothing finished for 30 s: the game starts'. Please compare against main, which on these cards stays at 0% until the window is closed (issues 1373 and 1405).
Beanstalk test farm
Commit
732aa8a· tier reviewed · approved by HumanBean · dashboardWhat will be tested · AI plan (
gpt-6.1-terra)On Windows with the AMD proprietary driver, this plan targets RDNA1 or Vega GPUs and compares two default-settings title starts per build against main, measuring startup time and requiring each start to reach the menu without a crash. No matching GPU is listed online, so the test will wait for one. The farm cannot guarantee that the second start reuses the first start’s cache or check the loading-screen pipeline counter; the requested progress and cache-reuse behavior therefore remain unverified. No tester online now matches windows+amd+amd-proprietary+gpu-rx-5xxx_vega-*: that target waits for one.
Done. Comment
@Beanstalkto run it again.WIP: fix(vk): a load-at-create start whose mesh fallback builds stall in the driver goes on to the title, with progress on the wayto fix(vk): a load-at-create start whose mesh fallback builds stall in the driver goes on to the title, with progress on the way