fix(vk): count a first start's load at create on the start-up bar #1575
No reviewers
Labels
No labels
area/accessibility
area/audio
area/input
area/install
area/menu
area/renderer
area/saves
area/tooling
area/upscaling
beanstalk: approved
beanstalk: failed
beanstalk: needs approval
beanstalk: running
beanstalk: tested
ci: approved
good first issue
gpu/amd
gpu/igpu
gpu/intel
gpu/nvidia
help wanted
kind
bug
kind
chore
kind
docs
kind
feature
kind
performance
kind
question
needs measurements
platform/linux
platform/macos
platform/steam-deck
platform/windows
preview
priority
critical
priority
high
priority
low
regression
risk
high
risk
low
risk
medium
status
blocked
status
confirmed
status
duplicate
status
invalid
status
needs-info
status
wontfix
ci: approved
No milestone
No project
No assignees
1 participant
Notifications
Due date
No due date set.
Dependencies
No dependencies set
Reference
MagicBeanDev/Longshot!1575
Loading…
Reference in a new issue
No description provided.
Delete branch "dukenheimer/Longshot:fix/start-bar-at-create"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Type: fix · Risk: low · Tested on: Windows 11 (llvm-mingw clang) and Linux (WSL Ubuntu 24.04, g++), game-free; in the game: Steam Deck OLED (RADV) · Needs: none
TL;DR
On the first start after preparing shaders, the start-up screen's bar read 100% while the game was still building pipelines: about 90 seconds on a Steam Deck, with "Loading pipelines 100%" on screen the whole time. The start does its work in two passes, and the bar counted only the first. This PR counts both passes in one total, so the bar stays under 100% and keeps moving until the second pass ends. A new test fails on main and passes here, on Windows and Linux. This changes the bar only: nothing is built differently and nothing gets faster.
Breakdown
Why
Under
RX_VK_LOAD=create, the first start after a preparation runs two passes before the title, both on the bar's last part:precompile_warm): every noted pipeline built into the pipeline cache;precompile_at_create): each mesh fallback program's compute pipelines, then the kept programs' noted pipelines.Each reported its own done of total. The warm reached its total,
PrepareBarnever goes back, so the screen read 100% for the whole second pass. On a Steam Deck at main7daafa7athat was 91 s, with the log saying "start-up: pipelines 1883 of 4568, 100% of all". This is the same plateau family as #714, which fixed it for the predicted pipelines.See #1522: its "stuck at Loading Pipelines 100%" matches this symptom. #1522 also asks about the slow build itself, a UI that stopped taking clicks, and whether a restart resumes the work. This PR does none of those, and #1522 has no log tying them to this cause, so it doesn't close the issue.
How to reproduce
Seen on: Steam Deck (SteamOS, RADV), main
7daafa7aSettings and switches: defaults (
RX_VK_LOAD=create); a GPU that draws mesh programs through the compute fallback, as the Deck doesSteps: prepare the shaders (
--prepare-shaders), then start the game for the first time after that preparation, and watch the start-up screen andlongshot_log.txtExpected: the bar under 100% while "load at create: X of Y steps" is still counting
Actual: "start-up: pipelines N of 4568, 100% of all" from N = 31 to 1883, for about 90 s, then "load at create: its pipelines took 91 s"
What changed
The two passes are counted as one, as #714 did for the predicted pipelines. The warm's steps come first. The second pass's steps are counted before the warm, from the same programs it will be given, and they stand in the total until the pass counts them itself.
gpu/vk/start_steps.h:begin/warm/create/endwrap the progress callback. The warm reportsdone of total + steps; load at create reportswarm + done of warm + total;endreports the full count when the pass returns.create_steps(taken, programs)counts the second pass's steps asprecompile_at_createdoes (total = fallbacks + kept):program_cache_load_some(..., fallbacks)takes;fallback_loaded::each).precompile_run(gpu/vk/vk_draw.cpp) wraps both passes' progress throughstart_steps:--prepare-shadersrun (LONGSHOT_PREPARE_ONLY) has no second pass, so it holds no steps and its warm still ends at 100%;program_cache_fallback_steps()(gpu/vk/vk_progcache.cpp, declared invk_progcache.handvk_pipelines.h): a snapshot of the recordsprogram_cache_load_some(..., fallbacks)would take (underdisk_mu), thencreate_stepsundershader_mu. The warm only borrows programs, so the count taken before it still holds.tests/test_start_progress_at_create.py:precompile_run's at-create branch from the source, runs it with stand-ins for the two passes, and feeds the realPrepareBar;create_steps, and source checks that the snapshot takes whatload_sometakes and thatprecompile_at_createstill countsfallbacks + kept.test_bounded_wait,test_cold_cache_checkandtest_warm_boundedpinnedprecompile_run's old call text. They now read the new calls and keep their guards: the lost-sample check still comes before the covered-notes decision.A follow-up idea for HumanBean, not in this PR: the preparation's own process returns before load at create (
LONGSHOT_PREPARE_ONLY), so the mesh fallback programs' compute pipelines (2284 programs, 4568 steps on the Deck) are compiled only at the game's first start after each preparation. That's the Deck's ~90 s first start; later starts find them in the caches ("none built", ~9 s). Building them in--prepare-shaderswould move that time into the preparation.Measurements
Verdict: nothing to time: this changes what the bar shows, not how long anything takes. The before and after are the new test's output, and the start-up log lines in the game run.
Hardware: Windows 11, llvm-mingw clang (the builder's); Linux: WSL Ubuntu 24.04, g++
Scene: game-free (the test's stand-ins for the two passes); in the game, the first windowed start after a Most preparation on a Steam Deck OLED, defaults (RX_VK_LOAD=create)
Method:
python -I -B tests/test_start_progress_at_create.pyon main and on this PR; the touched checks throughtests/run_all.py; in the game, the log'sstart-up: pipelines N of M, X% of alllines besideload at create: X of Y steps, one attended runHow to reproduce:
python -I -B tests/test_start_progress_at_create.py [--cxx g++]; in the game,--prepare-shaders, then start the game and readlongshot_log.txt's start-up lines430eef4f5c835025Teeth checks, each breaking the fix on purpose; each makes the new check fail:
Before the rebase onto
430eef4f,run_all.py --changed(151 checks) gave Windows 142 pass / 9 fail and Linux 117 pass / 8 fail. The same checks fail on main7daafa7aon each OS: on Windows test_fsr_uniform_ownership, test_draw_stream_visibility, test_dev_packs, test_pipeline_play_cache, test_compute_warm_apart, test_linux_pressure, test_upscaler_rows, test_vk_occlusion_queries and test_recorder_sync; on Linux byte_compile_tracked, test_bake_ds, test_dev_packs, test_linux_pressure, test_pipeline_cache_save, test_pipeline_play_cache, test_shader_module_guard and test_upscaler_rows. test_recorder_sync's failure is the missingvk_new.hthat #1363 fixes.A syntax build of
vk_draw.cppandvk_progcache.cppwith the build's flags gives the same diagnostics on main and here (117 and 1).After the rebase onto
430eef4f(which changed the warm, #1567): the new check fails on main430eef4fand passes here, on Windows and Linux. The new check, test_bounded_wait, test_cold_cache_check, test_warm_bounded, test_fallback_loaded, test_doc_anchors, test_compute_warm_apart, test_precompile_warm_computes and test_pipeline_state_computes all pass on both (9 of 9). The syntax build again matches main (117 and 1).Flags and settings
None. The diff reads
LONGSHOT_PREPARE_ONLY, which already exists (the preparation's own process;switches.def), and changes nothing about it.Testing
Tested: In the game on a Steam Deck OLED (AMD Custom GPU 0932, RADV VANGOGH, Mesa 26.2.0-devel git-5c65d4fc31, SteamOS Game Mode, 1280x800 at 90 Hz, 15 W), run by our Deck agent. The renderer was built from
5c835025on the m430 base (main430eef4f). The copy was freshly prepared on Most, and this was its first windowed start after that preparation, the case in How to reproduce.7daafa7athe same stretch logged "pipelines N of 4568, 100% of all" throughout.Game-free, on Windows (llvm-mingw clang) and Linux (WSL, g++): the new check fails on main and passes here.
test_bounded_wait,test_cold_cache_check,test_warm_bounded,test_fallback_loadedandtest_doc_anchorspass. The three teeth checks fail as expected.Not tested: the time-left estimate, which still jumps during load at create (about 15 s, then 1 to 4 min); that is outside this fix. The bar on a GPU with mesh shaders, where load at create has no fallback programs: the held step and
end()keep the bar's end where it was, and the test covers that case (an empty second pass ends at 100%). The fullrun_all.pywas not rerun after the rebase: the--changedrun was on7daafa7a, and the touched and warm-related checks were rerun on430eef4f.Must be tested: None.
Risks
What could break: the bar's movement on the first start after a preparation. A count that differs from the second pass's own total can only make the bar move unevenly; it can't stop the start, change what is built, or touch later starts.
Turning it off: revert the commit.
Compatibility: Nothing changes: no caches, saves, options or the vk ABI.
Issues
See #1522
Checklist
Everyone
type(scope): description) of the status line's type.can't be run by hand is in Testing's Not testable by hand, with why.
in Risks.
main.python -I -B tests/run_all.pypasses (any failure that also fails onmainis named).third_party/untouched (a new folder with aPROVENANCE.mdinstead).When it applies
Assisted-by:/Co-Authored-By:trailers.mainand passing here.tests/that reads changed source updated in this PR, not weakened.vk_draw.cppandvk_progcache.cppcompiled with the build's flags,-fsyntax-only, with the same diagnostics as main.)Under RX_VK_LOAD=create the first start after a preparation runs precompile_warm and then precompile_at_create on the bar's one part. Each reported its own done of total. The warm reached its total, the bar never goes back, and the screen read 100% through the second pass: 91 s on a Steam Deck ("start-up: pipelines 1883 of 4568, 100% of all"). start_steps (gpu/vk/start_steps.h) makes the two passes one count, as #714 did for the predicted pipelines. The warm's steps come first. The second pass's steps are counted before the warm by program_cache_fallback_steps(): a snapshot of what program_cache_load_some(..., fallbacks) will give precompile_at_create, through start_steps::create_steps. That is every variant of each key whose kind-4 records it takes, the variants of those keys already in memory, and the mesh fallback programs in memory under other keys (fallback_loaded.h), each a step and each fallback one more, as precompile_at_create counts them. At least one step is held for the second pass, so the warm's end is never the bar's. The pass's end reports the full count, so a pass with nothing to do ends the bar too. A --prepare-shaders run, which has no second pass, still ends its warm at 100%. A start whose notes are covered already runs the second pass alone, from 0. This changes the progress display only. It doesn't change what the start compiles, or how long it takes. The three checks that pin precompile_run's calls (test_bounded_wait, test_cold_cache_check, test_warm_bounded) now read the new calls. The lost-sample guard must still come before the covered-notes decision. Assisted-by: GitHub Copilot (Claude Opus 5.5) Co-authored-by: Copilot App <[email protected]>Tested in game: Steam Deck OLED (AMD Custom GPU 0932, RADV VANGOGH, Mesa 26.2.0-devel git-5c65d4fc31, SteamOS Game Mode, 1280x800 at 90 Hz, 15 W), renderer built from
5c835025on m430 (main430eef4f), first windowed start after a fresh Most --prepare-shaders. The bar total is the warm plus load at create (34,710 + 4,568 = 39,278). Through load at create the start-up lines rose from "pipelines 35041 of 39278, 89%" (205 of 4568 steps, 10 s) to "36533 of 39278, 93%" (1337 of 4568, 70 s), and reached the end of the count with "load at create: its pipelines took 88 s". On main7daafa7athe same stretch read "N of 4568, 100% of all" throughout. Screen captures showed "Loading pipelines" at 90% and 93%, then the title. Shaders took 101.6 s (m430: 101.0 s, one sample each). The game then played normally, with no GPU fault, timeout or reset. Run by our Deck agent.View command line instructions
Checkout
From your project repository, check out a new branch and test the changes.