fix(vk): the start names the driver's shader cache and says when the pipelines were built in another environment #902
No reviewers
Labels
No labels
area/accessibility
area/audio
area/input
area/install
area/menu
area/renderer
area/saves
area/tooling
area/upscaling
beanstalk: approved
beanstalk: failed
beanstalk: needs approval
beanstalk: running
beanstalk: tested
ci: approved
good first issue
gpu/amd
gpu/igpu
gpu/intel
gpu/nvidia
help wanted
kind
bug
kind
chore
kind
docs
kind
feature
kind
performance
kind
question
needs measurements
platform/linux
platform/macos
platform/steam-deck
platform/windows
preview
priority
critical
priority
high
priority
low
regression
risk
high
risk
low
risk
medium
status
blocked
status
confirmed
status
duplicate
status
invalid
status
needs-info
status
wontfix
ci: approved
No milestone
No project
No assignees
2 participants
Notifications
Due date
No due date set.
Dependencies
No dependencies set
Reference
MagicBeanDev/Longshot!902
Loading…
Reference in a new issue
No description provided.
Delete branch "zeldaootr/Longshot:driver-cache-diagnostics"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Type: fix · Risk: low · Tested on: Steam Deck OLED (SteamOS 3.9.5, RADV, Mesa 26.2.0-devel); Linux game-free checks · Needs: Linux NVIDIA, Windows (a start to the title)
Why
When a Steam Deck start from Game Mode compiled for 25-30 minutes the pipelines a start from the desktop loaded in seconds, the log gave no reason: its driver cache line read
0 MB (no folder found)beside a full Mesa cache, and it never said which cache folder or which driver switches the start ran with. This PR makes the start say both, and warn when its pipelines were built under other keys, so the next problem of this kind shows in the first log a player sends. It changes nothing in what is compiled.The cause of that compile (Game Mode's
RADV_FORCE_VRS_CONFIG_FILE) is fixed onmainby #900, which this PR now sits on. No issue on the tracker covers it (searched: VRS, RADV_FORCE_VRS, half rate shading, Game Mode, driver shader cache); See #741 and #802: their logs show the old line, which looked only inmesa_shader_cache_db.How to reproduce
Seen on: SteamOS 3.9.5 (Steam Deck OLED), RADV VANGOGH, Mesa 26.2.0-devel
Settings and switches: defaults
Steps: start from a shell with Mesa's cache under
XDG_CACHE_HOMEin a layout other thanmesa_shader_cache_db, or withRADV_FORCE_VRS_CONFIG_FILEnaming a file, and readlogs/longshot_log.txt.Expected: the log names the driver cache in use with its size, and says when RADV's VRS is forced and the pipelines were built without it.
Actual:
vk: driver shader cache at the start: 0 MB (no folder found), and nothing about the environment.What changed
The start reports the driver's shader cache as the environment chooses it, and warns when no completed build of the pipelines had its keys. The size count covers every Mesa layout and is bounded.
gpu/vk/driver_cache_env.h(new, standard library only): Mesa's folder (MESA_SHADER_CACHE_DIR,MESA_GLSL_CACHE_DIR,XDG_CACHE_HOME,~/.cache) and its three layouts, NVIDIA's folder, RADV's "VRS forced", the records of the environments that built the pipelines, and the verdict.gpu/vk/vk_draw.cpp:driver_cache_bytescountsmesa_shader_cache,mesa_shader_cache_sfandmesa_shader_cache_db, only Mesa's own folders (Steam's folder for a game holds Fossilize's and DXVK's too). It stops after 50,000 files or 250 ms and then says "at least".driver_cache_log("at the start")addsvk: driver shader cache: Mesa's cache in <folder> (<what chose it>), layout ...; RADV's VRS .... A variable named inLONGSHOT_DRIVER_ENV_SET(which #900, onmain, sets when it dropsRADV_FORCE_VRS_CONFIG_FILE) is called Longshot's, not the player's or Steam's.driver_env_note(), after a completedprecompile_run,precompile_warmorpredict_run, adds that build's keys and folder tocache/longshot_driver_env.txt(newest first, at most 8).docs/development.md, "The driver's own shader cache (Linux)": the two lines and the record.tests/test_driver_cache_env.pywithtests/driver_cache_env_test.cpp. The stand-ins intests/test_prepare_stall.pyfollow the new call.RADV's rules for "VRS forced", and how sure each is:
RADV_FORCE_VRS_CONFIG_FILEcounts only when it names a file that exists, whatever its content: measured (an empty file and1x1compiled; a name with no file didn't).RADV_FORCE_VRSis not read; without it,RADV_FORCE_VRSother than1x1forces VRS: from Mesa's source, not measured.VK_KHR_fragment_shading_rate(gfx10.3 and later): from Mesa's source; only the Deck (gfx10.3) was run.Measurements
Verdict: works on the Deck: the start names Mesa's folder and what chose it, and says whether RADV's VRS is forced; with #900's drop it names the drop as Longshot's. No warning where none is due. It adds no time a start can see (the precompile and the title within the run-to-run spread, below).
Baseline: the After column is the local integration
vrs-verify(bfc69759): main @aa95d35fwith this PR and its two siblings, so b shows the drop of #900 (then a sibling PR, merged since) as well. Before is the old line, as main writes it. Rebased since onto main @e4d035e9(head0816bf46): the lines measured are unchanged; the one conflict was beside them (Rebased, below).Hardware: Steam Deck OLED, AMD Custom GPU 0932 (RADV VANGOGH), Mesa 26.2.0-devel (git-5c65d4fc31), SteamOS 3.9.5 (20261023.100), 16 GB
Scene: the start-up, up to the title; scratch game folders, each start with a fresh copy of the same warm caches; headless, defaults
Method: the log's
vk: driver shader cachelines and the warning's presence; 4 starts interleaved a, b, b, a (a: no variable; b:RADV_FORCE_VRS_CONFIG_FILEnaming a1x1file)How to reproduce:
printf 1x1 > <scratch>/vrs/vrs-1x1.txt; arm b setsRADV_FORCE_VRS_CONFIG_FILEto itaa95d35f(the old line)1443161f(invrs-verifybfc69759)mesa_shader_cache_db(underXDG_CACHE_HOMEor~/.cache, orMESA_SHADER_CACHE_DIRitself), so0 MB (no folder found)on a Deck start from a shell whose Mesa cache was in another layout; nothing about the switchesMesa's cache in <scratch>/game/state/xdgcache (XDG_CACHE_HOME), layout Mesa's default; RADV's VRS not forced, and a size (at least 101 MB)RADV_FORCE_VRS_CONFIG_FILE dropped by Longshot, so RADV's VRS is not forced (RX_VK_HALF_RATE_SHADING=1 keeps it)Flags and settings
None for players. The renderer reads
LONGSHOT_DRIVER_ENV_SET, which only the host sets (#900).LONGSHOT_DRIVER_ENV_SETTesting
Tested:
tests/test_driver_cache_env.py, with g++ 13.3 and clang++ 23.1.5, after the rebase onto main @e4d035e9: fails at the test commit (a563a2d7) and at the mechanical move totests/srccut.py(e31af851), both withvk_draw.cpp does not include driver_cache_env.h, and passes at the fix (a21c4e0a); at the fix it also passes withSRCCUT_CHECK_NAIVE=1(the cuts are the same as its old brace counter's). With it,test_quit_not_crash,test_prepare_stalland #900'stest_driver_env_choosepass with both compilers.gpu/vk/driver_cache_env.hcompiled alone with g++ (libstdc++) and llvm-mingw clang (libc++, Windows target).gpu/vk/vk_draw.cppwhole-file syntax check with the build's renderer flags, Linux (clang) and Windows (llvm-mingw): the same as main @f637b42d(Linux 103 warnings and main's one narrowing error at an unchanged line; Windows 0 errors, 105 warnings).MAGICBEAN_TOOLING=<Tooling> python3 -I -B tests/run_all.py --changed forge/main, at the fix, with the Vulkan headers found: withCXX=g++and withCXX=clang++, 136 passed, 4 failed, 4 skipped each; the 4 (test_fsr_uniform_ownership,test_replay_write_order,test_vk_occlusion_queries,test_walk_key) fail on main @e4d035e9too, with both compilers.tests/run_all.pyon Linux (clang++), at the fix: 606 passed, 18 failed, 40 skipped, at a load average near 100 from other work on the PC. 16 fail on main @e4d035e9too, run there with the same headers and clang++:test_bug_report_defaults,test_crt_options,test_fsr_uniform_ownership,test_guest_imported,test_import_reach,test_linux_preflight,test_linux_update,test_make_game_cache,test_missing_nup,test_mods_size_table,test_parallel_init_order,test_pending_guest_import,test_replay_write_order,test_run_all_changed,test_vk_occlusion_queries,test_walk_key. The other two,test_fpx_stickyandtest_recorder_sync, ran past the runner's 600 s under that load; neither reads a file this PR changes. Run again alone at the fix, both pass (155 s and 213 s).Not tested: the warning itself in the game (a start with
RX_VK_HALF_RATE_SHADING=1and the file, which keeps RADV's VRS forced; the game-free test covers the verdict); the folder info line after a prediction; NVIDIA on Linux; a full renderer build and run on Windows (its Windows branch changed only in moving a lambda inside#ifdef _WIN32).Must be tested:
vk: driver shader cache:line naming NVIDIA's cache folder, a size on theat the startline, and no warning that no build had this start's keys on the second start (Test Farm: @Beanstalk linux nvidia, start to the title twice and read the vk: driver shader cache lines, no warning about the build's keys on the second start)Risks
What could break: the start's folder walk reads three Mesa folders instead of one, bounded at 50,000 files or 250 ms. One small file is written after each completed build of the pipelines. AMD's own Linux driver (AMDVLK,
VK_DRIVER_ID_AMD_OPEN_SOURCE) would be described as Mesa's; it only changes the wording of a log line.Turning it off: revert the commit.
Compatibility: nothing changes in the caches, saves,
longshot_options.inior the vk ABI. One new file,cache/longshot_driver_env.txt.Checklist
Everyone
type(scope): description) of the status line's type.main: one4d035e9(#1028, 2026-11-16), heada21c4e0a, with #900 (which setsLONGSHOT_DRIVER_ENV_SET) now on main. One conflict, invk_draw.cppprecompile_warm: main's #428 ends the warm pass's crash marker withcrash_markers::end(warming)where this PR hadremove(warming.c_str()); resolved asdriver_env_note()then main'scrash_markers::end(warming). #428'stests/test_quit_not_crash.pycuts that function, so the fix commit gives it adriver_env_notestand-in. The check moved totests/srccut.pyin a mechanical commit of its own between the test and the fix. Commits old → new:309c2610→a563a2d7,e31af851added,1443161f→a21c4e0a; the measured head in the table is1443161f.python -I -B tests/run_all.pypasses (any failure that also fails onmainis named).third_party/untouched (a new folder with aPROVENANCE.mdinstead).When it applies
Assisted-by:/Co-Authored-By:trailers.mainand passing here.tests/that reads changed source updated in this PR, not weakened.The Builder prepares in the desktop's environment and Steam starts the game in its own. In the Steam Deck's Game Mode that includes `RADV_FORCE_VRS_CONFIG_FILE`: measured on a Deck OLED, with that file there RADV gave every graphics pipeline another key, and the 32,438 noted pipelines compiled at about 20 a second instead of loading in 22 s. The log said nothing, and its cache line read "0 MB (no folder found)" beside a full cache. - `gpu/vk/driver_cache_env.h`: Mesa's folder and layouts as Mesa picks them, NVIDIA's, RADV's VRS (RADV with `VK_KHR_fragment_shading_rate`, the config file naming a file that exists, else the rate), the records and the verdict. - `driver_cache_bytes` counts Mesa's three layouts, only Mesa's folders inside Steam's, and stops at 50,000 files or 250 ms ("at least"). - The start logs the folder, what chose it, and RADV's VRS, naming what the host itself set or dropped (`LONGSHOT_DRIVER_ENV_SET`) as Longshot's. Completed builds record their keys and folder in `cache/longshot_driver_env.txt` (the last 8). The start warns when no build had its keys, and says as information when only the folder differs and predicted pipelines exist. Assisted-by: Claude1443161f56a21c4e0a1fRebased onto main
e4d035e9(heada21c4e0a, was1443161f), with #900 now on main: one conflict inprecompile_warm, where main's #428 ends the warm pass's crash marker withcrash_markers::end(warming), resolved by recording the build's environment (driver_env_note()) first and then ending the marker as main does; #428'stests/test_quit_not_crash.pycuts that function, so the fix commit gives it adriver_env_notestand-in. The check now cuts withtests/srccut.py, in its own mechanical commit between the test and the fix. Re-rantests/test_driver_cache_env.pyat each commit (it fails at the two test commits and passes at the fix, with g++ and clang++),run_all.py --changed forge/mainwith both compilers and the full suite: every failure also fails on main.Code-only review of
a21c4e0a1f; informational findings, with final approval left to another reviewer.One diagnostic correctness issue:
gpu/vk/vk_draw.cpp:10004returns false for bothVK_DRIVER_ID_NVIDIA_PROPRIETARYandVK_DRIVER_ID_AMD_PROPRIETARY, but the false branch ofdriver_cache_env::describe()andfingerprint()unconditionally selects NVIDIA's cache path and labels its keys as NVIDIA's. With the AMD proprietary driver, the new startup line and persisted environment therefore point support investigations at an unrelated cache. Use an explicit driver family (Mesa / NVIDIA / other), with an unknown-cache diagnostic and no NVIDIA-specific comparison for unsupported families. Add a classification-to-description test for AMD proprietary; the existing pure-helper tests supply amesaboolean and cannot detect this wiring error. This is source-derived, not a reproduced hardware failure.The Deck log evidence and compiler/test results are recorded in the description. Its required Linux NVIDIA two-start check and Windows startup check still have no posted result. The environment fixtures exercise record round-trips and cache-folder/key comparisons; the integration assertions mostly check source strings rather than actual filesystem or driver behavior. No tests, builds or game runs were performed for this review.
PR 903 is complementary: it saves compiled pipelines before the title, whereas this change explains the cache environment. Preserve both changes around
precompile_run/precompile_warm, including main's crash-marker handling, when combining them. I read the full diff, the startup/call-site context, test sources, description and completed-rebase comment; there was no current active-work claim in the checked activity.View command line instructions
Checkout
From your project repository, check out a new branch and test the changes.