The shader cache stamp hashes the translator's source byte for byte, so a comment edit makes every player prepare all shaders again #1611
Labels
No labels
area/accessibility
area/audio
area/input
area/install
area/menu
area/renderer
area/saves
area/tooling
area/upscaling
beanstalk: approved
beanstalk: failed
beanstalk: needs approval
beanstalk: running
beanstalk: tested
ci: approved
good first issue
gpu/amd
gpu/igpu
gpu/intel
gpu/nvidia
help wanted
kind
bug
kind
chore
kind
docs
kind
feature
kind
performance
kind
question
needs measurements
platform/linux
platform/macos
platform/steam-deck
platform/windows
preview
priority
critical
priority
high
priority
low
regression
risk
high
risk
low
risk
medium
status
blocked
status
confirmed
status
duplicate
status
invalid
status
needs-info
status
wontfix
ci: approved
No milestone
No project
No assignees
2 participants
Notifications
Due date
No due date set.
Dependencies
No dependencies set
Reference
MagicBeanDev/Longshot#1611
Loading…
Reference in a new issue
No description provided.
Delete branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Type: performance · Area: renderer, shader caches, build · Part of: #1493
TL;DR
The stamp that decides whether the shader caches survive an update hashes every file of the translator whole, comments and blank lines included. A change to one comment in
vk_shader.cpptherefore makes every player prepare all programs again at the next start. #1510 takes the struct size out of the key; this is the other input, the text. We propose to hash a conservative normal form of the source, so an edit that cannot change a translation keeps the caches, and a check that proves it on a sample of programs.Breakdown
stamp()reads the bytes of the three folders and three files and hashes them, with Windows line ends turned into Unix ones and nothing else.vk_shader.cppalone was touched by 112 commits since 6 Nov.Evidence
Code and history on
forge/mainat430eef4f2:nx2/translator_files.py:20: the docstring says files are stamped "byte for byte, comments included".:27-28: the folders (eden,sirit,compat) and the files (vk_shader.cpp,vk_bank.cpp,vk_state.h).:64-69:stamp()is a SHA-1 over each file's relative path and bytes.gpu/vk/vk_shader.cpp:1259:translator_stamp()mixesLONGSHOT_SHADER_STAMPwith the GPU facts and the switches.scripts/dev_packs.pyprepares a runtime again by the same files (stamped()at:59), so a normal form changes what a development pack re-prepares as well.git logonforge/mainsince 2026-11-06 touching a stamped file:vk_shader.cpp112 commits,vk_bank.cpp12,vk_state.h9, the three folders 138; 222 commits in all, and 131 of the 1,420 first-parent commits on main. A commit that touches a stamped file moves the stamp whether or not it changes a translation. How many of them changed no translation was not counted.vk_api.hout of the stamp; #1418 keeps the pipeline marker across an update that changes no shader; #1390 deleted switches the translation read; #1510 and #1542 takesizeof(Program)out of the key.The fork's claim, not ours: an outside audit of a fork says that in the fork 14 of 17 stamp changes between two releases changed no translation, and that each cost about 46,600 programs of preparation (75 seconds). It says the fork fixed this by hashing the sources without comments, blank lines and trailing spaces, with marked regions left out. We have not measured our own share.
Proposed change
nx2/translator_files.py, used bystamp(): it removes whole-line//comments,/* */blocks that start at the beginning of a line (after white space) and nothing else, blank lines, and trailing white space. It leaves any line it is unsure about, including lines with a quote or a backslash before a comment marker, exactly as it is.#includemoves it.How we would test it
vk_shader.cpp. The stamps must be equal; today they differ. A second pair differing by one token must differ.#if 0blocks). Each case is either left alone or removed, and a case that is left alone is correct by definition.vk_shader.cpp; this change with the same comment. The second drops the caches at its first start, the third keeps them. A yes or no in the log, not a timing.Related
New on this forge; not opened on GitHub.
Possibly related, for context: