feat(game): the shader caches and the button prompts in the game folder's cache, the logs in its logs #321
No reviewers
Labels
No labels
area/accessibility
area/audio
area/input
area/install
area/menu
area/renderer
area/saves
area/upscaling
good first issue
gpu/amd
gpu/igpu
gpu/intel
gpu/nvidia
help wanted
kind
bug
kind
chore
kind
docs
kind
feature
kind
performance
kind
question
needs measurements
platform/linux
platform/steam-deck
platform/windows
preview
priority
critical
priority
high
priority
low
status
blocked
status
confirmed
status
duplicate
status
invalid
status
needs-info
status
wontfix
testing wanted
No project
No assignees
2 participants
Notifications
Due date
No due date set.
Dependencies
No dependencies set
Reference
MagicBeanDev/Longshot!321
Loading…
Reference in a new issue
No description provided.
Delete branch "lndf/Longshot:feat/cache-logs-folders"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Why
The game folder fills up with loose files the game makes as it runs: several GB of shader and pipeline caches, the button prompts made from your controls, and five logs, all beside
longshot.exe. It's hard to tell what the build made from what can be deleted. This puts them in two folders:cache(safe to delete) andlogs.What changed
New folders (
host/game_layout.c, new): at the very start of bothmains (host/hos_main.c, before a--prepare-shadersrun too), the game makescache/,cache/prompts/andlogs/in the game folder. It moves what a build from before left loose: renames in the same folder, never over a file already there (the log names one that stays). The old prompt outputs, the pipeline cache's cut-short writes (.tmp,.tmp-<n>) and the program cache's old lock files are deleted.Renderer paths (
gpu/vk/vk_paths.h:cache_path,log_path): these go incache/:vk_progcache.cpp);vk_draw.cpp,vk_main.cpp);The frame trace goes in
logs/(vk_replay.cpp). The shipped pipeline state list and the shader seed stay beside the game.Pipeline cache name:
mkw_vk_pipelines*.bin(mkw-recomp's) is nowlongshot_vk_pipelines*.bin, with its stamp and play files (vk_draw.cpp,pcache_stamp.h,pipeline_cache_file.h). The first start renames the old files.Logs:
longshot_log.txtand the runs before it are inlogs/(host/hos_main.c).Button prompts: the files made from the player's face and controls go in
cache/prompts/(host/prompts.cmade_dir). The build's ownprompts/inputs stay.Tools:
nx2/make_game.pyseeds the pipeline list intocache/;logs/(vk_bug_report.cpp);cache/and the log inlogs/(nx2/pipeline_share.py,pipeline_share.h);scripts/dev_packs.pyseeds and warmscache/;scripts/validate_spirv.pyusage.Docs: the README, CONTRIBUTING,
docs/design.md,docs/development.md,docs/common-issues.md,docs/reporting-and-contributing.md,gpu/vk/VULKAN.md, and the game-problem issue template now say where the caches and logs are.Unchanged: the settings (
longshot_options.ini,longshot_env.txt),language.txt, the saves and the build's own files.Flags and settings
None.
Measurements
Not a performance change, so there's no fps or frame-time table. What matters is that a game folder from an older build keeps its caches across the move, so the first start after the update doesn't translate the shaders again. Measured on one update, file by file:
main:
514591a3(2026-11-13)this PR: feat/cache-logs-folders @
39694564Hardware: AMD Radeon RX 7800 XT (RADV NAVI32, Mesa 26.2.6), Fedora Linux 45 (kernel 7.3.3), the mesh shaders' compute path
Scene: none (the game folder's files)
Method: a copy of a game folder the launcher built from
main(nightly-80-7315957c, caches and logs loose beside the game), updated with this PR's Linux AppImage (--cli --update), the folder listed before and after.How to reproduce: copy a launcher-built game folder, run
Longshot-<version>-x86_64.AppImage --cli --update <copy>, then--play <copy>, and list the folder,cache/andlogs/.514591a3(before the update)39694564(after the update)longshot_programs.bincache/(kept: no shader translated again)mkw_vk_pipelines.bin374.3 MB + 4 play files, beside the gamecache/longshot_vk_pipelines.bin375.7 MB (the play files merged in, as at any start) + its stampcache/longshot_log.txtlogs/longshot_programs.bin.lockcache/(the old one deleted)Testing
Tested:
main(nightly-80-7315957c), updated with this PR's AppImage:e87e9685(--cli --update, then--play):cache/, renamed tolongshot_vk_pipelines*, the old log intologs/;cache/and the button prompts intocache/prompts/;The only problem found: the old
longshot_programs.bin.lockstayed loose. Fixed in39694564.39694564(--cli --update, a fresh copy): everything as above, and no lock file left beside the game.python -I -B tests/run_all.pyon Linux (CachyOS): 463 passed. The three failures (test_bake_portability,test_lift_cache,test_menu_i18n) fail on a cleanmaintoo.tests/test_game_layout.py, buildinghost/game_layout.cwith the POSIX shim on placeholder files, checks:cache/andlogs/, including the pipeline cache rename and its stamp and play files;bin/.test_bug_report(logs inlogs/),test_pipeline_share(list incache/, log inlogs/),test_mods_size_table(prompts served fromcache/prompts/),test_dev_packs,test_pcache_stamp,test_pipeline_cache_file,test_pipeline_play_cache,test_display_startup_failure.host/game_layout.ccompiles with no warnings for Windows (mingw-w64) and on Linux. The other host files pass a syntax check, and the Vulkan back end builds (Ubuntu 24.04 container).Screenshots
Not relevant on this PR.
Risk and rollback
A cache file this PR doesn't list stays loose beside the game, unused, and the game makes it again in
cache/. That's slower once, nothing worse. A file that can't be renamed stays where it is, and the log says so. Reverting the commit puts the paths back. The old loose files would then be gone (they were moved), so the game makes its caches again once.Reviewer notes
host/game_layout.cis the new part to read first, thengpu/vk/vk_paths.h.state_list_path(vk_draw.cpp) splits by name: the shipped list beside the game, the ones play adds incache/.Issues
None.
Checklist
For everyone
type(scope): description).main.python -I -B tests/run_all.pyrun and passing (any failure that also fails onmainis named above).third_party/untouched (a new folder with aPROVENANCE.mdinstead).If applicable
Assisted-by:/Co-Authored-By:trailers.tests/that reads changed source updated in this PR, not weakened.New commit
39694564: the program cache's old lock file (longshot_programs.bin.lock) beside the game is now deleted, since the game locks the one incache/. A test run left it loose.tests/test_game_layout.pycovers it, and fails without the fix.Tested in the game on Linux (Fedora 45, RX 7800 XT, RADV). I updated copies of a launcher-built game folder from
main(nightly-80) with this PR's AppImage, once ate87e9685and once at39694564:cache/, withmkw_vk_pipelines*renamed tolongshot_vk_pipelines*;logs/;cache/;The description has the before/after table. Still WIP: the Windows test request is next.
WIP: feat(game): the shader caches and the button prompts in the game folder's cache, the logs in its logsto feat(game): the shader caches and the button prompts in the game folder's cache, the logs in its logsCode review of
39694564(code only; I ran nothing). The move itself reads well: the migration is careful (a rename in the same folder, never over a file that's already there, and a second start does nothing). Themkw_vk_pipelines*tolongshot_vk_pipelines*rename keeps the stamp and play files together. The launcher's--updategoes through--prepare-shaders, so the move happens before the first play. A few things need fixing before it merges.Must fix
pr status 321reports that Testing has no Not tested: or Needs other testers: part and that Measured is unticked. That makes the ticked box "Testing filled in: what was tested, what wasn't, and what needs other testers" false. Your last comment says it's still WIP until the Windows test request, but no test request for 321 is open (test_requests.py list --all). Either runpr wip 321or fill in those parts. For Measured, the before/after file table is the right measurement for a change like this: tick the box, saying so.MoveFileExAandFindFirstFileAon NTFS (the*.tmp-*pattern), the log rotation intologs\ingame_main, andnarrow_paths_okrunning before the layout.tests/test_game_layout.pyis skipped on Windows. Please get a run that updates a launcher-built Windows folder with--updateand then plays it, ideally through a test request. Related box: Host code builds on both Windows (llvm-mingw) and Linux. The description says onlygame_layout.cwas compiled for Windows (mingw-w64), and the other host files got a syntax check. A full llvm-mingw build would settle it. The CI jobs for this head are still Blocked by required conditions (run 2771), so CI hasn't checked them either.assets/pipeline_states/README.mdlines 70 and 78 (<game folder>/longshot_pipeline_states.bin) anddocs/development.mdlines 532 and 537 ("writeslongshot_pipeline_states.binbeside the executable", and the sameaddcommand). The file is now incache/. These lines aren't in the diff, so I can't comment on them inline.Outside this repo (please coordinate)
Path.Combine(buildFolder, "longshot_log.txt")and.1.txt(CrashCheck.Find). With this PR the logs are inlogs/, so it would never find a crash. Whichever lands second has to look inlogs/first, then the folder. The launcher'smaindoesn't read the game's logs or caches today, so nothing that's merged breaks.longshot_startup.logis moved intologs/, anddocs/common-issues.mdnow sends players there. According to README line 195, the launcher's Play writes that file, but I can't find a writer on the launcher'smainor here. If something still writes it beside the game, the next start findslogs/longshot_startup.logalready there and leaves the new one loose. Then the newest file is beside the game while the docs point at the older one inlogs/. What writes it now?Suggestions (non-blocking)
cache/is described to players as "can be deleted at any time", but it also holds state that isn't a cache (inline).vk_progcache.cpplines 1 and 76,vk_draw.cpplines 1078, 1485 and 1523,driver_faults.hline 16,vk_main.cpplines 995 and 1010.docs/development.md.v3.0.0: I'd move this to v3.1.0. The PR is in the v3.0.0 milestone (due 2026-11-14), but rc.3 is already out, and
docs/versioning.mdsays the stable is the same commit as a release candidate that proved good. So taking this in would need an rc.4 and a Windows run in under a day, for a change whose failure mode (shaders prepared again, minutes to hours) would hit every player on the first stable. Nothing in v3.0.0 depends on it, and the migration code works the same from any older layout later.@ -0,0 +35,4 @@"longshot_pipelines.bin.predicted_keys", "longshot_pipelines.bin.built", "longshot_pipelines.bin.running","longshot_pipelines.bin.predicting", "longshot_pipelines.bin.built.warming", "longshot_computes.bin","longshot_compute_variants.bin", "longshot_pipeline_states.bin", "longshot_pipeline_states_v1.bin", "longshot_prepared.bin","longshot_prepared.bin.warming", "longshot_prepare_skipped", "longshot_driver_faults.txt", "longshot_load_start.txt",longshot_driver_faults.txt,longshot_prepare_skippedandlongshot_load_start.txtaren't caches. They're decisions the game made on this PC: the quarantine of programs the driver faulted on (driver_faults.h: on AMD's Windows driver a re-asked faulted program froze the game, issue 996), the player's skipped preparation, and the load-mode fallback. README line 252 now tells playerscachecan be deleted at any time. A player who clears it to fix something re-meets the driver fault on the next run. I'd keep these three beside the settings (or in a folder that isn't called cache), or make the README say which files incache/are worth keeping.longshot_pipeline_states.binis also the list players send in; deleting it is fine, but it's not really a cache either.@ -0,0 +67,4 @@moved++;return;}if (!kept++) snprintf(kept_first, sizeof kept_first, "%s", from);When the move fails (on Windows a file held open without FILE_SHARE_DELETE: an antivirus scan of a 7.8 GB file, an Explorer preview, a game still closing), this start writes a new cache in
cache/. From then on the loose one can never move, because the target exists. The player is left with a dead 7.8 GBlongshot_programs.binbeside the game, and the only sign is one log line at each start. Could a later start delete a loose cache file whosecache/copy exists (caches only, never logs), or at least name the bytes left in the log line? Non-blocking.Rebased onto current
main(0815d9843c). Main had moved 30 commits, among them #248. The two commits are now20b7bd5bandb797f590, still yours.Only
docs/development.mdconflicted.mainadded theRX_VK_PCACHE_FETCH_LOCKrow right afterRX_VK_PCACHE_STAMP, the row where this PR renamesmkw_vk_pipelinestolongshot_vk_pipelines. I kept main's new row and applied the rename in place. Each commit's added and removed lines are the same as before. No tests or builds were run on this PC.Our review's points still stand: #321 (comment)
396945644eb797f590f3