feat(build): an update to a version that translates shaders differently keeps the pipelines and the notes #220

Merged
HumanBean merged 2 commits from madebyjordan/LongshotLauncher:feat/keep-pipelines-across-translators into main 2026-11-20 22:41:52 +00:00
Member

An update to a version that translates shaders differently now keeps the old build's driver pipeline cache and pipeline notes. Today it keeps nothing, and the new build's preparation compiles every pipeline cold: 83 minutes on my RX 6700 XT, hours on many AMD PCs.

Why: ShaderCaches.Keep returned OtherTranslator and kept nothing when the two versions' translator stamps differ. But most of a new translator's SPIR-V is the old one's (99.6% between a stable and a nightly), so the driver's pipelines for it still serve. The game side, Longshot MagicBeanDev/Longshot#1927 (on MagicBeanDev/Longshot#1926), carries another translator's pipeline cache within a bound and keeps its notes; with this change the launcher hands them over. Measured there: the launcher's preparation after a translator change takes about 2¼ minutes instead of 83.

What changed (Core/ShaderCaches.cs, App/ViewModels/BuildStepViewModel.cs):

  • What's kept: for another translator, a version whose tools are gone, or an old build that translated no program, Keep keeps the pipeline side when the old build has a driver pipeline cache (CachesResult.PipelinesOnly). That's longshot_vk_pipelines.bin and its play files, moved or copied as before, plus its .stamp, its .base and the notes (longshot_pipelines.bin), always copied.
    • The stamp and base are copied so that a build that doesn't finish never hands the old build the new game's stamp, which would make it drop its pipeline cache. Undo deletes the copies and moves the rest back.
  • What's left behind: the programs, the prepare record, the module identifiers and the warm's record. The game translates again.
  • Unchanged: the same translator with programs keeps everything, as before. With no driver cache, notes alone aren't worth a copy (they may be just a seeded list), so it's still NoCaches or OtherTranslator.
  • The base file: longshot_vk_pipelines.base is a cache too (Free up space, cold build). It sits in its own Newer list until tests/longshot.ref pins a Longshot that has it, because Names is checked against the pin.
  • The build log says which was kept: shader caches: the pipelines and notes of nightly-… moved (…); its programs are not kept (another translator, none translated, or its tools are gone) and are translated again. No text the player sees changes.

A Longshot from before the game side drops a carried cache at its start by its stamp, as it did its own: a wasted move, nothing worse.

How it was tested

Windows 11 Home 10.0.26200, .NET SDK 10.0.404, on main 415358b.

  • New checks (ShaderCacheChecks, in a folder of their own):
    • another translator keeps exactly the pipeline side (driver cache, stamp, base, play file, notes), and the programs, prepare record, module IDs and warm record stay behind;
    • a version whose tools are gone does the same;
    • the same translator still keeps all 10 files;
    • another translator with no driver cache keeps nothing (OtherTranslator);
    • on one drive the pipeline side is moved, with the notes, stamp and base copied. Then the new build's game writes its own stamp and base and the build doesn't finish: Undo puts the cache back and leaves the old build its own stamp and base.
  • Broken variants, each caught: another translator keeping everything again; the stamp and base moved instead of copied (the Undo check fails).
  • The core suite, each check group run on past a failure (a local wrapper): the same 8 Windows-only groups fail as on main, nothing else. dotnet build -warnaserror is clean.
  • Reviewed cold by a second agent together with the game side. It found the Undo problem above, now fixed.

Not tested: a launcher update across a real translator change against the game side. That needs both merged and two nightlies on either side of a translator change.

🤖 Generated with Claude Code

An update to a version that translates shaders differently now keeps the old build's driver pipeline cache and pipeline notes. Today it keeps nothing, and the new build's preparation compiles every pipeline cold: 83 minutes on my RX 6700 XT, hours on many AMD PCs. **Why:** `ShaderCaches.Keep` returned `OtherTranslator` and kept nothing when the two versions' translator stamps differ. But most of a new translator's SPIR-V is the old one's (99.6% between a stable and a nightly), so the driver's pipelines for it still serve. The game side, Longshot https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/1927 (on https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/1926), carries another translator's pipeline cache within a bound and keeps its notes; with this change the launcher hands them over. Measured there: the launcher's preparation after a translator change takes about 2¼ minutes instead of 83. **What changed** (`Core/ShaderCaches.cs`, `App/ViewModels/BuildStepViewModel.cs`): - **What's kept:** for another translator, a version whose tools are gone, or an old build that translated no program, `Keep` keeps the **pipeline side** when the old build has a driver pipeline cache (`CachesResult.PipelinesOnly`). That's `longshot_vk_pipelines.bin` and its play files, moved or copied as before, plus its `.stamp`, its `.base` and the notes (`longshot_pipelines.bin`), always copied. - The stamp and base are copied so that a build that doesn't finish never hands the old build the new game's stamp, which would make it drop its pipeline cache. Undo deletes the copies and moves the rest back. - **What's left behind:** the programs, the prepare record, the module identifiers and the warm's record. The game translates again. - **Unchanged:** the same translator with programs keeps everything, as before. With no driver cache, notes alone aren't worth a copy (they may be just a seeded list), so it's still `NoCaches` or `OtherTranslator`. - **The base file:** `longshot_vk_pipelines.base` is a cache too (Free up space, cold build). It sits in its own `Newer` list until `tests/longshot.ref` pins a Longshot that has it, because `Names` is checked against the pin. - **The build log** says which was kept: `shader caches: the pipelines and notes of nightly-… moved (…); its programs are not kept (another translator, none translated, or its tools are gone) and are translated again`. No text the player sees changes. **A Longshot from before the game side** drops a carried cache at its start by its stamp, as it did its own: a wasted move, nothing worse. ## How it was tested Windows 11 Home 10.0.26200, .NET SDK 10.0.404, on main `415358b`. - **New checks** (`ShaderCacheChecks`, in a folder of their own): - another translator keeps exactly the pipeline side (driver cache, stamp, base, play file, notes), and the programs, prepare record, module IDs and warm record stay behind; - a version whose tools are gone does the same; - the same translator still keeps all 10 files; - another translator with no driver cache keeps nothing (`OtherTranslator`); - on one drive the pipeline side is moved, with the notes, stamp and base copied. Then the new build's game writes its own stamp and base and the build doesn't finish: Undo puts the cache back and leaves the old build its own stamp and base. - **Broken variants, each caught:** another translator keeping everything again; the stamp and base moved instead of copied (the Undo check fails). - **The core suite**, each check group run on past a failure (a local wrapper): the same 8 Windows-only groups fail as on main, nothing else. `dotnet build -warnaserror` is clean. - Reviewed cold by a second agent together with the game side. It found the Undo problem above, now fixed. **Not tested:** a launcher update across a real translator change against the game side. That needs both merged and two nightlies on either side of a translator change. 🤖 Generated with [Claude Code](https://claude.com/claude-code)
feat(build): an update to a version that translates shaders differently keeps the pipelines and the notes
All checks were successful
CI / Compile and test (pull_request) Has been skipped
CI / Conventional Commit title (pull_request) Successful in 7s
b0e80ffd67
ShaderCaches.Keep kept nothing when the two versions' translators
differed, so the new folder began with no caches and its preparation
compiled every pipeline cold (hours on AMD), though most of a new
translator's SPIR-V is the old one's (99.6% between a stable and a
nightly) and the driver's pipelines for it still serve.

Now another translator (or a version whose tools are gone, or one that
translated no program) keeps the pipeline side alone when there is a
driver's pipeline cache: that cache with its play files moved or copied
as before, and its stamp, its base and the pipeline notes copied, so a
build that does not finish never gives the old build the new game's
stamp (with which it would drop its pipeline cache): Undo deletes the
copies and moves the rest back. The programs, the prepare's record, the
module identifiers and the warm's record stay behind, and the game
translates again. A Longshot that carries another translator's pipeline
cache within a bound, and keeps its notes, uses them (Longshot's
gpu/vk/pcache_stamp.h); one from before drops them at its start, as it
did its own.

longshot_vk_pipelines.base (the cache's bound) is a cache too, in a list
of its own until tests/longshot.ref pins a Longshot that has it, as
Names is checked against the pin.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
BeanBot requested changes 2026-11-20 22:07:12 +00:00
Dismissed
BeanBot left a comment

Recommendation: Request changes. A failed update can leave the old build without its driver pipeline cache. The claimed preparation-time improvement is also unproven for this launcher change.

The change carries pipeline caches and notes into builds with a different or unknown shader translator, potentially reducing preparation time. It adds transfer and storage costs, and its rollback does not preserve the old cache if the new game discards it before the build fails.

Does it help: Unproven. The description reports 83 minutes versus about 2¼ minutes on an RX 6700 XT for the game-side change, but does not identify repeated comparable runs or demonstrate an update with this launcher change. It explicitly says a real update across a translator change was not tested.
Other systems: The transfer applies on Windows and Linux, including Steam Deck, and to all GPU vendors. Cross-drive copies consume space and time; their cost is not measured. Older game versions can discard the transferred cache, and the game-side compatibility and bounds could not be verified from this checkout.
Switches: None.

Reviewed: b0e80ffd67 against 415358bbcd: Read AGENTS.md and CONTRIBUTING.md, the commit and all three file diffs. Read the changed cache transfer and undo paths, their build caller, and the relevant test sections; other portions of the three changed files were skimmed because output was truncated. Checked cache clearing references and the README description. No review checklist or pull request template was present. No recorded run logs or translations were supplied; the game-side repository was unavailable.; nothing run (a code review).
Checked and fine:

  • The new tests exercise selection of pipeline files, missing tools, cross-drive copying and the copied stamp and base during undo.
  • The change adds no switches or network operations, and the commit uses the required Conventional Commit form.

Blocking (to change before merge)

  1. Core/ShaderCaches.cs:401: The pipeline cache is removed from the old folder on a same-drive transfer. If the target is an older Longshot that discards the carried cache at startup, as the description says it can, and the build then fails or is cancelled, Undo finds no target cache to move back. The still-installed old build loses its prepared driver cache and may need a long cold preparation. Preserve an independent old copy until the build succeeds, and test undo after the target deletes or replaces the transferred cache.

Suggestions (the author may take or leave)

  • Update the README cache-transfer description, which still says missing tools prevent transfer and only the notes are copied on the same drive.
  • Measure repeated end-to-end updates with and without this change, including cross-drive transfer costs and representative Windows and Linux GPU configurations.

Testing: Read-only review; no code, tests, builds or scripts were run. The author reports a clean build and core checks on Windows, with eight existing Windows-only groups failing, but no real translator-change update.
Overlaps and merge order: The described Longshot game-side change is a dependency for the intended gain; its actual pull request and merge status could not be verified from the supplied placeholder link. No other overlaps were available.


Automated code review by BeanBot (gpt-6.1-terra), on the pull request's opening. Nothing was run; a maintainer decides.

**Recommendation: Request changes.** A failed update can leave the old build without its driver pipeline cache. The claimed preparation-time improvement is also unproven for this launcher change. The change carries pipeline caches and notes into builds with a different or unknown shader translator, potentially reducing preparation time. It adds transfer and storage costs, and its rollback does not preserve the old cache if the new game discards it before the build fails. **Does it help:** Unproven. The description reports 83 minutes versus about 2¼ minutes on an RX 6700 XT for the game-side change, but does not identify repeated comparable runs or demonstrate an update with this launcher change. It explicitly says a real update across a translator change was not tested. **Other systems:** The transfer applies on Windows and Linux, including Steam Deck, and to all GPU vendors. Cross-drive copies consume space and time; their cost is not measured. Older game versions can discard the transferred cache, and the game-side compatibility and bounds could not be verified from this checkout. **Switches:** None. **Reviewed:** b0e80ffd67e2 against 415358bbcd86: Read AGENTS.md and CONTRIBUTING.md, the commit and all three file diffs. Read the changed cache transfer and undo paths, their build caller, and the relevant test sections; other portions of the three changed files were skimmed because output was truncated. Checked cache clearing references and the README description. No review checklist or pull request template was present. No recorded run logs or translations were supplied; the game-side repository was unavailable.; nothing run (a code review). **Checked and fine:** - The new tests exercise selection of pipeline files, missing tools, cross-drive copying and the copied stamp and base during undo. - The change adds no switches or network operations, and the commit uses the required Conventional Commit form. **Blocking** (to change before merge) 1. Core/ShaderCaches.cs:401: The pipeline cache is removed from the old folder on a same-drive transfer. If the target is an older Longshot that discards the carried cache at startup, as the description says it can, and the build then fails or is cancelled, Undo finds no target cache to move back. The still-installed old build loses its prepared driver cache and may need a long cold preparation. Preserve an independent old copy until the build succeeds, and test undo after the target deletes or replaces the transferred cache. **Suggestions** (the author may take or leave) - Update the README cache-transfer description, which still says missing tools prevent transfer and only the notes are copied on the same drive. - Measure repeated end-to-end updates with and without this change, including cross-drive transfer costs and representative Windows and Linux GPU configurations. **Testing:** Read-only review; no code, tests, builds or scripts were run. The author reports a clean build and core checks on Windows, with eight existing Windows-only groups failing, but no real translator-change update. **Overlaps and merge order:** The described Longshot game-side change is a dependency for the intended gain; its actual pull request and merge status could not be verified from the supplied placeholder link. No other overlaps were available. --- <sub>Automated code review by BeanBot (gpt-6.1-terra), on the pull request's opening. Nothing was run; a maintainer decides.</sub>
@ -361,3 +399,3 @@
// (the new folder has them all: the old names go. One that stays, a file something holds open, is the same
// file under two names, which two builds of one translator can both read)
foreach (var f in files.Where(c => c.Name != Notes))
foreach (var f in files.Where(c => c.Name != Notes && !(pipelinesOnly && CarriedStamps.Contains(c.Name))))
Member

Please preserve the old pipeline cache until the build succeeds. An older target game can discard the transferred cache at startup; if the build then fails, Undo has no file to move back, so the still-installed old build loses its cache. Please also test undo after the target deletes or replaces it.

Please preserve the old pipeline cache until the build succeeds. An older target game can discard the transferred cache at startup; if the build then fails, Undo has no file to move back, so the still-installed old build loses its cache. Please also test undo after the target deletes or replaces it.
Owner
@fix
Member

I changed this pull request where the review's points could be done here: the new head is 28c722180c.

Changed

  • Core/ShaderCaches.cs:401 — preserve the old pipeline cache on rollback: Pipeline-only transfers now copy the old cache even on one drive; rollback coverage deletes and replaces target files in tests/launcher_core/ShaderCacheChecks.cs:249. Tests were not run.
  • README cache-transfer description: Updated README.md:443 to describe pipeline-only transfers, missing tools, copies, and rollback.
  • Core/ShaderCaches.cs:401 — line comment on cache preservation and undo coverage: The independent copies in Core/ShaderCaches.cs:339 preserve the old cache, and tests/launcher_core/ShaderCacheChecks.cs:258 covers target deletion and replacement before undo. Tests were not run.

Left for the author

  • Measure end-to-end update costs and benefits: Run repeated comparable end-to-end updates with and without this launcher change, including cross-drive transfer time and space costs, on representative Windows and Linux GPU configurations. This requires builds and hardware runs, which were not performed here.

Nothing was built or run: check the change. It will be reviewed again.


Automated fix by BeanBot (gpt-6.1-terra), asked for by HumanBean. Nothing was run; a maintainer decides.

I changed this pull request where the review's points could be done here: the new head is 28c722180c72. **Changed** - **Core/ShaderCaches.cs:401 — preserve the old pipeline cache on rollback**: Pipeline-only transfers now copy the old cache even on one drive; rollback coverage deletes and replaces target files in tests/launcher_core/ShaderCacheChecks.cs:249. Tests were not run. - **README cache-transfer description**: Updated README.md:443 to describe pipeline-only transfers, missing tools, copies, and rollback. - **Core/ShaderCaches.cs:401 — line comment on cache preservation and undo coverage**: The independent copies in Core/ShaderCaches.cs:339 preserve the old cache, and tests/launcher_core/ShaderCacheChecks.cs:258 covers target deletion and replacement before undo. Tests were not run. **Left for the author** - **Measure end-to-end update costs and benefits**: Run repeated comparable end-to-end updates with and without this launcher change, including cross-drive transfer time and space costs, on representative Windows and Linux GPU configurations. This requires builds and hardware runs, which were not performed here. Nothing was built or run: check the change. It will be reviewed again. --- <sub>Automated fix by BeanBot (gpt-6.1-terra), asked for by HumanBean. Nothing was run; a maintainer decides.</sub>
fix(build): preserve pipeline caches on failed updates
All checks were successful
CI / Compile and test (pull_request) Has been skipped
CI / Conventional Commit title (pull_request) Successful in 7s
28c722180c
As the review on #220 asked.
BeanBot left a comment

Recommendation: Request changes. The reviewed head still has the previously reported rollback defect, and the claimed preparation-time gain has not been demonstrated end to end for this launcher change.

The change carries driver pipeline caches and notes across translator changes, potentially reducing shader preparation time at the cost of transfer time and storage. The checkout is still b0e80ffd, the commit reviewed previously; the reported fix at 28c722180c is not available here, so its rollback fix and README update cannot be verified.

Does it help: Unproven. The description reports 83 minutes versus about 2¼ minutes on an RX 6700 XT for the game-side change, but gives no repeated comparable launcher updates and says a real update across a translator change was not tested. Transfer time and storage costs are also unmeasured.
Other systems: The transfer applies on Windows and Linux, including Steam Deck, and across GPU vendors. Older game versions can discard the carried cache; compatibility and cache bounds in the game-side changes could not be verified from this checkout.
Switches: None.

Reviewed: b0e80ffd67 against 415358bbcd: Reviewed b0e80ffd against 415358bbcd and the supplied earlier review and comments. Read AGENTS.md, CONTRIBUTING.md, the commit and all three diffs; inspected the cache transfer and rollback, build caller, relevant test sections and README cache description. Other portions of the changed files were skimmed because output was truncated. No review checklist or pull request template was found. The reported later commit and game-side repository were unavailable; no recorded run logs or translations were supplied.; nothing run (a code review).
Checked and fine:

  • The new checks cover pipeline-side selection, missing tools, same-translator transfer, and preservation of copied stamps during rollback.
  • The change adds no switches or network operations, and its commit uses Conventional Commit form.

Blocking (to change before merge)

  1. Core/ShaderCaches.cs:401: The rollback defect reported in the previous review remains at this head. A failed update can leave the old build without its driver cache if the new game deletes or replaces the moved cache; preserve an independent old copy until success and cover that failure case.

Suggestions (the author may take or leave)

  • Review the reported 28c722180c fix when that commit is available in the checkout.
  • Measure repeated comparable end-to-end updates and transfer costs on representative Windows and Linux GPU configurations.

Testing: Read-only review; no code, tests, builds or scripts were run. The author reports a clean build and core checks on Windows, with eight existing groups failing, but no real translator-change update. The required Tooling release check and CI smoke run were not reported.
Overlaps and merge order: The described Longshot game-side changes are dependencies for the intended gain; their contents and merge status could not be verified. No other overlaps were available.


Automated code review by BeanBot (gpt-6.1-terra), asked for by BeanBot, after its fix. Nothing was run; a maintainer decides.

**Recommendation: Request changes.** The reviewed head still has the previously reported rollback defect, and the claimed preparation-time gain has not been demonstrated end to end for this launcher change. The change carries driver pipeline caches and notes across translator changes, potentially reducing shader preparation time at the cost of transfer time and storage. The checkout is still b0e80ffd, the commit reviewed previously; the reported fix at 28c722180c72 is not available here, so its rollback fix and README update cannot be verified. **Does it help:** Unproven. The description reports 83 minutes versus about 2¼ minutes on an RX 6700 XT for the game-side change, but gives no repeated comparable launcher updates and says a real update across a translator change was not tested. Transfer time and storage costs are also unmeasured. **Other systems:** The transfer applies on Windows and Linux, including Steam Deck, and across GPU vendors. Older game versions can discard the carried cache; compatibility and cache bounds in the game-side changes could not be verified from this checkout. **Switches:** None. **Reviewed:** b0e80ffd67e2 against 415358bbcd86: Reviewed b0e80ffd against 415358bbcd86 and the supplied earlier review and comments. Read AGENTS.md, CONTRIBUTING.md, the commit and all three diffs; inspected the cache transfer and rollback, build caller, relevant test sections and README cache description. Other portions of the changed files were skimmed because output was truncated. No review checklist or pull request template was found. The reported later commit and game-side repository were unavailable; no recorded run logs or translations were supplied.; nothing run (a code review). **Checked and fine:** - The new checks cover pipeline-side selection, missing tools, same-translator transfer, and preservation of copied stamps during rollback. - The change adds no switches or network operations, and its commit uses Conventional Commit form. **Blocking** (to change before merge) 1. Core/ShaderCaches.cs:401: The rollback defect reported in the previous review remains at this head. A failed update can leave the old build without its driver cache if the new game deletes or replaces the moved cache; preserve an independent old copy until success and cover that failure case. **Suggestions** (the author may take or leave) - Review the reported 28c722180c72 fix when that commit is available in the checkout. - Measure repeated comparable end-to-end updates and transfer costs on representative Windows and Linux GPU configurations. **Testing:** Read-only review; no code, tests, builds or scripts were run. The author reports a clean build and core checks on Windows, with eight existing groups failing, but no real translator-change update. The required Tooling release check and CI smoke run were not reported. **Overlaps and merge order:** The described Longshot game-side changes are dependencies for the intended gain; their contents and merge status could not be verified. No other overlaps were available. --- <sub>Automated code review by BeanBot (gpt-6.1-terra), asked for by BeanBot, after its fix. Nothing was run; a maintainer decides.</sub>
Owner
@fix
Member

I didn't change this pull request: what the review asks needs the author.

Already done

  • Core/ShaderCaches.cs:401 — preserve caches on rollback: Core/ShaderCaches.cs:339 copies the pipeline side when translators differ; tests/launcher_core/ShaderCacheChecks.cs:259 covers deletion and replacement before rollback.
  • Review the fix at 28c722180c: The checkout is at 28c722180c. The rollback fix and README description are present in Core/ShaderCaches.cs:339 and README.md:443.

Left for the author

  • Measure update and transfer costs: Run repeated comparable end-to-end updates across a translator change on representative Windows and Linux GPU configurations, including Steam Deck, and compare preparation time, transfer time, and storage use against updates without the carried caches. Runs and measurements cannot be performed here.

Nothing was pushed.


Automated fix by BeanBot (gpt-6.1-terra), asked for by HumanBean. Nothing was run; a maintainer decides.

I didn't change this pull request: what the review asks needs the author. **Already done** - **Core/ShaderCaches.cs:401 — preserve caches on rollback**: Core/ShaderCaches.cs:339 copies the pipeline side when translators differ; tests/launcher_core/ShaderCacheChecks.cs:259 covers deletion and replacement before rollback. - **Review the fix at 28c722180c72**: The checkout is at 28c722180c72. The rollback fix and README description are present in Core/ShaderCaches.cs:339 and README.md:443. **Left for the author** - **Measure update and transfer costs**: Run repeated comparable end-to-end updates across a translator change on representative Windows and Linux GPU configurations, including Steam Deck, and compare preparation time, transfer time, and storage use against updates without the carried caches. Runs and measurements cannot be performed here. Nothing was pushed. --- <sub>Automated fix by BeanBot (gpt-6.1-terra), asked for by HumanBean. Nothing was run; a maintainer decides.</sub>
HumanBean changed title from WIP: feat(build): an update to a version that translates shaders differently keeps the pipelines and the notes to feat(build): an update to a version that translates shaders differently keeps the pipelines and the notes 2026-11-20 22:41:42 +00:00
HumanBean deleted branch feat/keep-pipelines-across-translators 2026-11-20 22:41:53 +00:00
Sign in to join this conversation.
No reviewers
No milestone
No project
No assignees
3 participants
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
MagicBeanDev/LongshotLauncher!220
No description provided.