fix(build): estimate progress from the work left #213

Merged
HumanBean merged 5 commits from futondestroyer/LongshotLauncher:fix/build-progress-eta into main 2026-11-20 15:40:57 +00:00
Member

The Launcher could only infer shader preparation time from one percentage, and that percentage can jump when the renderer finds reusable work or changes phases. This PR keeps the earlier stage-aware progress fixes and combines the renderer's recent rate with the Launcher's cumulative rate when both are fresh.

What changed

  • Confirmed reuse, compile plans and retries, linking, and finishing affect the overall estimate. The visible pre-shader ETA covers work through Finish.
  • During shaders, an optional renderer signal supplies a countdown and can explicitly withdraw it or report that the pipeline cache is being saved. The Launcher accepts a fresh estimate before it has a cumulative rate. Once both estimates are fresh, the renderer can shorten the cumulative estimate by up to a quarter.
  • An expired progress sample cannot limit a fresh renderer ETA. Compile ETA and builder completion keep their existing events. Older builders continue to use the progress-based shader estimate.

Measurements

Windows 11 build 26300, i7-13700F, RTX 4070 driver 32.0.16.2101. The isolated runtimes use the same shared Longshot base. The separate producer PR adds the event in newer builders. Measurements used Launcher af8aa5f8. Rebased head d2f22c35 has identical progress, ETA and Build view source.

The comparison replays identical live events twice at one-second checkpoints, with and without the new shader signal. Mean absolute error is computed only where both estimators showed a number.

Run Actual shader phase Common samples Previous error Final error
Cold patched renderer 689.3 s 446 49.68 s 35.17 s
Warm patched renderer, run 1 23.7 s 6 1.88 s 0.85 s
Warm patched renderer, run 2 14.5 s 3 1.22 s 1.22 s

Using the raw renderer rate alone gave 88.23 s mean error on the cold trace. The final estimator also withheld a numeric estimate longer than the old one: it showed 446 versus 615 samples on that trace. At an early cold checkpoint with 530.1 s actual remaining, the estimate moved from 698.8 s to 524.1 s. At a late checkpoint with 10.1 s remaining, it still estimated 47.9 s. Both warm patched runs emitted separate cache-saving signals and reached successful completion. These are ETA results on one PC and shader setting, with no speedup claim.

Late shader countdown experiment

I tested a local variant that could shorten a fresh estimate by up to half after 90% based on roughly 30 seconds of progress. The table compares it with this PR's estimator on identical events at one-second intervals.

Trace Final minute, this PR Final minute, experiment Whole shader phase, this PR Whole shader phase, experiment
Original cold run 36.43 s 32.62 s 35.17 s 34.68 s
Independent run with a fresh configured driver-cache directory 28.58 s 28.58 s 201.70 s 201.76 s

The independent run reused the assembled development runtime, including its application cache, and completed shader preparation in 636.25 s. The last 5% took 130.60 s, compared with 3.85 s on the original patched run. Both short warm replays were unchanged. The experiment stays out of this PR. The next measurement needs to distinguish elapsed time spent preparing programs, compiling pipelines, waiting for the remaining workers and saving caches.

Testing

  • 140 focused Launcher assertions passed using .NET SDK 10.0.404. They cover old-builder fallback, protocol scope, warm and cold traces, stale and withdrawn forecasts, cache saving and final completion.
  • The exact-SDK locked Launcher build passed. Windows lint and locked build passed in the smoke run. Core reproduces UnlistedChecks on current main and this patch. The strict Longshot pin check also encounters an existing pipeline-cache filename mismatch. The Tooling release suite reports the same three tag failures on both revisions.
  • A game-free diagnostic copied the two exact source snapshots into temporary directories, preserved the signed fixture's LF bytes, and continued after the known UnlistedChecks assertion. Both reached the same next SavesChecks failure while importing the fixture that simulates a legacy Linux save moved into user/. The diagnostic changed one assertion in each temporary copy and gives baseline parity evidence.
  • Renderer ETA and saving screenshots
  • Earlier full stable cold and warm Launcher builds reached Ready. The new signal was validated on exact Longshot development runtimes with the Launcher replay, without repeating a full cold game build.
  • Related to Launcher #143.

Rebase validation

  • Replayed the published stack onto Launcher main 073e96e0. Git's independent merge calculation produced the same tree. The upstream tools.lock path fix and the ETA patch apply together cleanly.
  • Using pinned Python 3.12.16 and .NET SDK 10.0.404, d2f22c35 passed all 140 focused progress and clock assertions. They cover older-builder fallback, active renderer estimates, expiry, cache saving and completion. The base passed its 67 focused assertions. Lint and the locked Release build passed on both revisions.
  • The ordinary core harness failed at the same UnlistedChecks assertion on both revisions. The pinned producer contract reported the same pipeline-cache filename mismatch. mise run ci:smoke reproduced the same three Tooling release-test failures on both revisions, with 39 release tests passing on each. The failed cases are Bump.test_the_bump, Bump.test_the_next_stable and Publish.test_check_tag_against_the_published_releases.
  • Continuing past the known cleanup assertion in temporary copies reached the same Linux save-import assertion on both revisions. A separate strict run using Longshot main 89c3d21e stopped because the licence reader still expects scripts/ngx_sdk.py. The contract needs a coordinated reader update to move to that base.

Still to verify

  • Forgejo CI / Conventional Commit title fails on d2f22c35. It reads main's all-zero .tooling-rev before it checks the title. Launcher PR #215 changes just that file to Tooling main 0b0059ac, which contains scripts/release.py. #215 remains WIP and its own title job reads the same base pin. Forgejo CI / Compile and test is intentionally skipped on PR events.
  • The maintainer's ci: approved label starts the CircleCI fork workflow. Its ci/circleci: gate combines fork-lint, fork-build-test, fork-release-scripts and fork-package as applicable. The description check passes. The local core, release and producer-contract checks still reproduce baseline failures, so the repository's check requirement for leaving WIP remains open.
  • The latest published nightly resolves to 89c3d21e. Its Windows Builder appears in a checksum manifest whose signature verified against this Launcher's trust keys. The tagged builder and renderer sources lack @@shader_eta, and there is no PR-specific release tag for #1830. The portable end-to-end check still needs a published, signed builder that contains the producer signal. Shader error on additional drivers is still unmeasured.
The Launcher could only infer shader preparation time from one percentage, and that percentage can jump when the renderer finds reusable work or changes phases. This PR keeps the earlier stage-aware progress fixes and combines the renderer's recent rate with the Launcher's cumulative rate when both are fresh. ## What changed - Confirmed reuse, compile plans and retries, linking, and finishing affect the overall estimate. The visible pre-shader ETA covers work through Finish. - During shaders, an optional renderer signal supplies a countdown and can explicitly withdraw it or report that the pipeline cache is being saved. The Launcher accepts a fresh estimate before it has a cumulative rate. Once both estimates are fresh, the renderer can shorten the cumulative estimate by up to a quarter. - An expired progress sample cannot limit a fresh renderer ETA. Compile ETA and builder completion keep their existing events. Older builders continue to use the progress-based shader estimate. ## Measurements Windows 11 build 26300, i7-13700F, RTX 4070 driver 32.0.16.2101. The isolated runtimes use the same shared Longshot base. The [separate producer PR](https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/1830) adds the event in newer builders. Measurements used Launcher `af8aa5f8`. Rebased head `d2f22c35` has identical progress, ETA and Build view source. The comparison replays identical live events twice at one-second checkpoints, with and without the new shader signal. Mean absolute error is computed only where both estimators showed a number. | Run | Actual shader phase | Common samples | Previous error | Final error | | --- | ---: | ---: | ---: | ---: | | Cold patched renderer | 689.3 s | 446 | 49.68 s | **35.17 s** | | Warm patched renderer, run 1 | 23.7 s | 6 | 1.88 s | **0.85 s** | | Warm patched renderer, run 2 | 14.5 s | 3 | 1.22 s | 1.22 s | Using the raw renderer rate alone gave 88.23 s mean error on the cold trace. The final estimator also withheld a numeric estimate longer than the old one: it showed 446 versus 615 samples on that trace. At an early cold checkpoint with 530.1 s actual remaining, the estimate moved from 698.8 s to 524.1 s. At a late checkpoint with 10.1 s remaining, it still estimated 47.9 s. Both warm patched runs emitted separate cache-saving signals and reached successful completion. These are ETA results on one PC and shader setting, with no speedup claim. ### Late shader countdown experiment I tested a local variant that could shorten a fresh estimate by up to half after 90% based on roughly 30 seconds of progress. The table compares it with this PR's estimator on identical events at one-second intervals. | Trace | Final minute, this PR | Final minute, experiment | Whole shader phase, this PR | Whole shader phase, experiment | | --- | ---: | ---: | ---: | ---: | | Original cold run | 36.43 s | 32.62 s | 35.17 s | 34.68 s | | Independent run with a fresh configured driver-cache directory | 28.58 s | 28.58 s | 201.70 s | 201.76 s | The independent run reused the assembled development runtime, including its application cache, and completed shader preparation in 636.25 s. The last 5% took 130.60 s, compared with 3.85 s on the original patched run. Both short warm replays were unchanged. The experiment stays out of this PR. The next measurement needs to distinguish elapsed time spent preparing programs, compiling pipelines, waiting for the remaining workers and saving caches. ## Testing - 140 focused Launcher assertions passed using .NET SDK 10.0.404. They cover old-builder fallback, protocol scope, warm and cold traces, stale and withdrawn forecasts, cache saving and final completion. - The exact-SDK locked Launcher build passed. Windows lint and locked build passed in the smoke run. Core reproduces `UnlistedChecks` on current main and this patch. The strict Longshot pin check also encounters an existing pipeline-cache filename mismatch. The Tooling release suite reports the same three tag failures on both revisions. - A game-free diagnostic copied the two exact source snapshots into temporary directories, preserved the signed fixture's LF bytes, and continued after the known `UnlistedChecks` assertion. Both reached the same next `SavesChecks` failure while importing the fixture that simulates a legacy Linux save moved into `user/`. The diagnostic changed one assertion in each temporary copy and gives baseline parity evidence. - [Renderer ETA and saving screenshots](https://git.magicbean.dev/MagicBeanDev/LongshotLauncher/pulls/213#issuecomment-56777) - [Earlier full stable cold and warm Launcher builds](https://git.magicbean.dev/MagicBeanDev/LongshotLauncher/pulls/213#issuecomment-56353) reached Ready. The new signal was validated on exact Longshot development runtimes with the Launcher replay, without repeating a full cold game build. - Related to [Launcher #143](https://git.magicbean.dev/MagicBeanDev/LongshotLauncher/issues/143). ### Rebase validation - Replayed the published stack onto Launcher main `073e96e0`. Git's independent merge calculation produced the same tree. The upstream tools.lock path fix and the ETA patch apply together cleanly. - Using pinned Python 3.12.16 and .NET SDK 10.0.404, `d2f22c35` passed all 140 focused progress and clock assertions. They cover older-builder fallback, active renderer estimates, expiry, cache saving and completion. The base passed its 67 focused assertions. Lint and the locked Release build passed on both revisions. - The ordinary core harness failed at the same `UnlistedChecks` assertion on both revisions. The pinned producer contract reported the same pipeline-cache filename mismatch. `mise run ci:smoke` reproduced the same three Tooling release-test failures on both revisions, with 39 release tests passing on each. The failed cases are `Bump.test_the_bump`, `Bump.test_the_next_stable` and `Publish.test_check_tag_against_the_published_releases`. - Continuing past the known cleanup assertion in temporary copies reached the same Linux save-import assertion on both revisions. A separate strict run using Longshot main `89c3d21e` stopped because the licence reader still expects `scripts/ngx_sdk.py`. The contract needs a coordinated reader update to move to that base. ## Still to verify - Forgejo `CI / Conventional Commit title` fails on `d2f22c35`. It reads main's all-zero `.tooling-rev` before it checks the title. [Launcher PR #215](https://git.magicbean.dev/MagicBeanDev/LongshotLauncher/pulls/215) changes just that file to Tooling main `0b0059ac`, which contains `scripts/release.py`. #215 remains WIP and its own title job reads the same base pin. Forgejo `CI / Compile and test` is intentionally skipped on PR events. - The maintainer's `ci: approved` label starts the CircleCI fork workflow. Its `ci/circleci: gate` combines `fork-lint`, `fork-build-test`, `fork-release-scripts` and `fork-package` as applicable. The description check passes. The local core, release and producer-contract checks still reproduce baseline failures, so the repository's check requirement for leaving WIP remains open. - The [latest published nightly](https://git.magicbean.dev/MagicBeanDev/Longshot/releases/tag/nightly-612-89c3d21e) resolves to `89c3d21e`. Its Windows Builder appears in a checksum manifest whose signature verified against this Launcher's trust keys. The tagged builder and renderer sources lack `@@shader_eta`, and there is no PR-specific release tag for #1830. The portable end-to-end check still needs a published, signed builder that contains the producer signal. Shader error on additional drivers is still unmeasured.
fix(build): estimate progress from the work left
All checks were successful
CI / Compile and test (pull_request) Has been skipped
CI / Conventional Commit title (pull_request) Successful in 3s
109a3572b7
Use builder timing and reuse signals for the overall bar and reserve time for linking, finishing and shaders.

Assisted-by: OpenAI Codex
Member

Possibly related, for context:

  • issue #94 Idea: Let the build estimate settle before showing it — Open issue asks to delay showing the ETA while it settles; adjacent estimate reliability work.
Possibly related, for context: - issue #94 Idea: Let the build estimate settle before showing it — Open issue asks to delay showing the ETA while it settles; adjacent estimate reliability work.
Author
Member

Same synthetic cold-read transcript on main and the patch. Main shows fixed stage markers. The patch shows elapsed time against estimated work left and keeps the measured read percentage in its row.

main-fb775d73.png

patch-109a3572.png

Same synthetic cold-read transcript on main and the patch. Main shows fixed stage markers. The patch shows elapsed time against estimated work left and keeps the measured read percentage in its row. ![main-fb775d73.png](https://git.magicbean.dev/attachments/53cfff5c-968f-4106-99b1-2b81360f7563) ![patch-109a3572.png](https://git.magicbean.dev/attachments/d17e45b9-fc4c-46ee-92cb-0178cd8f8743)
Author
Member

Separated shader timing from translation, compile, disk pace and build worker count. Replayed four saved cold-build checkpoints: at shader start the estimate is 5m11s instead of 14m05s, with 1m18s actually remaining. 119 focused assertions passed. Warm rebuild is queued.

Separated shader timing from translation, compile, disk pace and build worker count. Replayed four saved cold-build checkpoints: at shader start the estimate is 5m11s instead of 14m05s, with 1m18s actually remaining. 119 focused assertions passed. Warm rebuild is queued.
fix(build): forecast shader preparation separately
All checks were successful
CI / Compile and test (pull_request) Has been skipped
CI / Conventional Commit title (pull_request) Successful in 4s
82b60eac78
Assisted-by: OpenAI Codex
Author
Member

Warm v3.0.0 rebuild on 82b60eac reached Ready in 26.2s with zero changed compile files. Shader preparation took 11.6s. At the final pre-completion sample the fallback ETA showed 301s with 1.5s left. This exposes a remaining warm shader forecast gap; PR stays WIP.

Warm v3.0.0 rebuild on 82b60eac reached Ready in 26.2s with zero changed compile files. Shader preparation took 11.6s. At the final pre-completion sample the fallback ETA showed 301s with 1.5s left. This exposes a remaining warm shader forecast gap; PR stays WIP.
Author
Member

The estimate now names the work before shaders and waits for shader progress before giving a total. A real 27.0s warm rebuild ended with 78% in the shader row and no false five-minute countdown. The cold Finish checkpoints and 128 focused assertions are recorded.

The estimate now names the work before shaders and waits for shader progress before giving a total. A real 27.0s warm rebuild ended with 78% in the shader row and no false five-minute countdown. The cold Finish checkpoints and 128 focused assertions are recorded.
Author
Member

Synthetic build scene showing the estimate before shaders and the stage duration rows.

build-eta-before-shaders.png

Synthetic build scene showing the estimate before shaders and the stage duration rows. ![build-eta-before-shaders.png](https://git.magicbean.dev/attachments/f110c032-e984-40e6-9f24-6b1157d16cab)
fix(build): wait for shader evidence before showing a total ETA
All checks were successful
CI / Compile and test (pull_request) Has been skipped
CI / Conventional Commit title (pull_request) Successful in 3s
f010114d6e
Assisted-by: OpenAI Codex
Author
Member

Rebased on current Launcher main and added the renderer shader ETA consumer. The exact-SDK locked build and 137 focused assertions passed. The paired renderer measurements are running; the existing Windows-suite failures are documented.

Rebased on current Launcher main and added the renderer shader ETA consumer. The exact-SDK locked build and 137 focused assertions passed. The paired renderer measurements are running; the existing Windows-suite failures are documented.
futondestroyer force-pushed fix/build-progress-eta from f010114d6e
All checks were successful
CI / Compile and test (pull_request) Has been skipped
CI / Conventional Commit title (pull_request) Successful in 3s
to 436194f7e6
Some checks failed
CI / Compile and test (pull_request) Has been skipped
CI / Conventional Commit title (pull_request) Failing after 3s
2026-11-20 02:32:11 +00:00
Compare
Author
Member

Synthetic scenes from the updated Build screen. The renderer countdown and the cache-write status are distinct, and build completion remains a separate event.

build-shader-eta.png

build-shader-saving.png

Synthetic scenes from the updated Build screen. The renderer countdown and the cache-write status are distinct, and build completion remains a separate event. ![build-shader-eta.png](https://git.magicbean.dev/attachments/06792e6e-4709-4183-895e-50a02a9107ef) ![build-shader-saving.png](https://git.magicbean.dev/attachments/4e2667f3-f289-4fb9-9271-c3952f3628ed)
Author
Member

Final synthetic build scenes from the bounded estimator. The renderer countdown and cache saving still have distinct states.

build-shader-eta.png

build-shader-saving.png

Final synthetic build scenes from the bounded estimator. The renderer countdown and cache saving still have distinct states. ![build-shader-eta.png](https://git.magicbean.dev/attachments/5f2fc256-20b2-4b46-aa08-14ce3fae514f) ![build-shader-saving.png](https://git.magicbean.dev/attachments/9c644c99-f5ce-43af-82d8-f386781d996a)
fix(build): limit short-window shader ETA swings
Some checks failed
CI / Compile and test (pull_request) Has been skipped
CI / Conventional Commit title (pull_request) Failing after 3s
af8aa5f8a9
Assisted-by: OpenAI Codex
Author
Member

Final guarded shader ETA: cold replay mean absolute error fell from 49.68s to 35.17s on 446 common samples. Both warm pairs completed. 140 focused checks passed. The paired Longshot producer is #1830 in its own repository. Full local checks and fork CI still have the documented baseline and maintainer gates.

Final guarded shader ETA: cold replay mean absolute error fell from 49.68s to 35.17s on 446 common samples. Both warm pairs completed. 140 focused checks passed. The paired Longshot producer is #1830 in its own repository. Full local checks and fork CI still have the documented baseline and maintainer gates.
Author
Member

A fresh isolated shader run finished in 636.25 s and took 130.60 s after reporting 95%. On identical holdout events the local tail adjustment left final-minute mean absolute error at 28.58 s and slightly worsened overall error, so the published estimator stays at af8aa5f8. The PR records the test limits and the separate CI prerequisites.

A fresh isolated shader run finished in 636.25 s and took 130.60 s after reporting 95%. On identical holdout events the local tail adjustment left final-minute mean absolute error at 28.58 s and slightly worsened overall error, so the published estimator stays at af8aa5f8. The PR records the test limits and the separate CI prerequisites.
Author
Member

The exact main and PR snapshots reached the same second Launcher core assertion when the local diagnostic continued past the known UnlistedChecks failure. The description records that comparison and distinguishes the intentionally skipped Forgejo compile job, the failed Forgejo title check, and the maintainer-approved CircleCI fork gate.

The exact main and PR snapshots reached the same second Launcher core assertion when the local diagnostic continued past the known UnlistedChecks failure. The description records that comparison and distinguishes the intentionally skipped Forgejo compile job, the failed Forgejo title check, and the maintainer-approved CircleCI fork gate.
futondestroyer force-pushed fix/build-progress-eta from af8aa5f8a9
Some checks failed
CI / Compile and test (pull_request) Has been skipped
CI / Conventional Commit title (pull_request) Failing after 3s
to d2f22c353b
Some checks failed
CI / Compile and test (pull_request) Has been skipped
CI / Conventional Commit title (pull_request) Failing after 8s
2026-11-20 12:38:15 +00:00
Compare
Author
Member

Rebased on 073e96e0. The progress, ETA and Build view source is byte-identical to the previous published head. All 140 focused checks passed, and both exact revisions reproduced the same core, contract and Tooling smoke failures. The PR records the pin fix in #215 and the separate fork CI approval.

Rebased on 073e96e0. The progress, ETA and Build view source is byte-identical to the previous published head. All 140 focused checks passed, and both exact revisions reproduced the same core, contract and Tooling smoke failures. The PR records the pin fix in #215 and the separate fork CI approval.
Author
Member

Verified both external blockers on d2f22c35. #215 is still WIP and points to the correct Tooling commit; the title job currently reads main's zero pin, and fork CI awaits ci: approved. The newest nightly's checksum manifest verified against the Launcher trust keys, but its tagged source lacks the shader ETA event. No portable installation or CI dispatch was used.

Verified both external blockers on d2f22c35. #215 is still WIP and points to the correct Tooling commit; the title job currently reads main's zero pin, and fork CI awaits ci: approved. The newest nightly's checksum manifest verified against the Launcher trust keys, but its tagged source lacks the shader ETA event. No portable installation or CI dispatch was used.
HumanBean changed title from WIP: fix(build): estimate progress from the work left to fix(build): estimate progress from the work left 2026-11-20 15:40:53 +00:00
HumanBean deleted branch fix/build-progress-eta 2026-11-20 15:40:58 +00:00
Sign in to join this conversation.
No reviewers
No milestone
No project
No assignees
2 participants
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
MagicBeanDev/LongshotLauncher!213
No description provided.