WIP: fix(vk): report the shader preparation countdown to builders #1830
No reviewers
Labels
No labels
area/accessibility
area/audio
area/input
area/install
area/menu
area/renderer
area/saves
area/tooling
area/upscaling
beanstalk: approved
beanstalk: failed
beanstalk: needs approval
beanstalk: running
beanstalk: tested
ci: approved
good first issue
gpu/amd
gpu/igpu
gpu/intel
gpu/nvidia
help wanted
kind
bug
kind
chore
kind
docs
kind
feature
kind
performance
kind
question
needs measurements
platform/linux
platform/macos
platform/steam-deck
platform/windows
preview
priority
critical
priority
high
priority
low
regression
risk
high
risk
low
risk
medium
status
blocked
status
confirmed
status
duplicate
status
invalid
status
needs-info
status
wontfix
ci: approved
No milestone
No project
No assignees
2 participants
Notifications
Due date
No due date set.
Dependencies
No dependencies set
Reference
MagicBeanDev/Longshot!1830
Loading…
Reference in a new issue
No description provided.
Delete branch "futondestroyer/Longshot:fix/renderer-shader-eta"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Summary
The Launcher can only infer shader preparation time from an overall fraction. That fraction can jump when the renderer finds reusable work or changes from shader programs to pipelines. The prepare-only renderer now emits its own recent-rate estimate and clears it as the work changes or the pipeline cache is written.
The producer is in
vk_progress.handvk_main.cpp. The existing saving intervals invk_draw.cppemit their status only in prepare-only mode.make_game.pyforwards the new shader event and keeps the renderer's exit separate from the builder's completion. The Launcher consumer is PR #213.Testing
Tested: Windows 11 build 26300, NVIDIA RTX 4070 driver 32.0.16.2101. Both earlier exact runtimes completed real
--prepare-shadersruns at Most. The patched renderer emitted 432 scoped signals, including two cache-saving events, and exited successfully. The relay regression failed on main because it stripped@@shader_eta unknownand passed on the patch. Five focused Windows checks passed after the first two assertions were updated. The test-only follow-up at06cb158ffixed the extracted prepare-cache and quit fixtures while preserving the renderer and builder source used by the game-tested runtime at3d544fc7.The interrupted Windows runner emitted
KeyboardInterruptand exited with WindowsSTATUS_CONTROL_C_EXIT(0xC000013A). Separate process groups completed all eight supportedtests/run_all.py --strict --shard K/8shards. The exact base068fb8ebhad 751 passes, 71 failures and 48 skips. The producer comparison had 752 passes, 70 failures and 48 skips across the same 870 checks. It reran the three shards containing the four changed fixture files and reused the results for the other, unchanged checks. The final revision also passed the tracked Python byte-compile check.tests/test_prepare_cache_save.pyis the sole outcome that changed. The repaired fixture exercises the actual saving helper, checks its flag during the forced write, and verifies thesavingandunknownevents. Its Most refresh expectation now matches the renderer's existing progress calculation. A second matched run with the full repository-pinned Python packages, the supported llvm-mingw configuration and .NET 8 clearedtest_fpx_sticky.py,test_make_game_cache.py,test_portable_builder_home.pyandtest_walk.pyon both sides.Rebase validation: Replayed the published stack onto
89c3d21e. Git's independent merge calculation matchesa74e3677exactly. The relay regression failed at the test-only commit with the expected dropped@@shader_eta unknownsignal. The rebased head passed all eight focused Windows checks and built the full Windows renderer withdev.py vk, including its export check. The latest main failed only the known Most-refresh expectation among the seven comparable focused tests. The 870-check comparison above applies to the earlier main and producer pair.The
Guide screenshots / Guide images and local UI source reviewcheck reports the same 33 source-review items on current main and the rebased head. The Guide source review and its recorded hashes need attention on main for green merge CI. The twoCI / Python source checks (ubuntu-24.04, part 1/2)andCI / Python source checks (ubuntu-24.04, part 2/2)jobs andCI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate)are manual dispatch jobs. A Linux runner is still needed for the applicable host build and UTF-8 path checklist item.70 named failures reproduced on the earlier main,
068fb8ebtests/test_appimage_player_build.pytests/test_auto_handheld.pytests/test_bug_report_defaults.pytests/test_build_long_paths.pytests/test_capture_read_once.pytests/test_capture_readback_visibility.pytests/test_crash_dialog.pytests/test_crt_options.pytests/test_damage_debug_menu.pytests/test_dev_ab.pytests/test_dev_build.pytests/test_dev_migrate.pytests/test_dev_packs.pytests/test_discover.pytests/test_display_startup_failure.pytests/test_dlss_feature_size.pytests/test_draw_stream_visibility.pytests/test_failed_compile_keeps_objects.pytests/test_fastfp_fma.pytests/test_fma_soft.pytests/test_fpx_sticky.pytests/test_frame_overlap.pytests/test_fsr3fi_hud_layer.pytests/test_fsr3fi_hudless_colour.pytests/test_fsr_image_retirement.pytests/test_fsr_uniform_ownership.pytests/test_game_menu_mouse_buttons.pytests/test_guest_imported.pytests/test_high_fps_fixes.pytests/test_host_guest_writes.pytests/test_hud_layout_row.pytests/test_insn_fixed_convert.pytests/test_job_queue.pytests/test_libc_hooks.pytests/test_lift_cache.pytests/test_lift_lookups.pytests/test_linux_preflight.pytests/test_linux_pressure.pytests/test_make_game_cache.pytests/test_make_game_ngx.pytests/test_mise_build.pytests/test_missing_nup.pytests/test_mods_list.pytests/test_ngx_sdk.pytests/test_pads_after_menu.pytests/test_parallel_init_order.pytests/test_pending_guest_import.pytests/test_pipeline_play_cache.pytests/test_portable_builder_home.pytests/test_present_queue_default.pytests/test_present_source_crop.pytests/test_present_thread.pytests/test_profiling_mode.pytests/test_psychov35_fsr3_fg.pytests/test_readback_timeout_source.pytests/test_replay_write_order.pytests/test_sdiv_overflow.pytests/test_slow_frames.pytests/test_song_note_cue.pytests/test_spawn_emit.pytests/test_staging_burst.pytests/test_texel_buffer_ownership.pytests/test_upscaler_rows.pytests/test_vbytes.pytests/test_vcore_escape.pytests/test_vfpx_elem.pytests/test_vk_occlusion_queries.pytests/test_walk.pytests/test_walk_key.pytests/test_work_lock.pyStill to test:
Measurements
Verdict: The renderer provides the measured countdown signal and distinguishes writing the cache from counting work. The raw renderer ETA is more responsive but less accurate overall on this cold trace, so the Launcher uses it as a bounded correction after its cumulative rate becomes available.
Setup: Windows 11 build 26300, i7-13700F, RTX 4070 driver 32.0.16.2101. Exact shared base and patched runtime, Most shader preparation, separate runtime caches. Launcher replay compares renderer ETA to the previous percentage-derived estimate on the same checkpoints.
068fb8eb06cb158f(runtime @3d544fc7)On the patched run, the initial raw ETA at the 160 s checkpoint was 447 s against 530 s actual remaining. The cumulative estimate at the same point was 699 s. Across the 446 common one-second samples, mean absolute error was 49.68 s for cumulative progress and 88.23 s for raw renderer ETA. The Launcher bounds the renderer correction after cumulative evidence is available, lowering the mean error to 35.17 s on the same samples, while preserving explicit unknown and saving signals. The warm order was main, patch, patch, main. Both runtimes reused their own cache from the cold run. These timings describe the tested cases. Raw traces stay local.
A third patched preparation used a fresh configured driver-cache directory and kept the assembled runtime's application cache. It completed in 636.25 s with 595 ETA events including two saving events. The last 5% of reported work took 130.60 s, compared with 3.85 s in the original patched cold run. The Launcher tested a local late-rate adjustment on both traces and kept it out of the PR after it failed to improve the independent trace. A more accurate tail estimate needs measurements for the actual work groups and for the time workers spend draining.
Risks
The extra output is limited to the prepare-only renderer and is independent of compile progress. Older clients can ignore the added event. The existing
LONGSHOT_PREPARE_ONLYswitch keeps its default off for normal game startup. Reverting this change removes the extra event.Checklist
Ready for review
main; the title is a Conventional Commit.mise run testpasses (any failure that also fails onmainis named). The earlier pair has a completed comparison. The current pair has focused checks and needs the full suite comparison.main, or Measurements says why there is nothing to measure.third_party/untouched; nothing GPL-incompatible.When it applies
mainand passes here.tests/that reads changed source is updated, not weakened.Assisted-by:/Co-Authored-By:trailers.Ready to merge
main, measured again or a comment says why the numbers still hold.Tested in game: Windows 11 build 26300, RTX 4070 driver 32.0.16.2101. Both exact dev runtimes ran real Most shader preparation to exit 0. The patched renderer sent its scoped ETA and distinct cache-save status.
Possibly related, for context:
Both warm repetitions passed on each exact runtime. The producer emitted separate saving events; replaying the same cold checkpoints shows the raw stage rate alone is too volatile, so the Launcher uses it as a bounded correction. Full Windows suite comparison is still running.
The independent isolated shader run passed on the same producer head, with 595 ETA events and both saving notifications. The three focused strict checks passed again. The full Windows run stopped without a completed suite result, and the Guide checker reports the same 25 source-review issues on official main and this head.
The interrupted Windows run returned STATUS_CONTROL_C_EXIT. Separate process groups completed all eight shards. Across the same 870 checks, main had 751 passes and 71 failures; the producer had 752 passes and 70 failures, all named and reproduced on main. The four repaired source checks and fixtures change only tests. Windows local validation is recorded, with the Linux host checklist and Guide merge check still open.
06cb158fa8a74e36773aRebased on
89c3d21e. The relay test fails for the expected missing event on the test-only commit, all eight focused Windows checks pass on the head, and the renderer builds and passes its export check. The prior full Windows comparison remains recorded for its exact pair. The current full suite and Linux host check remain open.View command line instructions
Checkout
From your project repository, check out a new branch and test the changes.