[Tracking] Longshot rolling full builds and benchmarks — new Windows baseline #69
Labels
No labels
accessibility
autorelease: pending
autorelease: tagged
bug
dependencies
documentation
duplicate
enhancement
github_actions
good first issue
help wanted
invalid
needs measurements
question
testing wanted
wontfix
No milestone
No project
No assignees
1 participant
Notifications
Due date
No due date set.
Dependencies
No dependencies set
Reference
MagicBeanDev/Longshot#69
Loading…
Reference in a new issue
No description provided.
Delete branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Purpose and current status
Track a separate Windows rolling full-build and gameplay record for MagicBeanDev/Longshot, continuing the methodology and historical record from Hookshot #2595: six-hour main rebuilds and in-game performance benchmarks.
The new fixed baseline for Longshot testing is Longshot main at 7afe6ae1b005ad4eb91dfcfb8545cdaac6bd1cfb, qualified and accepted on November 12, 2026 (America/Halifax). It is a new within-Longshot reference, not an imported Hookshot baseline, a claim that this commit is regression-free, or a claim that it is the latest main.
The local full build and all three qualification passes are complete. The owner authorized and enabled six-hour Longshot full builds and benchmarks on November 12, 2026: 05:30, 11:30, 17:30 and 23:30 America/Halifax. A scoped, deduplicated numeric-result publisher targets this issue. The first scheduled end-to-end run is pending; configuration checks are not a completed scheduled run. CircleCI remains a later step. Existing Hookshot automation and its #2595 reporting were not changed or repointed. The Longshot slots are 90 minutes after the Hookshot slots, and both use the shared OS lock.
New Longshot baseline
v2.1.0-rc.3-328-g7afe6ae1 (source 7cfff061), exact Longshot commit above; llvm-mingw clang 23.1.5; six compiler workers; release cold O1 / hot O2 with the shipped local PGO profile. The NVIDIA SDK is present in the local build; runtime DLSS is off.windows-checkpoints-v2, three fresh checkpoint sessions per scene; every session contains one 60-second stationary interval and one 60-second compact-motion interval, after separate stationary/camera-circle preconditioning. Twelve measured phases passed; all six measured sessions exited cleanly.FPS is frame count divided by aligned game-present-return time. Frame intervals come from
HOS_PRESENT_TRACE, not a once-per-second FPS summary, GPU duration or monitor scanout. Percentiles use nearest rank; the table's median/p95/p99 values are per-pass values averaged across the three passes. The observed range is not a confidence interval. Driver caches persist; per-build game caches start fresh and are warmed before measurement. Cold first-visit/startup work is not included in the steady-state FPS.All twelve measured phases, including tails and quietness
1% low is 1000 divided by the mean of the slowest
ceil(N/100)intervals. Slow-frame thresholds are strictly greater than 20/25 ms. Counts above are per approximately 60-second phase; they must not be compared directly to counts from differently sized windows. All outliers are retained, including the 115.81 ms Field-motion maximum.Validity and limitations: all measured trace populations/alignment passed; spans were 60.000088–60.025910 s. Effective settings were identical across passes, client/source/output geometry was verified, and all measured display intervals had stable topology/power with no transition or presentation-error/recreation line. Background CPU stayed below three cores on average and at most six over roughly five seconds; the largest measured phase mean/peak were 2.3447/3.6849 cores. Binary hashes and immutable checkpoint manifests were rechecked. Twelve before/after screenshots were reviewed outside timing: expected scenes, player and HUD were present; Market motion retains the known close-NPC camera occlusion. This is not a visual A/B or whole-game correctness verdict; guards are sampled and do not provide continuous absolute-position/collision detection. GPU duration, peak VRAM/RSS, per-thread CPU and readback-wait costs are not measured by this runner.
Retained startup/warm-up failures: the first unmeasured startup hit the former 240-second first-present limit during cold shader preparation. Only that unmeasured allowance was extended to 1200 seconds; measured phase clocks, statistics and validity/quietness guards were unchanged, and all 18 runner checks passed again. A subsequent unmeasured warm-up failed quietness at 3.9235 background cores on average. Both attempts were retained. After a separate quiet preflight passed, the successful attempt restarted every warm-up and every measured phase from fresh checkpoint copies, using the same hash-verified build and its warmed caches. No earlier measured samples were reused, and no second full-build time is claimed.
Hookshot continuity — historical arithmetic only
The prior record is Hookshot #2595; the most recent Windows result used here is the November 12 report at 4dd972a8. That thread also contains the Linux companion measurements; those are separate hardware/presentation experiments and are not pooled here.
For continuity, the following table preserves the Windows fixed reference at Hookshot 55b976c0, the historical rolling sample at Hookshot 4dd972a8, and the new Longshot reference. The original Hookshot fixed reference precedes the readback fix in Hookshot #2602; it must not be described as containing that fix.
The two delta columns use the explicitly named Hookshot column as their arithmetic baseline. These are unpaired, cross-project historical differences, not a performance regression/improvement verdict or evidence about any particular commit. The rolling Hookshot sample has one pass per workload, whereas both fixed references have three. The historical report discloses changed settings/instrumentation fingerprints (display-mode IDs/semantic validation and effective option rows); the Longshot port and startup guard also changed instrumentation identity. Background load differs even within the accepted quiet limits. Storage locations/cache state and machine load were not held in an interleaved A/B. Strict within-project comparability checks are not being relaxed to admit Hookshot data as a Longshot reference.
55b976c0, 3 passes, FPS4dd972a8, 1 pass, FPS7afe6ae1, 3 passes, FPS55b976c04dd972a8The new reference is descriptively higher than the original Hookshot fixed sample and lower than the last Hookshot rolling sample. Neither direction establishes a causal change in Longshot. Proper confirmation follows current Longshot guidance: at least two warm, interleaved runs each way on the same PC/save/scene/settings/build type, named commits, median results and actual log/test evidence.
55b976c04dd972a87afe6ae1The Longshot observation is 147.975 s (+11.4%) above the original Hookshot build and 233.396 s (+19.2%) above the later one. These are descriptive single-build differences with unmeasured build variance, not a controlled compile-speed regression or an FPS conclusion.
Rolling automation work to track
The current-main game-free suite was run at
d4a69456: 491 passed, 2 failed and 36 skipped initially. A leftover empty legacy issue-template directory caused the bug-report failure; that directory was removed and the affected test passed on rerun. The remaining failure isnx2/test_nup_paths.py, whose Windows symbolic-link fixtures lack the required privilege (WinError 1314). This is not a clean full-suite pass. Those source checks are separate from the pinned7afe6ae1runtime qualification. Game-free migration checks passed, and all 18 installed runner checks passed after the startup-only adjustment.This owner-controlled local rolling monitor is related to, but does not duplicate or activate, Longshot #43: opt-in hardware tests on self-hosted Forgejo runners. It does not register a personal PC as an upstream/shared runner or execute arbitrary PR jobs. CircleCI remains deferred; no CI integration is claimed.
Credit and data boundary
Credit to mooki000's hookshot-workbench, reviewed at f45e4a3fac9e197e67ac66edb2f704f78c42be68, for the measurement/comparison approach and reused MIT-licensed statistics/empirical-band helpers; Michael Schienbein's MIT notice is preserved. This Windows record complements the Linux work and does not imply endorsement or equivalent hardware/presentation.
Only reviewed numeric summaries are published here. Game inputs, generated code, binaries, caches, saves, maps, raw traces, local paths and credentials remain on the local PC. No artifacts or raw directories are attached. Automatic team progress updates through the repository's Matrix-to-Discord bridge are a separate opt-in from scheduled benchmark execution and automated result publication.
Rolling automation enabled
The owner has approved four daily full-build and benchmark slots: 05:30, 11:30, 17:30 and 23:30 America/Halifax, starting with the next scheduled slot (November 12 at 23:30). Existing Hookshot scheduling and #2595 reporting are unchanged. Longshot is offset by 90 minutes, and the shared OS lock prevents managed build/benchmark overlap.
Each cycle reviews current official-main guidance, pins the fetched commit, rebuilds fully even if unchanged, and runs the qualified four-phase screening protocol. The accepted three-pass Longshot reference remains
7afe6ae1, with comparisons to that fixed reference and the previous compatible successful Longshot run. Reports disclose gains or slowdowns, spread and frame-tail findings; incompatible results withhold deltas. Historical Hookshot comparisons above remain continuity context, not the rolling Longshot reference or a controlled speedup claim.The scoped publisher posts one deduplicated numeric report here per new scheduled run, including controlled blocked/failed outcomes. It checks all comment pages and aborts publication on failed reads. Fifteen game-free reporting checks and all eighteen existing runner checks passed; the accepted baseline hash and Hookshot automation definition were verified unchanged. The known repository-suite Windows symlink-privilege failure remains disclosed above.
The first scheduled end-to-end run is pending. This activation comment is not benchmark evidence. Keep the PC and Codex app running and provide an unlocked, quiet normal Windows desktop for gameplay. Locked/unavailable/disturbed sessions must fail or block rather than count invalid FPS. Public output is limited to approved numeric summaries/provenance; all game inputs, saves, generated content, raw evidence and credentials remain local. Meaningful dev-agents updates follow the repo bridge, hourly limit and brief performance-findings policy. No extra team post is forced for activation. CircleCI remains deferred.