[Tracking] Six-hour main rebuilds and in-game performance benchmarks #2595

Open
opened 2026-11-12 00:50:21 +00:00 by futondestroyer · 5 comments
futondestroyer commented 2026-11-12 00:50:21 +00:00 (Migrated from github.com)

[Tracking] Six-hour main rebuilds and in-game performance benchmarks

This issue tracks a planned six-hour full-rebuild and gameplay record for official main, alongside the Linux companion measurements in the comments. Each Windows cycle will rebuild even when the commit has not changed and identify the exact tested revision. Windows scheduling and publication of run results are currently paused pending review of the completed local baseline and runner.

Fixed starting baseline

The fixed starting-main baseline is 55b976c0e15f38fc10dcac757e332800d09b7c24. This replaces the originally proposed v2.1.0 release baseline. It is a starting reference, not a claim that this revision is regression-free.

Correction — November 12, 2026: this SHA is immediately before the readback fix in #2602. The fix's merge commit 76bbe4aa has 55b976c0 as its sole parent. The fixed reference stays unchanged; it must not be described as already including that fix.

Later comparable runs will be compared against both this fixed baseline and the previous comparable successful run, with absolute and percentage changes. The baseline will not slide forward with main. Three initial gameplay passes per phase have been collected locally to establish an average and observed variation; routine cycles may use one pass as a screening observation. A historical-reference difference alone is not a confirmed regression. Build repeat count and unmeasured build-time variance will be explicit.

Build measurements

  • Isolated source, work and output folders; fresh translation and compilation of all game and renderer objects. No incremental object reuse.
  • Fixed installed llvm-mingw toolchain, six compiler workers, release defaults (cold lifted code at O1, hot code at O2 with the shipped PGO profile). Development hot-code O1 overrides are excluded.
  • Complete build wall time and stage timestamps: reading inputs, translation, compilation, linking and packaging where their boundaries are available. Fetch/source setup, game-free tests and shader preconditioning are outside the build timer.
  • Per-file compiler timings/cache status and full-rebuild plan, exact SHA/version, compiler version, input/output hashes, local logs and linker maps. Per-file compiler seconds run in parallel and are not added up as wall time. Compiler CPU time and peak RAM are not currently measured.
  • PASS, BUILD FAILED, BENCHMARK FAILED or BLOCKED outcomes, with confounders and configuration changes stated. Failed or disturbed gameplay never feeds the performance baseline. Build/gameplay success is reported separately from game-free suite outcomes.
  • Build seconds are a within-machine observation. Different toolchains, worker counts and competing loads prevent a direct compile-speed comparison with another PC. Resource/load context and repeated suspicious builds are needed before claiming a compile-speed change.

Gameplay protocol

Hardware: Windows 11, Intel i7-13700F, RTX 4070 12 GB, 32 GB RAM. The current GPU check reports NVIDIA driver 621.01; each run records the driver again.

Fixed graphics: a non-activating borderless Windows window with verified 2560×1440 client/source/output, 16:9, 100% render resolution, dynamic resolution and handheld profile off, the game's own upscaler. DLSS, DLAA, FSR, frame generation, HDR, Reflex, Anti-Lag and V-Sync are off; game FPS is uncapped with real-time game speed. Quality, FOV, sensitivity and presentation mode remain fixed for comparable runs.

Four separate 60-second wall-clock phases, with gameplay running:

  1. Hyrule Field outside Lon Lon Ranch, facing Castle Town: stand still.
  2. The same checkpoint: compact circle with camera rotation.
  3. Market in front of the mask shop, facing the fountain: stand still.
  4. The same checkpoint: compact circle with camera rotation.

Saved checkpoints restore the approximate location and camera view. Field uses the preserved daytime checkpoint. Each checkpoint is copied into a disposable user folder; original saves/settings are preserved. The accepted motion uses 60% forward analog deflection with full right-stick rotation and camera speed 200%. Inputs are exclusive during stationary and motion phases and released at a monotonic deadline. Frame-based bridge holds are only an upper bound, not the measurement clock.

Startup, loading, a stationary warm-up and the complete camera-circle preconditioning are outside timing. Measurements use fresh checkpoint loads after preconditioning. Per-build game caches start fresh and are warmed; the local driver cache persists. First-visit compilation is excluded from steady-state numbers and is not yet a separately quantified metric.

The runner verifies effective options, actual client/output dimensions, expected map, non-null gameplay state, event/loading flags, stalls, exclusive inputs and competing CPU load, rejecting invalid phases. Screenshots for route review occur outside measured intervals; no injected overlays, screenshot readbacks or sampled profiling run inside them.

Mean FPS comes from aligned per-frame game-present return intervals and frame count/time. Median, p95 and p99 intervals come from those samples, never from once-per-second bridge FPS summaries. These are game-present return timings, not direct GPU duration or monitor scanout. All four workloads stay separate; repeats and observed spread accompany the averages. Reports will also show each measured pass, sample count/span, 1% low, frames strictly over 20/25 ms, longest interval, and competing CPU mean/peak with eligibility. Here, 1% low means 1000 divided by the mean of the slowest ceil(N/100) intervals, matching the workbench nightly reader; percentiles use nearest rank.

Credit to mooki000’s hookshot-workbench for its public harness, agent guidance, comparison methods and reusable tools. We reviewed its repository description, skills, agent brief, publication checks and relevant measurement/rolling code at revision f45e4a3f. Our local runner reuses its MIT-licensed frame-statistics and empirical-band helpers, preserving the license and attribution.

This project adds fixed-cadence Windows measurements to that Linux testing. The reports use the same game-present timing boundary, explicit cache/warm-up policy, quiet-load principle, exact build identities, per-pass evidence and honest missing/failed results. Windows CPU accounting uses system/process counters; quiet means other work below three cores on average with no approximately five-second sample above six cores.

The results should be read alongside each other as within-machine trends, not pooled or compared as raw FPS/build seconds. CPU/GPU, OS/driver, compiler, worker count, scene/save, actual render/output size and windowed versus headless presentation can differ. Our Windows builds include the locally accepted NGX SDK, with runtime DLSS/DLAA off. The Linux companion reports also include a separate DLAA test. SDK/build configuration and runtime settings must be recorded per tested build, rather than inferred from an older recipe. Per-thread CPU, GPU duration, peak memory and Linux kernel-driver diagnostics are not currently equivalent measurements in this runner.

The workbench’s paired new/previous/new/previous checks are stronger evidence for a regression than a single scheduled sample. Its first companion report distinguishes paired main comparisons from dated release samples; its next cycle judges changes against the spread between repeats. These provide complementary evidence, not Windows results.

Different platforms, cards/VRAM budgets and scenes broaden coverage. Agreement in within-machine trends is useful corroboration. A disagreement alone cannot identify the OS as the cause when hardware, scene, resolution, driver and presentation also differ. Neither raw FPS nor build seconds will be pooled across the two PCs.

Methodology changes before Windows scheduling resumes

These are agreed directions for the next revision of the protocol; they are not claims that the runner already implements or has measured them.

  • Paired confirmation: a suspected change needs clean old/new/old/new runs in the same quiet session, with both builds prepared and warmed under the same scene/settings/cache policy. Check changes against both the fixed reference and the previous comparable run, allowing for their observed variation. The current automatic confirmation repeats only the new build when it falls outside the fixed baseline's observed range; that is insufficient for a paired regression verdict and must be strengthened or followed by a separate controlled comparison. A commit range alone is not proof of causation.
  • Aligned reporting: add mean frame milliseconds and absolute/percentage changes alongside FPS. Label paired comparisons separately from arithmetic against dated historical samples. Retain each pass, duration, frame count and tail statistics.
  • Comparable hitch rates: retain strict >20/>25-ms counts and longest intervals, but also report counts per minute or affected-frame percentages. Our 60-second counts cannot be compared directly with the Linux report's 30-second counts.
  • Memory observations: add explicitly defined process-memory measurements with their sampling window and units. Windows working set, private commit and GPU memory are different measures and must not be substituted for one another or for an unspecified Linux peak. An increased short-run peak is not evidence of a leak.
  • Readback diagnostics: consider wait timing after verifying the same instrumentation and counted population on each compared build, recording its overhead. Unlike conditional-wait counts are not directly comparable. Renderer correctness waits remain enabled.
  • One optional common test: preserve the four Windows workloads above while considering an additional agreed scene/view/input, commit pair, settings, presentation mode and measurement window for cross-machine corroboration. Each tester uses their own local save. A different headless/resolution/settings profile requires its own versioned baseline; it will not silently replace this one.

A particularly useful proposed next Windows comparison is the retained 55b976c0 build against its direct child 76bbe4aa (#2602), alternating on this PC. That would test the fix on Windows without including the many later merges in the sampled range. This experiment has not been run. No automatic bisect or second scheduled campaign is being started.

Current status and reporting

The fixed starting-baseline full build and three measured passes for each of the four Windows phases have completed locally. Client/source/output dimensions, frame traces, saved-checkpoint manifests and clean process exits were checked. The full game-free suite still has remaining failures documented locally; successful build/gameplay validation does not mean the suite is green. Detailed Windows result tables remain unpublished pending review. Calibration FPS is not benchmark evidence.

Individual run reports will be concise, deduplicated comments here, including failures, UTC time, SHA/commit link, build stages, each valid gameplay phase, comparisons, repeat variation, configuration and candidate intervening commit range. A range between samples is not proof of causation, and build time alone is not evidence of an in-game regression.

Game files, generated code, binaries, caches, saves, maps and raw traces stay on the local PC. Public reports contain reviewed measurements and permitted log excerpts only.

# [Tracking] Six-hour main rebuilds and in-game performance benchmarks This issue tracks a planned six-hour full-rebuild and gameplay record for official `main`, alongside the Linux companion measurements in the comments. Each Windows cycle will rebuild even when the commit has not changed and identify the exact tested revision. **Windows scheduling and publication of run results are currently paused pending review of the completed local baseline and runner.** ## Fixed starting baseline The fixed starting-main baseline is [`55b976c0e15f38fc10dcac757e332800d09b7c24`](https://git.magicbean.dev/MagicBeanDev/Hookshot/commit/55b976c0e15f38fc10dcac757e332800d09b7c24). This replaces the originally proposed v2.1.0 release baseline. It is a starting reference, not a claim that this revision is regression-free. **Correction — November 12, 2026:** this SHA is immediately **before** the readback fix in #2602. The fix's merge commit [`76bbe4aa`](https://git.magicbean.dev/MagicBeanDev/Hookshot/commit/76bbe4aa58e3097a0350840badef1c4035ea8ea6) has `55b976c0` as its sole parent. The fixed reference stays unchanged; it must not be described as already including that fix. Later comparable runs will be compared against both this fixed baseline and the previous comparable successful run, with absolute and percentage changes. The baseline will not slide forward with main. Three initial gameplay passes per phase have been collected locally to establish an average and observed variation; routine cycles may use one pass as a screening observation. A historical-reference difference alone is not a confirmed regression. Build repeat count and unmeasured build-time variance will be explicit. ## Build measurements - Isolated source, work and output folders; fresh translation and compilation of all game and renderer objects. No incremental object reuse. - Fixed installed llvm-mingw toolchain, six compiler workers, release defaults (cold lifted code at O1, hot code at O2 with the shipped PGO profile). Development hot-code O1 overrides are excluded. - Complete build wall time and stage timestamps: reading inputs, translation, compilation, linking and packaging where their boundaries are available. Fetch/source setup, game-free tests and shader preconditioning are outside the build timer. - Per-file compiler timings/cache status and full-rebuild plan, exact SHA/version, compiler version, input/output hashes, local logs and linker maps. Per-file compiler seconds run in parallel and are not added up as wall time. Compiler CPU time and peak RAM are not currently measured. - PASS, BUILD FAILED, BENCHMARK FAILED or BLOCKED outcomes, with confounders and configuration changes stated. Failed or disturbed gameplay never feeds the performance baseline. Build/gameplay success is reported separately from game-free suite outcomes. - Build seconds are a within-machine observation. Different toolchains, worker counts and competing loads prevent a direct compile-speed comparison with another PC. Resource/load context and repeated suspicious builds are needed before claiming a compile-speed change. ## Gameplay protocol Hardware: Windows 11, Intel i7-13700F, RTX 4070 12 GB, 32 GB RAM. The current GPU check reports NVIDIA driver 621.01; each run records the driver again. Fixed graphics: a non-activating borderless Windows window with verified 2560×1440 client/source/output, 16:9, 100% render resolution, dynamic resolution and handheld profile off, the game's own upscaler. DLSS, DLAA, FSR, frame generation, HDR, Reflex, Anti-Lag and V-Sync are off; game FPS is uncapped with real-time game speed. Quality, FOV, sensitivity and presentation mode remain fixed for comparable runs. Four separate 60-second wall-clock phases, with gameplay running: 1. Hyrule Field outside Lon Lon Ranch, facing Castle Town: stand still. 2. The same checkpoint: compact circle with camera rotation. 3. Market in front of the mask shop, facing the fountain: stand still. 4. The same checkpoint: compact circle with camera rotation. Saved checkpoints restore the approximate location and camera view. Field uses the preserved daytime checkpoint. Each checkpoint is copied into a disposable user folder; original saves/settings are preserved. The accepted motion uses 60% forward analog deflection with full right-stick rotation and camera speed 200%. Inputs are exclusive during stationary and motion phases and released at a monotonic deadline. Frame-based bridge holds are only an upper bound, not the measurement clock. Startup, loading, a stationary warm-up and the complete camera-circle preconditioning are outside timing. Measurements use fresh checkpoint loads after preconditioning. Per-build game caches start fresh and are warmed; the local driver cache persists. First-visit compilation is excluded from steady-state numbers and is not yet a separately quantified metric. The runner verifies effective options, actual client/output dimensions, expected map, non-null gameplay state, event/loading flags, stalls, exclusive inputs and competing CPU load, rejecting invalid phases. Screenshots for route review occur outside measured intervals; no injected overlays, screenshot readbacks or sampled profiling run inside them. Mean FPS comes from aligned per-frame game-present return intervals and frame count/time. Median, p95 and p99 intervals come from those samples, never from once-per-second bridge FPS summaries. These are game-present return timings, not direct GPU duration or monitor scanout. All four workloads stay separate; repeats and observed spread accompany the averages. Reports will also show each measured pass, sample count/span, 1% low, frames strictly over 20/25 ms, longest interval, and competing CPU mean/peak with eligibility. Here, 1% low means 1000 divided by the mean of the slowest ceil(N/100) intervals, matching the workbench nightly reader; percentiles use nearest rank. ## Related work and credit Credit to [mooki000’s hookshot-workbench](https://github.com/mooki000/hookshot-workbench) for its public harness, agent guidance, comparison methods and reusable tools. We reviewed its repository description, skills, agent brief, publication checks and relevant measurement/rolling code at [revision f45e4a3f](https://github.com/mooki000/hookshot-workbench/tree/f45e4a3fac9e197e67ac66edb2f704f78c42be68). Our local runner reuses its MIT-licensed frame-statistics and empirical-band helpers, preserving the license and attribution. This project adds fixed-cadence Windows measurements to that Linux testing. The reports use the same game-present timing boundary, explicit cache/warm-up policy, quiet-load principle, exact build identities, per-pass evidence and honest missing/failed results. Windows CPU accounting uses system/process counters; quiet means other work below three cores on average with no approximately five-second sample above six cores. The results should be read alongside each other as within-machine trends, not pooled or compared as raw FPS/build seconds. CPU/GPU, OS/driver, compiler, worker count, scene/save, actual render/output size and windowed versus headless presentation can differ. Our Windows builds include the locally accepted NGX SDK, with runtime DLSS/DLAA off. The Linux companion reports also include a separate DLAA test. SDK/build configuration and runtime settings must be recorded per tested build, rather than inferred from an older recipe. Per-thread CPU, GPU duration, peak memory and Linux kernel-driver diagnostics are not currently equivalent measurements in this runner. The workbench’s paired new/previous/new/previous checks are stronger evidence for a regression than a single scheduled sample. Its [first companion report](https://git.magicbean.dev/MagicBeanDev/Hookshot/issues/2595) distinguishes paired main comparisons from dated release samples; its [next cycle](https://git.magicbean.dev/MagicBeanDev/Hookshot/issues/2595) judges changes against the spread between repeats. These provide complementary evidence, not Windows results. Different platforms, cards/VRAM budgets and scenes broaden coverage. Agreement in within-machine trends is useful corroboration. A disagreement alone cannot identify the OS as the cause when hardware, scene, resolution, driver and presentation also differ. Neither raw FPS nor build seconds will be pooled across the two PCs. ## Methodology changes before Windows scheduling resumes These are agreed directions for the next revision of the protocol; they are **not claims that the runner already implements or has measured them**. - **Paired confirmation:** a suspected change needs clean old/new/old/new runs in the same quiet session, with both builds prepared and warmed under the same scene/settings/cache policy. Check changes against both the fixed reference and the previous comparable run, allowing for their observed variation. The current automatic confirmation repeats only the new build when it falls outside the fixed baseline's observed range; that is insufficient for a paired regression verdict and must be strengthened or followed by a separate controlled comparison. A commit range alone is not proof of causation. - **Aligned reporting:** add mean frame milliseconds and absolute/percentage changes alongside FPS. Label paired comparisons separately from arithmetic against dated historical samples. Retain each pass, duration, frame count and tail statistics. - **Comparable hitch rates:** retain strict >20/>25-ms counts and longest intervals, but also report counts per minute or affected-frame percentages. Our 60-second counts cannot be compared directly with the Linux report's 30-second counts. - **Memory observations:** add explicitly defined process-memory measurements with their sampling window and units. Windows working set, private commit and GPU memory are different measures and must not be substituted for one another or for an unspecified Linux peak. An increased short-run peak is not evidence of a leak. - **Readback diagnostics:** consider wait timing after verifying the same instrumentation and counted population on each compared build, recording its overhead. Unlike conditional-wait counts are not directly comparable. Renderer correctness waits remain enabled. - **One optional common test:** preserve the four Windows workloads above while considering an additional agreed scene/view/input, commit pair, settings, presentation mode and measurement window for cross-machine corroboration. Each tester uses their own local save. A different headless/resolution/settings profile requires its own versioned baseline; it will not silently replace this one. A particularly useful proposed next Windows comparison is the retained `55b976c0` build against its direct child `76bbe4aa` (#2602), alternating on this PC. That would test the fix on Windows without including the many later merges in the sampled range. This experiment has not been run. No automatic bisect or second scheduled campaign is being started. ## Current status and reporting The fixed starting-baseline full build and three measured passes for each of the four Windows phases have completed locally. Client/source/output dimensions, frame traces, saved-checkpoint manifests and clean process exits were checked. The full game-free suite still has remaining failures documented locally; successful build/gameplay validation does not mean the suite is green. Detailed Windows result tables remain unpublished pending review. Calibration FPS is not benchmark evidence. Individual run reports will be concise, deduplicated comments here, including failures, UTC time, SHA/commit link, build stages, each valid gameplay phase, comparisons, repeat variation, configuration and candidate intervening commit range. A range between samples is not proof of causation, and build time alone is not evidence of an in-game regression. Game files, generated code, binaries, caches, saves, maps and raw traces stay on the local PC. Public reports contain reviewed measurements and permitted log excerpts only.
Owner

@mooki000 commented on GitHub · 2026-11-12 09:21 UTC · original

A Linux companion to the Windows cycles, from this PC's rolling rebuilds (the workbench tool the issue credits): the v2.1.0 release, main before the readback fix, and main as of 2026-11-12 08:14 UTC, each built fresh from source and measured in the same three scenes.

Machine: Arch Linux (kernel 7.2.15), i9-13900K, 94 GB, RTX 4090 24 GB, NVIDIA 615.78.08; no window (frames are the game's offscreen-present intervals, not scanout); 2048x1152.

Builds: 8ef84611 (v2.1.0), ef81aee7 (main of 2026-11-11, before #2602), 46780503 (main, with #2602 and #2698). Clang 23.1.3, LLD, 16 compile and 8 lift workers, release settings with the recorded profile.

Scene (fps, two runs each) v2.1.0 main before the fix main 46780503 new against before (mean frame time, paired) new against v2.1.0 (not paired)
Kokiri Forest, own upscaler, cap 240 79.5 / 79.7 63.8 / 63.4 79.3 / 81.1 -20.7 % -0.7 %
Hyrule Field, own upscaler, cap 240 94.7 / 94.0 69.4 / 68.9 93.0 / 92.3 -25.4 % +1.8 %
Kokiri Forest, DLAA, cap 60 59.0 / 59.0 54.6 / 54.6 59.8 / 59.5 -8.5 % -1.2 %
frames over 20 ms: v2.1.0 before new readback wait, ms a frame: v2.1.0 before new peak memory, GiB: v2.1.0 before new
Kokiri Forest, own upscaler, cap 240 26 / 16 396 / 412 19 / 16 0.02 / 0.02 5.04 / 5.00 0.54 / 0.54 12.9 14.8 16.4
Hyrule Field, own upscaler, cap 240 0 / 0 18 / 18 2 / 3 0.02 / 0.02 4.82 / 4.78 0.36 / 0.34 13.4 15.1 17.0
Kokiri Forest, DLAA, cap 60 49 / 55 435 / 470 33 / 38 0.02 / 0.02 6.00 / 6.01 0.67 / 0.72 13.2 15.1 16.9

What it says for this machine:

  • Main is back at the release's frame rate in all three scenes: its mean frame time is 20.7 and 25.4 % shorter than before the fix in the two uncapped scenes and 8.5 % shorter at the 60 cap with DLAA. That fits #2602, whose own pull request has the controlled comparison; a gap between two sampled commits is not by itself proof of a cause.
  • The readback wait is not back to the release's: about 0.5 ms a frame now, against 0.02 ms on v2.1.0 (5 ms before the fix). Brett-D's RTX 5090 numbers on #2602 show a remainder of the same kind (1.3 to 1.4 ms a frame there).
  • Peak memory rose across the three builds, by 3.5 to 3.8 GiB from v2.1.0 to main (the average of each build's two whole-run peaks). An observation only: it is not the judged metric, and nothing here says it is a leak.

How it was measured, and where it differs from this issue's protocol: each scene is one hold on the card with the two main builds alternating (new, before, new, before), a prepared cache and an unmeasured warm-up first, 30 s measured from 15 s after the load; all runs counted as quiet (other work under 3 cores on average, repeats within 5 %). The same opt-in readback logging runs on all three builds; its cost is not subtracted. The release's runs come from separate holds earlier the same night (2026-11-12 05:29 to 08:14 UTC), so the last column is arithmetic between dated samples, not a paired verdict. The route is this PC's timed input sweep, not the four standard phases, with a 240 cap on the uncapped arms (the measured rates stay well below it) and the camera not pixel-locked. So: a trend within this machine, not numbers to pool with the Windows ones.

Build wall time: 649 s, 753 s, 568 s for the three, under different machine loads; not a compile-speed comparison.

The tool and its report format: github.com/mooki000/hookshot-workbench@0febe1c99b/nightly

> [@mooki000](https://github.com/mooki000) commented on GitHub · 2026-11-12 09:21 UTC · [original](https://github.com/NotRexed/Hookshot/issues/2595#issuecomment-6819741343) A Linux companion to the Windows cycles, from this PC's rolling rebuilds (the workbench tool the issue credits): **the v2.1.0 release, main before the readback fix, and main as of 2026-11-12 08:14 UTC**, each built fresh from source and measured in the same three scenes. **Machine:** Arch Linux (kernel 7.2.15), i9-13900K, 94 GB, RTX 4090 24 GB, NVIDIA 615.78.08; no window (frames are the game's offscreen-present intervals, not scanout); 2048x1152. **Builds:** `8ef84611` (v2.1.0), `ef81aee7` (main of 2026-11-11, before #2602), `46780503` (main, with #2602 and #2698). Clang 23.1.3, LLD, 16 compile and 8 lift workers, release settings with the recorded profile. | Scene (fps, two runs each) | v2.1.0 | main before the fix | main `46780503` | new against before (mean frame time, paired) | new against v2.1.0 (not paired) | |---|---|---|---|---|---| | Kokiri Forest, own upscaler, cap 240 | 79.5 / 79.7 | 63.8 / 63.4 | **79.3 / 81.1** | -20.7 % | -0.7 % | | Hyrule Field, own upscaler, cap 240 | 94.7 / 94.0 | 69.4 / 68.9 | **93.0 / 92.3** | -25.4 % | +1.8 % | | Kokiri Forest, DLAA, cap 60 | 59.0 / 59.0 | 54.6 / 54.6 | **59.8 / 59.5** | -8.5 % | -1.2 % | | | frames over 20 ms: v2.1.0 | before | new | readback wait, ms a frame: v2.1.0 | before | new | peak memory, GiB: v2.1.0 | before | new | |---|---|---|---|---|---|---|---|---|---| | Kokiri Forest, own upscaler, cap 240 | 26 / 16 | 396 / 412 | 19 / 16 | 0.02 / 0.02 | 5.04 / 5.00 | 0.54 / 0.54 | 12.9 | 14.8 | 16.4 | | Hyrule Field, own upscaler, cap 240 | 0 / 0 | 18 / 18 | 2 / 3 | 0.02 / 0.02 | 4.82 / 4.78 | 0.36 / 0.34 | 13.4 | 15.1 | 17.0 | | Kokiri Forest, DLAA, cap 60 | 49 / 55 | 435 / 470 | 33 / 38 | 0.02 / 0.02 | 6.00 / 6.01 | 0.67 / 0.72 | 13.2 | 15.1 | 16.9 | **What it says for this machine:** - **Main is back at the release's frame rate** in all three scenes: its mean frame time is 20.7 and 25.4 % shorter than before the fix in the two uncapped scenes and 8.5 % shorter at the 60 cap with DLAA. That fits #2602, whose own pull request has the controlled comparison; a gap between two sampled commits is not by itself proof of a cause. - **The readback wait is not back to the release's**: about 0.5 ms a frame now, against 0.02 ms on v2.1.0 (5 ms before the fix). Brett-D's RTX 5090 numbers on #2602 show a remainder of the same kind (1.3 to 1.4 ms a frame there). - **Peak memory rose across the three builds**, by 3.5 to 3.8 GiB from v2.1.0 to main (the average of each build's two whole-run peaks). An observation only: it is not the judged metric, and nothing here says it is a leak. **How it was measured, and where it differs from this issue's protocol:** each scene is one hold on the card with the two main builds alternating (new, before, new, before), a prepared cache and an unmeasured warm-up first, 30 s measured from 15 s after the load; all runs counted as quiet (other work under 3 cores on average, repeats within 5 %). The same opt-in readback logging runs on all three builds; its cost is not subtracted. The release's runs come from separate holds earlier the same night (2026-11-12 05:29 to 08:14 UTC), so the last column is arithmetic between dated samples, not a paired verdict. The route is this PC's timed input sweep, not the four standard phases, with a 240 cap on the uncapped arms (the measured rates stay well below it) and the camera not pixel-locked. So: a trend within this machine, not numbers to pool with the Windows ones. **Build wall time:** 649 s, 753 s, 568 s for the three, under different machine loads; not a compile-speed comparison. The tool and its report format: https://github.com/mooki000/hookshot-workbench/tree/0febe1c99be8e054b19aec97815ac1a1314a990c/nightly <!-- github-sync:c:6819741343 -->
Owner

@mooki000 commented on GitHub · 2026-11-12 12:11 UTC · original

Next Linux cycle, same machine and method as above: main 51da0464 (2026-11-12 11:19 UTC) against the previous cycle's main 46780503, 6 commits apart, the two builds alternating in one hold per scene.

Scene (fps, two runs each) 46780503 51da0464 mean frame time, new against previous frames over 20 ms: previous new
Kokiri Forest, own upscaler, cap 240 79.6 / 80.7 79.6 / 79.9 +0.4 % 9 / 16 26 / 19
Hyrule Field, own upscaler, cap 240 92.8 / 90.6 91.1 / 90.6 +0.9 % 3 / 3 1 / 2
Kokiri Forest, DLAA, cap 60 59.4 / 59.2 59.5 / 59.4 -0.2 % 51 / 68 38 / 44

No change: all three differences are inside the spread between repeats (the tool's verdict for each is "within repeat spread"). All twelve runs counted as quiet. Build wall time 571 s (568 s last cycle).

> [@mooki000](https://github.com/mooki000) commented on GitHub · 2026-11-12 12:11 UTC · [original](https://github.com/NotRexed/Hookshot/issues/2595#issuecomment-6823250809) Next Linux cycle, same machine and method as above: main `51da0464` (2026-11-12 11:19 UTC) against the previous cycle's main `46780503`, 6 commits apart, the two builds alternating in one hold per scene. | Scene (fps, two runs each) | `46780503` | `51da0464` | mean frame time, new against previous | frames over 20 ms: previous | new | |---|---|---|---|---|---| | Kokiri Forest, own upscaler, cap 240 | 79.6 / 80.7 | 79.6 / 79.9 | +0.4 % | 9 / 16 | 26 / 19 | | Hyrule Field, own upscaler, cap 240 | 92.8 / 90.6 | 91.1 / 90.6 | +0.9 % | 3 / 3 | 1 / 2 | | Kokiri Forest, DLAA, cap 60 | 59.4 / 59.2 | 59.5 / 59.4 | -0.2 % | 51 / 68 | 38 / 44 | **No change**: all three differences are inside the spread between repeats (the tool's verdict for each is "within repeat spread"). All twelve runs counted as quiet. Build wall time 571 s (568 s last cycle). <!-- github-sync:c:6823250809 -->
Owner

@Revens2 commented on GitHub · 2026-11-12 12:55 UTC · original

Windows A/B results for three perf PRs, posted here to consolidate alongside the rolling cycles (requested in #dev-work).

Machine: Windows 11 build 26200, Intel i5-14600KF, NVIDIA RTX 5070 12 GB, driver 32.0.16.2114, Vulkan back end, warm caches. Control = the v2.1.0 release, not the fixed baseline 55b976c0. Different hardware, scenes and settings from this issue's four-phase protocol and from mooki000's Linux cycles: these are within-machine paired PR checks only, not poolable and not main-regression tracking.

Method: each PR is a paired control/patch A/B on the same route — rainy Kokiri Forest clearing, 80 s window after frame 3000, 2 clean runs per build, avg / 1% low / p99 from the game-frame trace. 2560x1440 output, frame generation on, 240 cap, default quality — except #2180 which runs a 3840x2160 output. #2156 and #2238 run under a synthetic 4700 MB allowance (RX_VK_TEST_ALLOWED_MB + RX_VK_VRAM_MB); the real driver allowance is 11175 MB, so the pressure path is exercised without real paging.

PR scene control v2.1.0 (avg / 1% low / p99 ms) patched (avg / 1% low / p99 ms) verdict
#2156 squeeze past the allowance 1440p + pressure 94.15 / 76.81 / 13.02 96.31 / 77.91 / 12.86 fps inside same-binary spread (repeats differ up to ~4.5%); state line not squeezed 12 -> 0 over the same harness = the fix itself
#2238 shadow give-back 1440p + pressure 94.15 / 76.81 / 13.02 94.88 / 77.61 / 12.89 neutral within spread; give-back fires in both builds, the added remake branch needs a scene transition this route has none of
#2180 -> 134b3ab render targets 4K, no synthetic pressure 79.18 / 68.85 / 14.53 79.16 / 69.09 / 14.48 neutral fps; made new 2441 -> 1339 mean, let-go +38% over control, pool reuse +44%, max frame 35.9 -> 23.0 ms

(#2242 + #2218 has no in-game surface — headless prepare path, no numbers to post.)

Limits, stated the way this issue asks: a 12 GB card cannot replay #2156's 8 GB paging repro (the honest evidence here is the squeeze state lines, not fps); the synthetic allowance never pages on this driver; #2180's original 16 GB AMD repro was not hit (graveyard save with headroom); none of these runs uses the four phases or the fixed baseline. Commit causation is not claimed from paired PR diffs alone.

Full run reports live in the fork PRs: https://github.com/Revens2/Hookshot/pull/6 (#2238), https://github.com/Revens2/Hookshot/pull/8 (#2156), https://github.com/Revens2/Hookshot/pull/9 (#2180), all heading upstream. Happy to keep dropping PR-level A/Bs here when they help alongside the rolling cycles.

> [@Revens2](https://github.com/Revens2) commented on GitHub · 2026-11-12 12:55 UTC · [original](https://github.com/NotRexed/Hookshot/issues/2595#issuecomment-6824187247) **Windows A/B results for three perf PRs**, posted here to consolidate alongside the rolling cycles (requested in #dev-work). **Machine:** Windows 11 build 26200, Intel i5-14600KF, NVIDIA RTX 5070 12 GB, driver 32.0.16.2114, Vulkan back end, warm caches. **Control = the v2.1.0 release**, not the fixed baseline `55b976c0`. Different hardware, scenes and settings from this issue's four-phase protocol and from mooki000's Linux cycles: these are within-machine paired PR checks only, not poolable and not main-regression tracking. **Method:** each PR is a paired control/patch A/B on the same route — rainy Kokiri Forest clearing, 80 s window after frame 3000, 2 clean runs per build, avg / 1% low / p99 from the game-frame trace. 2560x1440 output, frame generation on, 240 cap, default quality — except #2180 which runs a 3840x2160 output. #2156 and #2238 run under a synthetic 4700 MB allowance (`RX_VK_TEST_ALLOWED_MB` + `RX_VK_VRAM_MB`); the real driver allowance is 11175 MB, so the pressure path is exercised without real paging. | PR | scene | control v2.1.0 (avg / 1% low / p99 ms) | patched (avg / 1% low / p99 ms) | verdict | |---|---|---|---|---| | #2156 squeeze past the allowance | 1440p + pressure | 94.15 / 76.81 / 13.02 | 96.31 / 77.91 / 12.86 | fps inside same-binary spread (repeats differ up to ~4.5%); state line `not squeezed` 12 -> 0 over the same harness = the fix itself | | #2238 shadow give-back | 1440p + pressure | 94.15 / 76.81 / 13.02 | 94.88 / 77.61 / 12.89 | neutral within spread; give-back fires in both builds, the added remake branch needs a scene transition this route has none of | | #2180 -> `134b3ab` render targets | 4K, no synthetic pressure | 79.18 / 68.85 / 14.53 | 79.16 / 69.09 / 14.48 | neutral fps; `made new` 2441 -> 1339 mean, let-go +38% over control, pool reuse +44%, max frame 35.9 -> 23.0 ms | (#2242 + #2218 has no in-game surface — headless prepare path, no numbers to post.) **Limits, stated the way this issue asks:** a 12 GB card cannot replay #2156's 8 GB paging repro (the honest evidence here is the squeeze state lines, not fps); the synthetic allowance never pages on this driver; #2180's original 16 GB AMD repro was not hit (graveyard save with headroom); none of these runs uses the four phases or the fixed baseline. Commit causation is not claimed from paired PR diffs alone. Full run reports live in the fork PRs: https://github.com/Revens2/Hookshot/pull/6 (#2238), https://github.com/Revens2/Hookshot/pull/8 (#2156), https://github.com/Revens2/Hookshot/pull/9 (#2180), all heading upstream. Happy to keep dropping PR-level A/Bs here when they help alongside the rolling cycles. <!-- github-sync:c:6824187247 -->
Owner

@futondestroyer commented on GitHub · 2026-11-12 14:35 UTC · original

PASS — 2026-11-12T14:03:22.277990+00:00

Commit: 51da0464b4cf
Protocol: windows-checkpoints-v2; Windows 11, i7-13700F, RTX 4070; NVIDIA GeForce RTX 4070, 621.01.

Version: v2.1.0-258-g51da0464 (source 18ed87b5).
Compiler: clang version 23.1.5 (https://github.com/llvm/llvm-project.git 392dfff97d29b59d77e5cac6d4b6ef2585fe7117).

Full build: 1231.55 s (20.53 min), one repeat; six compiler workers; release cold O1 / hot O2 + shipped PGO.
Stages (wall seconds): read 106.67, lift 108.67, finish 133.22, compilation 879.89, linking 1.83, startup 1.26.
Fetch/setup, tests and shader preconditioning excluded. Build repeat variance is not measured.

Lift process observations: lift: emit and write (16 processes) in 23.8s (63s); lift: patches (16 processes) in 25.7s (90s).

Build versus fixed starting baseline: -71.75 (-5.51%) s.
Build versus previous comparable success: -71.75 (-5.51%) s.

Phase Mean FPS Median / p95 / p99 ms Repeats; FPS range vs fixed FPS (%) vs previous FPS (%)
field still 73.76 13.50 / 14.31 / 15.07 1; 73.76–73.76 +18.27 (+32.93%) +18.27 (+32.93%)
field motion 67.25 14.13 / 18.24 / 20.45 1; 67.25–67.25 +15.28 (+29.41%) +15.28 (+29.41%)
market still 57.49 17.30 / 19.30 / 21.79 1; 57.49–57.49 +14.71 (+34.37%) +14.71 (+34.37%)
market motion 54.73 17.42 / 25.38 / 30.44 1; 54.73–54.73 +7.66 (+16.27%) +7.66 (+16.27%)

Game-present return intervals, frame-count/time mean; neither GPU duration nor monitor scanout. Percentiles shown are per-pass percentiles averaged across repeats.
2560×1440 borderless client/output, 16:9, fixed 100%, dynamic resolution off, game’s own upscaler; DLSS/DLAA/FSR/FG/HDR/Reflex/Anti-Lag/V-Sync off, uncapped, real-time speed. Fresh checkpoint loads; stationary and complete camera-circle preconditioning outside timing.
Driver cache is persistent; per-build game caches start fresh and are warmed. First-visit compilation is excluded. Single-pass changes are candidates only; repeat suspicious results before calling a regression.

Phase / pass Frames / seconds FPS / 1% low >20 / >25 ms Max ms Game CPU ms/frame Other CPU mean / peak cores Eligible
field still / 1 4427 / 60.017 73.76 / 55.30 13 / 3 27.07 69.27 0.69 / 1.13 yes
field motion / 1 4035 / 60.001 67.25 / 35.00 47 / 14 92.12 80.41 0.68 / 1.15 yes
market still / 1 3451 / 60.024 57.49 / 39.54 77 / 18 30.41 111.59 0.75 / 1.24 yes
market motion / 1 3284 / 60.009 54.73 / 26.20 918 / 185 110.79 108.48 1.58 / 2.33 yes

1% low = 1000 / mean of the slowest ceil(N/100) frame intervals, matching the workbench nightly reader. Slow-frame thresholds are strictly greater than 20/25 ms. No peak-memory, GPU-duration or per-thread CPU claim is made.

Validation and observations

All expected phases completed with valid trace coverage/alignment, checkpoint/settings checks and background CPU eligibility.

  • Mean FPS versus fixed baseline: higher in every measured workload; workload changes range from +16.27% to +34.37%.
  • Outside the reference's observed repeat range: field still, field motion, market still, market motion.
  • Mean FPS versus previous comparable run: higher in every measured workload; workload changes range from +16.27% to +34.37%.
  • Outside the reference's observed repeat range: field still, field motion, market still, market motion.
  • These are descriptive comparisons, not a statistical significance test or a confirmed regression. A second new-build pass is screening evidence; regression confirmation requires paired old/new testing.
  • Build wall time was 5.51% lower than the original single build observation; build-time variance remains unmeasured.
  • Largest measured frame interval: 110.79 ms in market motion (pass 1).
  • Configuration disclaimer: this build newly reports the default language settings game_language=0 and menu_language=0, which were absent from the baseline configuration snapshot. These additions are not expected to affect the checkpoint performance tests; baseline comparisons are included under this disclosed assumption. Other configuration/instrumentation checks remain strict.

Intervening commits; candidate range, not proof of causation.

No visual regression claim: measured intervals have no screenshot readbacks. Build time is separate from in-game speed.

Method and metric-reader credit: mooki000’s hookshot-workbench, reviewed at f45e4a3f; reused helpers retain the MIT license. This Windows route complements its Linux testing. Different hardware/scenes/presentation modes make absolute FPS and build seconds unsuitable for direct comparison; compare within-machine trends on exact commits. A stronger regression conclusion needs clean alternating old/new runs.

> [@futondestroyer](https://github.com/futondestroyer) commented on GitHub · 2026-11-12 14:35 UTC · [original](https://github.com/NotRexed/Hookshot/issues/2595#issuecomment-6826527638) <!-- hookshot-six-hour:run-20261112T140322Z --> **PASS** — 2026-11-12T14:03:22.277990+00:00 Commit: [51da0464b4cf](https://git.magicbean.dev/MagicBeanDev/Hookshot/commit/51da0464b4cf0c7f56b81ebd6bb309325835e175) Protocol: `windows-checkpoints-v2`; Windows 11, i7-13700F, RTX 4070; NVIDIA GeForce RTX 4070, 621.01. Version: `v2.1.0-258-g51da0464 (source 18ed87b5)`. Compiler: `clang version 23.1.5 (https://github.com/llvm/llvm-project.git 392dfff97d29b59d77e5cac6d4b6ef2585fe7117)`. Full build: **1231.55 s** (20.53 min), one repeat; six compiler workers; release cold O1 / hot O2 + shipped PGO. Stages (wall seconds): read 106.67, lift 108.67, finish 133.22, compilation 879.89, linking 1.83, startup 1.26. Fetch/setup, tests and shader preconditioning excluded. Build repeat variance is not measured. Lift process observations: lift: emit and write (16 processes) in 23.8s (63s); lift: patches (16 processes) in 25.7s (90s). Build versus fixed starting baseline: -71.75 (-5.51%) s. Build versus previous comparable success: -71.75 (-5.51%) s. | Phase | Mean FPS | Median / p95 / p99 ms | Repeats; FPS range | vs fixed FPS (%) | vs previous FPS (%) | |---|---:|---:|---|---|---| | field still | 73.76 | 13.50 / 14.31 / 15.07 | 1; 73.76–73.76 | +18.27 (+32.93%) | +18.27 (+32.93%) | | field motion | 67.25 | 14.13 / 18.24 / 20.45 | 1; 67.25–67.25 | +15.28 (+29.41%) | +15.28 (+29.41%) | | market still | 57.49 | 17.30 / 19.30 / 21.79 | 1; 57.49–57.49 | +14.71 (+34.37%) | +14.71 (+34.37%) | | market motion | 54.73 | 17.42 / 25.38 / 30.44 | 1; 54.73–54.73 | +7.66 (+16.27%) | +7.66 (+16.27%) | Game-present return intervals, frame-count/time mean; neither GPU duration nor monitor scanout. Percentiles shown are per-pass percentiles averaged across repeats. 2560×1440 borderless client/output, 16:9, fixed 100%, dynamic resolution off, game’s own upscaler; DLSS/DLAA/FSR/FG/HDR/Reflex/Anti-Lag/V-Sync off, uncapped, real-time speed. Fresh checkpoint loads; stationary and complete camera-circle preconditioning outside timing. Driver cache is persistent; per-build game caches start fresh and are warmed. First-visit compilation is excluded. Single-pass changes are candidates only; repeat suspicious results before calling a regression. | Phase / pass | Frames / seconds | FPS / 1% low | >20 / >25 ms | Max ms | Game CPU ms/frame | Other CPU mean / peak cores | Eligible | |---|---:|---:|---:|---:|---:|---:|---| | field still / 1 | 4427 / 60.017 | 73.76 / 55.30 | 13 / 3 | 27.07 | 69.27 | 0.69 / 1.13 | yes | | field motion / 1 | 4035 / 60.001 | 67.25 / 35.00 | 47 / 14 | 92.12 | 80.41 | 0.68 / 1.15 | yes | | market still / 1 | 3451 / 60.024 | 57.49 / 39.54 | 77 / 18 | 30.41 | 111.59 | 0.75 / 1.24 | yes | | market motion / 1 | 3284 / 60.009 | 54.73 / 26.20 | 918 / 185 | 110.79 | 108.48 | 1.58 / 2.33 | yes | 1% low = 1000 / mean of the slowest ceil(N/100) frame intervals, matching the workbench nightly reader. Slow-frame thresholds are strictly greater than 20/25 ms. No peak-memory, GPU-duration or per-thread CPU claim is made. ### Validation and observations All expected phases completed with valid trace coverage/alignment, checkpoint/settings checks and background CPU eligibility. - Mean FPS versus fixed baseline: higher in every measured workload; workload changes range from +16.27% to +34.37%. - Outside the reference's observed repeat range: field still, field motion, market still, market motion. - Mean FPS versus previous comparable run: higher in every measured workload; workload changes range from +16.27% to +34.37%. - Outside the reference's observed repeat range: field still, field motion, market still, market motion. - These are descriptive comparisons, not a statistical significance test or a confirmed regression. A second new-build pass is screening evidence; regression confirmation requires paired old/new testing. - Build wall time was 5.51% lower than the original single build observation; build-time variance remains unmeasured. - Largest measured frame interval: 110.79 ms in market motion (pass 1). - Configuration disclaimer: this build newly reports the default language settings `game_language=0` and `menu_language=0`, which were absent from the baseline configuration snapshot. These additions are not expected to affect the checkpoint performance tests; baseline comparisons are included under this disclosed assumption. Other configuration/instrumentation checks remain strict. [Intervening commits](https://github.com/NotRexed/Hookshot/compare/55b976c0e15f38fc10dcac757e332800d09b7c24...51da0464b4cf0c7f56b81ebd6bb309325835e175); candidate range, not proof of causation. No visual regression claim: measured intervals have no screenshot readbacks. Build time is separate from in-game speed. Method and metric-reader credit: [mooki000’s hookshot-workbench](https://github.com/mooki000/hookshot-workbench), reviewed at `f45e4a3f`; reused helpers retain the MIT license. This Windows route complements its Linux testing. Different hardware/scenes/presentation modes make absolute FPS and build seconds unsuitable for direct comparison; compare within-machine trends on exact commits. A stronger regression conclusion needs clean alternating old/new runs. <!-- github-sync:c:6826527638 -->
Owner

@mooki000 commented on GitHub · 2026-11-12 16:42 UTC · original

Third Linux cycle, same PC and method as my two above (RTX 4090, NVIDIA 615.78.08, i9-13900K; no window; two alternated runs a scene in one exclusive hold). Main had not moved since the last cycle (51da0464), so this one is a fresh rebuild of the same commit measured against the previous cycle's build of it: a check of how far two builds of identical source differ here.

Scene (fps, two runs each) previous build of 51da0464 fresh rebuild of 51da0464 mean frame time
Kokiri Forest, game upscaler, 240 cap 79.4 / 80.2 80.6 / 79.9 -0.63%
Hyrule Field, game upscaler, 240 cap 93.2 / 94.5 94.5 / 93.5 -0.13%
Kokiri Forest, DLAA, 60 cap 59.7 / 59.5 59.6 / 59.5 +0.06%

All three are inside the spread between repeats, as they should be. So on this PC a difference under about 1% between two cycles says nothing; the 20% and 35% steps reported earlier in this thread are far outside that.

Peak resident memory was again 16.7 to 17.5 GiB in every run (v2.1.0: 13.3 to 13.7 GiB in the first cycle). The fresh build took 1,323 s, more than twice the last one, because the PC was busy with other builds: not a build-speed signal.

The next cycle will have a new main to compare (several pull requests have merged since).

> [@mooki000](https://github.com/mooki000) commented on GitHub · 2026-11-12 16:42 UTC · [original](https://github.com/NotRexed/Hookshot/issues/2595#issuecomment-6829790583) Third Linux cycle, same PC and method as my two above (RTX 4090, NVIDIA 615.78.08, i9-13900K; no window; two alternated runs a scene in one exclusive hold). **Main had not moved since the last cycle (`51da0464`), so this one is a fresh rebuild of the same commit measured against the previous cycle's build of it: a check of how far two builds of identical source differ here.** | Scene (fps, two runs each) | previous build of `51da0464` | fresh rebuild of `51da0464` | mean frame time | |---|---|---|---| | Kokiri Forest, game upscaler, 240 cap | 79.4 / 80.2 | 80.6 / 79.9 | -0.63% | | Hyrule Field, game upscaler, 240 cap | 93.2 / 94.5 | 94.5 / 93.5 | -0.13% | | Kokiri Forest, DLAA, 60 cap | 59.7 / 59.5 | 59.6 / 59.5 | +0.06% | All three are inside the spread between repeats, as they should be. So on this PC a difference under about 1% between two cycles says nothing; the 20% and 35% steps reported earlier in this thread are far outside that. Peak resident memory was again 16.7 to 17.5 GiB in every run (v2.1.0: 13.3 to 13.7 GiB in the first cycle). The fresh build took 1,323 s, more than twice the last one, because the PC was busy with other builds: not a build-speed signal. The next cycle will have a new main to compare (several pull requests have merged since). <!-- github-sync:c:6829790583 -->
Commenting is not possible because the repository is archived.
No milestone
No assignees
2 participants
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
MagicBeanDev/Hookshot#2595
No description provided.