WIP: feat(perf): sample performance every second and name the client #1372

Closed
gab9281 wants to merge 3 commits from gab9281/Longshot:codex/perf-position-telemetry into main
Member

Type: feat · Risk: medium · Tested on: Linux (game-free tests only) · Needs: Linux and Windows, any GPU, an in-game run

Why

The performance statistics don't say where in the world a frame was slow. Their per-map totals cover a whole map, so a stutter at one spot in Hyrule Field looks the same as one spread across the map. Crash and performance reports also mix the client and the game version into a single field, so the team can't filter the dashboards by each one separately. This PR has players who opted in to "Send performance statistics" record one sample per second of gameplay and send those samples in the existing five-minute batches. Both reports now name the client (Longshot) separately from the version.

What changed

The game now records a sample each second: frame and GPU time averages, hitches, the map, Link's position, and the camera's yaw and pitch. Each five-minute summary carries up to 300 of these samples. The crash report and the performance summary send client: Longshot beside version. The relay side, Longshot-api #12, #13 and #14, accepts both fields and keeps accepting older payloads that lack them.

  • perf: gpu/vk/perf_report.{h,cpp}: PositionSample and the samples in summary_json(); "client":"Longshot" comes first in the summary.
  • host: host/hos_main.c feeds the per-second position and view; host/rx_backend.c.
  • vk ABI: MkwVkHost changed, so MKWVK_API_VERSION is bumped to 30 (gpu/vk/vk_api.h).
  • crash: host/hos_crash_dialog.c build_report() sends "client": "Longshot" and shows "Client: Longshot" on the Send page.
  • docs: docs/performance-statistics.md (what is sent, including position and view) and docs/crash-report-window.md; README.

Measurements

Verdict: not measured yet: the per-second sampling's frame cost must be measured in game before this is ready (see Testing).

Size, measured with the game's own summary_json() (game-free): a five-minute summary is 51.8 KB on one map, 54.5 KB on four, and 62.2 KB on 16 (the most a summary holds). The relay's limit is 64 KiB, so the default interval fits, but an RX_VK_PERF_INTERVAL over about 6 minutes is refused (413): 600 s comes to 106 KB.

Flags and settings

None. This uses the existing "Send performance statistics" opt-in.

Testing

Tested: game-free checks on Linux only.

  • python3 -I -B tests/test_crash_dialog.py: pass. It checks that the report sends client and version as separate fields and that no private username appears in it.
  • python3 -I -B tests/test_perf_report.py: pass. It checks the summary JSON, the client field and the samples.
  • python3 -I -B tests/run_all.py: 607 passed, 37 failed, 77 skipped. The same 37 tests fail on main @ 544229a2 when run the same way. They fail on environment problems, such as MagicBeanDev/Tooling not sitting beside a /tmp checkout, so none of them is a regression.

Not tested: the game itself. No in-game or GPU run has been done, at the requester's request: this is opened as an untested WIP.

Must be tested:

  • Linux, any GPU: play about six minutes with Send performance statistics on and LONGSHOT_PERF_URL pointed at a local relay. Expect a summary with about 300 samples that have plausible positions, plus "client":"Longshot", and no frame-time change from main (Test Farm: @Beanstalk linux, five minutes of Hyrule Field, perf summary posted).
  • Windows, any GPU: the same six minutes with Send performance statistics on and LONGSHOT_PERF_URL pointed at a local relay, expecting a summary with about 300 samples that have plausible positions, plus "client":"Longshot", and no frame-time change from main (Test Farm: @Beanstalk windows, five minutes of Hyrule Field, perf summary posted).
  • Any OS: force a crash and open Send to the team. Expect "Client: Longshot" above the version, and sent.json with client and version as separate fields (by hand).

Risks

What could break: the per-second sampling in hos_main.c and the MkwVkHost ABI bump, so the host and the renderer must come from the same build. The summary is larger but stays under the relay's 64 KiB limit.

Turning it off: untick Send performance statistics; otherwise, revert.

Compatibility: the vk ABI changes (MKWVK_API_VERSION 30). Saves, options and caches are unchanged. Relays without Longshot-api #14 ignore client, and #13 already accepts samples.

Checklist

Everyone

  • One topic.
  • The title is a Conventional Commit (type(scope): description) of the status line's type.
  • Why gives the reason plainly and links the issues that fit (the tracker searched).
  • Tested on my own PC where it runs (in the game, for a change the game runs), as Testing's Tested says; what
    can't be run by hand is in Testing's Not testable by hand, with why. Not ticked yet: Testing names only game-free checks on Linux, and the change touches gpu/ and host/, which the game runs.
  • Measurements has a verdict, or one line saying why there is nothing to measure.
  • Nothing gets worse that I know of: any cost (speed, latency, memory, a setup that worked before) is measured and
    in Risks. Not ticked yet: Measurements says the per-second sampling's frame cost is not measured yet, and that cost is the open question.
  • Testing names what was tested, what wasn't, and what must still be tested, where and how.
  • Rebased on current main.
  • python -I -B tests/run_all.py passes (any failure that also fails on main is named).
  • No game files, dumps, build output or generated files.
  • third_party/ untouched (a new folder with a PROVENANCE.md instead).
  • The license is OK: GPL-3.0-or-later, nothing incompatible and nothing decompiled.

When it applies

  • A check under tests/ that reads changed source updated in this PR, not weakened.
  • AI assistance recorded with Assisted-by: / Co-Authored-By: trailers.
  • MKWVK_API_VERSION bumped (gpu/vk/vk_api.h's MkwVkHost or MkwVkRenderer changed).
  • Docs updated.

🤖 Generated with Claude Code

**Type:** feat · **Risk:** medium · **Tested on:** Linux (game-free tests only) · **Needs:** Linux and Windows, any GPU, an in-game run ## Why The performance statistics don't say where in the world a frame was slow. Their per-map totals cover a whole map, so a stutter at one spot in Hyrule Field looks the same as one spread across the map. Crash and performance reports also mix the client and the game version into a single field, so the team can't filter the dashboards by each one separately. This PR has players who opted in to "Send performance statistics" record one sample per second of gameplay and send those samples in the existing five-minute batches. Both reports now name the client (`Longshot`) separately from the version. ## What changed The game now records a sample each second: frame and GPU time averages, hitches, the map, Link's position, and the camera's yaw and pitch. Each five-minute summary carries up to 300 of these samples. The crash report and the performance summary send `client: Longshot` beside `version`. The relay side, Longshot-api #12, #13 and #14, accepts both fields and keeps accepting older payloads that lack them. - perf: `gpu/vk/perf_report.{h,cpp}`: `PositionSample` and the samples in `summary_json()`; `"client":"Longshot"` comes first in the summary. - host: `host/hos_main.c` feeds the per-second position and view; `host/rx_backend.c`. - vk ABI: `MkwVkHost` changed, so `MKWVK_API_VERSION` is bumped to 30 (`gpu/vk/vk_api.h`). - crash: `host/hos_crash_dialog.c` `build_report()` sends `"client": "Longshot"` and shows "Client: Longshot" on the Send page. - docs: `docs/performance-statistics.md` (what is sent, including position and view) and `docs/crash-report-window.md`; README. ## Measurements **Verdict:** not measured yet: the per-second sampling's frame cost must be measured in game before this is ready (see Testing). Size, measured with the game's own `summary_json()` (game-free): a five-minute summary is 51.8 KB on one map, 54.5 KB on four, and 62.2 KB on 16 (the most a summary holds). The relay's limit is 64 KiB, so the default interval fits, but an `RX_VK_PERF_INTERVAL` over about 6 minutes is refused (413): 600 s comes to 106 KB. ### Flags and settings None. This uses the existing "Send performance statistics" opt-in. ## Testing **Tested:** game-free checks on Linux only. - `python3 -I -B tests/test_crash_dialog.py`: pass. It checks that the report sends `client` and `version` as separate fields and that no private username appears in it. - `python3 -I -B tests/test_perf_report.py`: pass. It checks the summary JSON, the client field and the samples. - `python3 -I -B tests/run_all.py`: 607 passed, 37 failed, 77 skipped. The same 37 tests fail on `main` @ 544229a2 when run the same way. They fail on environment problems, such as MagicBeanDev/Tooling not sitting beside a `/tmp` checkout, so none of them is a regression. **Not tested:** the game itself. No in-game or GPU run has been done, at the requester's request: this is opened as an untested WIP. **Must be tested:** - Linux, any GPU: play about six minutes with Send performance statistics on and `LONGSHOT_PERF_URL` pointed at a local relay. Expect a summary with about 300 `samples` that have plausible positions, plus `"client":"Longshot"`, and no frame-time change from `main` (Test Farm: @Beanstalk linux, five minutes of Hyrule Field, perf summary posted). - Windows, any GPU: the same six minutes with Send performance statistics on and `LONGSHOT_PERF_URL` pointed at a local relay, expecting a summary with about 300 `samples` that have plausible positions, plus `"client":"Longshot"`, and no frame-time change from `main` (Test Farm: @Beanstalk windows, five minutes of Hyrule Field, perf summary posted). - Any OS: force a crash and open Send to the team. Expect "Client: Longshot" above the version, and `sent.json` with `client` and `version` as separate fields (by hand). ## Risks **What could break:** the per-second sampling in `hos_main.c` and the `MkwVkHost` ABI bump, so the host and the renderer must come from the same build. The summary is larger but stays under the relay's 64 KiB limit. **Turning it off:** untick Send performance statistics; otherwise, revert. **Compatibility:** the vk ABI changes (`MKWVK_API_VERSION` 30). Saves, options and caches are unchanged. Relays without Longshot-api #14 ignore `client`, and #13 already accepts `samples`. ## Checklist ### Everyone - [x] One topic. - [x] The title is a Conventional Commit (`type(scope): description`) of the status line's type. - [x] Why gives the reason plainly and links the issues that fit (the tracker searched). - [ ] Tested on my own PC where it runs (in the game, for a change the game runs), as Testing's Tested says; what can't be run by hand is in Testing's Not testable by hand, with why. Not ticked yet: Testing names only game-free checks on Linux, and the change touches gpu/ and host/, which the game runs. - [x] Measurements has a verdict, or one line saying why there is nothing to measure. - [ ] Nothing gets worse that I know of: any cost (speed, latency, memory, a setup that worked before) is measured and in Risks. Not ticked yet: Measurements says the per-second sampling's frame cost is not measured yet, and that cost is the open question. - [x] Testing names what was tested, what wasn't, and what must still be tested, where and how. - [x] Rebased on current `main`. - [x] `python -I -B tests/run_all.py` passes (any failure that also fails on `main` is named). - [x] No game files, dumps, build output or generated files. - [x] `third_party/` untouched (a new folder with a `PROVENANCE.md` instead). - [x] The license is OK: GPL-3.0-or-later, nothing incompatible and nothing decompiled. ### When it applies - [x] A check under `tests/` that reads changed source updated in this PR, not weakened. - [x] AI assistance recorded with `Assisted-by:` / `Co-Authored-By:` trailers. - [x] `MKWVK_API_VERSION` bumped (`gpu/vk/vk_api.h`'s `MkwVkHost` or `MkwVkRenderer` changed). - [x] Docs updated. 🤖 Generated with [Claude Code](https://claude.com/claude-code)
feat(telemetry): name the client in crash and perf reports
Some checks failed
CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Has been skipped
CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Has been skipped
CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Has been skipped
Third-party parts / The same as upstream (pull_request) Has been skipped
Website / Website on Cloudflare (pull_request) Has been skipped
CI / Conventional Commit title (pull_request) Successful in 7s
Guide screenshots / Guide images and local UI source review (pull_request) Failing after 10s
CI / Workflow lint (actionlint) (pull_request) Successful in 10s
Third-party parts / As their notes say (pull_request) Successful in 14s
Website / Website on Cloudflare (pull_request_target) Has been skipped
PR description / Description follows the template (pull_request_target) Successful in 3s
209dee90ea
Crash reports and performance summaries now send `client: Longshot` beside
the game version, so the relay can label every metric by client and by
version separately. The crash window's Send page shows it too.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Collaborator

Possibly related, for context:

  • issue #901 Game: Severe FPS drops while moving the camera on Death Mountain Trail (RTX 4080, nightly 189) — Location-tagged samples could aid investigation
  • PR #65 feat(runtime): HOS_COUNTERS, the memory and cache counters as one JSON line every N seconds — Overlapping periodic JSON telemetry feature
Possibly related, for context: - issue #901 Game: Severe FPS drops while moving the camera on Death Mountain Trail (RTX 4080, nightly 189) — Location-tagged samples could aid investigation - PR #65 feat(runtime): HOS_COUNTERS, the memory and cache counters as one JSON line every N seconds — Overlapping periodic JSON telemetry feature
dyst left a comment

I found 10 issues by reading the code. None were re-checked afterwards, and nothing was run in the game.

Items 1-4 should be fixed before this leaves WIP:

  1. Summaries can go over the launcher's 60 KiB limit, so whole 5-minute batches get refused.
  2. A stale pose is reported after Link stops updating.
  3. Yaw and pitch are read without checking the camera type.
  4. Up to a second of frames after each summary is never sampled.

Item 5 (exact positions leave the PC) is a privacy and design point. Items 6-10 are cleanup.

I found 10 issues by reading the code. None were re-checked afterwards, and nothing was run in the game. Items 1-4 should be fixed before this leaves WIP: 1. Summaries can go over the launcher's 60 KiB limit, so whole 5-minute batches get refused. 2. A stale pose is reported after Link stops updating. 3. Yaw and pitch are read without checking the camera type. 4. Up to a second of frames after each summary is never sampled. Item 5 (exact positions leave the PC) is a privacy and design point. Items 6-10 are cleanup.
@ -163,2 +172,4 @@
}
void finish_sample() {
if (collecting_pose_valid && collecting.frames > 0) samples.push_back(collecting);
Member

Summaries can go over the launcher's 60 KiB MAX_BODY. Nothing caps the number of samples, and each realistic one is about 177 bytes.

  • 300 samples come to about 53 KB. A few maps' totals (about 800 B each) push that past 61440 bytes.
  • With RX_VK_PERF_INTERVAL=840 there can be about 149 KB of samples, so every summary is refused.
  • When nx2/perf_report.py refuses a summary, it drops the per-map totals too, not just the samples.

The test at tests/test_perf_report.cpp:566 only uses 300 short samples on one map, so it doesn't catch this. The docs' "up to 300 samples" is also wrong for any interval other than 300.

**Summaries can go over the launcher's 60 KiB MAX_BODY.** Nothing caps the number of samples, and each realistic one is about 177 bytes. - 300 samples come to about 53 KB. A few maps' totals (about 800 B each) push that past 61440 bytes. - With `RX_VK_PERF_INTERVAL=840` there can be about 149 KB of samples, so every summary is refused. - When `nx2/perf_report.py` refuses a summary, it drops the per-map totals too, not just the samples. The test at `tests/test_perf_report.cpp:566` only uses 300 short samples on one map, so it doesn't catch this. The docs' "up to 300 samples" is also wrong for any interval other than 300.
@ -187,9 +213,11 @@ void write_summary(const std::filesystem::path& dir, const std::string& name, co
/// lost device: the process ends next)
void summarize(bool now) {
if (!current) return;
finish_sample();
Member

Frames just after each summary are never sampled. summarize() calls finish_sample(), which sets collecting_pose_valid=false and since_sample=0. No new sample starts until the next since_map>=1 check.

The 5-minute summary runs mid-frame, so for up to the next second the frames' hitches and frame times are missing from the samples, although they still count in the map totals. A stutter right after each summary is never placed on the map.

**Frames just after each summary are never sampled.** `summarize()` calls `finish_sample()`, which sets `collecting_pose_valid=false` and `since_sample=0`. No new sample starts until the next `since_map>=1` check. The 5-minute summary runs mid-frame, so for up to the next second the frames' hitches and frame times are missing from the samples, although they still count in the map totals. A stutter right after each summary is never placed on the map.
@ -282,3 +311,3 @@
std::lock_guard lk(mu);
if (!current || since_map >= kMapEvery) {
if (!current || since_map >= kMapEvery || since_sample >= 1.0) {
Member

since_sample duplicates since_map. The two advance together and are reset in the same block. The only difference is the reset in summarize's finish_sample, which is what causes the gap at line 216. Keep only since_map (kMapEvery is already 1 s) and start the new sample in the same block.

**`since_sample` duplicates `since_map`.** The two advance together and are reset in the same block. The only difference is the reset in `summarize`'s `finish_sample`, which is what causes the gap at line 216. Keep only `since_map` (`kMapEvery` is already 1 s) and start the new sample in the same block.
@ -292,9 +323,19 @@ void perf_frame(const fs::Frame& f, double wait_ms) {
avg_ms = usual * 0.98 + f.ms * 0.02;
const double cap_ms = menu_target_ms();
const double target = cap_ms > 0 ? cap_ms : usual;
int64_t hitches_before = 0;
Member

Two extra loops every frame to detect a hitch. This sums current->hitch[kCauses] before and after add_frame on every frame, which adds two 9-element loops to the hot path. Having add_frame return whether the frame was a hitch, or calling is_hitch(f.ms, std::max(target, usual)), does it in one comparison.

**Two extra loops every frame to detect a hitch.** This sums `current->hitch[kCauses]` before and after `add_frame` on every frame, which adds two 9-element loops to the hot path. Having `add_frame` return whether the frame was a hitch, or calling `is_hitch(f.ms, std::max(target, usual))`, does it in one comparison.
@ -167,6 +176,15 @@ inline std::string json_num(double v) {
return s;
}
inline std::string json_signed_num(double v) {
Member

json_signed_num copies json_num minus the std::max(v, 0.0) clamp, so the two have to be kept in step, including the shared 32-byte buffer truncation. json_num could become return json_signed_num(std::max(v, 0.0));.

**`json_signed_num` copies `json_num`** minus the `std::max(v, 0.0)` clamp, so the two have to be kept in step, including the shared 32-byte buffer truncation. `json_num` could become `return json_signed_num(std::max(v, 0.0));`.
@ -211,0 +228,4 @@
o += "},\"samples\":[";
for (size_t i = 0; i < samples.size(); i++) {
const auto& s = samples[i];
o += std::string(i ? "," : "") + "{\"map\":" + json_text(s.map) + ",\"x\":" + json_signed_num(s.x) +
Member

Exact positions leave the PC. Positions and angles are written to perf_pending with 3 decimals, and only the relay rounds them into 100-unit cells and 45/30-degree bins. The docs and README describe rounded cells as what is collected.

Round in summary_json instead. Sending the map as an index rather than repeating its name in every sample would also shrink the summary well under the 60 KiB limit (see the perf_report.cpp:175 comment).

**Exact positions leave the PC.** Positions and angles are written to `perf_pending` with 3 decimals, and only the relay rounds them into 100-unit cells and 45/30-degree bins. The docs and README describe rounded cells as what is collected. Round in `summary_json` instead. Sending the map as an index rather than repeating its name in every sample would also shrink the summary well under the 60 KiB limit (see the perf_report.cpp:175 comment).
@ -268,3 +268,3 @@
keep up with the sound, [2] shown late (over 50 ms after the one before), [3] decoding time in ns, [4] decoders that did
not start (another one took over). Returns the build's version (LONGSHOT_VERSION) */
const char *(*perf_info)(char *map, int cap, int64_t movie[5]);
const char *(*perf_info)(char *map, int cap, int64_t movie[5], float pose[5], int *has_pose);
Member

Outdated comments and docs for the API 30 change. The perf_info signature changed for API 30, but the Version-26 comment still only describes map and movie. It should give the pose order and units ({x, y, z, yaw, pitch}, angles in radians) and say that has_pose is 0 unless perf_tracking(1) was called.

docs/performance-statistics.md:89 also still says "five-minute batches" now that the interval can be changed, and no longer gives the reason for the 840 s cap.

**Outdated comments and docs for the API 30 change.** The `perf_info` signature changed for API 30, but the Version-26 comment still only describes map and movie. It should give the pose order and units ({x, y, z, yaw, pitch}, angles in radians) and say that `has_pose` is 0 unless `perf_tracking(1)` was called. `docs/performance-statistics.md:89` also still says "five-minute batches" now that the interval can be changed, and no longer gives the reason for the 840 s cap.
@ -64,2 +68,4 @@
ReleaseSRWLockShared(&map_lock);
hos_movie_stats(movie);
AcquireSRWLockShared(&perf_pose_lock);
*has_pose = perf_pose_valid;
Member

A stale pose is reported after Link stops updating. perf_pose_valid is only cleared when the camera view is missing or a value is not finite. After a return to the title or file select, a reset or a load, hos_perf_info keeps returning has_pose=1 with Link's last position.

Example: play in Hyrule Field, then go back to the title. Samples are tagged "Title and intro" but still carry Hyrule Field X/Y/Z and yaw/pitch every second, so title-screen frame times land on the field in the dashboards.

**A stale pose is reported after Link stops updating.** `perf_pose_valid` is only cleared when the camera view is missing or a value is not finite. After a return to the title or file select, a reset or a load, `hos_perf_info` keeps returning `has_pose=1` with Link's last position. Example: play in Hyrule Field, then go back to the title. Samples are tagged "Title and intro" but still carry Hyrule Field X/Y/Z and yaw/pitch every second, so title-screen frame times land on the field in the dashboards.
@ -1343,2 +1360,4 @@
}
/* The opt-in performance sample's latest Link position and gameplay camera angles. No game memory is read while unticked. */
static void oot_perf_pose(Ctx *c, uint64_t actor) {
Member

This does too much work on every tick. oot_perf_pose takes the exclusive SRW lock and follows the 5-load camera pointer chain on every Link update, competing with the present thread's shared read, but the renderer reads the pose only once a second. Limit it to about once a second with hos_now_ns (already called in this function), or publish the pose through a seqlock or atomics.

**This does too much work on every tick.** `oot_perf_pose` takes the exclusive SRW lock and follows the 5-load camera pointer chain on every Link update, competing with the present thread's shared read, but the renderer reads the pose only once a second. Limit it to about once a second with `hos_now_ns` (already called in this function), or publish the pose through a seqlock or atomics.
@ -1345,0 +1370,4 @@
return;
}
float pose[5] = {oot_load_float(c, actor + 796), oot_load_float(c, actor + 800), oot_load_float(c, actor + 804),
oot_load_float(c, view + 0x1a4), oot_load_float(c, view + 0x18c)};
Member

Yaw and pitch are read without checking the camera type. The flight and free-camera code check recomp_load64(c, view) == 0x84c18a88 before reading this view; oot_perf_pose reads view+0x1a4 and view+0x18c without that check.

During a cutscene, or with a different camera type active, the floats at those offsets are unrelated but finite, so they pass isfinite and get sent. Very large values (above about 1e28) are also silently truncated by json_signed_num's 32-byte buffer.

**Yaw and pitch are read without checking the camera type.** The flight and free-camera code check `recomp_load64(c, view) == 0x84c18a88` before reading this view; `oot_perf_pose` reads `view+0x1a4` and `view+0x18c` without that check. During a cutscene, or with a different camera type active, the floats at those offsets are unrelated but finite, so they pass `isfinite` and get sent. Very large values (above about 1e28) are also silently truncated by `json_signed_num`'s 32-byte buffer.
docs(guides): keep the Graphics image after the perf sample interface (ABI 30)
Some checks failed
CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Has been skipped
CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Has been skipped
CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Has been skipped
Third-party parts / The same as upstream (pull_request) Has been skipped
Website / Website on Cloudflare (pull_request) Has been skipped
CI / Conventional Commit title (pull_request) Successful in 7s
CI / Workflow lint (actionlint) (pull_request) Successful in 12s
Guide screenshots / Guide images and local UI source review (pull_request) Successful in 12s
Third-party parts / As their notes say (pull_request) Successful in 14s
Website / Website on Cloudflare (pull_request_target) Has been skipped
PR description / Description follows the template (pull_request_target) Successful in 11s
beanstalk/approval approved by gab9281 (reviewed)
beanstalk/linux 1/1 fail
beanstalk/windows 1/1 fail
0cece02f72
vk_api.h's perf_info pose outputs and perf_tracking callback change no
pictured row; the reviewed-source hash advances to the inspected file.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Author
Member

@Beanstalk Linux and Windows, any GPU: six minutes in Hyrule Field with Send performance statistics on, the same route on main and on this PR. Compare frame time and 1% lows, to see whether sampling every second costs anything, and check that a performance summary with about 300 samples and "client":"Longshot" is written.

@Beanstalk Linux and Windows, any GPU: six minutes in Hyrule Field with Send performance statistics on, the same route on main and on this PR. Compare frame time and 1% lows, to see whether sampling every second costs anything, and check that a performance summary with about 300 samples and "client":"Longshot" is written.
Member

Beanstalk test farm

Commit 0cece02 · tier reviewed · approved by gab9281 · dashboard

What will be tested · AI plan (gpt-6.1-terra)

On Linux and Windows, this plan compares frame time and 1% low frame time in Hyrule Field for six minutes per build, after the same brief movement on each. It fails on a crash or a frame-time regression above 10%. The farm cannot enable the launcher’s performance-statistics opt-in or inspect the written summary, so sampling overhead and the requested sample count and client field remain unverified; the six-minute measurement is stationary.

  • Where: linux (1-2 reports), windows (1-2 reports).
  • field-frame-time: from a new save, then teleport to Hyrule Field, move the stick, measure field for 6 min (after 5 s warm-up); 1 run on each side.
  • Measured: crashes, frame times, stutters and hitches.
  • The PR fails if: it crashes; or frame time (p99) is more than 10% above main; or frame time (worst 1%) is more than 10% above main.
  • Time: about 25 min of game runs per tester (2 game starts), plus the two builds.

Done (started 2026-11-17 07:51 UTC, last report 2026-11-17 09:20 UTC). Comment @Beanstalk to run it again.

Target Reports Crash Frame p99 (ms) vs main VRAM peak (MB) vs main Result
linux · 1: linux+amd+radv+dgpu AMD Radeon RX 7900 XT (RADV NAVI31), 26.2.6; 2: linux+nvidia+nvidia-proprietary+dgpu NVIDIA GeForce RTX 4070, 615.71.9 2/1 1 2 none 21.0 vs 19.3 (+9%) — fail ⚠ outlier
windows · 1: windows+amd+amd-proprietary+dgpu AMD Radeon RX 9070 XT, 2.0.416; 2: windows+nvidia+nvidia-proprietary+dgpu NVIDIA GeForce RTX 4070, 621.1.0 2/1 1 2 none 21.3 vs 18.5 (+15%) — fail ⚠ outlier

Measurements · linux

Verdict: fail on linux (2/1 report): field-frame-time field.frametime_p99_ms.max_delta_pct (+18.7%); field-frame-time field.frametime_1pct_low_ms.max_delta_pct (+16.9%) failed.

Hardware: 1: linux+amd+radv+dgpu AMD Radeon RX 7900 XT (RADV NAVI31), 26.2.6 AMD Ryzen 7 5700X3D 8-Core Processor, 32 GB RAM; 2: linux+nvidia+nvidia-proprietary+dgpu NVIDIA GeForce RTX 4070, 615.71.9 AMD Ryzen 7 5800X3D 8-Core Processor, 64 GB RAM
Scene: field-frame-time, from a new save; each cell's settings and switches as the table names them
Method: the Beanstalk test farm: main and this PR built and run on the same PC in one session, game starts alternating (ABBA); field-frame-time: 1 game start a side, field 360 s after 5 s warm-up; each number the median of the runs, then of 2 reports
How to reproduce: the farm's plan above (@Beanstalk on this PR runs it again)

Before: main @ 544229a After: this PR @ 0cece02
field-frame-time · field · frame time (p99) (ms) 19.3 21.0 (+9%)
field-frame-time · field · frame time (worst 1%) (ms) 20.1 21.6 (+8%)

Flags and settings

None: the plan varied no setting, switch or build option.

Markdown for the PR description
## Measurements

**Verdict:** fail on linux (2/1 report): `field-frame-time field.frametime_p99_ms.max_delta_pct (+18.7%)`; `field-frame-time field.frametime_1pct_low_ms.max_delta_pct (+16.9%)` failed.

Hardware: 1: linux+amd+radv+dgpu `AMD Radeon RX 7900 XT (RADV NAVI31), 26.2.6` `AMD Ryzen 7 5700X3D 8-Core Processor, 32 GB RAM`; 2: linux+nvidia+nvidia-proprietary+dgpu `NVIDIA GeForce RTX 4070, 615.71.9` `AMD Ryzen 7 5800X3D 8-Core Processor, 64 GB RAM`  
Scene: field-frame-time, from a new save; each cell's settings and switches as the table names them  
Method: the Beanstalk test farm: main and this PR built and run on the same PC in one session, game starts alternating (ABBA); field-frame-time: 1 game start a side, field 360 s after 5 s warm-up; each number the median of the runs, then of 2 reports  
How to reproduce: the farm's plan above (`@Beanstalk` on this PR runs it again)

| | Before: main @ 544229a | After: this PR @ 0cece02 |
|---|---|---|
| field-frame-time · field · frame time (p99) (ms) | 19.3 | 21.0 (+9%) |
| field-frame-time · field · frame time (worst 1%) (ms) | 20.1 | 21.6 (+8%) |

### Flags and settings

None: the plan varied no setting, switch or build option.

Measurements · windows

Verdict: fail on windows (2/1 report): field-frame-time field.frametime_p99_ms.max_delta_pct (+22.8%); field-frame-time field.frametime_1pct_low_ms.max_delta_pct (+29.5%) failed.

Hardware: 1: windows+amd+amd-proprietary+dgpu AMD Radeon RX 9070 XT, 2.0.416 AMD Ryzen 9 5900X 12-Core Processor, 32 GB RAM; 2: windows+nvidia+nvidia-proprietary+dgpu NVIDIA GeForce RTX 4070, 621.1.0 13th Gen Intel(R) Core(TM) i7-13700F, 32 GB RAM
Scene: field-frame-time, from a new save; each cell's settings and switches as the table names them
Method: the Beanstalk test farm: main and this PR built and run on the same PC in one session, game starts alternating (ABBA); field-frame-time: 1 game start a side, field 360 s after 5 s warm-up; each number the median of the runs, then of 2 reports
How to reproduce: the farm's plan above (@Beanstalk on this PR runs it again)

Before: main @ 544229a After: this PR @ 0cece02
field-frame-time · field · frame time (p99) (ms) 18.5 21.3 (+15%)
field-frame-time · field · frame time (worst 1%) (ms) 19.0 22.6 (+19%)

Flags and settings

None: the plan varied no setting, switch or build option.

Markdown for the PR description
## Measurements

**Verdict:** fail on windows (2/1 report): `field-frame-time field.frametime_p99_ms.max_delta_pct (+22.8%)`; `field-frame-time field.frametime_1pct_low_ms.max_delta_pct (+29.5%)` failed.

Hardware: 1: windows+amd+amd-proprietary+dgpu `AMD Radeon RX 9070 XT, 2.0.416` `AMD Ryzen 9 5900X 12-Core Processor, 32 GB RAM`; 2: windows+nvidia+nvidia-proprietary+dgpu `NVIDIA GeForce RTX 4070, 621.1.0` `13th Gen Intel(R) Core(TM) i7-13700F, 32 GB RAM`  
Scene: field-frame-time, from a new save; each cell's settings and switches as the table names them  
Method: the Beanstalk test farm: main and this PR built and run on the same PC in one session, game starts alternating (ABBA); field-frame-time: 1 game start a side, field 360 s after 5 s warm-up; each number the median of the runs, then of 2 reports  
How to reproduce: the farm's plan above (`@Beanstalk` on this PR runs it again)

| | Before: main @ 544229a | After: this PR @ 0cece02 |
|---|---|---|
| field-frame-time · field · frame time (p99) (ms) | 18.5 | 21.3 (+15%) |
| field-frame-time · field · frame time (worst 1%) (ms) | 19.0 | 22.6 (+19%) |

### Flags and settings

None: the plan varied no setting, switch or build option.
<!-- beanstalk:sticky --> ### Beanstalk test farm Commit `0cece02` · tier **reviewed** · approved by gab9281 · [dashboard](https://beanstalk.magicbean.dev/prs/1372) **What will be tested** · AI plan (`gpt-6.1-terra`) On Linux and Windows, this plan compares frame time and 1% low frame time in Hyrule Field for six minutes per build, after the same brief movement on each. It fails on a crash or a frame-time regression above 10%. The farm cannot enable the launcher’s performance-statistics opt-in or inspect the written summary, so sampling overhead and the requested sample count and client field remain unverified; the six-minute measurement is stationary. - **Where:** linux (1-2 reports), windows (1-2 reports). - **field-frame-time:** from a new save, then teleport to Hyrule Field, move the stick, measure field for 6 min (after 5 s warm-up); 1 run on each side. - **Measured:** crashes, frame times, stutters and hitches. - **The PR fails if:** it crashes; or frame time (p99) is more than 10% above main; or frame time (worst 1%) is more than 10% above main. - **Time:** about 25 min of game runs per tester (2 game starts), plus the two builds. Done (started 2026-11-17 07:51 UTC, last report 2026-11-17 09:20 UTC). Comment `@Beanstalk` to run it again. | Target | Reports | Crash | Frame p99 (ms) vs main | VRAM peak (MB) vs main | Result | | --- | --- | --- | --- | --- | --- | | linux · 1: linux+amd+radv+dgpu `AMD Radeon RX 7900 XT (RADV NAVI31), 26.2.6`; 2: linux+nvidia+nvidia-proprietary+dgpu `NVIDIA GeForce RTX 4070, 615.71.9` | 2/1 [1](https://beanstalk.magicbean.dev/reports/rp_4e04fd72f569e6ee) [2](https://beanstalk.magicbean.dev/reports/rp_639e77a7697c776a) | none | 21.0 vs 19.3 (+9%) | — | fail ⚠ outlier | | windows · 1: windows+amd+amd-proprietary+dgpu `AMD Radeon RX 9070 XT, 2.0.416`; 2: windows+nvidia+nvidia-proprietary+dgpu `NVIDIA GeForce RTX 4070, 621.1.0` | 2/1 [1](https://beanstalk.magicbean.dev/reports/rp_905572fccfb327cd) [2](https://beanstalk.magicbean.dev/reports/rp_1438243e52d546a9) | none | 21.3 vs 18.5 (+15%) | — | fail ⚠ outlier | #### Measurements · linux **Verdict:** fail on linux (2/1 report): `field-frame-time field.frametime_p99_ms.max_delta_pct (+18.7%)`; `field-frame-time field.frametime_1pct_low_ms.max_delta_pct (+16.9%)` failed. Hardware: 1: linux+amd+radv+dgpu `AMD Radeon RX 7900 XT (RADV NAVI31), 26.2.6` `AMD Ryzen 7 5700X3D 8-Core Processor, 32 GB RAM`; 2: linux+nvidia+nvidia-proprietary+dgpu `NVIDIA GeForce RTX 4070, 615.71.9` `AMD Ryzen 7 5800X3D 8-Core Processor, 64 GB RAM` Scene: field-frame-time, from a new save; each cell's settings and switches as the table names them Method: the Beanstalk test farm: main and this PR built and run on the same PC in one session, game starts alternating (ABBA); field-frame-time: 1 game start a side, field 360 s after 5 s warm-up; each number the median of the runs, then of 2 reports How to reproduce: the farm's plan above (`@Beanstalk` on this PR runs it again) | | Before: main @ 544229a | After: this PR @ 0cece02 | |---|---|---| | field-frame-time · field · frame time (p99) (ms) | 19.3 | 21.0 (+9%) | | field-frame-time · field · frame time (worst 1%) (ms) | 20.1 | 21.6 (+8%) | ### Flags and settings None: the plan varied no setting, switch or build option. <details><summary>Markdown for the PR description</summary> ```markdown ## Measurements **Verdict:** fail on linux (2/1 report): `field-frame-time field.frametime_p99_ms.max_delta_pct (+18.7%)`; `field-frame-time field.frametime_1pct_low_ms.max_delta_pct (+16.9%)` failed. Hardware: 1: linux+amd+radv+dgpu `AMD Radeon RX 7900 XT (RADV NAVI31), 26.2.6` `AMD Ryzen 7 5700X3D 8-Core Processor, 32 GB RAM`; 2: linux+nvidia+nvidia-proprietary+dgpu `NVIDIA GeForce RTX 4070, 615.71.9` `AMD Ryzen 7 5800X3D 8-Core Processor, 64 GB RAM` Scene: field-frame-time, from a new save; each cell's settings and switches as the table names them Method: the Beanstalk test farm: main and this PR built and run on the same PC in one session, game starts alternating (ABBA); field-frame-time: 1 game start a side, field 360 s after 5 s warm-up; each number the median of the runs, then of 2 reports How to reproduce: the farm's plan above (`@Beanstalk` on this PR runs it again) | | Before: main @ 544229a | After: this PR @ 0cece02 | |---|---|---| | field-frame-time · field · frame time (p99) (ms) | 19.3 | 21.0 (+9%) | | field-frame-time · field · frame time (worst 1%) (ms) | 20.1 | 21.6 (+8%) | ### Flags and settings None: the plan varied no setting, switch or build option. ``` </details> #### Measurements · windows **Verdict:** fail on windows (2/1 report): `field-frame-time field.frametime_p99_ms.max_delta_pct (+22.8%)`; `field-frame-time field.frametime_1pct_low_ms.max_delta_pct (+29.5%)` failed. Hardware: 1: windows+amd+amd-proprietary+dgpu `AMD Radeon RX 9070 XT, 2.0.416` `AMD Ryzen 9 5900X 12-Core Processor, 32 GB RAM`; 2: windows+nvidia+nvidia-proprietary+dgpu `NVIDIA GeForce RTX 4070, 621.1.0` `13th Gen Intel(R) Core(TM) i7-13700F, 32 GB RAM` Scene: field-frame-time, from a new save; each cell's settings and switches as the table names them Method: the Beanstalk test farm: main and this PR built and run on the same PC in one session, game starts alternating (ABBA); field-frame-time: 1 game start a side, field 360 s after 5 s warm-up; each number the median of the runs, then of 2 reports How to reproduce: the farm's plan above (`@Beanstalk` on this PR runs it again) | | Before: main @ 544229a | After: this PR @ 0cece02 | |---|---|---| | field-frame-time · field · frame time (p99) (ms) | 18.5 | 21.3 (+15%) | | field-frame-time · field · frame time (worst 1%) (ms) | 19.0 | 22.6 (+19%) | ### Flags and settings None: the plan varied no setting, switch or build option. <details><summary>Markdown for the PR description</summary> ```markdown ## Measurements **Verdict:** fail on windows (2/1 report): `field-frame-time field.frametime_p99_ms.max_delta_pct (+22.8%)`; `field-frame-time field.frametime_1pct_low_ms.max_delta_pct (+29.5%)` failed. Hardware: 1: windows+amd+amd-proprietary+dgpu `AMD Radeon RX 9070 XT, 2.0.416` `AMD Ryzen 9 5900X 12-Core Processor, 32 GB RAM`; 2: windows+nvidia+nvidia-proprietary+dgpu `NVIDIA GeForce RTX 4070, 621.1.0` `13th Gen Intel(R) Core(TM) i7-13700F, 32 GB RAM` Scene: field-frame-time, from a new save; each cell's settings and switches as the table names them Method: the Beanstalk test farm: main and this PR built and run on the same PC in one session, game starts alternating (ABBA); field-frame-time: 1 game start a side, field 360 s after 5 s warm-up; each number the median of the runs, then of 2 reports How to reproduce: the farm's plan above (`@Beanstalk` on this PR runs it again) | | Before: main @ 544229a | After: this PR @ 0cece02 | |---|---|---| | field-frame-time · field · frame time (p99) (ms) | 18.5 | 21.3 (+15%) | | field-frame-time · field · frame time (worst 1%) (ms) | 19.0 | 22.6 (+19%) | ### Flags and settings None: the plan varied no setting, switch or build option. ``` </details>
Member

Follow-up stacked on this PR: #1383. It fixes the 10 review findings: summaries fit the 60 KiB limit, no stale or cutscene poses, positions rounded to cells in the game, the sampling gap. It holds this PR's commits, so merge it after this one, or cherry-pick its two commits onto this branch. It's WIP and hasn't run in the game yet. One thing to check: the cell rounding assumes Longshot-api's relay rounds down into bins (floor); if it rounds to the nearest bin, bin_middle() needs a one-line change.

Follow-up stacked on this PR: #1383. It fixes the 10 review findings: summaries fit the 60 KiB limit, no stale or cutscene poses, positions rounded to cells in the game, the sampling gap. It holds this PR's commits, so merge it after this one, or cherry-pick its two commits onto this branch. It's WIP and hasn't run in the game yet. One thing to check: the cell rounding assumes Longshot-api's relay rounds down into bins (floor); if it rounds to the nearest bin, bin_middle() needs a one-line change.
Owner

Brought up to the ready-for-review standard (#1454): the checklist's Everyone list has two new boxes, added here from what this PR's description and comments show (please check they match what you know).

  • Tested on my own PC: not ticked: Testing names only game-free checks on Linux, and the change touches gpu/ and host/, which the game runs. Run it yourself on your PC (in the game, for a change the game runs) and say where and what you saw in Testing's Tested; or, when the fix's own case can't be run by hand, add a **Not testable by hand:** line saying why and what stands in for it. Then tick it.
  • Nothing gets worse that I know of: not ticked: Measurements says the per-second sampling's frame cost is not measured yet, and that cost is the open question. Measure that cost and state it in Risks, then tick it.
Brought up to the [ready-for-review standard](https://git.magicbean.dev/MagicBeanDev/Longshot/src/branch/main/CONTRIBUTING.md#ready-for-review-out-of-wip) (#1454): the checklist's Everyone list has two new boxes, added here from what this PR's description and comments show (please check they match what you know). - *Tested on my own PC*: not ticked: Testing names only game-free checks on Linux, and the change touches gpu/ and host/, which the game runs. Run it yourself on your PC (in the game, for a change the game runs) and say where and what you saw in Testing's *Tested*; or, when the fix's own case can't be run by hand, add a `**Not testable by hand:**` line saying why and what stands in for it. Then tick it. - *Nothing gets worse that I know of*: not ticked: Measurements says the per-second sampling's frame cost is not measured yet, and that cost is the open question. Measure that cost and state it in Risks, then tick it.
Author
Member

Closing this as superseded — the work from this PR is already on main.

b50f4977 (the squash-merge of #1383) carried this branch's commits into main, including the per-second PositionSamples and perf_tracking, the client: Longshot field in both performance and crash reports, and the API 30 bump. I verified all of these are present on main today. #1502 continues the sampling work (camera-following samples, relay-owned cells, skipping cutscenes and the title).

This PR itself stayed a draft: it was never run in the game, failed the Beanstalk farm on Linux and Windows (p99 frame time +9%/+15%, 1% lows up to ~+30%), had 10 review findings with 4 marked must-fix, and now conflicts with main. It should not be merged in its own right.

#1530 is a separate line of work — the game posting its own v2 summaries with replay and presentation data — and does not depend on this PR, so closing this does not block it.

Closing this as superseded — the work from this PR is already on `main`. `b50f4977` (the squash-merge of #1383) carried this branch's commits into `main`, including the per-second `PositionSample`s and `perf_tracking`, the `client: Longshot` field in both performance and crash reports, and the API 30 bump. I verified all of these are present on `main` today. #1502 continues the sampling work (camera-following samples, relay-owned cells, skipping cutscenes and the title). This PR itself stayed a draft: it was never run in the game, failed the Beanstalk farm on Linux and Windows (p99 frame time +9%/+15%, 1% lows up to ~+30%), had 10 review findings with 4 marked must-fix, and now conflicts with `main`. It should not be merged in its own right. #1530 is a separate line of work — the game posting its own v2 summaries with replay and presentation data — and does not depend on this PR, so closing this does not block it.
gab9281 closed this pull request 2026-11-18 17:24:29 +00:00
Some checks failed
CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Has been skipped
CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Has been skipped
CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Has been skipped
Third-party parts / The same as upstream (pull_request) Has been skipped
Website / Website on Cloudflare (pull_request) Has been skipped
CI / Conventional Commit title (pull_request) Successful in 7s
CI / Workflow lint (actionlint) (pull_request) Successful in 12s
Guide screenshots / Guide images and local UI source review (pull_request) Successful in 12s
Third-party parts / As their notes say (pull_request) Successful in 14s
Website / Website on Cloudflare (pull_request_target) Has been skipped
PR description / Description follows the template (pull_request_target) Successful in 11s
beanstalk/approval approved by gab9281 (reviewed)
beanstalk/linux 1/1 fail
beanstalk/windows 1/1 fail

Pull request closed

Sign in to join this conversation.
No reviewers
No milestone
No project
No assignees
5 participants
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
MagicBeanDev/Longshot!1372
No description provided.