WIP: feat(perf): sample performance every second and name the client #1372
No reviewers
Labels
No labels
area/accessibility
area/audio
area/input
area/install
area/menu
area/renderer
area/saves
area/tooling
area/upscaling
beanstalk: approved
beanstalk: failed
beanstalk: needs approval
beanstalk: running
beanstalk: tested
ci: approved
good first issue
gpu/amd
gpu/igpu
gpu/intel
gpu/nvidia
help wanted
kind
bug
kind
chore
kind
docs
kind
feature
kind
performance
kind
question
needs measurements
platform/linux
platform/macos
platform/steam-deck
platform/windows
preview
priority
critical
priority
high
priority
low
regression
risk
high
risk
low
risk
medium
status
blocked
status
confirmed
status
duplicate
status
invalid
status
needs-info
status
wontfix
ci: approved
No milestone
No project
No assignees
5 participants
Notifications
Due date
No due date set.
Dependencies
No dependencies set
Reference
MagicBeanDev/Longshot!1372
Loading…
Reference in a new issue
No description provided.
Delete branch "gab9281/Longshot:codex/perf-position-telemetry"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Type: feat · Risk: medium · Tested on: Linux (game-free tests only) · Needs: Linux and Windows, any GPU, an in-game run
Why
The performance statistics don't say where in the world a frame was slow. Their per-map totals cover a whole map, so a stutter at one spot in Hyrule Field looks the same as one spread across the map. Crash and performance reports also mix the client and the game version into a single field, so the team can't filter the dashboards by each one separately. This PR has players who opted in to "Send performance statistics" record one sample per second of gameplay and send those samples in the existing five-minute batches. Both reports now name the client (
Longshot) separately from the version.What changed
The game now records a sample each second: frame and GPU time averages, hitches, the map, Link's position, and the camera's yaw and pitch. Each five-minute summary carries up to 300 of these samples. The crash report and the performance summary send
client: Longshotbesideversion. The relay side, Longshot-api #12, #13 and #14, accepts both fields and keeps accepting older payloads that lack them.gpu/vk/perf_report.{h,cpp}:PositionSampleand the samples insummary_json();"client":"Longshot"comes first in the summary.host/hos_main.cfeeds the per-second position and view;host/rx_backend.c.MkwVkHostchanged, soMKWVK_API_VERSIONis bumped to 30 (gpu/vk/vk_api.h).host/hos_crash_dialog.cbuild_report()sends"client": "Longshot"and shows "Client: Longshot" on the Send page.docs/performance-statistics.md(what is sent, including position and view) anddocs/crash-report-window.md; README.Measurements
Verdict: not measured yet: the per-second sampling's frame cost must be measured in game before this is ready (see Testing).
Size, measured with the game's own
summary_json()(game-free): a five-minute summary is 51.8 KB on one map, 54.5 KB on four, and 62.2 KB on 16 (the most a summary holds). The relay's limit is 64 KiB, so the default interval fits, but anRX_VK_PERF_INTERVALover about 6 minutes is refused (413): 600 s comes to 106 KB.Flags and settings
None. This uses the existing "Send performance statistics" opt-in.
Testing
Tested: game-free checks on Linux only.
python3 -I -B tests/test_crash_dialog.py: pass. It checks that the report sendsclientandversionas separate fields and that no private username appears in it.python3 -I -B tests/test_perf_report.py: pass. It checks the summary JSON, the client field and the samples.python3 -I -B tests/run_all.py: 607 passed, 37 failed, 77 skipped. The same 37 tests fail onmain@544229a2when run the same way. They fail on environment problems, such as MagicBeanDev/Tooling not sitting beside a/tmpcheckout, so none of them is a regression.Not tested: the game itself. No in-game or GPU run has been done, at the requester's request: this is opened as an untested WIP.
Must be tested:
LONGSHOT_PERF_URLpointed at a local relay. Expect a summary with about 300samplesthat have plausible positions, plus"client":"Longshot", and no frame-time change frommain(Test Farm: @Beanstalk linux, five minutes of Hyrule Field, perf summary posted).LONGSHOT_PERF_URLpointed at a local relay, expecting a summary with about 300samplesthat have plausible positions, plus"client":"Longshot", and no frame-time change frommain(Test Farm: @Beanstalk windows, five minutes of Hyrule Field, perf summary posted).sent.jsonwithclientandversionas separate fields (by hand).Risks
What could break: the per-second sampling in
hos_main.cand theMkwVkHostABI bump, so the host and the renderer must come from the same build. The summary is larger but stays under the relay's 64 KiB limit.Turning it off: untick Send performance statistics; otherwise, revert.
Compatibility: the vk ABI changes (
MKWVK_API_VERSION30). Saves, options and caches are unchanged. Relays without Longshot-api #14 ignoreclient, and #13 already acceptssamples.Checklist
Everyone
type(scope): description) of the status line's type.can't be run by hand is in Testing's Not testable by hand, with why. Not ticked yet: Testing names only game-free checks on Linux, and the change touches gpu/ and host/, which the game runs.
in Risks. Not ticked yet: Measurements says the per-second sampling's frame cost is not measured yet, and that cost is the open question.
main.python -I -B tests/run_all.pypasses (any failure that also fails onmainis named).third_party/untouched (a new folder with aPROVENANCE.mdinstead).When it applies
tests/that reads changed source updated in this PR, not weakened.Assisted-by:/Co-Authored-By:trailers.MKWVK_API_VERSIONbumped (gpu/vk/vk_api.h'sMkwVkHostorMkwVkRendererchanged).🤖 Generated with Claude Code
Possibly related, for context:
I found 10 issues by reading the code. None were re-checked afterwards, and nothing was run in the game.
Items 1-4 should be fixed before this leaves WIP:
Item 5 (exact positions leave the PC) is a privacy and design point. Items 6-10 are cleanup.
@ -163,2 +172,4 @@}void finish_sample() {if (collecting_pose_valid && collecting.frames > 0) samples.push_back(collecting);Summaries can go over the launcher's 60 KiB MAX_BODY. Nothing caps the number of samples, and each realistic one is about 177 bytes.
RX_VK_PERF_INTERVAL=840there can be about 149 KB of samples, so every summary is refused.nx2/perf_report.pyrefuses a summary, it drops the per-map totals too, not just the samples.The test at
tests/test_perf_report.cpp:566only uses 300 short samples on one map, so it doesn't catch this. The docs' "up to 300 samples" is also wrong for any interval other than 300.@ -187,9 +213,11 @@ void write_summary(const std::filesystem::path& dir, const std::string& name, co/// lost device: the process ends next)void summarize(bool now) {if (!current) return;finish_sample();Frames just after each summary are never sampled.
summarize()callsfinish_sample(), which setscollecting_pose_valid=falseandsince_sample=0. No new sample starts until the nextsince_map>=1check.The 5-minute summary runs mid-frame, so for up to the next second the frames' hitches and frame times are missing from the samples, although they still count in the map totals. A stutter right after each summary is never placed on the map.
@ -282,3 +311,3 @@std::lock_guard lk(mu);if (!current || since_map >= kMapEvery) {if (!current || since_map >= kMapEvery || since_sample >= 1.0) {since_sampleduplicatessince_map. The two advance together and are reset in the same block. The only difference is the reset insummarize'sfinish_sample, which is what causes the gap at line 216. Keep onlysince_map(kMapEveryis already 1 s) and start the new sample in the same block.@ -292,9 +323,19 @@ void perf_frame(const fs::Frame& f, double wait_ms) {avg_ms = usual * 0.98 + f.ms * 0.02;const double cap_ms = menu_target_ms();const double target = cap_ms > 0 ? cap_ms : usual;int64_t hitches_before = 0;Two extra loops every frame to detect a hitch. This sums
current->hitch[kCauses]before and afteradd_frameon every frame, which adds two 9-element loops to the hot path. Havingadd_framereturn whether the frame was a hitch, or callingis_hitch(f.ms, std::max(target, usual)), does it in one comparison.@ -167,6 +176,15 @@ inline std::string json_num(double v) {return s;}inline std::string json_signed_num(double v) {json_signed_numcopiesjson_numminus thestd::max(v, 0.0)clamp, so the two have to be kept in step, including the shared 32-byte buffer truncation.json_numcould becomereturn json_signed_num(std::max(v, 0.0));.@ -211,0 +228,4 @@o += "},\"samples\":[";for (size_t i = 0; i < samples.size(); i++) {const auto& s = samples[i];o += std::string(i ? "," : "") + "{\"map\":" + json_text(s.map) + ",\"x\":" + json_signed_num(s.x) +Exact positions leave the PC. Positions and angles are written to
perf_pendingwith 3 decimals, and only the relay rounds them into 100-unit cells and 45/30-degree bins. The docs and README describe rounded cells as what is collected.Round in
summary_jsoninstead. Sending the map as an index rather than repeating its name in every sample would also shrink the summary well under the 60 KiB limit (see the perf_report.cpp:175 comment).@ -268,3 +268,3 @@keep up with the sound, [2] shown late (over 50 ms after the one before), [3] decoding time in ns, [4] decoders that didnot start (another one took over). Returns the build's version (LONGSHOT_VERSION) */const char *(*perf_info)(char *map, int cap, int64_t movie[5]);const char *(*perf_info)(char *map, int cap, int64_t movie[5], float pose[5], int *has_pose);Outdated comments and docs for the API 30 change. The
perf_infosignature changed for API 30, but the Version-26 comment still only describes map and movie. It should give the pose order and units ({x, y, z, yaw, pitch}, angles in radians) and say thathas_poseis 0 unlessperf_tracking(1)was called.docs/performance-statistics.md:89also still says "five-minute batches" now that the interval can be changed, and no longer gives the reason for the 840 s cap.@ -64,2 +68,4 @@ReleaseSRWLockShared(&map_lock);hos_movie_stats(movie);AcquireSRWLockShared(&perf_pose_lock);*has_pose = perf_pose_valid;A stale pose is reported after Link stops updating.
perf_pose_validis only cleared when the camera view is missing or a value is not finite. After a return to the title or file select, a reset or a load,hos_perf_infokeeps returninghas_pose=1with Link's last position.Example: play in Hyrule Field, then go back to the title. Samples are tagged "Title and intro" but still carry Hyrule Field X/Y/Z and yaw/pitch every second, so title-screen frame times land on the field in the dashboards.
@ -1343,2 +1360,4 @@}/* The opt-in performance sample's latest Link position and gameplay camera angles. No game memory is read while unticked. */static void oot_perf_pose(Ctx *c, uint64_t actor) {This does too much work on every tick.
oot_perf_posetakes the exclusive SRW lock and follows the 5-load camera pointer chain on every Link update, competing with the present thread's shared read, but the renderer reads the pose only once a second. Limit it to about once a second withhos_now_ns(already called in this function), or publish the pose through a seqlock or atomics.@ -1345,0 +1370,4 @@return;}float pose[5] = {oot_load_float(c, actor + 796), oot_load_float(c, actor + 800), oot_load_float(c, actor + 804),oot_load_float(c, view + 0x1a4), oot_load_float(c, view + 0x18c)};Yaw and pitch are read without checking the camera type. The flight and free-camera code check
recomp_load64(c, view) == 0x84c18a88before reading this view;oot_perf_posereadsview+0x1a4andview+0x18cwithout that check.During a cutscene, or with a different camera type active, the floats at those offsets are unrelated but finite, so they pass
isfiniteand get sent. Very large values (above about 1e28) are also silently truncated byjson_signed_num's 32-byte buffer.@Beanstalk Linux and Windows, any GPU: six minutes in Hyrule Field with Send performance statistics on, the same route on main and on this PR. Compare frame time and 1% lows, to see whether sampling every second costs anything, and check that a performance summary with about 300 samples and "client":"Longshot" is written.
Beanstalk test farm
Commit
0cece02· tier reviewed · approved by gab9281 · dashboardWhat will be tested · AI plan (
gpt-6.1-terra)On Linux and Windows, this plan compares frame time and 1% low frame time in Hyrule Field for six minutes per build, after the same brief movement on each. It fails on a crash or a frame-time regression above 10%. The farm cannot enable the launcher’s performance-statistics opt-in or inspect the written summary, so sampling overhead and the requested sample count and client field remain unverified; the six-minute measurement is stationary.
Done (started 2026-11-17 07:51 UTC, last report 2026-11-17 09:20 UTC). Comment
@Beanstalkto run it again.AMD Radeon RX 7900 XT (RADV NAVI31), 26.2.6; 2: linux+nvidia+nvidia-proprietary+dgpuNVIDIA GeForce RTX 4070, 615.71.9AMD Radeon RX 9070 XT, 2.0.416; 2: windows+nvidia+nvidia-proprietary+dgpuNVIDIA GeForce RTX 4070, 621.1.0Measurements · linux
Verdict: fail on linux (2/1 report):
field-frame-time field.frametime_p99_ms.max_delta_pct (+18.7%);field-frame-time field.frametime_1pct_low_ms.max_delta_pct (+16.9%)failed.Hardware: 1: linux+amd+radv+dgpu
AMD Radeon RX 7900 XT (RADV NAVI31), 26.2.6AMD Ryzen 7 5700X3D 8-Core Processor, 32 GB RAM; 2: linux+nvidia+nvidia-proprietary+dgpuNVIDIA GeForce RTX 4070, 615.71.9AMD Ryzen 7 5800X3D 8-Core Processor, 64 GB RAMScene: field-frame-time, from a new save; each cell's settings and switches as the table names them
Method: the Beanstalk test farm: main and this PR built and run on the same PC in one session, game starts alternating (ABBA); field-frame-time: 1 game start a side, field 360 s after 5 s warm-up; each number the median of the runs, then of 2 reports
How to reproduce: the farm's plan above (
@Beanstalkon this PR runs it again)544229a0cece02Flags and settings
None: the plan varied no setting, switch or build option.
Markdown for the PR description
Measurements · windows
Verdict: fail on windows (2/1 report):
field-frame-time field.frametime_p99_ms.max_delta_pct (+22.8%);field-frame-time field.frametime_1pct_low_ms.max_delta_pct (+29.5%)failed.Hardware: 1: windows+amd+amd-proprietary+dgpu
AMD Radeon RX 9070 XT, 2.0.416AMD Ryzen 9 5900X 12-Core Processor, 32 GB RAM; 2: windows+nvidia+nvidia-proprietary+dgpuNVIDIA GeForce RTX 4070, 621.1.013th Gen Intel(R) Core(TM) i7-13700F, 32 GB RAMScene: field-frame-time, from a new save; each cell's settings and switches as the table names them
Method: the Beanstalk test farm: main and this PR built and run on the same PC in one session, game starts alternating (ABBA); field-frame-time: 1 game start a side, field 360 s after 5 s warm-up; each number the median of the runs, then of 2 reports
How to reproduce: the farm's plan above (
@Beanstalkon this PR runs it again)544229a0cece02Flags and settings
None: the plan varied no setting, switch or build option.
Markdown for the PR description
Follow-up stacked on this PR: #1383. It fixes the 10 review findings: summaries fit the 60 KiB limit, no stale or cutscene poses, positions rounded to cells in the game, the sampling gap. It holds this PR's commits, so merge it after this one, or cherry-pick its two commits onto this branch. It's WIP and hasn't run in the game yet. One thing to check: the cell rounding assumes Longshot-api's relay rounds down into bins (floor); if it rounds to the nearest bin, bin_middle() needs a one-line change.
Brought up to the ready-for-review standard (#1454): the checklist's Everyone list has two new boxes, added here from what this PR's description and comments show (please check they match what you know).
**Not testable by hand:**line saying why and what stands in for it. Then tick it.Closing this as superseded — the work from this PR is already on
main.b50f4977(the squash-merge of #1383) carried this branch's commits intomain, including the per-secondPositionSamples andperf_tracking, theclient: Longshotfield in both performance and crash reports, and the API 30 bump. I verified all of these are present onmaintoday. #1502 continues the sampling work (camera-following samples, relay-owned cells, skipping cutscenes and the title).This PR itself stayed a draft: it was never run in the game, failed the Beanstalk farm on Linux and Windows (p99 frame time +9%/+15%, 1% lows up to ~+30%), had 10 review findings with 4 marked must-fix, and now conflicts with
main. It should not be merged in its own right.#1530 is a separate line of work — the game posting its own v2 summaries with replay and presentation data — and does not depend on this PR, so closing this does not block it.
Pull request closed