feat(perf): opt-in performance statistics, sent from the launcher to the team's Grafana #206

Open

gab9281 wants to merge 10 commits from gab9281/Longshot:feat/perf-metrics into main

pull from: gab9281/Longshot:feat/perf-metrics

merge into: MagicBeanDev:main

MagicBeanDev:main

Conversation 0 Commits 10 Files changed 29 +1372 -32

gab9281 commented 2026-11-13 14:13:40 +00:00

Member

• Copy link
• Copy Markdown

Why

Players report stutter, cutscenes that play badly and uneven frame pacing. The team can't tell which GPUs, drivers, CPUs, systems, renderers or maps these come from. With this PR, a player can tick Send performance statistics in the launcher. It is off by default. The game then counts how its frames went on each map, and the launcher sends the counts to the team's Grafana, where each of those can be filtered on its own.

What changed

• game, gpu/vk/perf_report.h and perf_report.cpp. With send=1 in the game folder's longshot_perf.ini, for each map the summary holds:
  • frame-time and GPU-time histograms, and the FPS counter's parts (game, render, GPU wait, pace);
  • hitches by what they waited on most (pipelines, shader translation, texture uploads, memory sync, the GPU, read-backs, the game, pacing), pacing misses, and draws left out;
  • rendering events: program failed, device lost, swapchain recreated, movie upscaler dropped, DLSS recreated, VRAM over budget;
  • cutscene playback: pictures shown, dropped and late, decoding time, decoder fallbacks;
  • VRAM and RAM, and the machine and settings. Every summary carries the GPU, driver, CPU, RAM, OS, kernel, upscaler, preset and cap.
  • A summary is written to perf_pending/ every 5 minutes and when the window closes (mkwvk_close_asked in vk_main.cpp, since ExitProcess is _exit on Linux and atexit never runs). One is also written, once, on a lost device (vk_replay.cpp device_lost).
  • The game reads the tick again before each summary. If it was unticked during play, the game stops measuring and writes nothing more.
  • At most 48 summaries wait on disk.
• game, vk_fps.cpp: while the summary is on, the FPS counter takes its Breakdown's timings and passes each frame to perf_frame() (add_frame: 6-16 ns). With the tick off, the only cost is one load of perf_measure per frame.
• host: the map Link is in and the movies' totals, through MkwVkHost::perf_info (API 22):
  • hos_main.c hos_perf_info;
  • nn_video.c counts each picture as the game takes it, so a movie still playing at a summary is counted;
  • dav1d_av1.c counts a decoder that didn't start.
• The launcher's side is nx2/perf_report.py (below). The launchers that were in builder/ left main in #171 and #181, so the tick itself goes to LongshotLauncher in a separate PR. That PR writes send=1 (or deletes the file and perf_pending/ when unticked) and, at its start, runs <python> -I nx2/perf_report.py <folder> --send in a separate process, so Play, which closes the launcher at once, can't cut a send short. Until it lands, send=1 in longshot_perf.ini turns it on by hand.
• Before the first map, the summary's map is "Preparing shaders" while the start-up screen prepares them, then "Title and intro". These are the same names #141's crash report uses.
• nx2: perf_report.py:
  • the tick;
  • sending, oldest first, with the version and the release flag filled in (as the crash report has them);
  • a summary the relay took or refused is deleted, and one it couldn't reach, or a 429/5xx, stays for next time;
  • each summary the relay took is kept, exactly as sent, in perf_sent/ (the newest 48), so a player or a tester can read what went to the team;
  • unticking deletes what is waiting, and those copies.
• nx2: addresses.ini/addresses.py hold the relay's address (perf_reports), out of the code.
• docs: docs/performance-statistics.md (what is sent and never sent, how the parts fit, the cost), a README line, and longshot_perf.ini in docs/development.md.
• tests:
  • tests/test_perf_report.cpp covers the summary's functions;
  • tests/test_perf_report.py: the launcher side against a stand-in relay, the detached sender, and the wiring in the game (the pre-map names included);
  • the video tests' fixtures get the new counter.

The relay and the Grafana dashboard are in Bean-Ops (deploy/perf-relay, longshot-performance.json), already merged and deployed: perf_reports in nx2/addresses.ini points at it. The dashboard also shows a perf index (average fps, 1% low or smooth frames) by map and graphics card, as a table and in 3D.

Flags and settings

Flag	What it does	Default now	Recommendation	Helps on	Doesn't help on	Untested on
longshot_perf.ini send=1 (the launcher's Send performance statistics tick)	The game measures its frames and writes a summary every 5 minutes and at close, and the launcher sends them	off (no file)	Off by default: it's the player's choice to send anything	finding which hardware, driver, system or map stutters or plays cutscenes badly	—	Windows, AMD and Intel GPUs
HOS_FPS_MAX=<61-1000> (existing)	The summary's fps_cap is the cap in effect: this flag's when set, the menu's otherwise	unset	Unchanged	the cap label matching what the run used	—	—

Measurements

main: fbd42024 (2026-11-13)
this PR: feat/perf-metrics @ ad4bce73 (rebased on that main). Since then: the review's fixes on the launcher's sender (nx2/perf_report.py, run after the game, not during it), and a rebase on main 514591a3 (#274 after d9648863: docs only) that changed none of this PR's code. Main's 9 commits since fbd42024 change the renderer for both sides alike (#234, #260, #247); this PR's per-frame work is the same, so the comparison above still holds
Hardware: NVIDIA GeForce RTX 3060 Laptop GPU (6 GB), NVIDIA 620.70.06, AMD Ryzen 7 6800H, 15 GB, CachyOS (Linux 7.3.3), KDE Wayland
Scene: the game's start, the intro and title (no save on this PC), 1920x1080, upscaler Off, no frame generation, HOS_FPS_MAX=1000 (uncapped)
Method: 75 s per run, started as the launcher starts it (through CachyOS's game-performance), with RX_VK_FRAME_LOG, ended with the control bridge's quit (the window's close path). Frames from 20 s on. Main, then this PR with the tick off, then on, repeated 3 times (interleaved). Each value is the median of the 3 runs.
How to reproduce: two game folders built from the same game files, one at each commit; HOS_FPS_MAX=1000; longshot_perf.ini with send=1 for "on"; quit with python -I scripts/longshot_ctl.py quit.

	main @ fbd42024	this PR @ ad4bce73, tick off	this PR @ ad4bce73, tick on
fps (uncapped)	150.9	149.1	149.1
1% low fps	96.7	95.2	95.1
median frame time	5.69 ms	5.73 ms	5.75 ms

{'run': 'main-1', 'frames': 8784, 'median ms': 5.52, 'fps': 153.88, '1% low fps': 98.6, 'end': 'quit', 'summaries': []}
{'run': 'off-1', 'frames': 8564, 'median ms': 5.69, 'fps': 150.36, '1% low fps': 96.9, 'end': 'quit', 'summaries': []}
{'run': 'on-1', 'frames': 8482, 'median ms': 5.75, 'fps': 149.33, '1% low fps': 96.7, 'end': 'quit', 'summaries': ['1794595369-0.json']}
{'run': 'main-2', 'frames': 8610, 'median ms': 5.69, 'fps': 150.95, '1% low fps': 96.7, 'end': 'quit', 'summaries': []}
{'run': 'off-2', 'frames': 8512, 'median ms': 5.73, 'fps': 149.06, '1% low fps': 95.2, 'end': 'quit', 'summaries': []}
{'run': 'on-2', 'frames': 8474, 'median ms': 5.75, 'fps': 148.92, '1% low fps': 95.0, 'end': 'quit', 'summaries': ['1794595596-0.json']}
{'run': 'main-3', 'frames': 8478, 'median ms': 5.78, 'fps': 148.57, '1% low fps': 96.5, 'end': 'quit', 'summaries': []}
{'run': 'off-3', 'frames': 8465, 'median ms': 5.83, 'fps': 147.98, '1% low fps': 94.3, 'end': 'quit', 'summaries': []}
{'run': 'on-3', 'frames': 8489, 'median ms': 5.76, 'fps': 149.06, '1% low fps': 95.1, 'end': 'quit', 'summaries': ['1794595823-0.json']}

Reading: main itself spread from 148.6 to 153.9 fps across the rounds, and every run of this PR is inside that spread. Round by round, though, the tick off was behind main each time (0.17, 0.04 and 0.05 ms a frame; 1.5 to 3.5 fps), so a cost of about 1% with the tick off can't be ruled out on this PC; on a laptop a 1% gap is the size of the drift between rounds. The tick on was about the same as the tick off (5.75 against 5.73 ms; faster in round 3). The earlier set at 88404d20 on main 36a12f6f, same method, gave main 148.4, off 150.4, on 149.8 fps: there the tick off was ahead of main. A bigger PC (test requests below) can settle the 1%.

Capped at 60 (the default cap, 3 rounds, same commits): all three were at 59.8-60.0 fps, with 1% lows of 41.6-42.1 (main), 41.5-44.2 (off) and 42.1-43.5 (on).

Testing

Tested:

• Game-free: RX_CPM=<the CPM cache> python -I -B tests/run_all.py at 65c63546 (main @ d9648863; b271368d is the same code on 514591a3, docs only): 518 passed, 1 failed, 15 skipped (Windows only). The failure, tests/test_menu_i18n.py (a pt_BR text of a menu line main changed), fails on main d9648863 too. At ad4bce73 on main fbd42024: 515 passed, 0 failed.
• In the game, Linux (the hardware above), built from this branch:
  • With the tick on, a summary is written when the game is quit through the window's close path.
  • In a run that reached play (a new game into Kokiri Forest), the summary splits the time by map: "KokiriForest_Child (child)" 44.4 s, and the intro 178.2 s.
  • After the rebase (ecfc377f), a run on the title: the summary's map is "Title and intro" (53.2 s).
  • Base data for the dashboard: a save continued in Hyrule Field, Link walking, 3 sessions of 4-5 minutes (one built before the second review's fixes, two with them). Their 3 summaries (Hyrule Field 268 s, 224 s and 272 s; "Title and intro" 30-40 s) were sent to the deployed relay and show on the dashboard's map × graphics card panels.
  • Longer runs on this PC (15 GB of RAM, other apps open) were ended by the kernel's out-of-memory killer after 2-5 minutes in Hyrule Field and right after a teleport to Kakariko. The game reaches about 6 GB there: its guest memory, 2.5 GB resident, and swap. This is the game's memory, not this PR's (the summary keeps a few kB per map), but it means the 5-minute summaries over a long session, and maps other than Hyrule Field, still need a PC with more memory.
  • The intro's movie is counted: 1506 pictures shown, 107 dropped, 29 late.
  • With the tick taken off during play, the game logs "performance statistics off" and writes nothing.
• Sending: the launcher's nx2/perf_report.py sent a real summary from these runs to a local copy of Bean-Ops' relay, which took it. Its /metrics showed:
  • the play time with every label: GPU, NVIDIA 620.70, CPU, CachyOS, kernel 7.3, upscaler Off, cap, map;
  • the hitches by cause;
  • the movie counts.
• The deployed relay: 6 real summaries from these runs, posted to perf_reports (as unofficial builds), were all taken, and show in Prometheus and on the Grafana dashboard.

Not tested:

• The tick on screen: it goes to LongshotLauncher (above).
• "Preparing shaders" in a summary. The start-up screen's preparation presents no game frames, so the name shows only for a preparation that runs behind play (an update).
• A summary from a lost device, and the 5-minute summaries in a long session. Only the close summary was seen in these runs.
• A release's flag: CI doesn't write source= yet, so every summary is "not a release" for now, as in the crash report.

Needs other testers:

• Windows 10/11: put send=1 in the game folder's longshot_perf.ini, play 10 minutes, close the game, then run python -I nx2/perf_report.py <folder> --send. Expect perf_pending\ to empty, and the same summaries in perf_sent\ (what was sent: check the gpu, driver, cpu, os and the maps). Request T-ULGD (waiting for the testing wanted label).
• A PC with 32 GB of RAM or more: a long session (an hour) over several maps, the tick on. Expect a summary in perf_pending/ every 5 minutes, each with the maps played since the one before, and after --send the same summaries in perf_sent/ and the maps on the dashboard's map × graphics card panels. Request T-AAC7 (waiting for the testing wanted label).
• AMD and Intel GPUs: the same, checking that the summary's gpu and driver name the card and its driver (for example radv Mesa 25.2). Request T-WNBH (waiting for the testing wanted label).

Screenshots

None: the game shows nothing new, and the tick is in LongshotLauncher's PR.

Risk and rollback

• Off by default: nothing is measured or written unless the player ticks it. With it off, the game reads one small file at its first frame.
• With it on: every frame takes the Breakdown's timings and add_frame(). See Measurements.
• Privacy: a summary holds no paths, user name or ID, the log, saves, or anything from the game files. docs/performance-statistics.md lists what is sent. Unticking deletes what is waiting, and a game already running stops at its next summary.
• Rollback: reverting the PR is enough. To stop sending, untick it, or remove perf_reports from nx2/addresses.ini.

Reviewer notes

• Start with gpu/vk/perf_report.cpp (what runs per frame and per summary), then nx2/perf_report.py (what leaves the PC).
• The bucket lists and the summary's format are the relay's. Change both together (Bean-Ops deploy/perf-relay/perf_relay.py).
• nx2/addresses.ini is also added by #141, with its crash_* keys. Whichever merges second has an add/add conflict to resolve by keeping both sections' keys.
• /code-review (high) on 36a12f6f..76bea1d7 found 10 issues. All are fixed (9abd14d5 after the rebase; see that commit's message). The most important were a send cut short by Play, and the summary missing at a window close on Linux. A second /code-review (high) on the rebased branch (0c74253d..6702528f) found 10 more, all fixed in 2c952130 (its message lists them). The most important were hitches blamed on translation done by the jobs behind the game, the title's time counted on the last map, and a summary's version replaced by the folder's at sending. A third /code-review (high) on the copy of each sent summary (9d8d6cfa..ad4bce73) found 8, all fixed in edda0942 (its message lists them).
• The summaries sent from this laptop, checked in Prometheus: every label filled and the same on every metric, frames and play time agreeing. One fix came from it: a hitch is now measured against the longer of the target and the recent average frame, so a game the GPU holds at 19 fps under a 60 cap no longer counts nearly every frame as a hitch (4,694 of 5,269 frames had been, in Hyrule Field). A check of it in tests/test_perf_report.cpp: the main ones were a copy kept after the player unticked during a send, a copy cut short taken for what was sent, and pruning by the summaries' names (a wrong clock kept the wrong ones).

Issues

None.

Checklist

For everyone

• One topic.
• The title is a Conventional Commit (type(scope): description).
• Rebased on current main.
• python -I -B tests/run_all.py run and passing (any failure that also fails on main is named above).
• Tested in the game, or what wasn't is named under Testing.
• Testing filled in: what was tested, what wasn't, and what needs other testers (with test requests).
• Measured: main and this PR at named commits, on the same PC, scene and settings, at least 2 interleaved runs each, the median reported.
• No game files, dumps, build output or generated files.
• third_party/ untouched (a new folder with a PROVENANCE.md instead).
• The license is OK: GPL-3.0-or-later, nothing incompatible and nothing decompiled.

If applicable

• AI assistance recorded with Assisted-by: / Co-Authored-By: trailers.
• Flags noted, and the switch reference updated (docs/development.md, the README).
• A row in docs/tested-configurations.md for each measured run.
• MKWVK_API_VERSION bumped (gpu/vk/vk_api.h's MkwVkHost or MkwVkRenderer changed).
• A game-free test added for the fix, failing on main and passing here.
• A check under tests/ that reads changed source updated in this PR, not weakened.
• Host code builds on both Windows (llvm-mingw) and Linux, with UTF-8 paths. (Both built here: Linux at ad4bce73 for the measurements, Windows with llvm-mingw at edda0942.)
• Docs updated.

🤖 Generated with Claude Code

https://claude.ai/code/session_015KgSKoUqUvce5UXG89NBgi

## Why Players report stutter, cutscenes that play badly and uneven frame pacing. The team can't tell which GPUs, drivers, CPUs, systems, renderers or maps these come from. With this PR, a player can tick **Send performance statistics** in the launcher. It is off by default. The game then counts how its frames went on each map, and the launcher sends the counts to the team's Grafana, where each of those can be filtered on its own. ## What changed - **game**, `gpu/vk/perf_report.h` and `perf_report.cpp`. With `send=1` in the game folder's `longshot_perf.ini`, for each map the summary holds: - frame-time and GPU-time histograms, and the FPS counter's parts (game, render, GPU wait, pace); - hitches by what they waited on most (pipelines, shader translation, texture uploads, memory sync, the GPU, read-backs, the game, pacing), pacing misses, and draws left out; - rendering events: program failed, device lost, swapchain recreated, movie upscaler dropped, DLSS recreated, VRAM over budget; - cutscene playback: pictures shown, dropped and late, decoding time, decoder fallbacks; - VRAM and RAM, and the machine and settings. Every summary carries the GPU, driver, CPU, RAM, OS, kernel, upscaler, preset and cap. - A summary is written to `perf_pending/` every 5 minutes and when the window closes (`mkwvk_close_asked` in `vk_main.cpp`, since `ExitProcess` is `_exit` on Linux and `atexit` never runs). One is also written, once, on a lost device (`vk_replay.cpp` `device_lost`). - The game reads the tick again before each summary. If it was unticked during play, the game stops measuring and writes nothing more. - At most 48 summaries wait on disk. - **game**, `vk_fps.cpp`: while the summary is on, the FPS counter takes its Breakdown's timings and passes each frame to `perf_frame()` (`add_frame`: 6-16 ns). With the tick off, the only cost is one load of `perf_measure` per frame. - **host**: the map Link is in and the movies' totals, through `MkwVkHost::perf_info` (API 22): - `hos_main.c` `hos_perf_info`; - `nn_video.c` counts each picture as the game takes it, so a movie still playing at a summary is counted; - `dav1d_av1.c` counts a decoder that didn't start. - **The launcher's side** is `nx2/perf_report.py` (below). The launchers that were in `builder/` left main in #171 and #181, so the tick itself goes to LongshotLauncher in a separate PR. That PR writes `send=1` (or deletes the file and `perf_pending/` when unticked) and, at its start, runs `<python> -I nx2/perf_report.py <folder> --send` in a separate process, so Play, which closes the launcher at once, can't cut a send short. Until it lands, `send=1` in `longshot_perf.ini` turns it on by hand. - **Before the first map**, the summary's map is "Preparing shaders" while the start-up screen prepares them, then "Title and intro". These are the same names #141's crash report uses. - **nx2**: `perf_report.py`: - the tick; - sending, oldest first, with the version and the release flag filled in (as the crash report has them); - a summary the relay took or refused is deleted, and one it couldn't reach, or a 429/5xx, stays for next time; - each summary the relay took is kept, exactly as sent, in `perf_sent/` (the newest 48), so a player or a tester can read what went to the team; - unticking deletes what is waiting, and those copies. - **nx2**: `addresses.ini`/`addresses.py` hold the relay's address (`perf_reports`), out of the code. - **docs**: `docs/performance-statistics.md` (what is sent and never sent, how the parts fit, the cost), a README line, and `longshot_perf.ini` in `docs/development.md`. - **tests**: - `tests/test_perf_report.cpp` covers the summary's functions; - `tests/test_perf_report.py`: the launcher side against a stand-in relay, the detached sender, and the wiring in the game (the pre-map names included); - the video tests' fixtures get the new counter. The relay and the Grafana dashboard are in Bean-Ops (`deploy/perf-relay`, `longshot-performance.json`), already merged and deployed: `perf_reports` in `nx2/addresses.ini` points at it. The dashboard also shows a perf index (average fps, 1% low or smooth frames) by map and graphics card, as a table and in 3D. ## Flags and settings | Flag | What it does | Default now | Recommendation | Helps on | Doesn't help on | Untested on | |---|---|---|---|---|---|---| | `longshot_perf.ini` `send=1` (the launcher's **Send performance statistics** tick) | The game measures its frames and writes a summary every 5 minutes and at close, and the launcher sends them | off (no file) | Off by default: it's the player's choice to send anything | finding which hardware, driver, system or map stutters or plays cutscenes badly | — | Windows, AMD and Intel GPUs | | `HOS_FPS_MAX=<61-1000>` (existing) | The summary's `fps_cap` is the cap in effect: this flag's when set, the menu's otherwise | unset | Unchanged | the cap label matching what the run used | — | — | ## Measurements main: fbd42024 (2026-11-13) this PR: feat/perf-metrics @ ad4bce73 (rebased on that main). Since then: the review's fixes on the launcher's sender (`nx2/perf_report.py`, run after the game, not during it), and a rebase on main 514591a3 (#274 after d9648863: docs only) that changed none of this PR's code. Main's 9 commits since fbd42024 change the renderer for both sides alike (#234, #260, #247); this PR's per-frame work is the same, so the comparison above still holds Hardware: NVIDIA GeForce RTX 3060 Laptop GPU (6 GB), NVIDIA 620.70.06, AMD Ryzen 7 6800H, 15 GB, CachyOS (Linux 7.3.3), KDE Wayland Scene: the game's start, the intro and title (no save on this PC), 1920x1080, upscaler Off, no frame generation, `HOS_FPS_MAX=1000` (uncapped) Method: 75 s per run, started as the launcher starts it (through CachyOS's `game-performance`), with `RX_VK_FRAME_LOG`, ended with the control bridge's quit (the window's close path). Frames from 20 s on. Main, then this PR with the tick off, then on, repeated 3 times (interleaved). Each value is the median of the 3 runs. How to reproduce: two game folders built from the same game files, one at each commit; `HOS_FPS_MAX=1000`; `longshot_perf.ini` with `send=1` for "on"; quit with `python -I scripts/longshot_ctl.py quit`. | | main @ fbd42024 | this PR @ ad4bce73, tick off | this PR @ ad4bce73, tick on | |---|---|---|---| | fps (uncapped) | 150.9 | 149.1 | 149.1 | | 1% low fps | 96.7 | 95.2 | 95.1 | | median frame time | 5.69 ms | 5.73 ms | 5.75 ms | ``` {'run': 'main-1', 'frames': 8784, 'median ms': 5.52, 'fps': 153.88, '1% low fps': 98.6, 'end': 'quit', 'summaries': []} {'run': 'off-1', 'frames': 8564, 'median ms': 5.69, 'fps': 150.36, '1% low fps': 96.9, 'end': 'quit', 'summaries': []} {'run': 'on-1', 'frames': 8482, 'median ms': 5.75, 'fps': 149.33, '1% low fps': 96.7, 'end': 'quit', 'summaries': ['1794595369-0.json']} {'run': 'main-2', 'frames': 8610, 'median ms': 5.69, 'fps': 150.95, '1% low fps': 96.7, 'end': 'quit', 'summaries': []} {'run': 'off-2', 'frames': 8512, 'median ms': 5.73, 'fps': 149.06, '1% low fps': 95.2, 'end': 'quit', 'summaries': []} {'run': 'on-2', 'frames': 8474, 'median ms': 5.75, 'fps': 148.92, '1% low fps': 95.0, 'end': 'quit', 'summaries': ['1794595596-0.json']} {'run': 'main-3', 'frames': 8478, 'median ms': 5.78, 'fps': 148.57, '1% low fps': 96.5, 'end': 'quit', 'summaries': []} {'run': 'off-3', 'frames': 8465, 'median ms': 5.83, 'fps': 147.98, '1% low fps': 94.3, 'end': 'quit', 'summaries': []} {'run': 'on-3', 'frames': 8489, 'median ms': 5.76, 'fps': 149.06, '1% low fps': 95.1, 'end': 'quit', 'summaries': ['1794595823-0.json']} ``` Reading: main itself spread from 148.6 to 153.9 fps across the rounds, and every run of this PR is inside that spread. Round by round, though, the tick off was behind main each time (0.17, 0.04 and 0.05 ms a frame; 1.5 to 3.5 fps), so a cost of about 1% with the tick off can't be ruled out on this PC; on a laptop a 1% gap is the size of the drift between rounds. The tick on was about the same as the tick off (5.75 against 5.73 ms; faster in round 3). The earlier set at 88404d20 on main 36a12f6f, same method, gave main 148.4, off 150.4, on 149.8 fps: there the tick off was ahead of main. A bigger PC (test requests below) can settle the 1%. Capped at 60 (the default cap, 3 rounds, same commits): all three were at 59.8-60.0 fps, with 1% lows of 41.6-42.1 (main), 41.5-44.2 (off) and 42.1-43.5 (on). ## Testing **Tested:** - Game-free: `RX_CPM=<the CPM cache> python -I -B tests/run_all.py` at 65c63546 (main @ d9648863; b271368d is the same code on 514591a3, docs only): 518 passed, 1 failed, 15 skipped (Windows only). The failure, `tests/test_menu_i18n.py` (a pt_BR text of a menu line main changed), fails on main d9648863 too. At ad4bce73 on main fbd42024: 515 passed, 0 failed. - In the game, Linux (the hardware above), built from this branch: - With the tick on, a summary is written when the game is quit through the window's close path. - In a run that reached play (a new game into Kokiri Forest), the summary splits the time by map: "KokiriForest_Child (child)" 44.4 s, and the intro 178.2 s. - After the rebase (ecfc377f), a run on the title: the summary's map is "Title and intro" (53.2 s). - Base data for the dashboard: a save continued in Hyrule Field, Link walking, 3 sessions of 4-5 minutes (one built before the second review's fixes, two with them). Their 3 summaries (Hyrule Field 268 s, 224 s and 272 s; "Title and intro" 30-40 s) were sent to the deployed relay and show on the dashboard's map × graphics card panels. - Longer runs on this PC (15 GB of RAM, other apps open) were ended by the kernel's out-of-memory killer after 2-5 minutes in Hyrule Field and right after a teleport to Kakariko. The game reaches about 6 GB there: its guest memory, 2.5 GB resident, and swap. This is the game's memory, not this PR's (the summary keeps a few kB per map), but it means the 5-minute summaries over a long session, and maps other than Hyrule Field, still need a PC with more memory. - The intro's movie is counted: 1506 pictures shown, 107 dropped, 29 late. - With the tick taken off during play, the game logs "performance statistics off" and writes nothing. - Sending: the launcher's `nx2/perf_report.py` sent a real summary from these runs to a local copy of Bean-Ops' relay, which took it. Its `/metrics` showed: - the play time with every label: GPU, `NVIDIA 620.70`, CPU, `CachyOS`, kernel `7.3`, upscaler `Off`, cap, map; - the hitches by cause; - the movie counts. - The deployed relay: 6 real summaries from these runs, posted to `perf_reports` (as unofficial builds), were all taken, and show in Prometheus and on the Grafana dashboard. **Not tested:** - The tick on screen: it goes to LongshotLauncher (above). - "Preparing shaders" in a summary. The start-up screen's preparation presents no game frames, so the name shows only for a preparation that runs behind play (an update). - A summary from a lost device, and the 5-minute summaries in a long session. Only the close summary was seen in these runs. - A release's flag: CI doesn't write `source=` yet, so every summary is "not a release" for now, as in the crash report. **Needs other testers:** - Windows 10/11: put `send=1` in the game folder's `longshot_perf.ini`, play 10 minutes, close the game, then run `python -I nx2/perf_report.py <folder> --send`. Expect `perf_pending\` to empty, and the same summaries in `perf_sent\` (what was sent: check the `gpu`, `driver`, `cpu`, `os` and the maps). Request T-ULGD (waiting for the `testing wanted` label). - A PC with 32 GB of RAM or more: a long session (an hour) over several maps, the tick on. Expect a summary in `perf_pending/` every 5 minutes, each with the maps played since the one before, and after `--send` the same summaries in `perf_sent/` and the maps on the dashboard's map × graphics card panels. Request T-AAC7 (waiting for the `testing wanted` label). - AMD and Intel GPUs: the same, checking that the summary's `gpu` and `driver` name the card and its driver (for example `radv Mesa 25.2`). Request T-WNBH (waiting for the `testing wanted` label). ## Screenshots None: the game shows nothing new, and the tick is in LongshotLauncher's PR. ## Risk and rollback - **Off by default:** nothing is measured or written unless the player ticks it. With it off, the game reads one small file at its first frame. - **With it on:** every frame takes the Breakdown's timings and `add_frame()`. See Measurements. - **Privacy:** a summary holds no paths, user name or ID, the log, saves, or anything from the game files. `docs/performance-statistics.md` lists what is sent. Unticking deletes what is waiting, and a game already running stops at its next summary. - **Rollback:** reverting the PR is enough. To stop sending, untick it, or remove `perf_reports` from `nx2/addresses.ini`. ## Reviewer notes - Start with `gpu/vk/perf_report.cpp` (what runs per frame and per summary), then `nx2/perf_report.py` (what leaves the PC). - The bucket lists and the summary's format are the relay's. Change both together (Bean-Ops `deploy/perf-relay/perf_relay.py`). - `nx2/addresses.ini` is also added by #141, with its `crash_*` keys. Whichever merges second has an add/add conflict to resolve by keeping both sections' keys. - `/code-review` (high) on 36a12f6f..76bea1d7 found 10 issues. All are fixed (9abd14d5 after the rebase; see that commit's message). The most important were a send cut short by Play, and the summary missing at a window close on Linux. A second `/code-review` (high) on the rebased branch (0c74253d..6702528f) found 10 more, all fixed in 2c952130 (its message lists them). The most important were hitches blamed on translation done by the jobs behind the game, the title's time counted on the last map, and a summary's version replaced by the folder's at sending. A third `/code-review` (high) on the copy of each sent summary (9d8d6cfa..ad4bce73) found 8, all fixed in edda0942 (its message lists them). - The summaries sent from this laptop, checked in Prometheus: every label filled and the same on every metric, frames and play time agreeing. One fix came from it: a hitch is now measured against the longer of the target and the recent average frame, so a game the GPU holds at 19 fps under a 60 cap no longer counts nearly every frame as a hitch (4,694 of 5,269 frames had been, in Hyrule Field). A check of it in `tests/test_perf_report.cpp`: the main ones were a copy kept after the player unticked during a send, a copy cut short taken for what was sent, and pruning by the summaries' names (a wrong clock kept the wrong ones). ## Issues None. ## Checklist ### For everyone - [x] One topic. - [x] The title is a Conventional Commit (`type(scope): description`). - [x] Rebased on current `main`. - [x] `python -I -B tests/run_all.py` run and passing (any failure that also fails on `main` is named above). - [x] Tested in the game, or what wasn't is named under Testing. - [x] Testing filled in: what was tested, what wasn't, and what needs other testers (with test requests). - [x] Measured: main and this PR at named commits, on the same PC, scene and settings, at least 2 interleaved runs each, the median reported. - [x] No game files, dumps, build output or generated files. - [x] `third_party/` untouched (a new folder with a `PROVENANCE.md` instead). - [x] The license is OK: GPL-3.0-or-later, nothing incompatible and nothing decompiled. ### If applicable - [x] AI assistance recorded with `Assisted-by:` / `Co-Authored-By:` trailers. - [x] Flags noted, and the switch reference updated (docs/development.md, the README). - [x] A row in `docs/tested-configurations.md` for each measured run. - [x] `MKWVK_API_VERSION` bumped (`gpu/vk/vk_api.h`'s `MkwVkHost` or `MkwVkRenderer` changed). - [x] A game-free test added for the fix, failing on `main` and passing here. - [x] A check under `tests/` that reads changed source updated in this PR, not weakened. - [x] Host code builds on both Windows (llvm-mingw) and Linux, with UTF-8 paths. (Both built here: Linux at ad4bce73 for the measurements, Windows with llvm-mingw at edda0942.) - [x] Docs updated. 🤖 Generated with [Claude Code](https://claude.com/claude-code) https://claude.ai/code/session_015KgSKoUqUvce5UXG89NBgi

perf-tick-linux.png

58 KiB

gab9281 added 3 commits 2026-11-13 14:13:41 +00:00

feat(perf): the game counts its frames for the opt-in performance summary ... 2d46386fb3

With send=1 in the game folder's longshot_perf.ini (the launcher's Send performance statistics tick, off by
default), the renderer adds each frame to its map's totals and writes a summary into perf_pending/ every 5
minutes and when the window closes:

- gpu/vk/perf_report.h, perf_report.cpp: frame-time and GPU-time histograms, the FPS counter's parts, hitches
  by what they waited on, pacing misses, draws left out, rendering events, movie playback, memory; the machine
  (GPU, driver, CPU, RAM, OS, kernel) and the upscaler, preset and cap.
- vk_fps.cpp: the Breakdown's timings while the summary is on; off, one load of perf_measure a frame.
- vk_main.cpp mkwvk_close_asked: the last summary (ExitProcess is _exit on Linux: atexit never runs there).
- host: the map and the movies' totals through MkwVkHost perf_info (API 22).
- tests/test_perf_report.cpp: the summary's functions; add_frame 16 ns at -O1.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_015KgSKoUqUvce5UXG89NBgi

feat(perf): the launcher's Send performance statistics tick, and sending the summaries ... 76bea1d701

- nx2/perf_report.py: the tick (longshot_perf.ini), and sending perf_pending/ to perf_reports in
  nx2/addresses.ini (Bean-Ops' perf relay) with the version and release flag, oldest first; one the relay
  couldn't take stays for next time. Unticking deletes what is waiting.
- nx2/addresses.ini, addresses.py: the external addresses, out of the code.
- Both launchers: the tick on the launch page, and sending on a thread at the launcher's start
  (builder/linux/longshot_builder.py, builder/py/longshot_builder/perf.py and window.py).
- docs/performance-statistics.md, the README, docs/development.md.
- tests/test_perf_report.py.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_015KgSKoUqUvce5UXG89NBgi

fix(perf): the review's findings ... 88404d20bb

- Windows: the shipped launcher (builder/LongshotBuilder) has the Send performance statistics tick and
  sends what waits, as the Linux and Python builders do.
- Every launcher sends in a process of its own (nx2/perf_report.py start_sending, --send): Play closes the
  launcher at once, and a summary the relay took was left to be sent, and counted, again.
- Unticked while the game runs: the next summary finds the tick off, and the game stops measuring and writes
  nothing more.
- At most 48 summaries wait in perf_pending, counted on disk (a player who never opens the launcher).
- A lost device's summary is written once, not once per thread that sees the loss.
- Swapchain and program-failure counts, and the first frame's waits, start from the game's counts when
  measuring starts, not from 0.
- Movie pictures are counted as the game takes them (submissions are not pictures), so a movie still
  playing at a summary or at exit counts too.
- JSON text is never cut inside a UTF-8 character; the sender reads a broken one as a replacement.
- MapTotals::any covers every movie count; dead code removed.
- The video tests' fixtures get the new counter.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_015KgSKoUqUvce5UXG89NBgi

gab9281 force-pushed feat/perf-metrics from 88404d20bb to 6702528faf 2026-11-13 15:03:07 +00:00 Compare

gab9281 added 2 commits 2026-11-13 15:27:19 +00:00

docs(perf): the measured runs in docs/tested-configurations.md ... 4e1f3907ad

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_015KgSKoUqUvce5UXG89NBgi

fix(perf): the second review's findings ... 910b654999

- Hitch causes: a draw waiting for a program's translation (draw_stats3)
  counts as shader_translate, not pipeline_wait; program_pick's in-place
  translation is counted once (it is timed inside PROF_PROGRAM too); the
  jobs' translations behind the game no longer count at all.
- The host's map is cleared back at the title and file select, so that
  time is "Title and intro", not the last map's.
- The sender keeps the version the game wrote (the folder may have been
  updated since); a release only when it is the folder's build.
- The game folder from GetModuleFileNameW (vk_paths.h game_dir_path,
  shared with vk_dlss.cpp): any folder name works on Windows.
- atexit only tries the lock: ExitProcess may have ended its holder.
- A map entered just before a summary stays new for the next one.
- A .tmp left by an exit mid-write is removed at the next start, and
  when the tick is taken off.
- The fps_cap label uses the menu's own cap rule (menu_fps_cap).
- perf_frame takes fps_presented's draw waits instead of reading again.
- movie_report's comment is back above it.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_015KgSKoUqUvce5UXG89NBgi

gab9281 force-pushed feat/perf-metrics from 910b654999 to ac41919db9 2026-11-13 15:27:56 +00:00 Compare

gab9281 force-pushed feat/perf-metrics from ac41919db9 to 2c95213061 2026-11-13 15:28:09 +00:00 Compare

gab9281 added 1 commit 2026-11-13 16:54:46 +00:00

feat(perf): keep a copy of each summary sent, in perf_sent/ ... 9e6c4c61ff

Each summary the relay took is written, exactly as it was posted, to
the game folder's perf_sent/ (the newest 48), so a player or a tester
can read what went to the team. `nx2/perf_report.py <folder>` says how
many are kept. Unticking deletes them with what is waiting.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_015KgSKoUqUvce5UXG89NBgi

gab9281 referenced this pull request 2026-11-13 17:58:18 +00:00

fix(vk): the start-up bar counts the compute pipelines, not 100% through them #256

gab9281 force-pushed feat/perf-metrics from 9e6c4c61ff to edda0942ae 2026-11-13 18:56:47 +00:00 Compare

gab9281 changed title from WIP: feat(perf): opt-in performance statistics, sent from the launcher to the team's Grafana to feat(perf): opt-in performance statistics, sent from the launcher to the team's Grafana 2026-11-13 18:57:08 +00:00

gab9281 force-pushed feat/perf-metrics from edda0942ae to 65c63546ca 2026-11-13 19:08:01 +00:00 Compare

gab9281 force-pushed feat/perf-metrics from 65c63546ca to b271368d0b 2026-11-13 19:08:14 +00:00 Compare

gab9281 added 1 commit 2026-11-13 19:26:21 +00:00

fix(perf): a hitch is a frame twice the usual, not every frame of a game held under its cap ... e5d64b9d12

The summaries sent from a laptop held at about 19 fps in Hyrule Field under
the 60 cap counted 4,694 hitches in 5,269 frames: against the cap's 16.7 ms,
every 50 ms frame was one, and the dashboard showed 34,000 hitches an hour.
A hitch is now measured against the longer of the target and the recent
average frame (before this one); slow frames still show in the fps and the
frame-time histograms. The pacing miss keeps the cap's target.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_015KgSKoUqUvce5UXG89NBgi

Some checks failed

Hide checks

CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Has been skipped

Details

CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Has been skipped

Details

CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Has been skipped

Details

Third-party parts / The same as upstream (pull_request) Has been skipped

Details

CI / Conventional Commit title (pull_request) Successful in 4s

Details

PR description / Description follows the template (pull_request_target) Has been cancelled

Details

CI / Workflow lint (actionlint) (pull_request) Successful in 5s

Details

Third-party parts / As their notes say (pull_request) Successful in 7s

Details

This pull request doesn't have enough approvals yet. 0 of 1 approvals granted.

You are not authorized to merge this pull request.

View command line instructions

Checkout

From your project repository, check out a new branch and test the changes.

git fetch -u 'feat/perf-metrics:gab9281-feat/perf-metrics'

git switch 'gab9281-feat/perf-metrics'

Sign in to join this conversation.

Reviewers

No reviewers

Labels

Clear labels   

area/accessibility

Barriers for people with disabilities

  

area/audio

Sound and music

  

area/input

Controllers, keyboard, mouse

  

area/install

Installing, updating, building from your game

  

area/menu

In-game menu and settings

  

area/renderer

Vulkan renderer, shaders, effects

  

area/saves

Save files

  

area/upscaling

DLSS, FSR, frame generation

  

good first issue

A good start for a new contributor

  

gpu/amd

AMD GPUs

  

gpu/igpu

Integrated GPUs

  

gpu/intel

Intel GPUs

  

gpu/nvidia

NVIDIA GPUs

  

help wanted

Extra hands or knowledge wanted

  

kind

bug

Something doesn't work as it should

  

kind

chore

Build, CI, dependencies, tidying

  

kind

docs

Documentation

  

kind

feature

A new feature or an improvement

  

kind

performance

Frame rate, frame time, loading, memory

  

kind

question

A question rather than a change

  

needs measurements

Waiting for before/after measurements

  

platform/linux

Linux only

  

platform/steam-deck

Steam Deck

  

platform/windows

Windows only

  

preview

Maintainers: deploy a preview of this PR's website (read its changes first). Removed after the run.

  

priority

critical

Crashes, data loss, or blocks most players

  

priority

high

Important: next in line

  

priority

low

Nice to have

  

status

blocked

Waits on something else first

  

status

confirmed

Reproduced or agreed on

  

status

duplicate

Already reported or proposed

  

status

invalid

Not a problem in this project

  

status

needs-info

Waiting for more information from the reporter

  

status

wontfix

Will not be worked on

  

testing wanted

Wanted: a test on other hardware (posts a test request in #test-requests)

No labels

area/accessibility

area/audio

area/input

area/install

area/menu

area/renderer

area/saves

area/upscaling

good first issue

gpu/amd

gpu/igpu

gpu/intel

gpu/nvidia

help wanted

kind

bug

kind

chore

kind

docs

kind

feature

kind

performance

kind

question

needs measurements

platform/linux

platform/steam-deck

platform/windows

preview

priority

critical

priority

high

priority

low

status

blocked

status

confirmed

status

duplicate

status

invalid

status

needs-info

status

wontfix

testing wanted

Milestone

Clear milestone

No items

No milestone

Projects

Clear projects

No items

No project

Assignees

Clear assignees

No assignees

1 participant

Notifications

Subscribe

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference

MagicBeanDev/Longshot!206

No description provided.