feat(control): an overlay command shows a test player's decisions beside the FPS counter #1358

Open
mooki wants to merge 5 commits from mooki/Longshot:control-overlay-text into main
Member

Type: feat · Risk: medium · Tested on: Linux, RTX 4090, headless (game-free checks; in-game pictures: see Testing) · Needs: Windows (build and run), a real window

Why

The test player (the "pilot", issue #1198) decides what to do next many times a minute, and until now the only way to see why was to read its run files afterwards. Someone watching a run, a recording or the game's own pictures could see the frame times (the FPS counter) but not the decisions. This adds a control-bridge command that puts a short text panel on the game's screen, drawn in the same overlay pass as the FPS counter, so both are visible at once. The pilot uses it to show its goal, the candidates with the chosen one marked, the step and its result, model and cost.

What changed

A client sends {"cmd":"overlay","text":"...","seconds":N} (overlay_clear or an empty text clears). The bridge checks and cleans the text (at most 2048 bytes and 16 lines of valid UTF-8, control characters dropped), keeps it only while the sending client is connected and for at most seconds, and hands it to the renderer, which draws it beside the FPS counter at every counter level and alone with the counter Off. Nothing is saved, logged or sent anywhere, and no game state changes.

  • host/control.c, host/control.h: the two commands, owner and expiry on the bridge's thread, unavailable without the renderer's hook.
  • gpu/vk/overlay_text.h (new, plain C): the text rules, used by the bridge and run again by the renderer.
  • gpu/vk/vk_api.h, host/rx_backend.c, gpu/vk/vk_replay.cpp: MkwVkRenderer.overlay_text, MKWVK_API_VERSION 29 to 30.
  • gpu/vk/fps_stats.h: the counter's column height, width and corner box as functions (the sums moved out of draw_fps; a test checks they agree with the old ones to the pixel); draw_fps publishes the box it drew.
  • gpu/vk/overlay_panel.h (new): the text store (its own lock, not the menu's) and the placement: in the counter's corner, under it (over it from a bottom corner) only while the panel ends within 70 % of the window's height, else beside it toward the middle, never over it, shrinking to 70 % or cutting lines narrower before giving up (and under it past that reach only when nothing else fits).
  • gpu/vk/vk_menu.cpp: draw_pilot_panel after draw_fps in both record paths (the window's and the headless RX_VK_SHOT_MENU pictures'); a translucent plate, the menu's font and Menu size scale; a new text size is measured once before the layout is settled and a line is cut to "..." only when it is more than 2 % over its room; a line starting with > is gold; not drawn while the Insert menu is open.
  • scripts/longshot_ctl.py: overlay LINE... [--seconds N] and --clear.
  • docs/control.md (the command, limits, privacy, pictures), docs/development.md (a row for the existing RX_VK_SHOT_MENU=1).
  • Unchanged: the bridge's shot (no overlay, so a picture sent to a vision model never carries the panel's text), no new switch, nothing when no client sends overlay.
  • The pilot side (a separate local tool, not in this repository) sends the text after each decision and each step result.

Measurements

Verdict: nothing to measure: this adds no work to a frame unless a client has set a panel (one atomic read in the overlay pass). No speed claim is made and none was measured.

Flags and settings

None: no new switch, no menu row and no ini key (the panel is drawn by gpu/vk/vk_menu.cpp's overlay pass but has no setting of its own). RX_VK_SHOT_MENU=1, an existing switch, is now documented in docs/development.md; it is what puts the FPS counter and the panel into a headless run's RX_VK_SHOTS pictures.

Testing

Tested: game-free checks on Linux (base main @ 6d72ca93b, head feat/control-overlay-text @ 38f10348f), a Windows-target syntax check and cross-build, and one windowless run of the game with the pilot (below):

  • tests/test_control.py: new cases against the fake game over the real socket (parsing, the 2048-byte and 16-line caps, UTF-8, dropped controls, expiry once, a later deadline, the owner's disconnect, unavailable). Red first: it did not compile without the header, then failed at the first overlay case with unknown_cmd; green after.
  • tests/test_overlay_panel.py: the text store under a concurrent writer and reader, the counter's column against the old sums, and 51,840 layouts (15 window shapes from 640x360 to 5120x1440 including 4:3, 5:4 and 21:9, three Menu sizes, four counter sizes, six counter levels, four corners): every placed panel is inside the window and clear of the counter; at 4:3 and wider a place is always found.
  • tests/test_longshot_ctl.py (the client, 52 tests).
  • The Windows-target syntax check (a local script that compiles each test's C fixture for Windows, llvm-mingw 20261104, idle gate, base main @ 6d72ca93b, head 38f10348f): 7 captured translation units, 1 failure, 1 unjudged driver. Compile for Windows: tests/test_control.py (control_test.c, host/control.c), tests/test_control_pipe_security.py, tests/test_fps_overlay_order.py, tests/test_overlay_panel.py (it defines _CRT_SECURE_NO_WARNINGS). The one failure, tests/test_audio_output_devices.py (memmove/tmpfile deprecated in parser.c: the same four error: lines on pristine main @ 6d72ca93b; this PR only moved the pinned API number in it); unjudged: tests/test_longshot_ctl.py (it starts the fake bridge with a direct Popen). Windows execution was not run.
  • The Windows cross-build on Linux (a local script that runs the project's own build plan for Windows, the production flags, llvm-mingw 20261104 / Clang 23.1.5, head 38f10348f, clean checkout): exit 0, 421 compile and link commands (103 new, 318 from matching cache entries), 0 failed, 0 warnings (under the build's own -Wno-everything: not a warning-clean claim). The renderer's 210 units compiled and longshot_vk.dll linked (14,367,744 B); the host group's 210 units compiled, including the changed host/control.c and host/rx_backend.c, and gpu/vk/vk_menu.cpp and gpu/vk/vk_replay.cpp compiled new. The game executable was not linked (no Linux lift registered for this head to link it from) and nothing was run on Windows or Wine.
  • tests/run_all.py (--jobs 6 --strict, fresh HOME, idle gate, both runs at the same time on a loaded PC): main @ 6d72ca93b 662 passed, 41 failed, 17 skipped in 319 s; this branch @ 38f10348f 663 passed, 41 failed, 17 skipped in 310 s. The failing files are the same 41 on both, compared by name (none fails only here): test_bug_report_defaults, test_cpu_report, test_crash_record, test_crt_options, test_damage_debug_menu, test_display_startup_failure, test_dlss_feature_size, test_failed_compile_keeps_objects, test_framegen_preset, test_fsr_uniform_ownership, test_guest_imported, test_hud_layout_row, test_import_reach, test_invalidate_vcore, test_linux_preflight, test_linux_pressure, test_linux_update, test_mirror_budget, test_missing_nup, test_mods_size_table, test_nonuniform_image_arrays, test_parallel_init_order, test_pending_guest_import, test_pipeline_play_cache, test_present_thread, test_profiling_mode, test_readback_timeout_source, test_replay_write_order, test_rumble_menu, test_run_all_changed, test_slow_frames, test_staging_burst, test_switch_table, test_target_image_pool, test_tested_configurations, test_upscaler_round_trip, test_upscaler_rows, test_vcore_escape, test_vk_occlusion_queries, test_vk_swapchain_compat, test_walk_key (they fail on this PC's main as it stands, for example test_import_reach on a std::vector::contains compile error with the system's clang++; not looked into further).
  • Built on this PC (Linux, normal nx2/make_game.py build, the rebased head 38f10348f): v3.0.0-rc.6-9-g38f10348f (source af14c9ad), longshot 546,801,056 B sha256 30c67c67939278304b9e345d1ed07d48057d82a7eb05a7e9ddbcb2d846b23026, longshot_vk.so 15,573,464 B sha256 4a3a176377ca14606928e58e7b62b614940bae8e89fe0925ecb805ae7a965702; 169 s, 96 objects new and 422 from the object cache of the earlier build (main's lift changed, so it was lifted again). The in-game pictures below come from this build.

In the game, on the rebased build (Linux 7.2.15, RTX 4090, driver 615.78.08, 2026-11-16, headless; the game's own first log line names the build: longshot v3.0.0-rc.6-9-g38f10348f (source af14c9ad), built from 38f10348f). This branch had been played before at an earlier head on an older main (c4aca39d3 on ac7787515); main has moved 37 commits since, so the game was built and played again after the rebase, and this is that run. One run of the pilot (hud3-talk) in Kokiri Forest, save explore17-kokiri-child: it walks to a Kokiri girl and talks to her, then goes to the signboard and reads it (rules decider, so no model call). Shader preparation first (45 s, exit 0, @@done shaders from the game itself), then the run: the pilot's run took 84.8 s and ended solved, the game ran 83.2 s, RX_VK_SHOT_MENU=1, a picture every 45 presents, 1280x720, 210 pictures. The game's log has control: overlay panel set (11 lines, 235 bytes; it goes with its client or its seconds) once and control: overlay panel cleared (asked) at the end. The wrapper's summary line: exit=143 (the pilot's own stop at the end of its run) fatal=0 crash=0 devlost=0 nocore=0 kernel=0 nvrm=0 journal=complete, and coredumpctl lists no core dump since the run started; its HOME-WRITES note names ~/.cache/longshot-builder, an existing empty folder from 13 November (the note tests existence only). Other work was running on the PC (load 12.7 at the start, 7.3 at the end), so the frame rates in the pictures say nothing about speed.

While the pilot played, a second small program changed the FPS counter's level and the Menu size over the run's own control socket, nine seconds per phase, in this order: Graph Medium, Breakdown Medium, Off Medium, Off Large, Graph Large (Breakdown Large, FPS and Frame time were next; the pilot solved its segment and the game was stopped before them). It recorded the renderer present of each change, so every picture is tied to its phase. 53 of the 210 pictures lie inside those phases (Graph Medium 11, Breakdown Medium 12, Off Medium 12, Off Large 12, Graph Large 6); the other 157 are the start-up (black, the opening field in moonlight, the title, the load menu, black again), with neither counter nor panel. All 210 were looked at: the 53 in sets of four at 800x450, the 157 in sets of twenty, and at full size the first picture after the size change and one picture of each of the other phases. What was seen:

  • Graph at Medium (11): the first five pictures, before the pilot's first decision (the Kokiri Forest title card is on screen), have the counter alone; from vk_007335, the first decision, the panel is under the counter, its lower edge at about y 335 of 720, with the hearts (top left) and the map (lower right) drawn and clear of it.
  • Breakdown at Medium (12; the first fault of the first run, fixed): the panel is beside the tall counter, to its left, its lower edge at about y 236 of 720 (the first run had it under the counter, down to y 685, in the corner where the map is drawn). The first picture of the phase is a walking scene and shows the hearts and the map clear of the panel; in the conversation that follows the map and hearts are hidden. All twelve look the same.
  • Counter Off at Medium (12): the panel alone in the counter's corner, every line whole, the dialogue box and its choices below it clear.
  • The size change Medium to Large with the counter Off (the case that had a line cut to "..." in the first run, the second fault, fixed): vk_008685, the first picture after the change (7 presents after it), has every line whole; its top line, GOAL talk-to-the-kokiri [open] FOR kokiri-013 (45 characters, the longest top line of this run), ends inside the plate. In the first run the same change with the same counter state (vk_005985) had a top line cut to "...". The other eleven Off-Large pictures have the signboard goal's lines whole (GOAL read-the-sign [open] FOR signboard-010, then [met] with a RUN solved row, up to 15 lines with +2 more). No picture of the 53 has a panel line cut to "...". The top lines of the two runs differ (the first run's was the signboard goal's, this run's the Kokiri talk goal's), so this is the same change but not the same text.
  • Graph at Large (6): from vk_009270 the panel is beside the counter (15 lines, RUN solved, +2 more), readable at the larger size, its right edge just left of the counter's, the hearts and the map clear. The first picture of the phase, vk_009225 (11 presents after the counter went from Off to Graph), has the counter still filling in (no frame-time row yet) and the panel under it, clear of the counter; the next picture has the counter whole and the panel beside it. A panel goes under the counter only while it ends within 70 % of the window's height, and with the counter's second row in, 15 lines at Large end past that, which I take to be why it moved (not measured): one picture of the six.

the panel at four counter states and two sizes

1. Graph at Medium (a walking scene): the panel under the counter, hearts and map clear. 2. Breakdown at Medium: beside the tall counter. 3. Counter Off at Medium: the panel alone, the dialogue's choices clear. 4. Counter Off, the first picture after the change to Large: every line whole. 5. Graph at Large: beside the counter, 15 lines, readable.

before and after the last commit

1 and 3 are from the first run (06096442a), 2 and 4 from this one (38f10348f). 1 and 2: Breakdown at Medium, under the counter down to the corner, and now beside it. 3 and 4: the first picture after the same size change (Medium to Large, counter Off): then a top line cut to "..." although it fitted, now every line whole (the top lines differ: the first run's was the signboard goal's, this run's the Kokiri talk goal's, which is longer).

Not tested: a real window (this PC's runs are windowless: the window path calls the same draw_pilot_panel, unseen); Windows (syntax and cross-build only, nothing run); the pilot's model rows (model, latency, cost) in the game: the run used the rules decider; Breakdown at Large, FPS and Frame time with the panel on this build (the run ended before those phases; the two earlier runs, at 06096442a and c4aca39d3 on the older main, saw Breakdown at Large and the FPS levels, but not on this base); the panel with the Insert menu open (hidden by design, not looked at).

Must be tested:

  • Windows, any GPU: build the game (the Linux cross-build here linked the renderer DLL but not the game executable), run the control tests, and send longshot_ctl.py overlay "hello" --seconds 30 in a window with the FPS counter on Graph, expecting the panel under the counter, every line whole and no overlap with the counter, the hearts or the map (by hand: look at the corner).
  • Linux with a real window (Wayland or X11), any GPU: at 4K and at 1280x720 with the Large Menu size, send longshot_ctl.py overlay "hello" --seconds 30 with the FPS counter in each corner, expecting the panel beside or under the counter, never over it, and inside the window (by hand: look at each corner).

Risks

What could break: draw_fps's height sum moved into fps_stats.h (checked to the pixel by a test, so the counter should look as before); a renderer built before this change with a newer host (or the other way round) is refused by the version check, as for every API bump. A client that keeps sending text can keep a panel on the screen over the game: the Insert menu hides it while open, and it goes with the client.

Turning it off: nothing is drawn unless a client sends overlay; revert the PR's commits to remove it.

Compatibility: MKWVK_API_VERSION 29 to 30 (a stale renderer library is refused); nothing changes for saves, longshot_options.ini or the caches. The number can conflict: main is at 29 (the fullscreen monitor choice, #1277) and this change takes 30, while the co-op playtest PR 355 is at 30 and PR 68 at 31 on their own branches (read from the local refs gs-pr/355 257463464 and gs-pr/68 79c9d411f), so whichever of them merges after another renumbers: a one-line change in gpu/vk/vk_api.h plus the one test that pins the exact number, tests/test_audio_output_devices.py (_Static_assert(MKWVK_API_VERSION == 30, ...); this PR moved it from 29 to 30, and tests/test_overlay_panel.py only asks for 30 or more).

Checklist

Everyone

  • One topic.
  • The title is a Conventional Commit (type(scope): description) of the status line's type.
  • Why gives the reason plainly and links the issues that fit (the tracker searched).
  • Measurements has a verdict, or one line saying why there is nothing to measure.
  • Testing names what was tested, what wasn't, and what must still be tested, where and how.
  • Rebased on current main (544229a26). The tests, builds and the game run above are on 38f10348f over main @ 6d72ca93b; main then gained one commit (#1332: gpu/vk/vk_main.cpp and a new test, no file shared with this branch, renderer API still 29), and after rebasing onto it test_control, test_overlay_panel, test_audio_output_devices and test_fps_overlay_order pass again. No rebuild on the new base.
  • python -I -B tests/run_all.py passes (any failure that also fails on main is named).
  • No game files, dumps, build output or generated files.
  • third_party/ untouched (a new folder with a PROVENANCE.md instead).
  • The license is OK: GPL-3.0-or-later, nothing incompatible and nothing decompiled.

When it applies

  • AI assistance recorded with Assisted-by: / Co-Authored-By: trailers.
  • Screenshots of the same spot, captioned like the Measurements columns, for a change that shows on screen.
  • MKWVK_API_VERSION bumped (gpu/vk/vk_api.h's MkwVkHost or MkwVkRenderer changed).
  • A game-free test added for the fix, failing on main and passing here.
  • A check under tests/ that reads changed source updated in this PR, not weakened.
  • Host code builds on both Windows (llvm-mingw) and Linux, with UTF-8 paths.
  • Docs updated.

New on this forge; not opened on GitHub.

**Type:** feat · **Risk:** medium · **Tested on:** Linux, RTX 4090, headless (game-free checks; in-game pictures: see Testing) · **Needs:** Windows (build and run), a real window ## Why The test player (the "pilot", issue #1198) decides what to do next many times a minute, and until now the only way to see why was to read its run files afterwards. Someone watching a run, a recording or the game's own pictures could see the frame times (the FPS counter) but not the decisions. This adds a control-bridge command that puts a short text panel on the game's screen, drawn in the same overlay pass as the FPS counter, so both are visible at once. The pilot uses it to show its goal, the candidates with the chosen one marked, the step and its result, model and cost. ## What changed A client sends `{"cmd":"overlay","text":"...","seconds":N}` (`overlay_clear` or an empty text clears). The bridge checks and cleans the text (at most 2048 bytes and 16 lines of valid UTF-8, control characters dropped), keeps it only while the sending client is connected and for at most `seconds`, and hands it to the renderer, which draws it beside the FPS counter at every counter level and alone with the counter Off. Nothing is saved, logged or sent anywhere, and no game state changes. - `host/control.c`, `host/control.h`: the two commands, owner and expiry on the bridge's thread, `unavailable` without the renderer's hook. - `gpu/vk/overlay_text.h` (new, plain C): the text rules, used by the bridge and run again by the renderer. - `gpu/vk/vk_api.h`, `host/rx_backend.c`, `gpu/vk/vk_replay.cpp`: `MkwVkRenderer.overlay_text`, **`MKWVK_API_VERSION` 29 to 30**. - `gpu/vk/fps_stats.h`: the counter's column height, width and corner box as functions (the sums moved out of `draw_fps`; a test checks they agree with the old ones to the pixel); `draw_fps` publishes the box it drew. - `gpu/vk/overlay_panel.h` (new): the text store (its own lock, not the menu's) and the placement: in the counter's corner, under it (over it from a bottom corner) only while the panel ends within 70 % of the window's height, else beside it toward the middle, never over it, shrinking to 70 % or cutting lines narrower before giving up (and under it past that reach only when nothing else fits). - `gpu/vk/vk_menu.cpp`: `draw_pilot_panel` after `draw_fps` in both record paths (the window's and the headless `RX_VK_SHOT_MENU` pictures'); a translucent plate, the menu's font and Menu size scale; a new text size is measured once before the layout is settled and a line is cut to "..." only when it is more than 2 % over its room; a line starting with `>` is gold; not drawn while the Insert menu is open. - `scripts/longshot_ctl.py`: `overlay LINE... [--seconds N]` and `--clear`. - `docs/control.md` (the command, limits, privacy, pictures), `docs/development.md` (a row for the existing `RX_VK_SHOT_MENU=1`). - Unchanged: the bridge's `shot` (no overlay, so a picture sent to a vision model never carries the panel's text), no new switch, nothing when no client sends `overlay`. - The pilot side (a separate local tool, not in this repository) sends the text after each decision and each step result. ## Measurements **Verdict:** nothing to measure: this adds no work to a frame unless a client has set a panel (one atomic read in the overlay pass). No speed claim is made and none was measured. ### Flags and settings None: no new switch, no menu row and no ini key (the panel is drawn by `gpu/vk/vk_menu.cpp`'s overlay pass but has no setting of its own). `RX_VK_SHOT_MENU=1`, an existing switch, is now documented in `docs/development.md`; it is what puts the FPS counter and the panel into a headless run's `RX_VK_SHOTS` pictures. ## Testing **Tested:** game-free checks on Linux (base `main @ 6d72ca93b`, head `feat/control-overlay-text @ 38f10348f`), a Windows-target syntax check and cross-build, and one windowless run of the game with the pilot (below): - `tests/test_control.py`: new cases against the fake game over the real socket (parsing, the 2048-byte and 16-line caps, UTF-8, dropped controls, expiry once, a later deadline, the owner's disconnect, `unavailable`). Red first: it did not compile without the header, then failed at the first overlay case with `unknown_cmd`; green after. - `tests/test_overlay_panel.py`: the text store under a concurrent writer and reader, the counter's column against the old sums, and 51,840 layouts (15 window shapes from 640x360 to 5120x1440 including 4:3, 5:4 and 21:9, three Menu sizes, four counter sizes, six counter levels, four corners): every placed panel is inside the window and clear of the counter; at 4:3 and wider a place is always found. - `tests/test_longshot_ctl.py` (the client, 52 tests). - The Windows-target syntax check (a local script that compiles each test's C fixture for Windows, llvm-mingw 20261104, idle gate, base `main @ 6d72ca93b`, head `38f10348f`): 7 captured translation units, 1 failure, 1 unjudged driver. Compile for Windows: `tests/test_control.py` (`control_test.c`, `host/control.c`), `tests/test_control_pipe_security.py`, `tests/test_fps_overlay_order.py`, `tests/test_overlay_panel.py` (it defines `_CRT_SECURE_NO_WARNINGS`). The one failure, `tests/test_audio_output_devices.py` (`memmove`/`tmpfile` deprecated in `parser.c`: the same four `error:` lines on pristine `main @ 6d72ca93b`; this PR only moved the pinned API number in it); unjudged: `tests/test_longshot_ctl.py` (it starts the fake bridge with a direct `Popen`). Windows execution was not run. - The Windows cross-build on Linux (a local script that runs the project's own build plan for Windows, the production flags, llvm-mingw 20261104 / Clang 23.1.5, head `38f10348f`, clean checkout): exit 0, 421 compile and link commands (103 new, 318 from matching cache entries), 0 failed, 0 warnings (under the build's own `-Wno-everything`: not a warning-clean claim). The renderer's 210 units compiled and `longshot_vk.dll` linked (14,367,744 B); the host group's 210 units compiled, including the changed `host/control.c` and `host/rx_backend.c`, and `gpu/vk/vk_menu.cpp` and `gpu/vk/vk_replay.cpp` compiled new. **The game executable was not linked** (no Linux lift registered for this head to link it from) and nothing was run on Windows or Wine. - `tests/run_all.py` (`--jobs 6 --strict`, fresh HOME, idle gate, both runs at the same time on a loaded PC): `main @ 6d72ca93b` 662 passed, 41 failed, 17 skipped in 319 s; this branch `@ 38f10348f` 663 passed, 41 failed, 17 skipped in 310 s. The failing files are the same 41 on both, compared by name (none fails only here): `test_bug_report_defaults`, `test_cpu_report`, `test_crash_record`, `test_crt_options`, `test_damage_debug_menu`, `test_display_startup_failure`, `test_dlss_feature_size`, `test_failed_compile_keeps_objects`, `test_framegen_preset`, `test_fsr_uniform_ownership`, `test_guest_imported`, `test_hud_layout_row`, `test_import_reach`, `test_invalidate_vcore`, `test_linux_preflight`, `test_linux_pressure`, `test_linux_update`, `test_mirror_budget`, `test_missing_nup`, `test_mods_size_table`, `test_nonuniform_image_arrays`, `test_parallel_init_order`, `test_pending_guest_import`, `test_pipeline_play_cache`, `test_present_thread`, `test_profiling_mode`, `test_readback_timeout_source`, `test_replay_write_order`, `test_rumble_menu`, `test_run_all_changed`, `test_slow_frames`, `test_staging_burst`, `test_switch_table`, `test_target_image_pool`, `test_tested_configurations`, `test_upscaler_round_trip`, `test_upscaler_rows`, `test_vcore_escape`, `test_vk_occlusion_queries`, `test_vk_swapchain_compat`, `test_walk_key` (they fail on this PC's `main` as it stands, for example `test_import_reach` on a `std::vector::contains` compile error with the system's `clang++`; not looked into further). - Built on this PC (Linux, normal `nx2/make_game.py` build, the rebased head `38f10348f`): `v3.0.0-rc.6-9-g38f10348f (source af14c9ad)`, `longshot` 546,801,056 B sha256 `30c67c67939278304b9e345d1ed07d48057d82a7eb05a7e9ddbcb2d846b23026`, `longshot_vk.so` 15,573,464 B sha256 `4a3a176377ca14606928e58e7b62b614940bae8e89fe0925ecb805ae7a965702`; 169 s, 96 objects new and 422 from the object cache of the earlier build (main's lift changed, so it was lifted again). The in-game pictures below come from this build. **In the game, on the rebased build** (Linux 7.2.15, RTX 4090, driver 615.78.08, 2026-11-16, headless; the game's own first log line names the build: `longshot v3.0.0-rc.6-9-g38f10348f (source af14c9ad)`, built from `38f10348f`). This branch had been played before at an earlier head on an older `main` (`c4aca39d3` on `ac7787515`); `main` has moved 37 commits since, so the game was built and played again after the rebase, and this is that run. One run of the pilot (`hud3-talk`) in Kokiri Forest, save `explore17-kokiri-child`: it walks to a Kokiri girl and talks to her, then goes to the signboard and reads it (rules decider, so no model call). Shader preparation first (45 s, exit 0, `@@done shaders` from the game itself), then the run: the pilot's run took 84.8 s and ended solved, the game ran 83.2 s, `RX_VK_SHOT_MENU=1`, a picture every 45 presents, 1280x720, 210 pictures. The game's log has `control: overlay panel set (11 lines, 235 bytes; it goes with its client or its seconds)` once and `control: overlay panel cleared (asked)` at the end. The wrapper's summary line: `exit=143` (the pilot's own stop at the end of its run) `fatal=0 crash=0 devlost=0 nocore=0 kernel=0 nvrm=0 journal=complete`, and `coredumpctl` lists no core dump since the run started; its `HOME-WRITES` note names `~/.cache/longshot-builder`, an existing empty folder from 13 November (the note tests existence only). Other work was running on the PC (load 12.7 at the start, 7.3 at the end), so the frame rates in the pictures say nothing about speed. While the pilot played, a second small program changed the FPS counter's level and the Menu size over the run's own control socket, nine seconds per phase, in this order: Graph Medium, Breakdown Medium, Off Medium, Off Large, Graph Large (Breakdown Large, FPS and Frame time were next; the pilot solved its segment and the game was stopped before them). It recorded the renderer present of each change, so every picture is tied to its phase. 53 of the 210 pictures lie inside those phases (Graph Medium 11, Breakdown Medium 12, Off Medium 12, Off Large 12, Graph Large 6); the other 157 are the start-up (black, the opening field in moonlight, the title, the load menu, black again), with neither counter nor panel. **All 210 were looked at**: the 53 in sets of four at 800x450, the 157 in sets of twenty, and at full size the first picture after the size change and one picture of each of the other phases. What was seen: - **Graph at Medium** (11): the first five pictures, before the pilot's first decision (the Kokiri Forest title card is on screen), have the counter alone; from `vk_007335`, the first decision, the panel is under the counter, its lower edge at about y 335 of 720, with the hearts (top left) and the map (lower right) drawn and clear of it. - **Breakdown at Medium** (12; the first fault of the first run, fixed): the panel is **beside** the tall counter, to its left, its lower edge at about y 236 of 720 (the first run had it under the counter, down to y 685, in the corner where the map is drawn). The first picture of the phase is a walking scene and shows the hearts and the map clear of the panel; in the conversation that follows the map and hearts are hidden. All twelve look the same. - **Counter Off at Medium** (12): the panel alone in the counter's corner, every line whole, the dialogue box and its choices below it clear. - **The size change Medium to Large with the counter Off** (the case that had a line cut to "..." in the first run, the second fault, fixed): `vk_008685`, the first picture after the change (7 presents after it), has every line whole; its top line, `GOAL talk-to-the-kokiri [open] FOR kokiri-013` (45 characters, the longest top line of this run), ends inside the plate. In the first run the same change with the same counter state (`vk_005985`) had a top line cut to "...". The other eleven Off-Large pictures have the signboard goal's lines whole (`GOAL read-the-sign [open] FOR signboard-010`, then `[met]` with a `RUN solved` row, up to 15 lines with `+2 more`). **No picture of the 53 has a panel line cut to "..."**. The top lines of the two runs differ (the first run's was the signboard goal's, this run's the Kokiri talk goal's), so this is the same change but not the same text. - **Graph at Large** (6): from `vk_009270` the panel is beside the counter (15 lines, `RUN solved`, `+2 more`), readable at the larger size, its right edge just left of the counter's, the hearts and the map clear. The first picture of the phase, `vk_009225` (11 presents after the counter went from Off to Graph), has the counter still filling in (no frame-time row yet) and the panel under it, clear of the counter; the next picture has the counter whole and the panel beside it. A panel goes under the counter only while it ends within 70 % of the window's height, and with the counter's second row in, 15 lines at Large end past that, which I take to be why it moved (not measured): one picture of the six. ![the panel at four counter states and two sizes](https://git.magicbean.dev/attachments/48eb04f1-4c53-4071-83a6-f7f4781df369) *1. Graph at Medium (a walking scene): the panel under the counter, hearts and map clear. 2. Breakdown at Medium: beside the tall counter. 3. Counter Off at Medium: the panel alone, the dialogue's choices clear. 4. Counter Off, the first picture after the change to Large: every line whole. 5. Graph at Large: beside the counter, 15 lines, readable.* ![before and after the last commit](https://git.magicbean.dev/attachments/9b5cbbed-4a47-47f4-b1fd-8f83fda8c924) *1 and 3 are from the first run (`06096442a`), 2 and 4 from this one (`38f10348f`). 1 and 2: Breakdown at Medium, under the counter down to the corner, and now beside it. 3 and 4: the first picture after the same size change (Medium to Large, counter Off): then a top line cut to "..." although it fitted, now every line whole (the top lines differ: the first run's was the signboard goal's, this run's the Kokiri talk goal's, which is longer).* **Not tested:** a real window (this PC's runs are windowless: the window path calls the same `draw_pilot_panel`, unseen); Windows (syntax and cross-build only, nothing run); the pilot's model rows (model, latency, cost) in the game: the run used the rules decider; Breakdown at Large, FPS and Frame time with the panel on this build (the run ended before those phases; the two earlier runs, at `06096442a` and `c4aca39d3` on the older `main`, saw Breakdown at Large and the FPS levels, but not on this base); the panel with the Insert menu open (hidden by design, not looked at). **Must be tested:** - Windows, any GPU: build the game (the Linux cross-build here linked the renderer DLL but not the game executable), run the control tests, and send `longshot_ctl.py overlay "hello" --seconds 30` in a window with the FPS counter on Graph, expecting the panel under the counter, every line whole and no overlap with the counter, the hearts or the map (by hand: look at the corner). - Linux with a real window (Wayland or X11), any GPU: at 4K and at 1280x720 with the Large Menu size, send `longshot_ctl.py overlay "hello" --seconds 30` with the FPS counter in each corner, expecting the panel beside or under the counter, never over it, and inside the window (by hand: look at each corner). ## Risks **What could break:** `draw_fps`'s height sum moved into `fps_stats.h` (checked to the pixel by a test, so the counter should look as before); a renderer built before this change with a newer host (or the other way round) is refused by the version check, as for every API bump. A client that keeps sending text can keep a panel on the screen over the game: the Insert menu hides it while open, and it goes with the client. **Turning it off:** nothing is drawn unless a client sends `overlay`; revert the PR's commits to remove it. **Compatibility:** `MKWVK_API_VERSION` 29 to 30 (a stale renderer library is refused); nothing changes for saves, `longshot_options.ini` or the caches. **The number can conflict:** `main` is at 29 (the fullscreen monitor choice, #1277) and this change takes 30, while the co-op playtest PR 355 is at 30 and PR 68 at 31 on their own branches (read from the local refs `gs-pr/355` `257463464` and `gs-pr/68` `79c9d411f`), so whichever of them merges after another renumbers: a one-line change in `gpu/vk/vk_api.h` plus the one test that pins the exact number, `tests/test_audio_output_devices.py` (`_Static_assert(MKWVK_API_VERSION == 30, ...)`; this PR moved it from 29 to 30, and `tests/test_overlay_panel.py` only asks for 30 or more). ## Checklist ### Everyone - [x] One topic. - [x] The title is a Conventional Commit (`type(scope): description`) of the status line's type. - [x] Why gives the reason plainly and links the issues that fit (the tracker searched). - [x] Measurements has a verdict, or one line saying why there is nothing to measure. - [x] Testing names what was tested, what wasn't, and what must still be tested, where and how. - [x] Rebased on current `main` (544229a26). The tests, builds and the game run above are on `38f10348f` over `main @ 6d72ca93b`; `main` then gained one commit (#1332: `gpu/vk/vk_main.cpp` and a new test, no file shared with this branch, renderer API still 29), and after rebasing onto it `test_control`, `test_overlay_panel`, `test_audio_output_devices` and `test_fps_overlay_order` pass again. No rebuild on the new base. - [x] `python -I -B tests/run_all.py` passes (any failure that also fails on `main` is named). - [x] No game files, dumps, build output or generated files. - [x] `third_party/` untouched (a new folder with a `PROVENANCE.md` instead). - [x] The license is OK: GPL-3.0-or-later, nothing incompatible and nothing decompiled. ### When it applies - [x] AI assistance recorded with `Assisted-by:` / `Co-Authored-By:` trailers. - [x] Screenshots of the same spot, captioned like the Measurements columns, for a change that shows on screen. - [x] `MKWVK_API_VERSION` bumped (`gpu/vk/vk_api.h`'s `MkwVkHost` or `MkwVkRenderer` changed). - [x] A game-free test added for the fix, failing on `main` and passing here. - [x] A check under `tests/` that reads changed source updated in this PR, not weakened. - [x] Host code builds on both Windows (llvm-mingw) and Linux, with UTF-8 paths. - [x] Docs updated. New on this forge; not opened on GitHub.
A client of the control bridge can set a short multi-line panel (at most 2048 bytes of UTF-8 and 16 lines, control characters dropped) that the renderer draws every frame: a test player shows what it decides, next to the frame times. It goes when its seconds are over, when the client that set it disconnects, or when it is cleared; it is never saved, logged or sent anywhere, and changes nothing in the game.

- host/control.c: overlay and overlay_clear; the text rules live in gpu/vk/overlay_text.h (the renderer runs the same function over what it is handed); expiry and owner on the bridge's thread; unavailable when the renderer has no hook
- gpu/vk/vk_api.h: MkwVkRenderer.overlay_text, MKWVK_API_VERSION 29 -> 30 (host and renderer compare it); host/rx_backend.c's wrapper
- scripts/longshot_ctl.py: overlay LINE... [--seconds N] and --clear (the panel goes with the connection, so the command holds it)
- tests: control_test.c (the text rules; parsing, the caps, expiry once, a later deadline, owner's disconnect, unavailable), the client against its fake bridge, the pinned version in test_audio_output_devices.py
- docs/control.md

The renderer's drawing follows in the next commit.

Assisted-by: Claude Sonnet 5.5
Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
The renderer draws the control bridge's overlay text (the commit before) in the overlay pass of the FPS counter, so both show at once at every counter level, and with the counter Off. It sits in the counter's corner, under it (over it from a bottom corner) or, where there is no room, beside it, never over it; a translucent plate, the menu's font and its Menu size scale; not while the Insert menu is open.

- gpu/vk/fps_stats.h: the counter's column height, width and box as functions (draw_fps's own sums, moved; a test checks they agree to the pixel); draw_fps publishes the box it drew
- gpu/vk/overlay_panel.h: the text store (its own lock, never the menu's) and the placement, swept by tests/test_overlay_panel.cpp at every level, corner, counter size and Menu size over many window shapes
- gpu/vk/vk_menu.cpp: draw_pilot_panel after draw_fps in both record paths (the window's, and the headless RX_VK_SHOT_MENU pictures); a panel alone keeps the frame from being skipped; menu_overlay_text is the renderer API's hook (vk_replay.cpp)
- docs/development.md: RX_VK_SHOT_MENU, the switch that puts the counter and the panel into a headless run's pictures

Assisted-by: Claude Sonnet 5.5
Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
llvm-mingw marks memcpy deprecated without it; the Windows-target syntax check failed the new fixture, passes now.

Assisted-by: Claude Sonnet 5.5
Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
tests/test_fps_overlay_order.py runs the real menu_record with stubs; with the panel's text set the old fixture did not build (no pilot_store, draw_pilot_panel). It now has them and a case for every combination. Against main's vk_menu.cpp (no panel) the new case fails; the suite on the branch had this file as its one failure beyond main's 19.

Assisted-by: Claude Sonnet 5.5
Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
fix(vk): the overlay panel stays off the HUD's map, and a line that fits is not cut after a Menu size change
Some checks failed
CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Has been skipped
CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Has been skipped
CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Has been skipped
Third-party parts / The same as upstream (pull_request) Has been skipped
Website / Website on Cloudflare (pull_request) Has been skipped
CI / Conventional Commit title (pull_request) Successful in 8s
Guide screenshots / Guide images and local UI source review (pull_request) Failing after 16s
CI / Workflow lint (actionlint) (pull_request) Successful in 17s
Website / Website on Cloudflare (pull_request_target) Has been skipped
Third-party parts / As their notes say (pull_request) Successful in 19s
PR description / Description follows the template (pull_request_target) Successful in 11s
8e1f34ab45
Found by looking at the first in-game pictures (headless, RX_VK_SHOT_MENU=1):
- With the counter at Breakdown and the Medium Menu size the panel went under the counter and reached the corner where the HUD's map is. It now goes under the counter only while it ends within 70% of the window's height (mirrored from a bottom corner) and otherwise beside it; only when nothing else fits is it put under the counter past that.
- The first pictures after a Menu size change had a line ending in '...' that fitted (the first measure of a new size was a pixel or two short of what is drawn). A new size is measured once to settle, and a line is cut only past 2% over its room.

tests/test_overlay_panel.cpp first: a new property in the sweep (at 16:9 and wider, the counter's own size, up to Breakdown: under the counter only within reach) failed at 640x360 Menu size 1 Breakdown (the panel ended at 257 of 360) before the change, passes after; the 51,840 layouts are otherwise unchanged. The draw change has a wiring check; it was not re-seen in the game yet.

Assisted-by: Claude Sonnet 5.5
Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
mooki changed title from WIP: feat(control): an overlay command shows a test player's decisions beside the FPS counter to feat(control): an overlay command shows a test player's decisions beside the FPS counter 2026-11-17 05:55:06 +00:00
TrashPanda left a comment

A client can put a short text panel beside the FPS counter, and it leaves with that client or with its seconds. I read head 8e1f34ab. It applies onto current main. The window runs it still names are open, so this is not an approval.

The bridge keeps at most 2048 bytes and 16 lines of UTF-8, drops control characters, and clears the panel when that client disconnects or when seconds passes. The renderer draws it after the counter, in both the window and the headless RX_VK_SHOT_MENU pictures, and not while the Insert menu or the start-up screen is up. The bridge's own shot still records the picture without this panel. The panel sits under the counter while it ends within 70% of the window, and beside it otherwise, and a line is cut to "..." only when it is more than 2% over its room. MkwVkRenderer gained overlay_text, and the interface is 30. Main is still 29. If #1034 lands first, this bump is the next number after that one.

The headless Linux pictures show the panel clear of the counter, the hearts and the map. Must be tested still asks for a Windows window (the panel under Graph, every line whole) and a Linux window at 4K and at 1280x720, the counter in each corner.

A client can put a short text panel beside the FPS counter, and it leaves with that client or with its seconds. I read head `8e1f34ab`. It applies onto current main. The window runs it still names are open, so this is not an approval. The bridge keeps at most 2048 bytes and 16 lines of UTF-8, drops control characters, and clears the panel when that client disconnects or when `seconds` passes. The renderer draws it after the counter, in both the window and the headless `RX_VK_SHOT_MENU` pictures, and not while the Insert menu or the start-up screen is up. The bridge's own `shot` still records the picture without this panel. The panel sits under the counter while it ends within 70% of the window, and beside it otherwise, and a line is cut to "..." only when it is more than 2% over its room. `MkwVkRenderer` gained `overlay_text`, and the interface is 30. Main is still 29. If https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/1034 lands first, this bump is the next number after that one. The headless Linux pictures show the panel clear of the counter, the hearts and the map. Must be tested still asks for a Windows window (the panel under Graph, every line whole) and a Linux window at 4K and at 1280x720, the counter in each corner.
Some checks failed
CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Has been skipped
CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Has been skipped
CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Has been skipped
Third-party parts / The same as upstream (pull_request) Has been skipped
Website / Website on Cloudflare (pull_request) Has been skipped
CI / Conventional Commit title (pull_request) Successful in 8s
Guide screenshots / Guide images and local UI source review (pull_request) Failing after 16s
CI / Workflow lint (actionlint) (pull_request) Successful in 17s
Website / Website on Cloudflare (pull_request_target) Has been skipped
Third-party parts / As their notes say (pull_request) Successful in 19s
PR description / Description follows the template (pull_request_target) Successful in 11s
This pull request has changes conflicting with the target branch.
  • tests/test_audio_output_devices.py
View command line instructions

Manual merge helper

Use this merge commit message when completing the merge manually.

Checkout

From your project repository, check out a new branch and test the changes.
git fetch -u 'control-overlay-text:mooki-control-overlay-text'
git switch 'mooki-control-overlay-text'
Sign in to join this conversation.
No reviewers
No milestone
No project
No assignees
2 participants
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
MagicBeanDev/Longshot!1358
No description provided.