feat(control): an overlay command shows a test player's decisions beside the FPS counter #1358
No reviewers
Labels
No labels
area/accessibility
area/audio
area/input
area/install
area/menu
area/renderer
area/saves
area/tooling
area/upscaling
beanstalk: approved
beanstalk: failed
beanstalk: needs approval
beanstalk: running
beanstalk: tested
ci: approved
good first issue
gpu/amd
gpu/igpu
gpu/intel
gpu/nvidia
help wanted
kind
bug
kind
chore
kind
docs
kind
feature
kind
performance
kind
question
needs measurements
platform/linux
platform/macos
platform/steam-deck
platform/windows
preview
priority
critical
priority
high
priority
low
regression
risk
high
risk
low
risk
medium
status
blocked
status
confirmed
status
duplicate
status
invalid
status
needs-info
status
wontfix
ci: approved
No milestone
No project
No assignees
2 participants
Notifications
Due date
No due date set.
Dependencies
No dependencies set
Reference
MagicBeanDev/Longshot!1358
Loading…
Reference in a new issue
No description provided.
Delete branch "mooki/Longshot:control-overlay-text"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Type: feat · Risk: medium · Tested on: Linux, RTX 4090, headless (game-free checks; in-game pictures: see Testing) · Needs: Windows (build and run), a real window
Why
The test player (the "pilot", issue #1198) decides what to do next many times a minute, and until now the only way to see why was to read its run files afterwards. Someone watching a run, a recording or the game's own pictures could see the frame times (the FPS counter) but not the decisions. This adds a control-bridge command that puts a short text panel on the game's screen, drawn in the same overlay pass as the FPS counter, so both are visible at once. The pilot uses it to show its goal, the candidates with the chosen one marked, the step and its result, model and cost.
What changed
A client sends
{"cmd":"overlay","text":"...","seconds":N}(overlay_clearor an empty text clears). The bridge checks and cleans the text (at most 2048 bytes and 16 lines of valid UTF-8, control characters dropped), keeps it only while the sending client is connected and for at mostseconds, and hands it to the renderer, which draws it beside the FPS counter at every counter level and alone with the counter Off. Nothing is saved, logged or sent anywhere, and no game state changes.host/control.c,host/control.h: the two commands, owner and expiry on the bridge's thread,unavailablewithout the renderer's hook.gpu/vk/overlay_text.h(new, plain C): the text rules, used by the bridge and run again by the renderer.gpu/vk/vk_api.h,host/rx_backend.c,gpu/vk/vk_replay.cpp:MkwVkRenderer.overlay_text,MKWVK_API_VERSION29 to 30.gpu/vk/fps_stats.h: the counter's column height, width and corner box as functions (the sums moved out ofdraw_fps; a test checks they agree with the old ones to the pixel);draw_fpspublishes the box it drew.gpu/vk/overlay_panel.h(new): the text store (its own lock, not the menu's) and the placement: in the counter's corner, under it (over it from a bottom corner) only while the panel ends within 70 % of the window's height, else beside it toward the middle, never over it, shrinking to 70 % or cutting lines narrower before giving up (and under it past that reach only when nothing else fits).gpu/vk/vk_menu.cpp:draw_pilot_panelafterdraw_fpsin both record paths (the window's and the headlessRX_VK_SHOT_MENUpictures'); a translucent plate, the menu's font and Menu size scale; a new text size is measured once before the layout is settled and a line is cut to "..." only when it is more than 2 % over its room; a line starting with>is gold; not drawn while the Insert menu is open.scripts/longshot_ctl.py:overlay LINE... [--seconds N]and--clear.docs/control.md(the command, limits, privacy, pictures),docs/development.md(a row for the existingRX_VK_SHOT_MENU=1).shot(no overlay, so a picture sent to a vision model never carries the panel's text), no new switch, nothing when no client sendsoverlay.Measurements
Verdict: nothing to measure: this adds no work to a frame unless a client has set a panel (one atomic read in the overlay pass). No speed claim is made and none was measured.
Flags and settings
None: no new switch, no menu row and no ini key (the panel is drawn by
gpu/vk/vk_menu.cpp's overlay pass but has no setting of its own).RX_VK_SHOT_MENU=1, an existing switch, is now documented indocs/development.md; it is what puts the FPS counter and the panel into a headless run'sRX_VK_SHOTSpictures.Testing
Tested: game-free checks on Linux (base
main @ 6d72ca93b, headfeat/control-overlay-text @ 38f10348f), a Windows-target syntax check and cross-build, and one windowless run of the game with the pilot (below):tests/test_control.py: new cases against the fake game over the real socket (parsing, the 2048-byte and 16-line caps, UTF-8, dropped controls, expiry once, a later deadline, the owner's disconnect,unavailable). Red first: it did not compile without the header, then failed at the first overlay case withunknown_cmd; green after.tests/test_overlay_panel.py: the text store under a concurrent writer and reader, the counter's column against the old sums, and 51,840 layouts (15 window shapes from 640x360 to 5120x1440 including 4:3, 5:4 and 21:9, three Menu sizes, four counter sizes, six counter levels, four corners): every placed panel is inside the window and clear of the counter; at 4:3 and wider a place is always found.tests/test_longshot_ctl.py(the client, 52 tests).main @ 6d72ca93b, head38f10348f): 7 captured translation units, 1 failure, 1 unjudged driver. Compile for Windows:tests/test_control.py(control_test.c,host/control.c),tests/test_control_pipe_security.py,tests/test_fps_overlay_order.py,tests/test_overlay_panel.py(it defines_CRT_SECURE_NO_WARNINGS). The one failure,tests/test_audio_output_devices.py(memmove/tmpfiledeprecated inparser.c: the same fourerror:lines on pristinemain @ 6d72ca93b; this PR only moved the pinned API number in it); unjudged:tests/test_longshot_ctl.py(it starts the fake bridge with a directPopen). Windows execution was not run.38f10348f, clean checkout): exit 0, 421 compile and link commands (103 new, 318 from matching cache entries), 0 failed, 0 warnings (under the build's own-Wno-everything: not a warning-clean claim). The renderer's 210 units compiled andlongshot_vk.dlllinked (14,367,744 B); the host group's 210 units compiled, including the changedhost/control.candhost/rx_backend.c, andgpu/vk/vk_menu.cppandgpu/vk/vk_replay.cppcompiled new. The game executable was not linked (no Linux lift registered for this head to link it from) and nothing was run on Windows or Wine.tests/run_all.py(--jobs 6 --strict, fresh HOME, idle gate, both runs at the same time on a loaded PC):main @ 6d72ca93b662 passed, 41 failed, 17 skipped in 319 s; this branch@ 38f10348f663 passed, 41 failed, 17 skipped in 310 s. The failing files are the same 41 on both, compared by name (none fails only here):test_bug_report_defaults,test_cpu_report,test_crash_record,test_crt_options,test_damage_debug_menu,test_display_startup_failure,test_dlss_feature_size,test_failed_compile_keeps_objects,test_framegen_preset,test_fsr_uniform_ownership,test_guest_imported,test_hud_layout_row,test_import_reach,test_invalidate_vcore,test_linux_preflight,test_linux_pressure,test_linux_update,test_mirror_budget,test_missing_nup,test_mods_size_table,test_nonuniform_image_arrays,test_parallel_init_order,test_pending_guest_import,test_pipeline_play_cache,test_present_thread,test_profiling_mode,test_readback_timeout_source,test_replay_write_order,test_rumble_menu,test_run_all_changed,test_slow_frames,test_staging_burst,test_switch_table,test_target_image_pool,test_tested_configurations,test_upscaler_round_trip,test_upscaler_rows,test_vcore_escape,test_vk_occlusion_queries,test_vk_swapchain_compat,test_walk_key(they fail on this PC'smainas it stands, for exampletest_import_reachon astd::vector::containscompile error with the system'sclang++; not looked into further).nx2/make_game.pybuild, the rebased head38f10348f):v3.0.0-rc.6-9-g38f10348f (source af14c9ad),longshot546,801,056 B sha25630c67c67939278304b9e345d1ed07d48057d82a7eb05a7e9ddbcb2d846b23026,longshot_vk.so15,573,464 B sha2564a3a176377ca14606928e58e7b62b614940bae8e89fe0925ecb805ae7a965702; 169 s, 96 objects new and 422 from the object cache of the earlier build (main's lift changed, so it was lifted again). The in-game pictures below come from this build.In the game, on the rebased build (Linux 7.2.15, RTX 4090, driver 615.78.08, 2026-11-16, headless; the game's own first log line names the build:
longshot v3.0.0-rc.6-9-g38f10348f (source af14c9ad), built from38f10348f). This branch had been played before at an earlier head on an oldermain(c4aca39d3onac7787515);mainhas moved 37 commits since, so the game was built and played again after the rebase, and this is that run. One run of the pilot (hud3-talk) in Kokiri Forest, saveexplore17-kokiri-child: it walks to a Kokiri girl and talks to her, then goes to the signboard and reads it (rules decider, so no model call). Shader preparation first (45 s, exit 0,@@done shadersfrom the game itself), then the run: the pilot's run took 84.8 s and ended solved, the game ran 83.2 s,RX_VK_SHOT_MENU=1, a picture every 45 presents, 1280x720, 210 pictures. The game's log hascontrol: overlay panel set (11 lines, 235 bytes; it goes with its client or its seconds)once andcontrol: overlay panel cleared (asked)at the end. The wrapper's summary line:exit=143(the pilot's own stop at the end of its run)fatal=0 crash=0 devlost=0 nocore=0 kernel=0 nvrm=0 journal=complete, andcoredumpctllists no core dump since the run started; itsHOME-WRITESnote names~/.cache/longshot-builder, an existing empty folder from 13 November (the note tests existence only). Other work was running on the PC (load 12.7 at the start, 7.3 at the end), so the frame rates in the pictures say nothing about speed.While the pilot played, a second small program changed the FPS counter's level and the Menu size over the run's own control socket, nine seconds per phase, in this order: Graph Medium, Breakdown Medium, Off Medium, Off Large, Graph Large (Breakdown Large, FPS and Frame time were next; the pilot solved its segment and the game was stopped before them). It recorded the renderer present of each change, so every picture is tied to its phase. 53 of the 210 pictures lie inside those phases (Graph Medium 11, Breakdown Medium 12, Off Medium 12, Off Large 12, Graph Large 6); the other 157 are the start-up (black, the opening field in moonlight, the title, the load menu, black again), with neither counter nor panel. All 210 were looked at: the 53 in sets of four at 800x450, the 157 in sets of twenty, and at full size the first picture after the size change and one picture of each of the other phases. What was seen:
vk_007335, the first decision, the panel is under the counter, its lower edge at about y 335 of 720, with the hearts (top left) and the map (lower right) drawn and clear of it.vk_008685, the first picture after the change (7 presents after it), has every line whole; its top line,GOAL talk-to-the-kokiri [open] FOR kokiri-013(45 characters, the longest top line of this run), ends inside the plate. In the first run the same change with the same counter state (vk_005985) had a top line cut to "...". The other eleven Off-Large pictures have the signboard goal's lines whole (GOAL read-the-sign [open] FOR signboard-010, then[met]with aRUN solvedrow, up to 15 lines with+2 more). No picture of the 53 has a panel line cut to "...". The top lines of the two runs differ (the first run's was the signboard goal's, this run's the Kokiri talk goal's), so this is the same change but not the same text.vk_009270the panel is beside the counter (15 lines,RUN solved,+2 more), readable at the larger size, its right edge just left of the counter's, the hearts and the map clear. The first picture of the phase,vk_009225(11 presents after the counter went from Off to Graph), has the counter still filling in (no frame-time row yet) and the panel under it, clear of the counter; the next picture has the counter whole and the panel beside it. A panel goes under the counter only while it ends within 70 % of the window's height, and with the counter's second row in, 15 lines at Large end past that, which I take to be why it moved (not measured): one picture of the six.1. Graph at Medium (a walking scene): the panel under the counter, hearts and map clear. 2. Breakdown at Medium: beside the tall counter. 3. Counter Off at Medium: the panel alone, the dialogue's choices clear. 4. Counter Off, the first picture after the change to Large: every line whole. 5. Graph at Large: beside the counter, 15 lines, readable.
1 and 3 are from the first run (
06096442a), 2 and 4 from this one (38f10348f). 1 and 2: Breakdown at Medium, under the counter down to the corner, and now beside it. 3 and 4: the first picture after the same size change (Medium to Large, counter Off): then a top line cut to "..." although it fitted, now every line whole (the top lines differ: the first run's was the signboard goal's, this run's the Kokiri talk goal's, which is longer).Not tested: a real window (this PC's runs are windowless: the window path calls the same
draw_pilot_panel, unseen); Windows (syntax and cross-build only, nothing run); the pilot's model rows (model, latency, cost) in the game: the run used the rules decider; Breakdown at Large, FPS and Frame time with the panel on this build (the run ended before those phases; the two earlier runs, at06096442aandc4aca39d3on the oldermain, saw Breakdown at Large and the FPS levels, but not on this base); the panel with the Insert menu open (hidden by design, not looked at).Must be tested:
longshot_ctl.py overlay "hello" --seconds 30in a window with the FPS counter on Graph, expecting the panel under the counter, every line whole and no overlap with the counter, the hearts or the map (by hand: look at the corner).longshot_ctl.py overlay "hello" --seconds 30with the FPS counter in each corner, expecting the panel beside or under the counter, never over it, and inside the window (by hand: look at each corner).Risks
What could break:
draw_fps's height sum moved intofps_stats.h(checked to the pixel by a test, so the counter should look as before); a renderer built before this change with a newer host (or the other way round) is refused by the version check, as for every API bump. A client that keeps sending text can keep a panel on the screen over the game: the Insert menu hides it while open, and it goes with the client.Turning it off: nothing is drawn unless a client sends
overlay; revert the PR's commits to remove it.Compatibility:
MKWVK_API_VERSION29 to 30 (a stale renderer library is refused); nothing changes for saves,longshot_options.inior the caches. The number can conflict:mainis at 29 (the fullscreen monitor choice, #1277) and this change takes 30, while the co-op playtest PR 355 is at 30 and PR 68 at 31 on their own branches (read from the local refsgs-pr/355257463464andgs-pr/6879c9d411f), so whichever of them merges after another renumbers: a one-line change ingpu/vk/vk_api.hplus the one test that pins the exact number,tests/test_audio_output_devices.py(_Static_assert(MKWVK_API_VERSION == 30, ...); this PR moved it from 29 to 30, andtests/test_overlay_panel.pyonly asks for 30 or more).Checklist
Everyone
type(scope): description) of the status line's type.main(544229a26). The tests, builds and the game run above are on38f10348fovermain @ 6d72ca93b;mainthen gained one commit (#1332:gpu/vk/vk_main.cppand a new test, no file shared with this branch, renderer API still 29), and after rebasing onto ittest_control,test_overlay_panel,test_audio_output_devicesandtest_fps_overlay_orderpass again. No rebuild on the new base.python -I -B tests/run_all.pypasses (any failure that also fails onmainis named).third_party/untouched (a new folder with aPROVENANCE.mdinstead).When it applies
Assisted-by:/Co-Authored-By:trailers.MKWVK_API_VERSIONbumped (gpu/vk/vk_api.h'sMkwVkHostorMkwVkRendererchanged).mainand passing here.tests/that reads changed source updated in this PR, not weakened.New on this forge; not opened on GitHub.
WIP: feat(control): an overlay command shows a test player's decisions beside the FPS counterto feat(control): an overlay command shows a test player's decisions beside the FPS counterA client can put a short text panel beside the FPS counter, and it leaves with that client or with its seconds. I read head
8e1f34ab. It applies onto current main. The window runs it still names are open, so this is not an approval.The bridge keeps at most 2048 bytes and 16 lines of UTF-8, drops control characters, and clears the panel when that client disconnects or when
secondspasses. The renderer draws it after the counter, in both the window and the headlessRX_VK_SHOT_MENUpictures, and not while the Insert menu or the start-up screen is up. The bridge's ownshotstill records the picture without this panel. The panel sits under the counter while it ends within 70% of the window, and beside it otherwise, and a line is cut to "..." only when it is more than 2% over its room.MkwVkRenderergainedoverlay_text, and the interface is 30. Main is still 29. If #1034 lands first, this bump is the next number after that one.The headless Linux pictures show the panel clear of the counter, the hearts and the map. Must be tested still asks for a Windows window (the panel under Graph, every line whole) and a Linux window at 4K and at 1280x720, the counter in each corner.
View command line instructions
Manual merge helper
Use this merge commit message when completing the merge manually.
Checkout
From your project repository, check out a new branch and test the changes.