feat(console): a Quake-style Debug tools console that drops from the top of the screen, over one command registry #1641
No reviewers
Labels
No labels
area/accessibility
area/audio
area/input
area/install
area/menu
area/renderer
area/saves
area/tooling
area/upscaling
beanstalk: approved
beanstalk: failed
beanstalk: needs approval
beanstalk: running
beanstalk: tested
ci: approved
good first issue
gpu/amd
gpu/igpu
gpu/intel
gpu/nvidia
help wanted
kind
bug
kind
chore
kind
docs
kind
feature
kind
performance
kind
question
needs measurements
platform/linux
platform/macos
platform/steam-deck
platform/windows
preview
priority
critical
priority
high
priority
low
regression
risk
high
risk
low
risk
medium
status
blocked
status
confirmed
status
duplicate
status
invalid
status
needs-info
status
wontfix
ci: approved
No milestone
No project
No assignees
2 participants
Notifications
Due date
No due date set.
Dependencies
No dependencies set
Reference
MagicBeanDev/Longshot!1641
Loading…
Reference in a new issue
No description provided.
Delete branch "mooki/Longshot:feat/console"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Type: feat · Risk: medium · Tested on: Linux (Arch, kernel 7.2.15): game-free checks and one windowless game run on an RTX 4090, with pictures · Needs: Windows, any GPU, a real controller
Stacked on #1351 (head
3a191e809,!pos X Y Z, on main430eef4f2): its commit is the base of this branch. Review only the 17 commits above it: the command registry (8), the console's font and drop-down (9). If #1351 changes in review, this branch is rebased onto its new head.TL;DR
The live input file's
!lines had no answer, and no command could be typed in the game. This PR puts one command registry behind the file (an answer for every line in the log, a 10 s deadline on a request Link never takes, handlers forpos,age,teleport,time,give,take,items,status) and adds a Quake-style console on it: with Debug tools on, the key above Tab drops a monospace console from the top of the screen in 200 ms, with Tab completion, history, a controller's picker and the last lines left at the top after it closes. Game-free tests, mutation runs and one windowless run (pictures below) show it working.Breakdown
pos,mark,give Item_Bomb 5orhelpin the game and read the answer.strncmpand parser, with no answer and no deadline; the Debug tab's rows had no name; nothing in the game took a typed line.!line tohost/console.c, which checks it against a table (gpu/vk/console_schema.h) and either runs it or queues a request that Link's update takes and answers by ticket. The console is a second door on the same registry: a plate that slides down from the top edge, an output, a]line, Tab, history, scrolling, notify lines, a picker for a controller.hid: live command at ...,cheats: ...), the Debug tab's actions, the bridge, saves and options; nothing opens without Debug tools; the Insert menu works as before and the console gives it the screen.overlay(needs #1358), the bridge's ownconsolecommand (the bridge has none yet: the run opens the console with the menu script's key), the co-op room hook (the room refusal is checked game-free only).menu_tickpolls two more keys and the console takes the game's input while it is open;MKWVK_API_VERSIONgoes from 33 to 34, the number #1408 (headf08b5eece) also takes, so whichever of the two lands second renumbers.Why
A tester or a script that writes
!pos,!cheat 122or!shots 500into the live file gets no answer but a log line, cannot ask what exists, and loses lines (two!cheatlines in one 30 ms poll keep only the last; a line sent at the title fires when play starts, minutes later). Each new command added a parser of its own. The registry puts one table and one queue behind the file; the handlers are the rows testers asked for first (where Link is, his age, the clock to the minute, items by name, the state of a run); and the console is the way to use them by hand in the game, the way Quake's console is. The design (the drop-down, the monospace face, the open questions and their defaults) is in the developer console report.What changed
Three parts, in this order.
The registry (
4f7cb68a4to818ce6ff4):test(input)pins every!line of the live file (138 polls) before anything moves;gpu/vk/console_core.handconsole_schema.hare the table, the tokenizer, the argument checks,help, completion and the room gate;host/console.cis the queue with tickets (32 waiting, a request Link does not take within 10 s is withdrawn), the log line (console: live: !pos 1 2 3 -> ok: returned to 1.00 2.00 3.00) and the three doors (the live file, the console, the bridge's laterconsole);docs/console.mdlists the table.The handlers (
50e207e00to3e605b5d0):pos,ageandteleportread when called with no argument;timereads and sets H[:MM],noon,midnight;give,takeanditemschange the Inventory page's items by name;status. Link's update samples what the reads answer from twice a second (oot_console_sample); the Windows-target drivers build.The console (nine commits from
948b15912):third_party/fonts/libertinus/LibertinusMono-Regular.ttf(Libertinus 7.051, the release the folder already has; the zip's sha256 is checked againstPROVENANCE.mdandOFL.txtis the one in the folder),nx2/ui_fonts.py,gpu/vk/vk_imgui.cpp. Its own commit.gpu/vk/console_ui.h(slide, line, history, Tab, scrolled output, notify lines, the picker's lists) andconsole_layout.h(the frame, the plate from the bottom up, the notify strip); no drawing and no game in them.gpu/vk/console_draw.h(the draw calls on a sink),vk_console.cpp(the ImGui sink, called frommenu_recordandmenu_record_imageafter the FPS counter, so the open console covers the HUD).MKWVK_API_VERSION34 withMkwVkHost.console_run,console_result,console_name,console_feed,console_room;host/console.ckeeps the other doors' lines (8 a second, 32 held; the bridge's and the live file's lines show in the output and the notify strip) and answers the maps' names;gpu/vk/console_pump.his the console's half.menu_tickpolls two more keys (the one above Tab, and L) and hands the console its presses, text, pointer and pad;menu_is_open()includes the console, so the host drops the game's input while it is open; the Debug tab's Developer rows get Console, Console height and Notify time (con_height,con_notifytime). The switches are read-only in this version (getandsetare not rows of the table yet)....and the chosen one shows its whole text on up to two lines under the choices (the last commit).RX_VK_MENU_SCRIPTtakesCONSOLE,TAB,PGUPand the like,RX_VK_MENU_SCRIPT_GO=<file>starts its clock when a file exists, the console's clock is 1/60 s a tick during a headless burst of shots;docs/console.md, the README's rows.Measurements
Verdict: no speed is claimed. With Debug tools off the closed console does nothing per frame (the console's tick returns before it touches the model); with Debug tools on it costs a few nanoseconds a frame (timed in isolation, below); closed, with no slide running and no notify line alive, it draws nothing (a test pins the empty call list).
Hardware: Linux (Arch, kernel 7.2.15); the closed frame on the host's CPU, the slides on an NVIDIA RTX 4090 with the headless renderer (no window), one run.
Scene: the closed frame in isolation:
Pump::poll,Console::tickand the draw side'swants_frametest with the registry's real feed and room calls fromhost/console.c, 20 million frames, the best of five runs, on a machine at load 30 (other work running). The slides: the field of a library save (a child Link in the grass), 1280x720, FPS counter on, Debug tools on; Console height Half, Third, Full, then Half with a controller's picker.Method: the closed frame is timed in isolation, not in the game: a paired in-game run is not part of this PR. The slides are measured from the pixels of a burst of shots (a shot of every present, the console's clock fixed at 1/60 s a tick, so the frames repeat exactly): the gold line's bottom edge on ticks 1 to 11 of every slide in and out (px at 720p), against the eased-out formula at that tick's time within 1 ms (the clock counts whole milliseconds), 2 px of slack; the first slides on the real clock, from the log.
How to reproduce: Debug tools on (
HOS_OOT_CHEATS=1), a library save,RX_VK_SHOT_MENU=1,RX_VK_SHOT_GO=<file>,RX_VK_MENU_SCRIPT_GO=<file>andRX_VK_MENU_SCRIPT=1:CONSOLE,2.6:CONSOLE,5:CONSOLE,...(the keys are listed indocs/development.md), touch the files, read the gold line's row in the shots.c507b98c884532af32wants_frame)The real-clock lengths are the design's 200 ms plus the tick that sees the slide finished (at 43 presents a second a tick is 23 ms, so 9 ticks; at 60 an earlier run of this stack measured 12 ticks, 216 ms).
Flags and settings
This PR adds one environment switch to the registry (
HOS_CONSOLE_ROOM), two options (con_height,con_notifytime) and one test switch (RX_VK_MENU_SCRIPT_GO), andRX_VK_SHOT_GOnow also fixes the console's clock during a headless burst of shots.HOS_CONSOLE_ROOM=allowcon_height(Debug tab, Developer: Console height)con_notifytime(Developer: Notify time)RX_VK_MENU_SCRIPT_GO=<file>RX_VK_SHOT_GO=<file>Testing
Tested: Linux (Arch, kernel 7.2.15), game-free unless said:
python -I -B tests/test_console_registry.py(the table, the tokenizer, the checks, the gate's 1,584 lines over every row, door, Debug tools and room state, each handler's cases),tests/test_console_legacy_lines.py(the golden test of 138 live-file polls),tests/test_console_queue.py(ThreadSanitizer with a sampler thread writing while the reads run, then ASan and UBSan; the feed and the names).python -I -B tests/test_console_ui.pyruns three programs with 416 checks (the model and layout: slide timing and reversal, the line, the history file, wrapping, scrolling, notify clocks, Tab, the picker,two_linesand the layout's rows, a sweep of over 1,000 window, size, height, pad and scroll cases; the draw calls on a fake sink; the registry link against a fake host);tests/test_console_wiring.pyreads the wiring from the source (the keys, the Debug tools gate, host signals, drawing order, rows, history file, host calls, font); the menu tests that readmenu_tick(test_menu_keys.py,test_menu_text_field.py,test_fps_overlay_order.py) and two fixtures that cutini_key()out of the menu (test_audio_output_menu.py,test_stick_dead_zone.py) are brought to the new code, none removed.!shotsanswer), all killed; the 58 of them inhost/console.c, the one registry file this branch added code to (the feed and the names), re-run on this head: 58 killed, 0 survived; the other 56 are in files this branch does not change and are carried from the registry's runs. The console: 155 mutants of the model, layout and drawing (console_ui.h,console_layout.h,console_draw.h,console_pump.h), 34 of the picker's cut and whole-text lines, 34 of the wiring: all 223 killed, re-run on this head.giveandtakeread the counts throughrx_option, which the Vulkan back end answers 0 for): the registry keeps its own copy of the counts,tests/console_host_test.cstands in for the getter as the back end answers it.test_console_ui,test_console_registry,test_console_queue,test_console_legacy_lines,test_console_wiring,test_console_docs,test_menu_keys,test_menu_text_field,test_fps_overlay_order,test_audio_output_menu,test_stick_dead_zone,test_pos_handoff; one that does not exist yet is skipped): all 17 commits above #1351 pass at their own commit. The two fixtures that cutini_key()out of the menu are in the commit that adds the keys they name; before that they failed in three commits. Under a load of 60 the threadedtest_console_queueandtest_pos_handofftimed out once each and pass when run alone.test_console_queue.pydeclares itself Linux-only (the sanitizers); the model, drawing, pump and registry-host programs were compiled by hand for the mingw target against its libc++ (all four OK); two older menu fixtures this PR touched for their stub options (test_audio_output_menu.py,test_stick_dead_zone.py) stop at the Windows CRT's deprecation warnings under-Werrorin their ownfopenandgetenvcalls and in two headers'getenvandsscanf(the same lines are on main). Windows execution is not tested.python -I -B tests/run_all.py --strict, 4 at once: this branch 728 passed, 42 failed, 17 skipped; forge main (c507b98c8) 725 passed, 46 failed, 17 skipped. Every one of this branch's 42 failures also fails on main (main fails four more under the same load:test_bug_report,test_crash_groups,test_hardware_report,test_prepare_cache_save), so none is this branch's; the console, registry and menu tests all pass. The 42:test_bug_report_defaults,test_capture_readback_visibility,test_cpu_report,test_crash_record(a 10 s wait for a child on a loaded machine; passes alone),test_crt_options,test_damage_debug_menu,test_dev_packs,test_display_startup_failure,test_dlss_feature_size,test_draw_stream_visibility,test_failed_compile_keeps_objects,test_fsr_image_retirement,test_fsr_uniform_ownership,test_guest_imported,test_hud_layout_row,test_invalidate_quick,test_invalidate_vcore,test_libc_hooks,test_linux_preflight,test_linux_pressure,test_linux_update,test_log_open_failure,test_mirror_budget,test_missing_nup,test_mods_size_table,test_nonuniform_image_arrays,test_parallel_init_order,test_pending_guest_import,test_pipeline_play_cache,test_present_source_crop,test_present_thread,test_profiling_mode,test_psychov35_fsr3_fg,test_readback_timeout_source,test_replay_write_order,test_run_all_changed,test_slow_frames,test_staging_burst,test_switch_table,test_upscaler_rows,test_vk_occlusion_queries,test_walk_key. The suite ran on the head before the last history change (the same tree).The picture run: one windowless run in the side lane, run
caf8a3d9df5d03e6edb2a6793a875c27of the exact head of this branch,84532af32(the game's first log line islongshot v3.0.0-rc.7-99-g84532af32 (source 9ab32666); RTX 4090, 1280x720, a library save in the field, Debug tools on, the FPS counter on; the machine was at load 45 to 65, so the game ran at about 43 presents a second before the script, not 60). The control bridge has noconsolecommand yet, so the menu script'sCONSOLEkey opens the console and the bridge sets the Console height and the FPS counter. The script opened and closed the console at Half, Third, Full and Half again (the controller's picker) and typedhelp,items bomb,give Item_Bomb 5,take Item_Bomb 5,status,pos 1 2(a usage error), Tab and PgUp; the live file ran!mark,!statusand!pos 1 2with the console closed. The log has each answer (console: ui: give Item_Bomb 5 -> ok: inventory Item_Bomb +5 (had 0, now 0, event status 2),take,items bomb,status).exit=0 fatal=0 crash=0 devlost=0 nocore=0 kernel=0 nvrm=0 journal=complete, nokmemsysDoCacheOporNVRM: Xidline in the journal, 1724 shots. An earlier run of this stack, before its last commit, showedhelpcutting thegiverow in two mid-word (world comm/and [world]: the output kept lines of 160 bytes, the row is 175); that commit fixes it, test first, and this run shows it fixed (Full). The same tree ran before the last history change (the fixtures' commit folded into the input commit) with the same pictures. Each picture below is a shot of the real rendering, read against the design's four mockups.Known differences from the mockups: none that matter: the plate colour, the gold line, the
]line and caret, the monospace output, the version, the^row, the picker's boxes and chips and the notify strip match. Two things the mockups do not show: a Suggested description longer than its box is cut with...(the registry's help strings are full sentences) and the chosen one's whole text sits on two lines under the lists (a later request); and the notify strip's oldest line was not caught fading (the shot is 3 s after the lines).Not testable by hand: nothing.
Not tested: Windows execution (the key is the layout's on Windows), a real controller, a real window, ultrawide in the game, the Debug tab rows' look in the game (only their source and the tests), the room refusal in a real co-op room, a bomb count changing through
giveandtake(the small bag starts full once unlocked).Must be tested:
help, press Tab, scroll with PgUp, close with Esc, expecting a plate that drops in about 200 ms, the answer in the output, no Insert menu opening when Esc closes it, and the key being the one above Tab on this layout (by hand)...., the chosen one whole under the lists, A to run the chosen line and B to close (by hand, or the Test Farm).HOS_INPUT_LIVE=<file>appending!give Item_BombBag_Small, then!take Item_BombBag_Smalland!time 19:30, expectingokanswers in the log and the clock set (by hand, the log).Risks
What could break: the live reader and
oot_update_player_cheatsare the path of every test-player line and of the Debug tab's actions, so a mistake there would stop!cheator!pos(the old lines are pinned by the golden test; the update itself runs only in the game). A script that relied on the old reader's looseness (!dumpfoo,!cheat 3x, extra arguments) now getsusagein the log.menu_tickpolls two more keys and the console takes the game's input while open (the host's "a menu is open" signal): a wrong state there would leave the game without input, and the pictures show it closing.MKWVK_API_VERSIONis 34 and is numbered last on merge. The sample taken twice a second inside Link's update is about ten guest reads, read from the code, not measured.Turning it off: Debug tools off closes the console and nothing else uses it; or revert the commits (the registry's first, then the handlers, then the console, in reverse).
Compatibility: one more file in the player's folder,
longshot_console_history.txt, written only while Debug tools is on; two new options-file keys; a font file (153,992 bytes) in the build; the shader and pipeline caches are not touched (the translator's stamp does not change:test_shader_stamp_inputs.pypasses). Nothing else changes for a player: the live file only a test run sets up.Cost: the closed console costs nothing with Debug tools off and 6.5 to 9.7 ns a frame with it on (the registry's feed and room calls and the model's tick, timed in isolation: Measurements); it draws nothing while closed with no notify line alive; the sample in Link's update is about ten guest reads twice a second (read from the code, not timed). A paired in-game run was not made; the console adds one font file (153,992 bytes) to the build and, while open, the draw calls of one plate.
Issues
None found that this fixes (the tracker was searched for #1351, #1358, #1408 and #1411, which become rows of the table).
Checklist
Everyone
type(scope): description) of the status line's type.can't be run by hand is in Testing's Not testable by hand, with why.
in Risks.
main.python -I -B tests/run_all.pypasses (any failure that also fails onmainis named).third_party/untouched (a new folder with aPROVENANCE.mdinstead).When it applies
Assisted-by:/Co-Authored-By:trailers.MKWVK_API_VERSIONbumped (gpu/vk/vk_api.h'sMkwVkHostorMkwVkRendererchanged).mainand passing here.tests/that reads changed source updated in this PR, not weakened.New on this forge; not opened on GitHub.
A test run or a bug report that gives coordinates could only be followed by walking there, or by a teleport that lands at the map's entrance. "!pos X Y Z" in HOS_INPUT_LIVE's file asks the host to put Link at that position (metres, the frame of the control bridge's pos) by the Debug tab's Return to mark path: the asked position becomes the mark and the return runs at Link's next update, so the log says "cheats: returned to X Y Z" and the game's own position change moves him. Only exactly three finite numbers are taken (blanks, tabs and a CRLF line end around them are fine); any other line is ignored, and a refused "!pos" line says so in the log ("hid: !pos takes exactly X Y Z (three finite numbers, metres): line ignored"), so a test run sees why nothing moved. It needs neither HOS_OOT_CHEATS nor the Debug tools row. It is a line of the live input file, not a switch: no new HOS_ name is read, so gpu/vk/switches.def has nothing to add (HOS_INPUT_LIVE is its row already). What is not in it: Link's facing and the camera's angles. They were tried as optional extra numbers on the same line and run in the game: the position held, the facing did not (the game works out his heading again at his next update, so the turn did not stay) and the camera could not be set (the camera object the host's hook is given has no view to read). Rather than ship them half working, a line with more than three numbers is refused, and the position is the whole command. The command was called "!place" while this change was open; that name is not read. nn_hid.c reads the line (host/nn_hid.c, beside "!cheat"): parse_pos takes strtof's numbers one by one and stores nothing unless all three are finite, each ends at a blank or the line's end, and nothing follows them. hos_main.c's oot_update_player_cheats takes the ask as the return to mark's action when no other action is waiting. tests/input_background_test.c, tests/test_input_background.py: the live-pos case runs the real read_input against a live file. A good line stores its three numbers and asks once (blanks, tabs, a CRLF, exponents). It refuses, stores nothing and, for a "!pos" line, logs the refusal for: 0, 1, 2, 4, 5, 6 and 7 numbers, nan, inf, -inf, an overflowing number, letters, text after the numbers, "1 2 3,4", two numbers glued together ("1-2 3", "4 5.5.5"); another command's name ("!posx", "!position", "!pos1 2 3") and the old name "!place 1 2 3" are neither acted on nor logged as a refused "!pos". A refused line does not stop the next good one. It fails on main (the ask is never set). docs/development.md: the HOS_INPUT_LIVE row. Handoff (review of the first head): the reader (a game thread) and Link's update (another) shared three volatile floats and a flag raised with a release store, and Link's update cleared the flag first and copied the floats after, so a second "!pos" line read in between could leave X from one request with Y and Z from the next (and was a data race). The request now crosses as one value: one spin lock around "write the numbers and raise the flag" (hos_oot_pos_ask) and around "copy the numbers and clear the flag" (oot_pos_take), the guest call made from Link's update's local copy after the lock is released. tests/test_pos_handoff.py compiles that section of host/hos_main.c alone: of two requests the later is taken whole, and a producer publishing (k, k, k) against a consumer taking shows no torn tuple in 3,000,000 requests; the old protocol (flag first, copy after) shown by the same driver tears, and so does a mutant of the real code with the lock taken out (3,161 torn tuples). tests/input_background_test.c checks one hand-over per good line. Assisted-by: Claude Sonnet 5.5 Co-Authored-By: Claude Sonnet 5.5 <[email protected]>@Beanstalk
WIP: feat(console): a Quake-style Debug tools console that drops from the top of the screen, over one command registryto feat(console): a Quake-style Debug tools console that drops from the top of the screen, over one command registry@Beanstalk
@mooki approval of
84532afrecorded. This diff touches sensitive paths, so a second member ofBigBeansmust approve it too.Beanstalk test farm
Commit
84532af· tier untrusted · approved by mooki · dashboardThis diff touches sensitive paths, so it needs two members of
BigBeans:nx2/build.py,nx2/ui_fonts.pyA member of
BigBeanscan review this commit and comment@Beanstalkto test it.View command line instructions
Manual merge helper
Use this merge commit message when completing the merge manually.
Checkout
From your project repository, check out a new branch and test the changes.