Add MCP server for scripted in-game testing

nicholasgower commented

2026-11-08 20:24:51 +00:00

(Migrated from github.com)

What this adds

scripts/hookshot_mcp.py: a stdio MCP server built on the official Python SDK (mcp==2.3.0).
- Launches a built Hookshot headless, muted, with a dedicated run/cache/save directory.
- Tools: launch, stop, status, press, stick, sequence, screenshot, wait, logs.
- Screenshots are returned inline as MCP image content and saved to the run directory.
- Resources: hookshot://log, hookshot://last-screenshot, hookshot://status.
Game-side test seams:
- HOS_INPUT_SCRIPT_ONLY=1 ignores physical pads, keyboard and mouse.
- Analog stick tokens LX=, LY=, RX=, RY=.
- !quit live command for a clean stop.
- HOOKSHOT_NO_ENV_FILE=1 lets the MCP server own the environment.
- RX_VK_SHOT_FILE and RX_SCREENSHOT_FILE request one PNG of the next present from the Vulkan and WebGPU renderers.
Tests: tests/test_mcp_server.py (protocol over stdio, input encoding, env handling, native seams, fake-game end to end), plus analog and script-only cases in tests/test_input_background.py.
Docs: docs/mcp.md with host configuration and tool usage; updated development, design, contributing and Vulkan notes.
CI installs mcp==2.3.0 for the new test.

Follow-up checks

scripts/pr_check.py: a read-only pre-PR gate. It rejects game-derived paths, CRLF endings, non-Conventional commit subjects and missing commit trailers, then runs the game-free suite unless --no-tests is given. It never stages, commits, pushes or calls GitHub.
tests/test_pr_check.py: throwaway-repository coverage for clean and bad PRs, including a .nup, CRLF content, missing trailers and an untracked game file.
tests/test_shot_request.py: extracts the production read_shot_request() from vk_replay.cpp and compiles a fixture for the missing, empty, trimmed, one-shot and buffer-edge cases behind RX_VK_SHOT_FILE.
tests/test_docs.py: resolves relative Markdown links, verifies documented python/powershell commands name tracked files, and compares this page's MCP tool/resource tables with the live server.
CONTRIBUTING.md and docs/development.md now point at scripts/pr_check.py; gpu/vk/VULKAN.md and docs/mcp.md point at the new fixtures.

Verification

python -I -B tests/test_mcp_server.py
python -I -B tests/run_all.py mcp --strict
python -I -B tests/test_input_background.py --cc gcc (all cases, including analog and script-only)
python -I scripts/hookshot_mcp.py --check --game-dir <fake-folder>
python -I -B tests/test_pr_check.py
python -I -B tests/test_shot_request.py --cxx g++
python -I -B tests/test_docs.py
python -I -B tests/run_all.py pr_check shot_request docs --strict

The Vulkan/WebGPU C++ changes could not be compiled in this environment because there is no game build, Vulkan headers or dump here. The input-parser changes were compiled and exercised through the existing fixture; the extracted shot-request parser was compiled and tested separately.

The full local tests/run_all.py --strict run gave 71 passed, 11 failed, 9 skipped; the failures are environment/tool gaps here (missing SPIR-V/Vulkan headers, clang/clang++, and Windows symlink privileges in nx2/test_nup_paths.py), not the new tests. All three new checks passed.

Known issue in the first commit

The first commit e9c2b0d predates the new pr_check gate. It is not a Conventional Commit and lacks the Co-Authored-By trailer, so python -I -B scripts/pr_check.py --base main --no-tests reports it. The three follow-up commits pass. I did not rewrite the published branch; if a rebase/amend is acceptable, the subject should become feat: add the Hookshot MCP server for scripted in-game testing and the body should end with both trailers from docs/design.md section 8.

No game files, builds, saves or generated screenshots are included.

## What this adds - `scripts/hookshot_mcp.py`: a stdio MCP server built on the official Python SDK (`mcp==2.3.0`). - Launches a built Hookshot headless, muted, with a dedicated run/cache/save directory. - Tools: `launch`, `stop`, `status`, `press`, `stick`, `sequence`, `screenshot`, `wait`, `logs`. - Screenshots are returned inline as MCP image content and saved to the run directory. - Resources: `hookshot://log`, `hookshot://last-screenshot`, `hookshot://status`. - Game-side test seams: - `HOS_INPUT_SCRIPT_ONLY=1` ignores physical pads, keyboard and mouse. - Analog stick tokens `LX=`, `LY=`, `RX=`, `RY=`. - `!quit` live command for a clean stop. - `HOOKSHOT_NO_ENV_FILE=1` lets the MCP server own the environment. - `RX_VK_SHOT_FILE` and `RX_SCREENSHOT_FILE` request one PNG of the next present from the Vulkan and WebGPU renderers. - Tests: `tests/test_mcp_server.py` (protocol over stdio, input encoding, env handling, native seams, fake-game end to end), plus analog and script-only cases in `tests/test_input_background.py`. - Docs: `docs/mcp.md` with host configuration and tool usage; updated development, design, contributing and Vulkan notes. - CI installs `mcp==2.3.0` for the new test. ## Follow-up checks - `scripts/pr_check.py`: a read-only pre-PR gate. It rejects game-derived paths, CRLF endings, non-Conventional commit subjects and missing commit trailers, then runs the game-free suite unless `--no-tests` is given. It never stages, commits, pushes or calls GitHub. - `tests/test_pr_check.py`: throwaway-repository coverage for clean and bad PRs, including a `.nup`, CRLF content, missing trailers and an untracked game file. - `tests/test_shot_request.py`: extracts the production `read_shot_request()` from `vk_replay.cpp` and compiles a fixture for the missing, empty, trimmed, one-shot and buffer-edge cases behind `RX_VK_SHOT_FILE`. - `tests/test_docs.py`: resolves relative Markdown links, verifies documented `python`/`powershell` commands name tracked files, and compares this page's MCP tool/resource tables with the live server. - `CONTRIBUTING.md` and `docs/development.md` now point at `scripts/pr_check.py`; `gpu/vk/VULKAN.md` and `docs/mcp.md` point at the new fixtures. ## Verification - `python -I -B tests/test_mcp_server.py` - `python -I -B tests/run_all.py mcp --strict` - `python -I -B tests/test_input_background.py --cc gcc` (all cases, including `analog` and `script-only`) - `python -I scripts/hookshot_mcp.py --check --game-dir <fake-folder>` - `python -I -B tests/test_pr_check.py` - `python -I -B tests/test_shot_request.py --cxx g++` - `python -I -B tests/test_docs.py` - `python -I -B tests/run_all.py pr_check shot_request docs --strict` The Vulkan/WebGPU C++ changes could not be compiled in this environment because there is no game build, Vulkan headers or dump here. The input-parser changes were compiled and exercised through the existing fixture; the extracted shot-request parser was compiled and tested separately. The full local `tests/run_all.py --strict` run gave 71 passed, 11 failed, 9 skipped; the failures are environment/tool gaps here (missing SPIR-V/Vulkan headers, `clang`/`clang++`, and Windows symlink privileges in `nx2/test_nup_paths.py`), not the new tests. All three new checks passed. ## Known issue in the first commit The first commit `e9c2b0d` predates the new `pr_check` gate. It is not a Conventional Commit and lacks the `Co-Authored-By` trailer, so `python -I -B scripts/pr_check.py --base main --no-tests` reports it. The three follow-up commits pass. I did not rewrite the published branch; if a rebase/amend is acceptable, the subject should become `feat: add the Hookshot MCP server for scripted in-game testing` and the body should end with both trailers from `docs/design.md` section 8. No game files, builds, saves or generated screenshots are included.

👍 1

nicholasgower commented

2026-11-08 23:12:50 +00:00

(Migrated from github.com)

I'm also working on a save file editor for this project, and I've decided I will not continue work on this until after the MCP server is merged. Easily automating user input is impossible without a functional MCP server, and I'd rather not combine PRs prematurely. #784

I'm also working on a save file editor for this project, and I've decided I will not continue work on this until after the MCP server is merged. Easily automating user input is impossible without a functional MCP server, and I'd rather not combine PRs prematurely. https://git.magicbean.dev/MagicBeanDev/Hookshot/pulls/784

hckrman101 commented

2026-11-09 00:49:15 +00:00

(Migrated from github.com)

Nice work on this; the game-side seams in particular (script-only input, !quit, the one-shot screenshot request) are things I've wanted too.

I've been running something similar locally for a while, but as a few small command-line tools instead of an MCP server, so I wanted to share how that's turned out. I think the core of this would be more useful as a CLI, with MCP (if wanted) as a thin layer on top.

What I use, roughly:

oot hookshot [--dir D] starts a build with its own data folder (XDG_DATA_HOME), then gets into the newest save by itself. It appends presses to the HOS_INPUT_LIVE file, and decides when to press from the log rather than from sleeps: the per-second a frame: N draws line is about 2 in the intro, 15–70 in menus, about 5 while loading, and hundreds in play.
A run harness: every run gets its own folder with a copy of a pristine save, a log, and screenshots (RX_VK_SHOTS). Results are files, plus one summary line per run in a text file (fps, flash count from the frame log, …), so two builds can be A/B'd by diffing.
oot-status is a small GPU queue/lock. With several builds and test runs going at once, I needed runs to stop stepping on each other:
- exclusive (alone on the GPU), shared (screenshot runs), timing (exclusive, and no builds running beside it, so fps numbers mean something), and build (no GPU slot);
- FIFO with priorities;
- "only while I'm not playing" for background runs.
hookshot-build wraps the build with a shared object cache across checkouts. A second checkout of the same commit builds in about 20 s instead of about 12 min.
oot-compare loads the same save in Hookshot and in an emulator and takes side-by-side screenshots. The save files are the same format in both. I wrote it up here: https://gist.github.com/hckrman101/701e15b643b7ce32f778116f3004b1a4

Why a CLI rather than only MCP:

Anyone can use it: a person in a terminal, CI, a shell script, or any coding agent, since all of them can run commands. An MCP server needs an MCP host to be useful, and it's harder to poke at by hand when something goes wrong.
It composes. Exit codes, stdout, and files on disk mean you can write launch && wait-for play && shot && stop, loop over builds, or diff two runs' logs, without a protocol in between.
Screenshots as files, not inline images. A path is cheap. Returning the PNG as image content each time is expensive for an agent's context and gone once the session ends. With files, you can look at the ones that matter, keep them for a PR, or compare them later.
State lives in a run folder, not in a long-lived server. If the tool or the agent restarts, the run is still there: pid, live-input file, log, shots. status just reads it back.
Easier to test. Each command is a plain function with arguments and output.

What I'd suggest: keep the seams as they are, and make scripts/hookshot_mcp.py's tools into subcommands of one script. For example, hookshot-test launch|press|stick|sequence|shot|wait|logs|status|stop --run <dir>, where launch writes the run folder and the rest read it. Then hookshot_mcp.py can be ~100 lines that call those same functions. Two things I'd add that have saved me a lot of time:

wait on game state (log signals: draw counts, "loading", the first present) instead of seconds.
A lock around GPU use, so parallel runs and builds don't skew each other's timings.

Happy to compare notes or help with any of it.

Nice work on this; the game-side seams in particular (script-only input, `!quit`, the one-shot screenshot request) are things I've wanted too. I've been running something similar locally for a while, but as a few small command-line tools instead of an MCP server, so I wanted to share how that's turned out. I think the core of this would be more useful as a CLI, with MCP (if wanted) as a thin layer on top. **What I use, roughly:** - **`oot hookshot [--dir D]`** starts a build with its own data folder (`XDG_DATA_HOME`), then gets into the newest save by itself. It appends presses to the `HOS_INPUT_LIVE` file, and decides when to press from the log rather than from sleeps: the per-second `a frame: N draws` line is about 2 in the intro, 15–70 in menus, about 5 while loading, and hundreds in play. - **A run harness:** every run gets its own folder with a copy of a pristine save, a log, and screenshots (`RX_VK_SHOTS`). Results are files, plus one summary line per run in a text file (fps, flash count from the frame log, …), so two builds can be A/B'd by diffing. - **`oot-status`** is a small GPU queue/lock. With several builds and test runs going at once, I needed runs to stop stepping on each other: - exclusive (alone on the GPU), shared (screenshot runs), timing (exclusive, and no builds running beside it, so fps numbers mean something), and build (no GPU slot); - FIFO with priorities; - "only while I'm not playing" for background runs. - **`hookshot-build`** wraps the build with a shared object cache across checkouts. A second checkout of the same commit builds in about 20 s instead of about 12 min. - **`oot-compare`** loads the same save in Hookshot and in an emulator and takes side-by-side screenshots. The save files are the same format in both. I wrote it up here: https://gist.github.com/hckrman101/701e15b643b7ce32f778116f3004b1a4 **Why a CLI rather than only MCP:** - **Anyone can use it:** a person in a terminal, CI, a shell script, or any coding agent, since all of them can run commands. An MCP server needs an MCP host to be useful, and it's harder to poke at by hand when something goes wrong. - **It composes.** Exit codes, stdout, and files on disk mean you can write `launch && wait-for play && shot && stop`, loop over builds, or diff two runs' logs, without a protocol in between. - **Screenshots as files, not inline images.** A path is cheap. Returning the PNG as image content each time is expensive for an agent's context and gone once the session ends. With files, you can look at the ones that matter, keep them for a PR, or compare them later. - **State lives in a run folder, not in a long-lived server.** If the tool or the agent restarts, the run is still there: pid, live-input file, log, shots. `status` just reads it back. - **Easier to test.** Each command is a plain function with arguments and output. **What I'd suggest:** keep the seams as they are, and make `scripts/hookshot_mcp.py`'s tools into subcommands of one script. For example, `hookshot-test launch|press|stick|sequence|shot|wait|logs|status|stop --run <dir>`, where `launch` writes the run folder and the rest read it. Then `hookshot_mcp.py` can be ~100 lines that call those same functions. Two things I'd add that have saved me a lot of time: 1. **`wait` on game state** (log signals: draw counts, "loading", the first present) instead of seconds. 2. **A lock around GPU use**, so parallel runs and builds don't skew each other's timings. Happy to compare notes or help with any of it.

NotRexed commented

2026-11-09 05:30:20 +00:00

(Migrated from github.com)

Closing to keep the queue clear — please rebase on current main and reopen; thanks!

nicholasgower commented

2026-11-09 06:48:24 +00:00

(Migrated from github.com)

Rebased onto current main (b2765ba) and opened again as #1025, with the real-game verification and the --game-dir fix from that testing included.

Rebased onto current `main` (`b2765ba`) and opened again as #1025, with the real-game verification and the `--game-dir` fix from that testing included.

github-sync referenced this pull request

2026-11-12 17:57:37 +00:00

Add an MCP server for scripted in-game testing #1025

Rows
Columns

Add MCP server for scripted in-game testing #764

What this adds

Follow-up checks

Verification

Known issue in the first commit