Game menus: point and click with the mouse in the title, pause menu, panel pages, options and dialogs #1500

2026-11-10T05:21:17Z

jordanella commented

2026-11-10 05:21:17 +00:00

(Migrated from github.com)

Summary

Point-and-click mouse support in Ocarina of Time's own menus (Windows):

Menus covered: the title screen and file list, the pause menu (all four pages), the panel's Key Items / Threads of Time / Map pages, the in-game options, special actions and the save dialogs.
Hover: gives an entry the focus through the game's own focus setter, so it's silent and the game's state stays consistent.
Clicks: left-click is A and right-click is B. The tabs, the L/R signs and the Back prompt can be clicked.
Wheel: turns the pause menu's pages. It scrolls lists by focusing the row just out of view, the same way the pad scrolls them. On the map it changes floors and zooms.
Drag: a left drag pans the map and the tapestry, turns a stone on Key Items, and turns the chosen item on the pause menu's Items page.
R item row: the wheel steps through the items you choose for ZR. Each notch is one D-pad press, queued so presses never overlap. The mouse stays with the camera there: no cursor, no pointing.
Cursor: stays hidden while a pad or the keyboard is in use, until the mouse moves. The wheel still works while it is hidden.
Leaving a menu: the camera gets the mouse back about 0.1 s after the menu goes.

How it works

nx2/oot_patches.py adds two calls at the heads of lifted functions, in the supported build only (protect_hearts):
- the pause menu's focus handler (f_802ed660)
- every menu screen's update (f_802f1380)
A lift that is patched again is left as it is.
host/oot_menu.c:
- Tracks the screens and gives the mouse to the one in front.
- Hit-tests the buttons' own hit boxes (button+0x58) in the layout's 3840×2160 units.
- Calls the game's setters on the game's thread:
  - the focus setter f_8088ffc0
  - on Key Items, the page's own key and refresh (f_833fda00)
The pad is used only where the game has no setter: A/B, L/R, the map's floors.
host/display.c: frees the mouse while a game menu runs (300 ms without a frame captures it again) and hides the cursor until the mouse is used.
host/nn_hid.c: host-injected presses (hos_hid_tap, in the input script's syntax), a held stick, and a drag mapped like the camera's mouse (HOS_MENU_DRAG_GAIN, default 6).
Guest reads are bounds-checked memcpy (commit_on_touch covers reserved pages), not ReadProcessMemory.
One lock serialises the mouse state across the game's worker threads. The focus hook never waits for it: when it is busy, the change is kept and applied at the next tick.

Tests

tests/test_oot_menu_mouse.py (Windows): compiles oot_menu.c against a fake guest memory. It covers:
- pause menu: hover → setter, click → A, right-click → B, the Back prompt over a button's box, the L/R signs, the wheel
- mirrored half-width slot boxes
- list rows hit across the line and in the gaps between rows
- the wheel moving the list focus through the setter
tests/test_oot_menu_mouse.py also covers:
- a focus change reported from another thread while the setter holds the lock
- the R item row's queued D-pad presses and their cap
- the wheel with the cursor hidden
- the menus-closed notice that hands the mouse back
tests/test_menu_hooks.py: each hook is inserted first and once, re-applying is harmless, and only the supported build's lift gets the hooks.
tests/test_input_background.py and tests/screensaver_test.c: stand-ins for the new code inside the parts of nn_hid.c and display.c they compile.
tests/run_all.py: no new failures compared with main on the same machine.
Played through by hand on the supported build: child and adult saves, dungeon and area maps, options, title, saves.

Since the review

Done:
- The focus hook no longer blocks on the mouse lock.
- pause_screen is read and written atomically.
- The cursor is hidden as soon as a game menu opens, so there's no flash for controller players.
Not done:
- No off switch: left out on purpose.
- Swapped mouse buttons: still read from the physical button.
- A single host-input entry point: not attempted; it can follow.
#1428: whichever PR lands second reworks the pointer globals.

Known limits / follow-ups

Addresses are this build's. The hooks and setters are fixed addresses, gated to the supported build. A follow-up will find them by signature at lift time and fail closed.
Positions measured from screenshots: some positions are measured in English:
- the tab bar
- the Back/Y prompts
- the map's floor rows and area-map picture
- Key Items' item centres
Other languages' layouts haven't been checked yet.
Native mouse camera: it doesn't respond with the slingshot or boomerang out. That's an existing issue, separate from this PR.

🤖 Generated with Claude Code

## Summary Point-and-click mouse support in Ocarina of Time's own menus (Windows): - **Menus covered:** the title screen and file list, the pause menu (all four pages), the panel's Key Items / Threads of Time / Map pages, the in-game options, special actions and the save dialogs. - **Hover:** gives an entry the focus through the game's own focus setter, so it's silent and the game's state stays consistent. - **Clicks:** left-click is A and right-click is B. The tabs, the L/R signs and the Back prompt can be clicked. - **Wheel:** turns the pause menu's pages. It scrolls lists by focusing the row just out of view, the same way the pad scrolls them. On the map it changes floors and zooms. - **Drag:** a left drag pans the map and the tapestry, turns a stone on Key Items, and turns the chosen item on the pause menu's Items page. - **R item row:** the wheel steps through the items you choose for ZR. Each notch is one D-pad press, queued so presses never overlap. The mouse stays with the camera there: no cursor, no pointing. - **Cursor:** stays hidden while a pad or the keyboard is in use, until the mouse moves. The wheel still works while it is hidden. - **Leaving a menu:** the camera gets the mouse back about 0.1 s after the menu goes. ## How it works - `nx2/oot_patches.py` adds two calls at the heads of lifted functions, in the supported build only (`protect_hearts`): - the pause menu's focus handler (`f_802ed660`) - every menu screen's update (`f_802f1380`) A lift that is patched again is left as it is. - **`host/oot_menu.c`:** - Tracks the screens and gives the mouse to the one in front. - Hit-tests the buttons' own hit boxes (`button+0x58`) in the layout's 3840×2160 units. - Calls the game's setters on the game's thread: - the focus setter `f_8088ffc0` - on Key Items, the page's own key and refresh (`f_833fda00`) The pad is used only where the game has no setter: A/B, L/R, the map's floors. - **`host/display.c`:** frees the mouse while a game menu runs (300 ms without a frame captures it again) and hides the cursor until the mouse is used. - **`host/nn_hid.c`:** host-injected presses (`hos_hid_tap`, in the input script's syntax), a held stick, and a drag mapped like the camera's mouse (`HOS_MENU_DRAG_GAIN`, default 6). - Guest reads are bounds-checked `memcpy` (`commit_on_touch` covers reserved pages), not `ReadProcessMemory`. - One lock serialises the mouse state across the game's worker threads. The focus hook never waits for it: when it is busy, the change is kept and applied at the next tick. ## Tests - `tests/test_oot_menu_mouse.py` (Windows): compiles `oot_menu.c` against a fake guest memory. It covers: - pause menu: hover → setter, click → A, right-click → B, the Back prompt over a button's box, the L/R signs, the wheel - mirrored half-width slot boxes - list rows hit across the line and in the gaps between rows - the wheel moving the list focus through the setter - `tests/test_oot_menu_mouse.py` also covers: - a focus change reported from another thread while the setter holds the lock - the R item row's queued D-pad presses and their cap - the wheel with the cursor hidden - the menus-closed notice that hands the mouse back - `tests/test_menu_hooks.py`: each hook is inserted first and once, re-applying is harmless, and only the supported build's lift gets the hooks. - `tests/test_input_background.py` and `tests/screensaver_test.c`: stand-ins for the new code inside the parts of `nn_hid.c` and `display.c` they compile. - `tests/run_all.py`: no new failures compared with `main` on the same machine. - Played through by hand on the supported build: child and adult saves, dungeon and area maps, options, title, saves. ## Since the review - **Done:** - The focus hook no longer blocks on the mouse lock. - `pause_screen` is read and written atomically. - The cursor is hidden as soon as a game menu opens, so there's no flash for controller players. - **Not done:** - **No off switch:** left out on purpose. - **Swapped mouse buttons:** still read from the physical button. - **A single host-input entry point:** not attempted; it can follow. - **#1428:** whichever PR lands second reworks the pointer globals. ## Known limits / follow-ups - **Addresses are this build's.** The hooks and setters are fixed addresses, gated to the supported build. A follow-up will find them by signature at lift time and fail closed. - **Positions measured from screenshots:** some positions are measured in English: - the tab bar - the Back/Y prompts - the map's floor rows and area-map picture - Key Items' item centres Other languages' layouts haven't been checked yet. - **Native mouse camera:** it doesn't respond with the slingshot or boomerang out. That's an existing issue, separate from this PR. 🤖 Generated with [Claude Code](https://claude.com/claude-code)

hckrman101 (Migrated from github.com) approved these changes

2026-11-10 05:49:07 +00:00

hckrman101 (Migrated from github.com) left a comment

Nice work. The way it's built (hooks at the heads of f_802ed660 / f_802f1380, gated to the supported build, then driving the game's own focus setter and Key Items refresh instead of faking pad input everywhere) is the right call, and the Ctx save/restore around the guest calls follows the pattern hos_main.c already uses. I also checked the plumbing:

The hooks are idempotent and counted in done like the other protect_hearts patches.
On Linux the hooks still get inserted, but oot_menu.c's #else stubs keep the link happy, and nn_hid.c's new SRW lock and 64-bit interlocked calls map onto host/posix/windows.h.
Host taps don't change script_end_ms, so HOS_INPUT_EXCLUSIVE / scripted runs behave as before. HOS_INPUT_BACKGROUND is unaffected because the pointer is only read while the window is in front.
No new option ids. The new tests are picked up by run_all.py's glob.

Things worth a look, none blocking:

No off switch. This changes mouse behaviour for every Windows player: the mouse is freed on the title screen, in the pause menu and in in-game dialogs. It also rests on fixed addresses and layout positions measured in English. An Insert-menu row or an env var to turn it off would give people a way out if a screen misbehaves. If you add a row, ids 10, 11, 23, 27 and 28 are taken by open PRs.
Swapped mouse buttons (inline): the drag check reads the physical left button.
Focus hook lock (inline): a blocking acquire in a guest hook while another game thread can hold the same lock across guest calls. Low risk, but cheap to avoid.
Overlap with #1428: it replaces display.c's ptr_x/ptr_y/... globals with a HosPointer sample. This PR reads and writes those globals directly, so whichever lands second will need a rework there, not just a rebase.
Nit: pause_screen and ticks are written under mouse_lock but read in hos_oot_menu_tick under seen_lock only. That's harmless on x64 with aligned 64-bit fields, but it's worth a comment or an atomic read.
Nit: when a game menu opens while the mouse was captured, mouse_capture(w, 0) calls ShowCursor(TRUE), and the cursor stays visible until the next WM_SETCURSOR or the 250 ms timer hides it. A controller player may see a brief flash. Calling SetCursor(NULL) right in menus_mouse would avoid that.

Optional simplification: hos_hid_stick (held stick until a deadline) and hos_hid_drag (decaying stick) are two separate host-stick injectors, and hos_hid_tap is a third input path into nn_hid.c. #926 (control bridge) and #1025 (MCP server) also inject input there. One small "host input" entry point (taps plus a stick sample with an optional decay) would keep that surface smaller. It's fine as it is if you'd rather not.

CI was still pending when I looked.

Nice work. The way it's built (hooks at the heads of `f_802ed660` / `f_802f1380`, gated to the supported build, then driving the game's own focus setter and Key Items refresh instead of faking pad input everywhere) is the right call, and the Ctx save/restore around the guest calls follows the pattern `hos_main.c` already uses. I also checked the plumbing: - The hooks are idempotent and counted in `done` like the other protect_hearts patches. - On Linux the hooks still get inserted, but `oot_menu.c`'s `#else` stubs keep the link happy, and `nn_hid.c`'s new SRW lock and 64-bit interlocked calls map onto `host/posix/windows.h`. - Host taps don't change `script_end_ms`, so `HOS_INPUT_EXCLUSIVE` / scripted runs behave as before. HOS_INPUT_BACKGROUND is unaffected because the pointer is only read while the window is in front. - No new option ids. The new tests are picked up by `run_all.py`'s glob. Things worth a look, none blocking: - **No off switch.** This changes mouse behaviour for every Windows player: the mouse is freed on the title screen, in the pause menu and in in-game dialogs. It also rests on fixed addresses and layout positions measured in English. An Insert-menu row or an env var to turn it off would give people a way out if a screen misbehaves. If you add a row, ids 10, 11, 23, 27 and 28 are taken by open PRs. - **Swapped mouse buttons** (inline): the drag check reads the physical left button. - **Focus hook lock** (inline): a blocking acquire in a guest hook while another game thread can hold the same lock across guest calls. Low risk, but cheap to avoid. - **Overlap with #1428**: it replaces display.c's `ptr_x/ptr_y/...` globals with a `HosPointer` sample. This PR reads and writes those globals directly, so whichever lands second will need a rework there, not just a rebase. - Nit: `pause_screen` and `ticks` are written under `mouse_lock` but read in `hos_oot_menu_tick` under `seen_lock` only. That's harmless on x64 with aligned 64-bit fields, but it's worth a comment or an atomic read. - Nit: when a game menu opens while the mouse was captured, `mouse_capture(w, 0)` calls `ShowCursor(TRUE)`, and the cursor stays visible until the next WM_SETCURSOR or the 250 ms timer hides it. A controller player may see a brief flash. Calling `SetCursor(NULL)` right in `menus_mouse` would avoid that. Optional simplification: `hos_hid_stick` (held stick until a deadline) and `hos_hid_drag` (decaying stick) are two separate host-stick injectors, and `hos_hid_tap` is a third input path into nn_hid.c. #926 (control bridge) and #1025 (MCP server) also inject input there. One small "host input" entry point (taps plus a stick sample with an optional decay) would keep that surface smaller. It's fine as it is if you'd rather not. CI was still pending when I looked.

host/oot_menu.c

					
				@ -0,0 +1,817 @@

				/* The mouse in the game's own menus: the title and its lists, the pause menu, the panel of

This is a blocking acquire inside a guest function (the pause menu's focus handler). Meanwhile mouse_tick can hold the same lock on another game thread while it runs guest code (set_focus, guest_call). If that guest code ever waits on something the thread blocked here holds (a job, a guest mutex), the two deadlock. I doubt it happens with today's call graph, but focus_locked only writes a handful of words. Making those plain atomics, or using TryAcquire here and stashing the pane for the next tick, would remove the risk completely.

This is a blocking acquire inside a guest function (the pause menu's focus handler). Meanwhile `mouse_tick` can hold the same lock on another game thread while it runs guest code (`set_focus`, `guest_call`). If that guest code ever waits on something the thread blocked here holds (a job, a guest mutex), the two deadlock. I doubt it happens with today's call graph, but `focus_locked` only writes a handful of words. Making those plain atomics, or using `TryAcquire` here and stashing the pane for the next tick, would remove the risk completely.

host/oot_menu.c

					
				@ -0,0 +395,4 @@

				    const int item_model = pause && page_now == 2, pan = page >= 0 || item_model;

				    static int ldown, ldrag, lx0, ly0, llx, lly;

				    if (pan) {

				        const int held = (GetAsyncKeyState(VK_LBUTTON) & 0x8000) != 0;

GetAsyncKeyState reports the physical buttons, but WM_LBUTTONDOWN is the logical one. With "swap primary and secondary buttons" on (left-handed setups), the press arrives from the physical right button, held reads 0 on the next tick, and every drag turns into an immediate click. You could track the logical left-button state in display.c (WM_LBUTTONDOWN/WM_LBUTTONUP are already handled there) and hand it out through hos_mouse_game_pointer. Or check GetSystemMetrics(SM_SWAPBUTTON) and read VK_RBUTTON instead.

`GetAsyncKeyState` reports the *physical* buttons, but `WM_LBUTTONDOWN` is the logical one. With "swap primary and secondary buttons" on (left-handed setups), the press arrives from the physical right button, `held` reads 0 on the next tick, and every drag turns into an immediate click. You could track the logical left-button state in display.c (`WM_LBUTTONDOWN`/`WM_LBUTTONUP` are already handled there) and hand it out through `hos_mouse_game_pointer`. Or check `GetSystemMetrics(SM_SWAPBUTTON)` and read `VK_RBUTTON` instead.

jordanella commented

2026-11-10 15:44:19 +00:00

(Migrated from github.com)

Thanks for the review. Follow-ups are pushed:

Focus hook lock: it no longer blocks. It tries the lock, and if mouse_tick holds it the focus change is kept and applied at the next tick. A test covers it: a focus change reported from another thread while the setter holds the lock.
pause_screen: now read and written atomically, with a comment. ticks is counted the same way.
Cursor flash: the cursor is hidden as soon as a game menu opens.
Off switch: left out on purpose.
Swapped mouse buttons and a single host-input entry point: not in this PR.
#1428: noted. Whichever lands second reworks the pointer globals.

Since then I've also added a drag to turn the item on the Items page and a wheel for the R item row (one D-pad press per notch, with the mouse kept for the camera). The wheel now works while the cursor is hidden, and the mouse returns to the camera about 0.1 s after a menu closes. Main is merged in, and the branch is conflict-free again.

Thanks for the review. Follow-ups are pushed: - **Focus hook lock:** it no longer blocks. It tries the lock, and if `mouse_tick` holds it the focus change is kept and applied at the next tick. A test covers it: a focus change reported from another thread while the setter holds the lock. - **`pause_screen`:** now read and written atomically, with a comment. `ticks` is counted the same way. - **Cursor flash:** the cursor is hidden as soon as a game menu opens. - **Off switch:** left out on purpose. - **Swapped mouse buttons and a single host-input entry point:** not in this PR. - **#1428:** noted. Whichever lands second reworks the pointer globals. Since then I've also added a drag to turn the item on the Items page and a wheel for the R item row (one D-pad press per notch, with the mouse kept for the camera). The wheel now works while the cursor is hidden, and the mouse returns to the camera about 0.1 s after a menu closes. Main is merged in, and the branch is conflict-free again.

jordanella (Migrated from github.com) reviewed

2026-11-10 15:44:21 +00:00

host/oot_menu.c

					
				@ -0,0 +1,817 @@

				/* The mouse in the game's own menus: the title and its lists, the pause menu, the panel of

Fixed: the hook only tries the lock now, and keeps the change for the next tick when it's busy. tests/test_oot_menu_mouse.py reproduces the two-thread case.

Fixed: the hook only tries the lock now, and keeps the change for the next tick when it's busy. `tests/test_oot_menu_mouse.py` reproduces the two-thread case.

hckrman101 (Migrated from github.com) approved these changes

2026-11-10 15:56:05 +00:00

hckrman101 (Migrated from github.com) left a comment

Re-reviewed the new commits (ba20577..891d6a6, plus the main merge). Looks good to me.

Focus hook: fixed. Try-lock plus the kept change works, and a direct hand-over clears any older kept one, so a late pending change can't overwrite a newer focus. The two-thread test covers the case I was worried about.
pause_screen / ticks atomics and hiding the cursor on open both look right.
The Items-page drag, the R item-row wheel (the queued D-pad taps fit the hos_hid_tap script syntax and the 256-byte buffer) and the 50 ms "menus gone" hand-back all read correctly. Hidden-cursor mode returning 2 (wheel only, no presses or pointing) makes sense.
Swapped buttons: fine to leave for a follow-up. Just note that the new Items drag also goes through GetAsyncKeyState(VK_LBUTTON), so it has the same left-handed caveat.

CI: the 8 failures from before are gone after the merge. The one left, test_shader_stamp_cache.py (NameError: logo_rgba), comes from main at 6add2f1 (#1567) and is already fixed on main by #1575. It isn't caused by this PR, and merging main again should turn both Python jobs green.

Re-reviewed the new commits (ba20577..891d6a6, plus the main merge). Looks good to me. - Focus hook: fixed. Try-lock plus the kept change works, and a direct hand-over clears any older kept one, so a late pending change can't overwrite a newer focus. The two-thread test covers the case I was worried about. - `pause_screen` / `ticks` atomics and hiding the cursor on open both look right. - The Items-page drag, the R item-row wheel (the queued D-pad taps fit the `hos_hid_tap` script syntax and the 256-byte buffer) and the 50 ms "menus gone" hand-back all read correctly. Hidden-cursor mode returning 2 (wheel only, no presses or pointing) makes sense. - Swapped buttons: fine to leave for a follow-up. Just note that the new Items drag also goes through `GetAsyncKeyState(VK_LBUTTON)`, so it has the same left-handed caveat. CI: the 8 failures from before are gone after the merge. The one left, `test_shader_stamp_cache.py` (`NameError: logo_rgba`), comes from main at 6add2f1 (#1567) and is already fixed on main by #1575. It isn't caused by this PR, and merging main again should turn both Python jobs green.

Commenting is not possible because the repository is archived.

No reviewers

hckrman101