WIP: fix(hle): sizes and names the game passes are checked to lie in guest memory in more host functions #1179

Draft
TrashPanda wants to merge 4 commits from TrashPanda/Longshot:fix/hle-guest-ranges into main
Contributor

Type: fix · Risk: low · Tested on: Linux, RTX 4070 Ti (game-free checks) · Needs: none

Why

Several host functions copied or read through a pointer and size the game gave them without checking they stay inside the game's memory, so a bad value could read or write the host's own memory (wrong sound, a crash in the mixer). This is the guest-memory part of #218, split out so it can be reviewed and merged on its own.

What changed

nn::audio wave buffers and coefficients, GenerateRandomBytes, the HID state copies (at most 16 states, as the console keeps), the socket Poll stand-in, the debug log lines, the event-name copies, the item key and NVN's proc names go through hos_guest_range or the new guest_name.

Commits, each with its original author and message:

d2f82d90 fix(hle): spans the game names are checked to lie in guest memory in more host functions (hckrman101)

GP() turns a guest address into a host pointer without a check, so a
size from the game walks a copy past the guest window into the host's
own memory. nn::fs and the libc copy hooks were moved onto
hos_guest_range before; these were not:

  • nn::audio AppendWaveBuffer took a wave buffer's data and ADPCM context
    as given, and the mixer read them for as long as the voice played
    (the host's memory into the sound, or a fault in the mixer); a buffer
    outside guest memory is now refused. AcquireVoiceSlot's coefficients
    likewise;
  • nn::os::GenerateRandomBytes wrote ARG(1) bytes from ARG(0);
  • nn::hid GetNpadStates wrote any count of states (now at most 16, as
    the console keeps) and GetSixAxisSensorStates its states unchecked;
  • the socket Poll stand-in cleared up to 4096 descriptors unchecked;
  • OutputDebugString and nn::diag's log lines copied up to 4 KB into the
    log from an unchecked pointer;
  • the event-name copies (hos_main.c) and the item key (oot_menu.c, whose
    own bound could wrap) wrote through pointers read from game objects.

New tests/test_hle_guest_ranges.py runs wb_in_guest and checks the
functions' text; the wave-buffer and input fixtures have hos_guest_range
over their own guest memory.

Co-Authored-By: Claude Opus 5.5 [email protected]

5a0cf75f fix(nvn): read the proc names the game asks for only inside guest memory (hckrman101)

nvnDeviceGetProcAddress and nvnBootstrapLoader read the name the game
passes in place, to its end: a pointer near the guest window's end, or
a name with no end, read past the guest's memory into the host's. A new
guest_name copies at most 255 characters, only from inside the window,
and gives "" when there is no string there (which asks for no proc).

tests/test_hle_guest_ranges.py checks both callers and runs guest_name.

Co-Authored-By: Claude Opus 5.5 [email protected]

62d066f0 test: the oot_menu mouse fixture's stand-in hos.h has hos_guest_range (Joey Magnus)
6263697b test: the biquad and teleport fixtures have a hos_guest_range over their own guest memory (Joey Magnus)

Measurements

Verdict: nothing to measure: a correctness fix off the frame path, covered by game-free checks.

Flags and settings

None.

Testing

Tested: On Linux (kernel 7.0), RTX 4070 Ti, main @ 71b1edd8, this PR @ 6263697b: tests/test_hle_guest_ranges.py, tests/test_audio_wave_buffer_release.py, tests/test_oot_menu_mouse.py pass. The full suite (tests/run_all.py) has not been run on this branch yet; a comment will follow when it has.

Not tested: A play session with this build. Opened as work in progress until it runs in the game.

Must be tested:

  • Any OS (by hand): play 10 minutes with sound, the ocarina and the Insert menu, expecting no change

Risks

What could break: audio, input and NVN calls if a check refused a valid span.

Turning it off: revert the commit.

Compatibility: Nothing changes.

Checklist

Everyone

  • One topic.
  • The title is a Conventional Commit (type(scope): description) of the status line's type.
  • Why gives the reason plainly and links the issues that fit (the tracker searched).
  • Tested on my own PC where it runs (in the game, for a change the game runs), as Testing's Tested says; what
    can't be run by hand is in Testing's Not testable by hand, with why.
  • Measurements has a verdict, or one line saying why there is nothing to measure.
  • Nothing gets worse that I know of: any cost (speed, latency, memory, a setup that worked before) is measured and
    in Risks.
  • Testing names what was tested, what wasn't, and what must still be tested, where and how.
  • Rebased on current main.
  • No game files, dumps, build output or generated files.
  • third_party/ untouched (a new folder with a PROVENANCE.md instead).
  • The license is OK: GPL-3.0-or-later, nothing incompatible and nothing decompiled.
  • python -I -B tests/run_all.py passes (any failure that also fails on main is named).

When it applies

  • A game-free test added for the fix, failing on main and passing here.
  • AI assistance recorded with Assisted-by: / Co-Authored-By: trailers.
**Type:** fix · **Risk:** low · **Tested on:** Linux, RTX 4070 Ti (game-free checks) · **Needs:** none ## Why Several host functions copied or read through a pointer and size the game gave them without checking they stay inside the game's memory, so a bad value could read or write the host's own memory (wrong sound, a crash in the mixer). This is the guest-memory part of [#218](https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/218), split out so it can be reviewed and merged on its own. ## What changed nn::audio wave buffers and coefficients, `GenerateRandomBytes`, the HID state copies (at most 16 states, as the console keeps), the socket Poll stand-in, the debug log lines, the event-name copies, the item key and NVN's proc names go through `hos_guest_range` or the new `guest_name`. Commits, each with its original author and message: <details><summary><code>d2f82d90</code> fix(hle): spans the game names are checked to lie in guest memory in more host functions (hckrman101)</summary> GP() turns a guest address into a host pointer without a check, so a size from the game walks a copy past the guest window into the host's own memory. nn::fs and the libc copy hooks were moved onto hos_guest_range before; these were not: - nn::audio AppendWaveBuffer took a wave buffer's data and ADPCM context as given, and the mixer read them for as long as the voice played (the host's memory into the sound, or a fault in the mixer); a buffer outside guest memory is now refused. AcquireVoiceSlot's coefficients likewise; - nn::os::GenerateRandomBytes wrote ARG(1) bytes from ARG(0); - nn::hid GetNpadStates wrote any count of states (now at most 16, as the console keeps) and GetSixAxisSensorStates its states unchecked; - the socket Poll stand-in cleared up to 4096 descriptors unchecked; - OutputDebugString and nn::diag's log lines copied up to 4 KB into the log from an unchecked pointer; - the event-name copies (hos_main.c) and the item key (oot_menu.c, whose own bound could wrap) wrote through pointers read from game objects. New tests/test_hle_guest_ranges.py runs wb_in_guest and checks the functions' text; the wave-buffer and input fixtures have hos_guest_range over their own guest memory. Co-Authored-By: Claude Opus 5.5 <[email protected]> </details> <details><summary><code>5a0cf75f</code> fix(nvn): read the proc names the game asks for only inside guest memory (hckrman101)</summary> nvnDeviceGetProcAddress and nvnBootstrapLoader read the name the game passes in place, to its end: a pointer near the guest window's end, or a name with no end, read past the guest's memory into the host's. A new guest_name copies at most 255 characters, only from inside the window, and gives "" when there is no string there (which asks for no proc). tests/test_hle_guest_ranges.py checks both callers and runs guest_name. Co-Authored-By: Claude Opus 5.5 <[email protected]> </details> <details><summary><code>62d066f0</code> test: the oot_menu mouse fixture's stand-in hos.h has hos_guest_range (Joey Magnus)</summary> </details> <details><summary><code>6263697b</code> test: the biquad and teleport fixtures have a hos_guest_range over their own guest memory (Joey Magnus)</summary> </details> ## Measurements **Verdict:** nothing to measure: a correctness fix off the frame path, covered by game-free checks. ### Flags and settings None. ## Testing **Tested:** On Linux (kernel 7.0), RTX 4070 Ti, `main` @ 71b1edd8, this PR @ 6263697b: `tests/test_hle_guest_ranges.py`, `tests/test_audio_wave_buffer_release.py`, `tests/test_oot_menu_mouse.py` pass. The full suite (`tests/run_all.py`) has not been run on this branch yet; a comment will follow when it has. **Not tested:** A play session with this build. Opened as work in progress until it runs in the game. **Must be tested:** - Any OS (by hand): play 10 minutes with sound, the ocarina and the Insert menu, expecting no change ## Risks **What could break:** audio, input and NVN calls if a check refused a valid span. **Turning it off:** revert the commit. **Compatibility:** Nothing changes. ## Checklist ### Everyone - [x] One topic. - [x] The title is a Conventional Commit (`type(scope): description`) of the status line's type. - [x] Why gives the reason plainly and links the issues that fit (the tracker searched). - [x] Tested on my own PC where it runs (in the game, for a change the game runs), as Testing's Tested says; what can't be run by hand is in Testing's Not testable by hand, with why. - [x] Measurements has a verdict, or one line saying why there is nothing to measure. - [x] Nothing gets worse that I know of: any cost (speed, latency, memory, a setup that worked before) is measured and in Risks. - [x] Testing names what was tested, what wasn't, and what must still be tested, where and how. - [x] Rebased on current `main`. - [x] No game files, dumps, build output or generated files. - [x] `third_party/` untouched (a new folder with a `PROVENANCE.md` instead). - [x] The license is OK: GPL-3.0-or-later, nothing incompatible and nothing decompiled. - [ ] `python -I -B tests/run_all.py` passes (any failure that also fails on `main` is named). ### When it applies - [x] A game-free test added for the fix, failing on `main` and passing here. - [x] AI assistance recorded with `Assisted-by:` / `Co-Authored-By:` trailers.
GP() turns a guest address into a host pointer without a check, so a
size from the game walks a copy past the guest window into the host's
own memory. nn::fs and the libc copy hooks were moved onto
hos_guest_range before; these were not:
- nn::audio AppendWaveBuffer took a wave buffer's data and ADPCM context
  as given, and the mixer read them for as long as the voice played
  (the host's memory into the sound, or a fault in the mixer); a buffer
  outside guest memory is now refused. AcquireVoiceSlot's coefficients
  likewise;
- nn::os::GenerateRandomBytes wrote ARG(1) bytes from ARG(0);
- nn::hid GetNpadStates wrote any count of states (now at most 16, as
  the console keeps) and GetSixAxisSensorStates its states unchecked;
- the socket Poll stand-in cleared up to 4096 descriptors unchecked;
- OutputDebugString and nn::diag's log lines copied up to 4 KB into the
  log from an unchecked pointer;
- the event-name copies (hos_main.c) and the item key (oot_menu.c, whose
  own bound could wrap) wrote through pointers read from game objects.

New tests/test_hle_guest_ranges.py runs wb_in_guest and checks the
functions' text; the wave-buffer and input fixtures have hos_guest_range
over their own guest memory.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
nvnDeviceGetProcAddress and nvnBootstrapLoader read the name the game
passes in place, to its end: a pointer near the guest window's end, or
a name with no end, read past the guest's memory into the host's. A new
guest_name copies at most 255 characters, only from inside the window,
and gives "" when there is no string there (which asks for no proc).

tests/test_hle_guest_ranges.py checks both callers and runs guest_name.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
test: the biquad and teleport fixtures have a hos_guest_range over their own guest memory
Some checks are pending
CI / Conventional Commit title (pull_request) Blocked by required conditions
CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Blocked by required conditions
CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Blocked by required conditions
CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Blocked by required conditions
CI / Workflow lint (actionlint) (pull_request) Blocked by required conditions
Guide screenshots / Guide images and local UI source review (pull_request) Blocked by required conditions
Third-party parts / As their notes say (pull_request) Blocked by required conditions
Third-party parts / The same as upstream (pull_request) Blocked by required conditions
Website / Website on Cloudflare (pull_request_target) Has been skipped
PR description / Description follows the template (pull_request_target) Successful in 8s
6263697bcf
Author
Contributor

Tested in game: Linux (Ubuntu 26.04.1, kernel 7.0.0-43), RTX 4070 Ti 12 GB, NVIDIA 595.99.02, Ryzen 7 9800X3D: this branch built together with the other 18 split from #218 and #284-#288, merged onto main @ 71b1edd8 (make_game.py --update, build v3.0.0-rc.5-82-g5952bb3d), then played by a scripted tour through the control bridge (scripts/memtour.py: a save loaded, Zora's Domain, the Market, Hyrule Field and Death Mountain Trail, 40 s each with walking and camera swings). All 4 stops reached, a clean quit, and no device loss or Vulkan error in the log. The HLE calls the checks guard ran through the tour (audio renderer, applet, hid) with valid ranges, and nothing was refused.

Tested in game: Linux (Ubuntu 26.04.1, kernel 7.0.0-43), RTX 4070 Ti 12 GB, NVIDIA 595.99.02, Ryzen 7 9800X3D: this branch built together with the other 18 split from #218 and #284-#288, merged onto main @ 71b1edd8 (make_game.py --update, build v3.0.0-rc.5-82-g5952bb3d), then played by a scripted tour through the control bridge (scripts/memtour.py: a save loaded, Zora's Domain, the Market, Hyrule Field and Death Mountain Trail, 40 s each with walking and camera swings). All 4 stops reached, a clean quit, and no device loss or Vulkan error in the log. The HLE calls the checks guard ran through the tour (audio renderer, applet, hid) with valid ranges, and nothing was refused.
Collaborator

Possibly related, for context:

  • issue #432 Idea: guest-memory fixes from the macOS port on Windows and Linux too, once timed (dirty-page atomics, zeroed pages marked, fatal allocation failures) — Related guest-memory hardening idea; its scope is separate allocation and dirty-mark changes
Possibly related, for context: - issue #432 Idea: guest-memory fixes from the macOS port on Windows and Linux too, once timed (dirty-page atomics, zeroed pages marked, fatal allocation failures) — Related guest-memory hardening idea; its scope is separate allocation and dirty-mark changes
Owner

Brought up to the ready-for-review standard (#1454): the checklist's Everyone list has two new boxes, added here from what this PR's description and comments show (please check they match what you know).

  • Tested on my own PC: ticked: Author's comment reports an in-game tour on their Linux RTX 4070 Ti with the guarded HLE calls running.
  • Nothing gets worse that I know of: ticked: Risks names only a check refusing a valid span, and the in-game tour reports none refused.
Brought up to the [ready-for-review standard](https://git.magicbean.dev/MagicBeanDev/Longshot/src/branch/main/CONTRIBUTING.md#ready-for-review-out-of-wip) (#1454): the checklist's Everyone list has two new boxes, added here from what this PR's description and comments show (please check they match what you know). - *Tested on my own PC*: ticked: Author's comment reports an in-game tour on their Linux RTX 4070 Ti with the guarded HLE calls running. - *Nothing gets worse that I know of*: ticked: Risks names only a check refusing a valid span, and the in-game tour reports none refused.
Some checks are pending
CI / Conventional Commit title (pull_request) Blocked by required conditions
CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Blocked by required conditions
CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Blocked by required conditions
CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Blocked by required conditions
CI / Workflow lint (actionlint) (pull_request) Blocked by required conditions
Guide screenshots / Guide images and local UI source review (pull_request) Blocked by required conditions
Third-party parts / As their notes say (pull_request) Blocked by required conditions
Third-party parts / The same as upstream (pull_request) Blocked by required conditions
Website / Website on Cloudflare (pull_request_target) Has been skipped
PR description / Description follows the template (pull_request_target) Successful in 8s
This pull request is marked as a work in progress.
Some workflows are waiting to be reviewed.
This branch is out-of-date with the base branch
View command line instructions

Checkout

From your project repository, check out a new branch and test the changes.
git fetch -u 'fix/hle-guest-ranges:TrashPanda-fix/hle-guest-ranges'
git switch 'TrashPanda-fix/hle-guest-ranges'
Sign in to join this conversation.
No reviewers
No milestone
No project
No assignees
3 participants
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
MagicBeanDev/Longshot!1179
No description provided.