Freeing a guest address region lets go of the lock before it discards the pages, so a new region can be handed the same range and lose its first contents #1610

Open
opened 2026-11-18 15:43:49 +00:00 by mooki · 1 comment
Member

Type: correctness · Area: host, guest address regions · Part of: #1493

TL;DR

hos_region_free removes a region from the list, releases the lock, and only then discards the region's pages. A thread that allocates a region in between can be given the same address range and start writing to it, and the discard then wipes what it wrote. We found this by reading the code; no run has shown it. We propose to keep the range unavailable to allocations until the discard has finished, and a fixture that checks the order.

Breakdown

  • Who it affects: any player, whenever the game frees and allocates address regions on different threads (resource memory is loaded and unloaded in the background). The window is short, so the likely symptom is rare: a freshly loaded resource that reads as zeros, or a later crash from it. Nobody has reported one with this cause.
  • What is wrong: the removal and the discard are two steps with the lock released between them, and the range is in no list during the gap.
  • What we propose: a freeing state. The entry stays in the list, marked, so the allocator skips it. The discard runs without the lock (it can be slow). Then a short second critical section removes the entry.
  • What does not change: first-fit placement from the top, region sizes, which pages are discarded, how the discard is done on each system.
  • What we do not claim: that it has ever happened. It is a missing ordering rule, found by reading.
  • How we would prove it: a fixture that checks, at the moment of the discard, that the range cannot be allocated; then a threaded stress with an ownership table.

Evidence

Code on forge/main at 430eef4f2:

  • host/hos_mem.c:190-205, hos_region_free: under LOCK() it finds the region, marks its pages used (pages_mark_used_locked), removes the entry with memmove and decrements nregions. UNLOCK() is at :203. Only then, at :204, if (freed_size) discard_pages(freed, freed_size).
  • :97-141, hos_region_alloc: first fit from the top over the sorted list under LOCK(), inserts the new entry, UNLOCK() at :139, then hos_commit(found, size) at :140. Between :203 and :204 the freed range is in no list, so this function can return it.
  • :152-177, discard_pages: on Windows DiscardVirtualMemory (or a MEM_RESET), on Linux a hole punched in the guest window (hos_posix_guest_discard), and a memset of zeros when a Vulkan import holds the pages. Any of these removes data another thread has just written.
  • No test names hos_region_free or hos_region_alloc. The two checks under tests that mention host/hos_mem.c are about hos_mark_written.

The fork's claim, not ours: an outside audit of a fork says the fork fixed the same race by marking the region as freeing, discarding while it was still listed, then removing it, with the lock dropped for the discard. We have not read that code and do not rely on it. The rule is simple enough to write from the problem.

Proposed change

  1. Add a freeing mark to a region entry. hos_region_free sets it under the lock, releases the lock, discards the pages, takes the lock again and removes the entry. hos_region_alloc treats a marked entry as occupied. hos_regions and hos_mem_region (the control bridge's range checks) keep listing it as mapped until it is removed.
  2. Keep the discard outside the lock: it can take a long time on a large region, and the lock is taken on every allocation.
  3. A second free of the same address while it is marked does nothing.
  4. Not part of this change, and already tracked in #432 item 3 (a failed guest commit is fatal, not a logged pretend-success): hos_commit returns 0 when it is refused (host/hos.h:75), and its result is ignored at hos_mem.c:140 and in MapPhysicalMemory (host/hos_svc.c:471-475).

How we would test it

  • Game-free, fails today: a host fixture, in the style of the existing host fixtures that compile C files from host, that builds the region functions with hooks for the lock and for the discard. It frees a region and, when the discard hook fires, asks whether an allocation of the same size could return that range. Today it can, because the list no longer holds the entry. After the change it cannot.
  • Game-free, second step: a stress with several threads that allocate, write a pattern, free and verify, over a small window, with an ownership table that fails if two threads hold one range. Run it first against main's order to see whether it can show the race at all; a fixture that never fails on main is worth nothing and is reported as such.
  • Both run on Linux in the project's suite. A Windows run of the same fixture is wanted, since the discard call differs.
  • No speed or stability claim follows from either. A player-visible claim needs a report with this cause.
  • #1493, track B (one owner for each range of guest memory): this is a missing rule in the host's own region manager, below the renderer.
  • #432 (open idea): guest-memory fixes from the macOS port, including explicit allocation failures.
  • #1034 and #1339 (open): write marks for host writes to guest memory; a different rule on the same memory.
  • #480 (closed, Apple only): serialised guest memory ownership on that port's page size.

New on this forge; not opened on GitHub.

**Type:** correctness · **Area:** host, guest address regions · **Part of:** #1493 ## TL;DR `hos_region_free` removes a region from the list, releases the lock, and only then discards the region's pages. A thread that allocates a region in between can be given the same address range and start writing to it, and the discard then wipes what it wrote. We found this by reading the code; no run has shown it. We propose to keep the range unavailable to allocations until the discard has finished, and a fixture that checks the order. ## Breakdown - **Who it affects:** any player, whenever the game frees and allocates address regions on different threads (resource memory is loaded and unloaded in the background). The window is short, so the likely symptom is rare: a freshly loaded resource that reads as zeros, or a later crash from it. Nobody has reported one with this cause. - **What is wrong:** the removal and the discard are two steps with the lock released between them, and the range is in no list during the gap. - **What we propose:** a freeing state. The entry stays in the list, marked, so the allocator skips it. The discard runs without the lock (it can be slow). Then a short second critical section removes the entry. - **What does not change:** first-fit placement from the top, region sizes, which pages are discarded, how the discard is done on each system. - **What we do not claim:** that it has ever happened. It is a missing ordering rule, found by reading. - **How we would prove it:** a fixture that checks, at the moment of the discard, that the range cannot be allocated; then a threaded stress with an ownership table. ## Evidence Code on `forge/main` at `430eef4f2`: - `host/hos_mem.c:190-205`, `hos_region_free`: under `LOCK()` it finds the region, marks its pages used (`pages_mark_used_locked`), removes the entry with `memmove` and decrements `nregions`. `UNLOCK()` is at `:203`. Only then, at `:204`, `if (freed_size) discard_pages(freed, freed_size)`. - `:97-141`, `hos_region_alloc`: first fit from the top over the sorted list under `LOCK()`, inserts the new entry, `UNLOCK()` at `:139`, then `hos_commit(found, size)` at `:140`. Between `:203` and `:204` the freed range is in no list, so this function can return it. - `:152-177`, `discard_pages`: on Windows `DiscardVirtualMemory` (or a `MEM_RESET`), on Linux a hole punched in the guest window (`hos_posix_guest_discard`), and a memset of zeros when a Vulkan import holds the pages. Any of these removes data another thread has just written. - No test names `hos_region_free` or `hos_region_alloc`. The two checks under `tests` that mention `host/hos_mem.c` are about `hos_mark_written`. The fork's claim, not ours: an outside audit of a fork says the fork fixed the same race by marking the region as freeing, discarding while it was still listed, then removing it, with the lock dropped for the discard. We have not read that code and do not rely on it. The rule is simple enough to write from the problem. ## Proposed change 1. Add a freeing mark to a region entry. `hos_region_free` sets it under the lock, releases the lock, discards the pages, takes the lock again and removes the entry. `hos_region_alloc` treats a marked entry as occupied. `hos_regions` and `hos_mem_region` (the control bridge's range checks) keep listing it as mapped until it is removed. 2. Keep the discard outside the lock: it can take a long time on a large region, and the lock is taken on every allocation. 3. A second free of the same address while it is marked does nothing. 4. Not part of this change, and already tracked in #432 item 3 (a failed guest commit is fatal, not a logged pretend-success): `hos_commit` returns 0 when it is refused (`host/hos.h:75`), and its result is ignored at `hos_mem.c:140` and in `MapPhysicalMemory` (`host/hos_svc.c:471-475`). ## How we would test it - Game-free, fails today: a host fixture, in the style of the existing host fixtures that compile C files from `host`, that builds the region functions with hooks for the lock and for the discard. It frees a region and, when the discard hook fires, asks whether an allocation of the same size could return that range. Today it can, because the list no longer holds the entry. After the change it cannot. - Game-free, second step: a stress with several threads that allocate, write a pattern, free and verify, over a small window, with an ownership table that fails if two threads hold one range. Run it first against main's order to see whether it can show the race at all; a fixture that never fails on main is worth nothing and is reported as such. - Both run on Linux in the project's suite. A Windows run of the same fixture is wanted, since the discard call differs. - No speed or stability claim follows from either. A player-visible claim needs a report with this cause. ## Related - #1493, track B (one owner for each range of guest memory): this is a missing rule in the host's own region manager, below the renderer. - #432 (open idea): guest-memory fixes from the macOS port, including explicit allocation failures. - #1034 and #1339 (open): write marks for host writes to guest memory; a different rule on the same memory. - #480 (closed, Apple only): serialised guest memory ownership on that port's page size. New on this forge; not opened on GitHub.
Collaborator

Possibly related, for context:

  • issue #432 Idea: guest-memory fixes from the macOS port on Windows and Linux too, once timed (dirty-page atomics, zeroed pages marked, fatal allocation failures) — Related guest-memory correctness and allocation failure tracking
  • issue #1493 Renderer rewrite — Parent renderer and host-layer rework track
  • PR #468 fix(vk): retain guest pages during import handoff and retirement — Related guest-page ownership and discard lifetime, but different race
Possibly related, for context: - issue #432 Idea: guest-memory fixes from the macOS port on Windows and Linux too, once timed (dirty-page atomics, zeroed pages marked, fatal allocation failures) — Related guest-memory correctness and allocation failure tracking - issue #1493 Renderer rewrite — Parent renderer and host-layer rework track - PR #468 fix(vk): retain guest pages during import handoff and retirement — Related guest-page ownership and discard lifetime, but different race
Sign in to join this conversation.
No milestone
No project
No assignees
2 participants
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
MagicBeanDev/Longshot#1610
No description provided.