Idea: guest-memory fixes from the macOS port on Windows and Linux too, once timed (dirty-page atomics, zeroed pages marked, fatal allocation failures) #432

New issue

Open

opened 2026-11-14 13:46:24 +00:00 by mooki · 2 comments

mooki commented

2026-11-14 13:46:24 +00:00

Member

The idea

The macOS port (#420) changes four things in guest memory handling. They are correct on every platform, but they touch
code that Windows and Linux run every frame, so they need timing and testing there before they leave the Apple path. In
the macOS pull request they are Apple-only. This issue tracks turning them on everywhere once they have been measured
on Windows (NVIDIA, AMD, Intel) and Linux (NVIDIA, AMD).

Dirty collection reads the marks as byte atomics (host/rx_backend.c, collect_in). This runs every frame.
- Today the collection reads the atomic byte marks through bulk memcpy and word loads, while game threads write them
  one byte at a time.
- With the change, the wide zero-skip uses byte-atomic loads, matching the writer's granule, and keeps the atomic
  exchange when it consumes marks.
- Empty ranges consume nothing. Host dirty marking clamps an oversized length without overflow.
- Measure: frame time in a busy scene, main against the change, at least 2 interleaved runs each, median
  reported, on the same PC, scene and settings.
Renderer-owned guest zeroing is marked written (host_guest_alloc).
Allocation failures are explicit:
- a failed guest commit is fatal rather than a logged pretend-success;
- collector allocation failures are reported.
- On Windows this must keep #324's ERROR_COMMITMENT_LIMIT retry (host/mkw_glue.c).
Stricter checks of what the host returns in hos_mem.c, plus VirtualQuery's last_error. They must keep
#324's mapped-pages accounting.

Each will be its own pull request once measured. On macOS all four are required: 16 KiB native pages and arm64 memory
ordering.

Anything else

Tester: @mooki (Windows, Linux and macOS machines).

### The idea The macOS port (#420) changes four things in guest memory handling. They are correct on every platform, but they touch code that Windows and Linux run every frame, so they need timing and testing there before they leave the Apple path. In the macOS pull request they are **Apple-only**. This issue tracks turning them on everywhere once they have been measured on Windows (NVIDIA, AMD, Intel) and Linux (NVIDIA, AMD). 1. **Dirty collection reads the marks as byte atomics** (`host/rx_backend.c`, `collect_in`). This runs every frame. - Today the collection reads the atomic byte marks through bulk `memcpy` and word loads, while game threads write them one byte at a time. - With the change, the wide zero-skip uses byte-atomic loads, matching the writer's granule, and keeps the atomic exchange when it consumes marks. - Empty ranges consume nothing. Host dirty marking clamps an oversized length without overflow. - **Measure:** frame time in a busy scene, main against the change, at least 2 interleaved runs each, median reported, on the same PC, scene and settings. 2. **Renderer-owned guest zeroing is marked written** (`host_guest_alloc`). 3. **Allocation failures are explicit:** - a failed guest commit is fatal rather than a logged pretend-success; - collector allocation failures are reported. - On Windows this must keep #324's `ERROR_COMMITMENT_LIMIT` retry (`host/mkw_glue.c`). 4. **Stricter checks of what the host returns** in `hos_mem.c`, plus `VirtualQuery`'s `last_error`. They must keep #324's mapped-pages accounting. Each will be its own pull request once measured. On macOS all four are required: 16 KiB native pages and arm64 memory ordering. ### Anything else Tester: @mooki (Windows, Linux and macOS machines).

mooki commented

2026-11-14 15:43:53 +00:00

Author

Member

Claiming the Windows and Linux side of this from a second PC: the timing, and one pull request per item once it is measured.

Where it gets measured here: Linux with an RTX 4090; Linux with an Intel UHD 770; Windows with Intel Arc integrated graphics (Core Ultra 9 185H). Main against the change, interleaved runs, medians, the same scene and settings, as the issue asks.
Not available here: AMD on either system, and NVIDIA on Windows. I am asking in the team room for those two.
What it needs first: the four changes as commits on a pushed branch. They exist only in the macOS work so far and are in none of its open pull requests. Once they are pushed (a work-in-progress pull request is enough), each is taken from there unchanged, switched on for every platform on its own branch, and measured.
Order: item 1 first (the dirty collection runs every frame, so it is the one that can cost something), then 2, 3 and 4.

Claiming the Windows and Linux side of this from a second PC: the timing, and one pull request per item once it is measured. - **Where it gets measured here:** Linux with an RTX 4090; Linux with an Intel UHD 770; Windows with Intel Arc integrated graphics (Core Ultra 9 185H). Main against the change, interleaved runs, medians, the same scene and settings, as the issue asks. - **Not available here:** AMD on either system, and NVIDIA on Windows. I am asking in the team room for those two. - **What it needs first:** the four changes as commits on a pushed branch. They exist only in the macOS work so far and are in none of its open pull requests. Once they are pushed (a work-in-progress pull request is enough), each is taken from there unchanged, switched on for every platform on its own branch, and measured. - **Order:** item 1 first (the dirty collection runs every frame, so it is the one that can cost something), then 2, 3 and 4.

mooki referenced this issue

2026-11-14 16:12:17 +00:00

WIP: fix(macos): serialize guest memory ownership on 16 KiB pages and pin imported backing #480

mooki commented

2026-11-14 16:13:03 +00:00

Author

Member

The four changes are now pushed, in two WIP pull requests from the macOS side:

#478 (mac/guest-memory @ 9c679145): the macOS guest-memory groundwork.
#480 (mac/guest-16k @ 647e90e4, on #478 and #436): carries all four items. Commit 3366b61b puts every part Windows or Linux could see under #if defined(__APPLE__), with main's code in the #else. Linux and Windows preprocess to the same tokens as main (14 of 14 comparisons), so there is nothing to time there yet.

To save you separating them, I'm preparing a branch from current main (9c9d69ed) with one commit per item, in this issue's order, each switched on for every platform and each passing its own test when cherry-picked alone. I'll post the branch and the four SHAs here once its suite has run on the Mac.

The four changes are now pushed, in two WIP pull requests from the macOS side: - #478 (`mac/guest-memory` @ `9c679145`): the macOS guest-memory groundwork. - #480 (`mac/guest-16k` @ `647e90e4`, on #478 and #436): carries all four items. Commit `3366b61b` puts every part Windows or Linux could see under `#if defined(__APPLE__)`, with main's code in the `#else`. Linux and Windows preprocess to the same tokens as main (14 of 14 comparisons), so there is nothing to time there yet. To save you separating them, I'm preparing a branch from current main (`9c9d69ed`) with **one commit per item, in this issue's order, each switched on for every platform** and each passing its own test when cherry-picked alone. I'll post the branch and the four SHAs here once its suite has run on the Mac.

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference

MagicBeanDev/Longshot#432

No description provided.

Rows
Columns