WIP: feat(mem): HOS_HEAP_LAZY=1 commits the game's heap where it is first used (Windows, opt-in) #2558

Draft

NotRexed wants to merge 1 commit from feat/heap-lazy into main

NotRexed commented

2026-11-11 23:12:35 +00:00

(Migrated from github.com)

Why. Windows counts every committed page against RAM plus the page file, used or not. The game sizes its heap whole at the start (6.9 GB here), and Hookshot commits all of it then. That is the largest item in the commit-limit crashes of long sessions: an RTX 3060 crashed at 30.7 GB of commit after 54 minutes. Similar work elsewhere measured 4.7 instead of 6.8 GB of heap committed in Castle Town, with ~12,900 first-use faults costing 440 ms over four minutes.

What (opt-in, HOS_HEAP_LAZY=1, Windows):

SetMemoryHeapSize (host/hos_mem.c) commits nothing up front; its log line says so.
The existing commit-on-touch fault handler (host/mkw_glue.c) commits a heap page's 64 KB on its first use. These faults are counted (a log line every 16,384) and kept out of the "nothing handed it out" lines.
Host APIs that touch guest memory without faulting call hos_commit_reserved first: the save/file system's ReadFile (which fails on reserved pages rather than faulting) and WriteFile. It commits only reserved pages, so a page that a write watch or the shadow-write guard protects keeps its protection.
The GPU's host-pointer imports already commit their range (vk_memory.cpp).
Off by default. Linux commits on first touch anyway.

Tests: tests/test_heap_lazy.py runs hos_commit_reserved on a real reserved region:

with the switch off, nothing changes;
reserved pages are committed read-write;
a PAGE_READONLY page keeps its protection;
unaligned ranges and ranges across a page boundary commit only the pages they touch;
a range outside the window does nothing.

It also checks the three source sites. The changed files compile; the mkw_glue.c poll error with plain clang flags is already on main.

Not yet verified in-game: commit charge with and without it over a long route, and stutter from the first-use faults.

🤖 Generated with Claude Code

**Why.** Windows counts every committed page against RAM plus the page file, used or not. The game sizes its heap whole at the start (6.9 GB here), and Hookshot commits all of it then. That is the largest item in the commit-limit crashes of long sessions: an RTX 3060 crashed at 30.7 GB of commit after 54 minutes. Similar work elsewhere measured 4.7 instead of 6.8 GB of heap committed in Castle Town, with ~12,900 first-use faults costing 440 ms over four minutes. **What** (opt-in, `HOS_HEAP_LAZY=1`, Windows): - `SetMemoryHeapSize` (`host/hos_mem.c`) commits nothing up front; its log line says so. - The existing commit-on-touch fault handler (`host/mkw_glue.c`) commits a heap page's 64 KB on its first use. These faults are counted (a log line every 16,384) and kept out of the "nothing handed it out" lines. - Host APIs that touch guest memory without faulting call `hos_commit_reserved` first: the save/file system's `ReadFile` (which fails on reserved pages rather than faulting) and `WriteFile`. It commits only reserved pages, so a page that a write watch or the shadow-write guard protects keeps its protection. - The GPU's host-pointer imports already commit their range (`vk_memory.cpp`). - Off by default. Linux commits on first touch anyway. **Tests:** `tests/test_heap_lazy.py` runs `hos_commit_reserved` on a real reserved region: - with the switch off, nothing changes; - reserved pages are committed read-write; - a `PAGE_READONLY` page keeps its protection; - unaligned ranges and ranges across a page boundary commit only the pages they touch; - a range outside the window does nothing. It also checks the three source sites. The changed files compile; the `mkw_glue.c` `poll` error with plain clang flags is already on main. **Not yet verified in-game:** commit charge with and without it over a long route, and stutter from the first-use faults. 🤖 Generated with [Claude Code](https://claude.com/claude-code)