MagicBeanDev/Longshot

Watch

Fork 71

WIP: fix(hos): wait at the commit limit for the host's own allocations, then say out of memory #341

Draft

llun wants to merge 4 commits from llun/Longshot:fix/host-alloc-commit-limit into main

llun commented

2026-11-14 02:39:51 +00:00

Member

Why

At Windows' commit limit (RAM plus the page file full), the NVN layer's own malloc/calloc failing ended the game at once with FATAL: native NVN: recording handle set, with no wait for commit to come back and no out-of-memory notice, so a player saw an unexplained crash (#333: RTX 3060 12 GB, 16 GB RAM with a 16 GB page file, Hyrule Field). With this PR a failed host allocation is treated like a guest commit at the limit: it waits, and if memory doesn't come back the game says it is out of memory, shows the notice and only then stops.

The cause, commit growing over play, is #324's; this PR is only about how the game ends at the limit.

What changed

hos: hos_xmalloc / hos_xcalloc / hos_xrealloc in host/mkw_glue.c (declared in host/hos.h), never NULL. A failed allocation goes to hos_alloc_failed(), which tries again every 50 ms for up to 5 s as commit_wait() does (#77), while the renderer's textures give commit back. If it gets the memory, one log line (... at the commit limit allocated after N ms). If not: the line, hos_out_of_memory(1), then hos_fatal("out of memory (RAM plus the page file): <what> (<n> bytes)"). When over 1 GB more than it asked for is still free, it isn't the limit (a size gone wrong), so the fatal says how much was free and shows no notice. A calloc whose count × size overflows stops at once.
hos: hos_out_of_memory(int wait) is the out-of-memory line and notice, once a run, shared by commit_refused() (guest commits, as before) and the host allocations. The notice is always on a thread of its own: shown on the game thread that ran out (sead::MainThread), MessageBoxA came back at once and showed nothing (found in the game run, fixed in the second commit). With wait the caller waits until the notice is closed, its own or one already up, before the game stops. HOS_HEADLESS shows no notice. On Linux the line and the notice speak of RAM and swap, not the page file.
nvn: host/nvn_native.c's five sites from the issue (command buffer table ×2, recording buffer, recording handle set, recording copy, through pool_get()), and also texture copies table (pins_grow()), use the helpers instead of die() on NULL.
video: host/nn_video.c's decoder table and image pool use them instead of hos_fatal("video: out of memory"). Its picture buffers that drop a frame on NULL (issue 342) are unchanged.
tests: tests/test_host_alloc_wait.py (new) compiles the real hos_out_of_memory and hos_alloc_failed cut out of mkw_glue.c with a fake malloc, clock and notice (tests/host_alloc_wait_test.c), and checks the call sites by text. tests/test_commit_limit.py follows commit_refused's notice into hos_out_of_memory; tests/test_nvn_host_tables.py's fixture gets stand-ins for the helpers.
docs: HOS_TEST_ALLOC_REFUSE in docs/development.md's switches; docs/tested/2026-11-13-host-alloc-oom-llun-run01.md.

Flags and settings

Flag	What it does	Default now	Recommendation	Helps on	Doesn't help on	Untested on
`HOS_HEADLESS` (existing)	Now also leaves out the out-of-memory notice when a guest commit is refused (it already left out other boxes); the line is logged as before	unchanged	keep: a headless run has nobody to close a box	test runs	n/a	Linux
`HOS_TEST_ALLOC_REFUSE=<text>`	A test: every host allocation through the helpers whose label contains the text is refused, as at the commit limit, to see the wait, the line, the notice and the stop	off (unset)	off: it is only for testing	any PC	n/a	Linux

Measurements

main: cf605c85 (2026-11-14)
this PR: fix/host-alloc-commit-limit @ d60e26c5 (the game was built from 50077318: d60e26c5 only adds the docs/tested file, so the code is the same)
Hardware: NVIDIA RTX 5080, driver 621.24, AMD Ryzen 9 9900X3D, 64 GB RAM, 9 GB page file, Windows 11 Pro 26300
Scene: boot, before the title (the game's first recording); 1280x720 window, default settings, HOS_AUDIO=0, RX_VK_PRECOMPILE=0 (shaders not prepared)
Method: this is a fix, so the proof is the bug's path before and after, not fps. This PC doesn't reach the commit limit (27 GB of commit was free), so HOS_TEST_ALLOC_REFUSE refuses the allocation the way Windows does at the limit. That switch isn't on main, so on main the before is the player's log from #333 (nightly-80-7315957c; main has the same code at these sites). Two forced runs at two different sites, one headless, and one normal 90 s boot with no switch. A run script watched the game's windows, captured the notice, and closed it 2 s later.
How to reproduce: build this branch, then start longshot.exe from the game folder with HOS_TEST_ALLOC_REFUSE=recording handle set (or recording copy). After about 7 s the notice appears; OK ends the game. python -I -B tests/test_host_alloc_wait.py --cc <clang or gcc> for the game-free check.

	main @ `cf605c85` (the player's log, #333)	this PR @ `d60e26c5`
a host allocation refused at the commit limit	the game ends at once	tried again for 5 s, then out of memory
log	`FATAL: native NVN: recording handle set`	`mem: ... still refused after 5 s`, `mem: Windows is out of memory (RAM and page file) ...`, `FATAL: out of memory (RAM plus the page file): native NVN: recording handle set (524288 bytes)`
the out-of-memory notice	none	shown; the game waits until it is closed and exits 0.3 s after (forced4, forced5)
`HOS_TEST_ALLOC_REFUSE=recording copy`	n/a (no switch)	the same path, `native NVN: recording copy (128 bytes)` (forced5)
headless (`HOS_HEADLESS=1`)	n/a	the same lines, no notice, no wait for one (forced6)
normal boot, no switch, 90 s	n/a	5040 presents at 60 fps, no fatal, no allocation-wait line (normal1)
`tests/test_host_alloc_wait.py`	FAIL (`commit_refused does not say out of memory through hos_out_of_memory`)	PASS, 7 cases

main (the player's log, #333):
[hos Resource Memory] mem: commit of 0x3f0000 bytes at the commit limit made after 200 ms (2 so far)
[hos ModuleSystemWorker3] FATAL: native NVN: recording handle set (lr 0x80600de0 sp 0x10062cb960)

this PR @ 50077318, HOS_TEST_ALLOC_REFUSE="recording handle set" (forced4):
[hos sead::MainThread] mem: native NVN: recording handle set: 524288 bytes of host memory still refused after 5 s at the commit limit (26861 MB of commit left)
[hos sead::MainThread] mem: Windows is out of memory (RAM and page file): close other programs, or let Windows manage the page file size
[hos sead::MainThread] FATAL: out of memory (RAM plus the page file): native NVN: recording handle set (524288 bytes) (lr 0x80c4dfc0 sp 0x10ff1e0)
run script: box up 6.8 s after start; process still running: True
run script: process still running with the notice up 2 s on: True
run script: exit code 3; 9.4 s in all; exit 0.3 s after the notice was closed

this PR @ 50077318, HOS_TEST_ALLOC_REFUSE="recording copy" (forced5):
[hos sead::MainThread] mem: native NVN: recording copy: 128 bytes of host memory still refused after 5 s at the commit limit (27308 MB of commit left)
[hos sead::MainThread] mem: Windows is out of memory (RAM and page file): close other programs, or let Windows manage the page file size
[hos sead::MainThread] FATAL: out of memory (RAM plus the page file): native NVN: recording copy (128 bytes) (lr 0x80c4dfc0 sp 0x10ff1e0)

this PR @ 50077318, no switch, headless (normal1, at 90 s):
[hos Presentation Thread] vk: 5040 presents, 60.0 fps; a frame: 16 draws (0 skipped), 14 dispatches, 3 passes, 14 barriers, 6 submits, ...

tests/test_host_alloc_wait.py, main @ cf605c85 (the test copied in):
FAIL: commit_refused does not say out of memory through hos_out_of_memory
this PR:
PASS recovers / limit / commit-first / not-limit / overflow / headless / test-switch
PASS: a host allocation at the commit limit waits, then says out of memory and shows the notice before it stops

Testing

Tested:

In the game, on the RTX 5080 above (Windows 11, NVIDIA 621.24), from a build of this branch made with make_game.py: the four runs above. The first build showed the notice on the game's own thread, where MessageBoxA showed nothing: the game waited the 5 s, logged the lines and stopped with no notice. The second commit moves the notice to its own thread, and the runs above are from that build. A normal boot with no switch ran 90 s at 60 fps with no fatal.
Game-free: python -I -B tests/run_all.py: 476 passed, 8 failed, 76 skipped (Windows, llvm-mingw's clang as gcc). All 8 fail the same way on main @ cf605c85 on this PC: test_bake_portability, test_fpx_sticky, test_lift_cache, test_module_ids, test_oot_state_tap, test_pad_layout, test_spawn_emit, test_tested_configurations (that one is CRLF line endings in its own CLI output). The new test_host_alloc_wait.py passes on this branch and fails on main. Mutations of the fix are caught: notice not waited for → limit fails; no retries → recovers fails.
Linux: gcc 13.3 (Ubuntu 24.04 under WSL) compiles host/nvn_native.c cleanly, and host/mkw_glue.c with only an error that main's mkw_glue.c has too (Ctx has no member poll at a64_poll: a define the lift's own build gives). Nothing is reported in the new code. That is a compile of the changed files with the posix windows.h, not a full Linux build.

Not tested:

The real commit limit: this PC has 64 GB of RAM. The path is the same once malloc returns NULL, and the switch makes it return NULL at the same call.
An allocation recovering inside the 5 s in the game: covered by the game-free check (recovers), not seen in a run.
A full Linux build and run, and the notice through SDL on Linux.
host/nn_video.c's two sites in a movie (two-line changes, built on Windows).

Needs other testers:

Linux (any GPU): build this branch and run with HOS_TEST_ALLOC_REFUSE="recording handle set". Expected: the line mem: out of memory (RAM and swap), the SDL notice, and the stop once it's closed (T-9A3Y; it waits for the testing wanted label).

Screenshots

The notice in forced4 (this PR @ 50077318), on the RTX 5080 PC: the comment with the picture. Nothing else on screen changes. On main @ cf605c85 there is no notice on this path.

Risk and rollback

A thread that runs out now holds its locks for up to 5 s while it waits (the NVN layer's cmd_lock or copy_lock, the video service's table lock). Before, the process ended at that point, so nothing that worked before waits longer; another thread that needs that lock waits too, and if the renderer's give-back needs it, the wait ends in the notice as before.
The notice no longer shows under HOS_HEADLESS for a refused guest commit either (it popped up before). Headless runs are tests, and the log line is unchanged.
Reverting the PR restores the old behaviour.

Reviewer notes

Start at hos_alloc_failed() and hos_out_of_memory() in host/mkw_glue.c.
The labels keep the old die() texts behind a native NVN: prefix, so a log from before and after can be grepped for the same name.
Not done here: other host allocations that already handle NULL themselves (drop a frame, refuse a call) are left as they are; only sites that ended the game on NULL were changed.

Issues

Fixes #333
See #324

Checklist

For everyone

One topic.
The title is a Conventional Commit (type(scope): description).
Rebased on current main.
python -I -B tests/run_all.py run and passing (any failure that also fails on main is named above).
Tested in the game, or what wasn't is named under Testing.
Testing filled in: what was tested, what wasn't, and what needs other testers (with test requests).
Measured: main and this PR at named commits, on the same PC, scene and settings, at least 2 interleaved runs each, the median reported. (A fix, so no numbers to take a median of: the new check fails on main @ cf605c85 and passes here, on this PC. In the game, two forced runs here, and the player's log for main, where the switch doesn't exist.)
No game files, dumps, build output or generated files.
third_party/ untouched (a new folder with a PROVENANCE.md instead).
The license is OK: GPL-3.0-or-later, nothing incompatible and nothing decompiled.

If applicable

AI assistance recorded with Assisted-by: / Co-Authored-By: trailers.
Flags noted, and the switch reference updated (docs/development.md, the README).
Screenshots for a change that shows on screen.
A file in docs/tested/ for each new measured run (format in docs/tested-configurations.md; no appended row or assembled view).
A game-free test added for the fix, failing on main and passing here.
A check under tests/ that reads changed source updated in this PR, not weakened.
Host code builds on both Windows (llvm-mingw) and Linux, with UTF-8 paths. (Windows: a full build. Linux: only a gcc compile of the changed files, not a full build; asked of a Linux tester, T-9A3Y.)
Docs updated.

🤖 Generated with Claude Code

## Why At Windows' commit limit (RAM plus the page file full), the NVN layer's own `malloc`/`calloc` failing ended the game at once with `FATAL: native NVN: recording handle set`, with no wait for commit to come back and no out-of-memory notice, so a player saw an unexplained crash (#333: RTX 3060 12 GB, 16 GB RAM with a 16 GB page file, Hyrule Field). With this PR a failed host allocation is treated like a guest commit at the limit: it waits, and if memory doesn't come back the game says it is out of memory, shows the notice and only then stops. The cause, commit growing over play, is #324's; this PR is only about how the game ends at the limit. ## What changed - hos: `hos_xmalloc` / `hos_xcalloc` / `hos_xrealloc` in `host/mkw_glue.c` (declared in `host/hos.h`), never NULL. A failed allocation goes to `hos_alloc_failed()`, which tries again every 50 ms for up to 5 s as `commit_wait()` does (#77), while the renderer's textures give commit back. If it gets the memory, one log line (`... at the commit limit allocated after N ms`). If not: the line, `hos_out_of_memory(1)`, then `hos_fatal("out of memory (RAM plus the page file): <what> (<n> bytes)")`. When over 1 GB more than it asked for is still free, it isn't the limit (a size gone wrong), so the fatal says how much was free and shows no notice. A `calloc` whose count × size overflows stops at once. - hos: `hos_out_of_memory(int wait)` is the out-of-memory line and notice, once a run, shared by `commit_refused()` (guest commits, as before) and the host allocations. The notice is always on a thread of its own: shown on the game thread that ran out (`sead::MainThread`), `MessageBoxA` came back at once and showed nothing (found in the game run, fixed in the second commit). With `wait` the caller waits until the notice is closed, its own or one already up, before the game stops. `HOS_HEADLESS` shows no notice. On Linux the line and the notice speak of RAM and swap, not the page file. - nvn: `host/nvn_native.c`'s five sites from the issue (`command buffer table` ×2, `recording buffer`, `recording handle set`, `recording copy`, through `pool_get()`), and also `texture copies table` (`pins_grow()`), use the helpers instead of `die()` on NULL. - video: `host/nn_video.c`'s decoder table and image pool use them instead of `hos_fatal("video: out of memory")`. Its picture buffers that drop a frame on NULL (issue 342) are unchanged. - tests: `tests/test_host_alloc_wait.py` (new) compiles the real `hos_out_of_memory` and `hos_alloc_failed` cut out of `mkw_glue.c` with a fake `malloc`, clock and notice (`tests/host_alloc_wait_test.c`), and checks the call sites by text. `tests/test_commit_limit.py` follows `commit_refused`'s notice into `hos_out_of_memory`; `tests/test_nvn_host_tables.py`'s fixture gets stand-ins for the helpers. - docs: `HOS_TEST_ALLOC_REFUSE` in `docs/development.md`'s switches; `docs/tested/2026-11-13-host-alloc-oom-llun-run01.md`. ## Flags and settings | Flag | What it does | Default now | Recommendation | Helps on | Doesn't help on | Untested on | |---|---|---|---|---|---|---| | `HOS_HEADLESS` (existing) | Now also leaves out the out-of-memory notice when a guest commit is refused (it already left out other boxes); the line is logged as before | unchanged | keep: a headless run has nobody to close a box | test runs | n/a | Linux | | `HOS_TEST_ALLOC_REFUSE=<text>` | A test: every host allocation through the helpers whose label contains the text is refused, as at the commit limit, to see the wait, the line, the notice and the stop | off (unset) | off: it is only for testing | any PC | n/a | Linux | ## Measurements main: cf605c85 (2026-11-14) this PR: fix/host-alloc-commit-limit @ d60e26c5 (the game was built from 50077318: d60e26c5 only adds the `docs/tested` file, so the code is the same) Hardware: NVIDIA RTX 5080, driver 621.24, AMD Ryzen 9 9900X3D, 64 GB RAM, 9 GB page file, Windows 11 Pro 26300 Scene: boot, before the title (the game's first recording); 1280x720 window, default settings, `HOS_AUDIO=0`, `RX_VK_PRECOMPILE=0` (shaders not prepared) Method: this is a fix, so the proof is the bug's path before and after, not fps. This PC doesn't reach the commit limit (27 GB of commit was free), so `HOS_TEST_ALLOC_REFUSE` refuses the allocation the way Windows does at the limit. That switch isn't on `main`, so on `main` the before is the player's log from #333 (nightly-80-7315957c; `main` has the same code at these sites). Two forced runs at two different sites, one headless, and one normal 90 s boot with no switch. A run script watched the game's windows, captured the notice, and closed it 2 s later. How to reproduce: build this branch, then start `longshot.exe` from the game folder with `HOS_TEST_ALLOC_REFUSE=recording handle set` (or `recording copy`). After about 7 s the notice appears; OK ends the game. `python -I -B tests/test_host_alloc_wait.py --cc <clang or gcc>` for the game-free check. | | main @ cf605c85 (the player's log, #333) | this PR @ d60e26c5 | |---|---|---| | a host allocation refused at the commit limit | the game ends at once | tried again for 5 s, then out of memory | | log | `FATAL: native NVN: recording handle set` | `mem: ... still refused after 5 s`, `mem: Windows is out of memory (RAM and page file) ...`, `FATAL: out of memory (RAM plus the page file): native NVN: recording handle set (524288 bytes)` | | the out-of-memory notice | none | shown; the game waits until it is closed and exits 0.3 s after (forced4, forced5) | | `HOS_TEST_ALLOC_REFUSE=recording copy` | n/a (no switch) | the same path, `native NVN: recording copy (128 bytes)` (forced5) | | headless (`HOS_HEADLESS=1`) | n/a | the same lines, no notice, no wait for one (forced6) | | normal boot, no switch, 90 s | n/a | 5040 presents at 60 fps, no fatal, no allocation-wait line (normal1) | | `tests/test_host_alloc_wait.py` | FAIL (`commit_refused does not say out of memory through hos_out_of_memory`) | PASS, 7 cases | ``` main (the player's log, #333): [hos Resource Memory] mem: commit of 0x3f0000 bytes at the commit limit made after 200 ms (2 so far) [hos ModuleSystemWorker3] FATAL: native NVN: recording handle set (lr 0x80600de0 sp 0x10062cb960) this PR @ 50077318, HOS_TEST_ALLOC_REFUSE="recording handle set" (forced4): [hos sead::MainThread] mem: native NVN: recording handle set: 524288 bytes of host memory still refused after 5 s at the commit limit (26861 MB of commit left) [hos sead::MainThread] mem: Windows is out of memory (RAM and page file): close other programs, or let Windows manage the page file size [hos sead::MainThread] FATAL: out of memory (RAM plus the page file): native NVN: recording handle set (524288 bytes) (lr 0x80c4dfc0 sp 0x10ff1e0) run script: box up 6.8 s after start; process still running: True run script: process still running with the notice up 2 s on: True run script: exit code 3; 9.4 s in all; exit 0.3 s after the notice was closed this PR @ 50077318, HOS_TEST_ALLOC_REFUSE="recording copy" (forced5): [hos sead::MainThread] mem: native NVN: recording copy: 128 bytes of host memory still refused after 5 s at the commit limit (27308 MB of commit left) [hos sead::MainThread] mem: Windows is out of memory (RAM and page file): close other programs, or let Windows manage the page file size [hos sead::MainThread] FATAL: out of memory (RAM plus the page file): native NVN: recording copy (128 bytes) (lr 0x80c4dfc0 sp 0x10ff1e0) this PR @ 50077318, no switch, headless (normal1, at 90 s): [hos Presentation Thread] vk: 5040 presents, 60.0 fps; a frame: 16 draws (0 skipped), 14 dispatches, 3 passes, 14 barriers, 6 submits, ... tests/test_host_alloc_wait.py, main @ cf605c85 (the test copied in): FAIL: commit_refused does not say out of memory through hos_out_of_memory this PR: PASS recovers / limit / commit-first / not-limit / overflow / headless / test-switch PASS: a host allocation at the commit limit waits, then says out of memory and shows the notice before it stops ``` ## Testing **Tested:** - In the game, on the RTX 5080 above (Windows 11, NVIDIA 621.24), from a build of this branch made with `make_game.py`: the four runs above. The first build showed the notice on the game's own thread, where `MessageBoxA` showed nothing: the game waited the 5 s, logged the lines and stopped with no notice. The second commit moves the notice to its own thread, and the runs above are from that build. A normal boot with no switch ran 90 s at 60 fps with no fatal. - Game-free: `python -I -B tests/run_all.py`: 476 passed, 8 failed, 76 skipped (Windows, llvm-mingw's clang as `gcc`). All 8 fail the same way on `main` @ cf605c85 on this PC: `test_bake_portability`, `test_fpx_sticky`, `test_lift_cache`, `test_module_ids`, `test_oot_state_tap`, `test_pad_layout`, `test_spawn_emit`, `test_tested_configurations` (that one is CRLF line endings in its own CLI output). The new `test_host_alloc_wait.py` passes on this branch and fails on `main`. Mutations of the fix are caught: notice not waited for → `limit` fails; no retries → `recovers` fails. - Linux: `gcc 13.3` (Ubuntu 24.04 under WSL) compiles `host/nvn_native.c` cleanly, and `host/mkw_glue.c` with only an error that `main`'s `mkw_glue.c` has too (`Ctx` has no member `poll` at `a64_poll`: a define the lift's own build gives). Nothing is reported in the new code. That is a compile of the changed files with the posix `windows.h`, not a full Linux build. **Not tested:** - The real commit limit: this PC has 64 GB of RAM. The path is the same once `malloc` returns NULL, and the switch makes it return NULL at the same call. - An allocation recovering inside the 5 s in the game: covered by the game-free check (`recovers`), not seen in a run. - A full Linux build and run, and the notice through SDL on Linux. - `host/nn_video.c`'s two sites in a movie (two-line changes, built on Windows). **Needs other testers:** - Linux (any GPU): build this branch and run with `HOS_TEST_ALLOC_REFUSE="recording handle set"`. Expected: the line `mem: out of memory (RAM and swap)`, the SDL notice, and the stop once it's closed (T-9A3Y; it waits for the `testing wanted` label). ## Screenshots The notice in forced4 (this PR @ 50077318), on the RTX 5080 PC: [the comment with the picture](https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/341#issuecomment-16766). Nothing else on screen changes. On `main` @ cf605c85 there is no notice on this path. ## Risk and rollback - A thread that runs out now holds its locks for up to 5 s while it waits (the NVN layer's `cmd_lock` or `copy_lock`, the video service's table lock). Before, the process ended at that point, so nothing that worked before waits longer; another thread that needs that lock waits too, and if the renderer's give-back needs it, the wait ends in the notice as before. - The notice no longer shows under `HOS_HEADLESS` for a refused guest commit either (it popped up before). Headless runs are tests, and the log line is unchanged. - Reverting the PR restores the old behaviour. ## Reviewer notes - Start at `hos_alloc_failed()` and `hos_out_of_memory()` in `host/mkw_glue.c`. - The labels keep the old `die()` texts behind a `native NVN: ` prefix, so a log from before and after can be grepped for the same name. - Not done here: other host allocations that already handle NULL themselves (drop a frame, refuse a call) are left as they are; only sites that ended the game on NULL were changed. ## Issues Fixes #333 See #324 ## Checklist ### For everyone - [x] One topic. - [x] The title is a Conventional Commit (`type(scope): description`). - [x] Rebased on current `main`. - [x] `python -I -B tests/run_all.py` run and passing (any failure that also fails on `main` is named above). - [x] Tested in the game, or what wasn't is named under Testing. - [x] Testing filled in: what was tested, what wasn't, and what needs other testers (with test requests). - [x] Measured: main and this PR at named commits, on the same PC, scene and settings, at least 2 interleaved runs each, the median reported. (A fix, so no numbers to take a median of: the new check fails on `main` @ cf605c85 and passes here, on this PC. In the game, two forced runs here, and the player's log for `main`, where the switch doesn't exist.) - [x] No game files, dumps, build output or generated files. - [x] `third_party/` untouched (a new folder with a `PROVENANCE.md` instead). - [x] The license is OK: GPL-3.0-or-later, nothing incompatible and nothing decompiled. ### If applicable - [x] AI assistance recorded with `Assisted-by:` / `Co-Authored-By:` trailers. - [x] Flags noted, and the switch reference updated (docs/development.md, the README). - [x] Screenshots for a change that shows on screen. - [x] A file in `docs/tested/` for each new measured run (format in `docs/tested-configurations.md`; no appended row or assembled view). - [x] A game-free test added for the fix, failing on `main` and passing here. - [x] A check under `tests/` that reads changed source updated in this PR, not weakened. - [ ] Host code builds on both Windows (llvm-mingw) and Linux, with UTF-8 paths. (Windows: a full build. Linux: only a gcc compile of the changed files, not a full build; asked of a Linux tester, T-9A3Y.) - [x] Docs updated. 🤖 Generated with [Claude Code](https://claude.com/claude-code)

llun added 4 commits

2026-11-14 02:39:52 +00:00

fix(hos): wait at the commit limit for the host's own allocations, then say out of memory f72efa654d

A malloc or calloc in the NVN layer that failed at Windows' commit limit (RAM plus the page file full) ended the game at once
with "FATAL: native NVN: recording handle set", with no wait and no out-of-memory notice (issue 333). hos_xmalloc,
hos_xcalloc and hos_xrealloc (host/mkw_glue.c) try a failed allocation again for up to 5 s, as hos_commit does, while the
renderer's textures give commit back; past that comes the out-of-memory line and notice (shared with commit_refused, said
once a run, and waited for before the game stops), then a fatal that says out of memory. Used at the NVN layer's command
buffer table, recording buffers and copies, recording handle set and texture copies table, and the video service's decoder
table and image pool. HOS_TEST_ALLOC_REFUSE=<text> refuses the allocations whose label holds the text, to see the path.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

docs(dev): HOS_TEST_ALLOC_REFUSE in the switch reference 68d121863d

Co-Authored-By: Claude Opus 5.5 <[email protected]>

fix(hos): the out-of-memory notice on a thread of its own when the game stops after it 5007731877

Shown on the game's thread that ran out (sead::MainThread), MessageBoxA came back at once and showed nothing, so the game
stopped with no notice. hos_out_of_memory always shows it on a new thread; a caller that stops the game waits until it is
closed.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

docs(tested): issue 333's forced out-of-memory runs on an RTX 5080

CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Has been skipped

Details

CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Has been skipped

Details

CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Has been skipped

Details

Third-party parts / The same as upstream (pull_request) Has been skipped

Details

Website / Website on Cloudflare (pull_request) Has been skipped

Details

CI / Conventional Commit title (pull_request) Successful in 4s

Details

CI / Workflow lint (actionlint) (pull_request) Successful in 5s

Details

Third-party parts / As their notes say (pull_request) Successful in 6s

Details

PR description / Description follows the template (pull_request_target) Successful in 3s

Details

Website / Website on Cloudflare (pull_request_target) Has been skipped

Details

d60e26c52a

Co-Authored-By: Claude Opus 5.5 <[email protected]>

llun commented

2026-11-14 02:39:59 +00:00

Author

Member

The out-of-memory notice in forced4 (this PR @ 50077318, RTX 5080, Windows 11): HOS_TEST_ALLOC_REFUSE=recording handle set, up while the game waits; OK ends it (exit 3, 0.3 s after).

The out-of-memory notice in forced4 (this PR @ 50077318, RTX 5080, Windows 11): `HOS_TEST_ALLOC_REFUSE=recording handle set`, up while the game waits; OK ends it (exit 3, 0.3 s after). ![this PR @ 50077318: the out-of-memory notice](https://git.magicbean.dev/attachments/0225b4ac-ecfa-46b6-852f-6c591c85ff5d)

oom-notice.png

12 KiB

llun commented

2026-11-14 02:40:07 +00:00

Author

Member

Description: the screenshot linked (comment 16766).

llun commented

2026-11-14 02:40:23 +00:00

Author

Member

Description: the Measurements name the head d60e26c5 (built from 50077318, the same code); HOS_HEADLESS noted under Flags. The Linux box stays unticked: a gcc compile of the changed files only, so far.

Description: the Measurements name the head d60e26c5 (built from 50077318, the same code); HOS_HEADLESS noted under Flags. The Linux box stays unticked: a gcc compile of the changed files only, so far.

llun commented

2026-11-14 02:43:28 +00:00

Author

Member

Linux test request opened: T-9A3Y (Linux, any GPU: the forced out-of-memory path and a normal start). It needs the testing wanted label before anyone can claim it; the PR stays WIP until the Linux box is settled.

Linux test request opened: T-9A3Y (Linux, any GPU: the forced out-of-memory path and a normal start). It needs the `testing wanted` label before anyone can claim it; the PR stays WIP until the Linux box is settled.

llun added the

testing wanted

label

2026-11-14 02:47:48 +00:00

jarynclouatre referenced this pull request

2026-11-14 03:03:33 +00:00

fix(vk): the page files' room to grow counts before the commit rule cuts the textures #346

mooki added the

labels

2026-11-14 03:55:07 +00:00

mooki requested changes

2026-11-14 04:22:44 +00:00

mooki left a comment

Read head d60e26c5 against main cf605c85: the complete patch, allocation callers, Linux thread and notice code, description, all four comments, issue 333 and the notice picture. Ran the three changed Python checks, with GCC and Clang for both C fixtures; all passed.

What's good

The retry keeps the old realloc block intact. The check covers recovery, persistent failure, overflow, headless mode and waiting for an existing notice.
The change stays with allocations that already ended the game on failure. The video buffers that can refuse a picture are unchanged.
The Windows forced-failure runs are clearly distinguished from a real commit-limit test. The picture shows the closing notice, and the only difference from the tested 50077318 to this head is the added test record.

Must change

Make the Linux notice-thread allocation safe when memory is exhausted. host/mkw_glue.c:65-67 now calls CreateThread after a host allocation has kept failing. On Linux, that calls thread_new() in host/posix/win32_posix.c:188-192, which calls calloc and immediately writes t->h.type without checking the result. If this allocation also fails, the new error path dereferences NULL before it can reach the notice or the final out-of-memory fatal. The NULL-handle fallback here does not catch that. Previously these host allocation failures went straight to die() or hos_fatal() without creating a thread. Make thread creation return failure safely, or prepare the notice resources before exhaustion, and add a case that refuses this allocation and still reaches the intended fatal without a signal. HOS_TEST_ALLOC_REFUSE does not refuse this plain calloc, and the current fixture replaces CreateThread with a version that always succeeds, so neither covers it.

Smaller points

host/mkw_glue.c:127 still says “RAM plus the page file” on Linux, although the preceding line and notice correctly say swap. Use the same platform wording in the final fatal.
The allocator fixture defines _WIN32 even when compiled on Linux. Its passing Linux runs exercise the Windows branch with substitutes, not the sysinfo branch or the real Linux thread and SDL calls.

Readiness and limits

The branch contains current main cf605c85; there is no intervening-main conflict to resolve. The forge's source-check jobs were skipped, not passed: https://git.magicbean.dev/MagicBeanDev/Longshot/actions/runs/3826/jobs/1.

No game, full build, real memory exhaustion or Windows execution was run here. The requested Linux build, ordinary start and notice test in T-9A3Y remain open. A forced-label run would establish that the notice works with memory available; it would not settle the failed thread-allocation case above. I did not test AMD, Intel, Steam Deck or the changed video allocation paths in a movie.

Read head `d60e26c5` against main `cf605c85`: the complete patch, allocation callers, Linux thread and notice code, description, all four comments, issue 333 and the notice picture. Ran the three changed Python checks, with GCC and Clang for both C fixtures; all passed. ### What's good - The retry keeps the old `realloc` block intact. The check covers recovery, persistent failure, overflow, headless mode and waiting for an existing notice. - The change stays with allocations that already ended the game on failure. The video buffers that can refuse a picture are unchanged. - The Windows forced-failure runs are clearly distinguished from a real commit-limit test. The picture shows the closing notice, and the only difference from the tested `50077318` to this head is the added test record. ### Must change 1. **Make the Linux notice-thread allocation safe when memory is exhausted.** `host/mkw_glue.c:65-67` now calls `CreateThread` after a host allocation has kept failing. On Linux, that calls `thread_new()` in `host/posix/win32_posix.c:188-192`, which calls `calloc` and immediately writes `t->h.type` without checking the result. If this allocation also fails, the new error path dereferences NULL before it can reach the notice or the final out-of-memory fatal. The NULL-handle fallback here does not catch that. Previously these host allocation failures went straight to `die()` or `hos_fatal()` without creating a thread. Make thread creation return failure safely, or prepare the notice resources before exhaustion, and add a case that refuses this allocation and still reaches the intended fatal without a signal. `HOS_TEST_ALLOC_REFUSE` does not refuse this plain `calloc`, and the current fixture replaces `CreateThread` with a version that always succeeds, so neither covers it. ### Smaller points - `host/mkw_glue.c:127` still says “RAM plus the page file” on Linux, although the preceding line and notice correctly say swap. Use the same platform wording in the final fatal. - The allocator fixture defines `_WIN32` even when compiled on Linux. Its passing Linux runs exercise the Windows branch with substitutes, not the `sysinfo` branch or the real Linux thread and SDL calls. ### Readiness and limits The branch contains current main `cf605c85`; there is no intervening-main conflict to resolve. The forge's source-check jobs were skipped, not passed: https://git.magicbean.dev/MagicBeanDev/Longshot/actions/runs/3826/jobs/1. No game, full build, real memory exhaustion or Windows execution was run here. The requested Linux build, ordinary start and notice test in T-9A3Y remain open. A forced-label run would establish that the notice works with memory available; it would not settle the failed thread-allocation case above. I did not test AMD, Intel, Steam Deck or the changed video allocation paths in a movie.

llun commented

2026-11-14 05:43:21 +00:00

Author

Member

Thanks @mooki for review 1238. Agreed on all three:

The Linux notice thread: thread_new() will return NULL when its calloc fails, CreateThread passes that on, and the notice path falls through to the final out-of-memory fatal. A fixture case will refuse that allocation and check the fatal is reached without a signal.
The final fatal's wording will say swap on Linux, as the notice does.
The allocator fixture will stop defining _WIN32 on Linux, so its Linux run covers the sysinfo branch.

The fix follows in the next push; the PR stays WIP until then and until T-9A3Y is settled.

Thanks @mooki for review 1238. Agreed on all three: 1. The Linux notice thread: `thread_new()` will return NULL when its `calloc` fails, `CreateThread` passes that on, and the notice path falls through to the final out-of-memory fatal. A fixture case will refuse that allocation and check the fatal is reached without a signal. 2. The final fatal's wording will say swap on Linux, as the notice does. 3. The allocator fixture will stop defining `_WIN32` on Linux, so its Linux run covers the `sysinfo` branch. The fix follows in the next push; the PR stays WIP until then and until T-9A3Y is settled.

CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Has been skipped

Details

CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Has been skipped

Details

CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Has been skipped

Details

Third-party parts / The same as upstream (pull_request) Has been skipped

Details

Website / Website on Cloudflare (pull_request) Has been skipped

Details

CI / Conventional Commit title (pull_request) Successful in 4s

Details

CI / Workflow lint (actionlint) (pull_request) Successful in 5s

Details

Third-party parts / As their notes say (pull_request) Successful in 6s

Details

PR description / Description follows the template (pull_request_target) Successful in 3s

Details

Website / Website on Cloudflare (pull_request_target) Has been skipped

Details

This pull request is marked as a work in progress.

This branch is out-of-date with the base branch

View command line instructions

Checkout

From your project repository, check out a new branch and test the changes.

git fetch -u 'fix/host-alloc-commit-limit:llun-fix/host-alloc-commit-limit'

git switch 'llun-fix/host-alloc-commit-limit'

No reviewers

No milestone

No project

No assignees

2 participants

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference

MagicBeanDev/Longshot!341

No description provided.

Rows
Columns