fix(vk): a C++ exception out of the renderer is said and stops the game as out of memory or a FATAL line, not 0x20474343 #2170

Merged
HumanBean merged 2 commits from HumanBean/Longshot:fix/renderer-cxx-escape into main 2026-11-22 00:15:15 +00:00
Owner

Summary

When a C++ exception gets out of the renderer (longshot_vk.dll), nothing catches it. The host is C, and a renderer thread's body is the top of its stack. So the game ends with exception 0x20474343 (llvm-mingw's code for a C++ exception nothing caught) and "An unexpected error". The log doesn't say what was thrown or where.

#1291 gave the renderer a new_handler, so a refused operator new waits and then says out of memory. But the crash bot on #81 still lists 0x20474343 on v3.0.0, which has #1291:

  • KERNELBASE.dll+0xc569a: 12 reports;
  • +0xc408a: 4 reports;
  • +0x25369: 3 reports.

Several of these logs reach the commit limit with no vk: a C++ allocation refused line before the crash, so what was thrown there wasn't a refused operator new. It's also one of the two crash kinds in #947 (AMD APUs and handhelds, loading a save). #1280 itself is closed, with its duplicates.

The likely candidates, from reading the code:

  • A thread that couldn't be started. At the commit limit a new thread's stack gets no commit, and std::thread's constructor throws std::system_error. The texture decode starts helper threads during play, from a std::vector<std::thread>: a failed start there destroys the vector while the other threads are still running, which is std::terminate.
  • A size gone wrong (std::length_error or std::bad_array_new_length from a vector).
  • .at() (std::out_of_range).
  • A std::filesystem call without an error_code.

This PR catches them at the ways out and stops the game with what was thrown (gpu/vk/cxx_escape.h):

  • Every function the renderer gives the host (MkwVkRenderer, wrapped by gpu/vk/renderer_guard.h when mkwvk_init returns the table), and mkwvk_init itself.
  • The render thread, the presenter and the order watch are started with spawn_needed. Their bodies catch, and a thread that can't be started stops the game as out of memory. A thread that fails to start inside one of these, or inside a host call, comes out as a std::system_error into the same catch.
  • The texture decode's helper threads (the gather, the hole check, the unswizzle) go through fork_join. A helper that can't be started leaves its part to the thread that asked, so the game no longer hits std::terminate.

What's caught goes into the log as vk: a C++ exception came out of <where>: <type>: <what()>. Then:

  • std::bad_alloc, or a std::system_error for no memory or a thread that couldn't be started: the host's out-of-memory line and notice, as #1291 does.
  • Anything else: the renderer's vk: FATAL: line (exit 3), which the crash window shows.

Only std::exception is caught, never catch (...). An access violation, or a driver's MSVC exception (0xe06d7363: nvgpucomp64.dll, AudioDevProps2.dll in #81), stays the crash record's, with its backtrace.

The game still stops: a half-done replay isn't safe to go on from. What changes is that the player sees "out of memory" when that's the cause, and the log names everything else.

On main: an exception out of any MkwVkRenderer function crashes with 0x20474343 and no line saying what it was. tests/test_cxx_escape.py shows each kind.

Read first: gpu/vk/cxx_escape.h, gpu/vk/renderer_guard.h, mkwvk_init / init_body in gpu/vk/vk_main.cpp.

See #1280 (closed, still seen on v3.0.0). See #947.

Testing

Tested: Linux, RTX 3090, NVIDIA 620.70.06, this PR @ 1a3655a9 as a pack over main's base, through the job queue:

  1. A memory tour from kokiri-start through Kokiri Forest, Hyrule Field and the Market: in play, clean quit, no came out of or could not be started line, no crash.
  2. The same with HOS_TEST_ALLOC_REFUSE=renderer (#1291's hook): HOS_TEST_ALLOC_REFUSE: one C++ allocation of 1 PB, then a C++ allocation refused ... tried again for up to 5 s, the renderer's C++ allocations: still refused after 5 s, the out-of-memory line and FATAL: out of memory (RAM plus swap): the renderer's C++ allocations: the game stopped itself with the notice, as on main, with the table wrapped and the replay thread started through spawn_needed; no 0x20474343.

The game-free test (tests/test_cxx_escape.py with tests/cxx_escape_test.cpp) was not run on this PC (no test suites here); CI runs it. Only a compile check of the test file was made (g++ and clang++ -fsyntax-only -Wall -Wextra -Werror, both clean). On main the test fails because gpu/vk/cxx_escape.h is missing and nothing guards the table: the reason for the bug.

Still to test:

  • Windows, any GPU (llvm-mingw, SEH unwinding): an exception thrown under nvn_present or on the render thread is caught by the guard, not passed on as 0x20474343. There's no switch that throws on demand, so the check is that a normal run plays as on main, with no vk: a C++ exception came out of line (@Beanstalk a normal 10-minute run on Windows NVIDIA and AMD, and a Steam Deck, comparing with main).
  • Windows at the commit limit (the #1280 reports): the game says out of memory, or names the exception in its FATAL line, not "An unexpected error" (by hand, by whoever can reproduce #388's growth on a 16 GB PC).

Measurements

Nothing to measure for speed. Each MkwVkRenderer call goes through one more function call with a table-based try, which costs nothing unless something throws. A normal run's fps against main is in the run plan, to confirm.

Risks

  • A path that used to throw through the host and somehow went on now stops the game. Nothing in the host catches C++ exceptions, so before, these ended the game anyway.
  • The log line is a little off. A caught std::bad_alloc goes through the host's hos_alloc_gave_up, whose line says "still refused after 5 s". With more than 1 GB of commit left, it's that function's "refused for 5 s with N MB of commit left" fatal instead.
  • Thread starts this PR leaves alone. Start-up preparation, shots, perf summary, bug report, menu, FSR 4 builds, capture helpers, imports, frame generation's display, the reaper and the jobs' workers are not converted. Their bodies still aren't guarded, except the jobs' workers and the recorder, which catch their own, and a failed start inside a guarded caller is now caught there. They can follow in a later PR, as can mkwvk_close_asked, which tests/test_quit_not_crash.py reads statement by statement.
  • To undo it, revert the PR. Nothing changes in saves, longshot_options.ini or the caches.

Checklist

Ready for review

  • One topic, on current main; the title is a Conventional Commit. (Rebased onto main: 499fcda2 test, 1a3655a9 fix.)
  • Tested on my own PC, in the game for a change the game runs, as Testing says.
  • mise run test passes (any failure that also fails on main is named). (Not run on this PC: CI runs it. The tests that cut changed code were updated by reading them.)
  • Measured against main, or Measurements says why there is nothing to measure.
  • No game files, build output or hand-edited generated files; third_party/ untouched; nothing GPL-incompatible.

When it applies

  • A fix: a game-free test that fails on main and passes here. (It fails on main by its source check; whether it passes here is for CI.)
  • A check under tests/ that reads changed source is updated, not weakened. These follow the new text: test_renderer_new_handler and test_mesh_mode_switch read init_body; test_present_acquire_ahead reads the presenter's spawn_needed line; test_texture_staging_publication reads the fork_join. The test_guest_holes and test_reserved_core fixtures include cxx_escape.h.
  • Host code builds on Windows (llvm-mingw) and Linux, with UTF-8 paths. (Renderer code only; the main session's build shows it.)
  • Docs updated (docs/dev/reference/threads.md, "A C++ exception out of a thread").
  • AI assistance recorded with Assisted-by: / Co-Authored-By: trailers.

Ready to merge

  • Everything in Still to test has run, its results in Tested.
  • Tier 1 hardware it can affect is tested (docs/dev/standards.md, "Hardware tiers").
  • Approved in review, every comment answered.
  • CI passes on the head; on current main, measured again or a comment says why the numbers still hold.

🤖 Generated with Claude Code

## Summary When a C++ exception gets out of the renderer (`longshot_vk.dll`), nothing catches it. The host is C, and a renderer thread's body is the top of its stack. So the game ends with exception `0x20474343` (llvm-mingw's code for a C++ exception nothing caught) and "An unexpected error". The log doesn't say what was thrown or where. #1291 gave the renderer a `new_handler`, so a refused `operator new` waits and then says out of memory. But the crash bot on #81 still lists `0x20474343` on v3.0.0, which has #1291: - `KERNELBASE.dll+0xc569a`: 12 reports; - `+0xc408a`: 4 reports; - `+0x25369`: 3 reports. Several of these logs reach the commit limit with no `vk: a C++ allocation refused` line before the crash, so what was thrown there wasn't a refused `operator new`. It's also one of the two crash kinds in #947 (AMD APUs and handhelds, loading a save). #1280 itself is closed, with its duplicates. The likely candidates, from reading the code: - **A thread that couldn't be started.** At the commit limit a new thread's stack gets no commit, and `std::thread`'s constructor throws `std::system_error`. The texture decode starts helper threads during play, from a `std::vector<std::thread>`: a failed start there destroys the vector while the other threads are still running, which is `std::terminate`. - **A size gone wrong** (`std::length_error` or `std::bad_array_new_length` from a vector). - **`.at()`** (`std::out_of_range`). - **A `std::filesystem` call without an `error_code`.** This PR catches them at the ways out and stops the game with what was thrown (`gpu/vk/cxx_escape.h`): - **Every function the renderer gives the host** (`MkwVkRenderer`, wrapped by `gpu/vk/renderer_guard.h` when `mkwvk_init` returns the table), and `mkwvk_init` itself. - **The render thread, the presenter and the order watch** are started with `spawn_needed`. Their bodies catch, and a thread that can't be started stops the game as out of memory. A thread that fails to start inside one of these, or inside a host call, comes out as a `std::system_error` into the same catch. - **The texture decode's helper threads** (the gather, the hole check, the unswizzle) go through `fork_join`. A helper that can't be started leaves its part to the thread that asked, so the game no longer hits `std::terminate`. What's caught goes into the log as `vk: a C++ exception came out of <where>: <type>: <what()>`. Then: - `std::bad_alloc`, or a `std::system_error` for no memory or a thread that couldn't be started: the host's out-of-memory line and notice, as #1291 does. - Anything else: the renderer's `vk: FATAL:` line (exit 3), which the crash window shows. Only `std::exception` is caught, never `catch (...)`. An access violation, or a driver's MSVC exception (`0xe06d7363`: `nvgpucomp64.dll`, `AudioDevProps2.dll` in #81), stays the crash record's, with its backtrace. The game still stops: a half-done replay isn't safe to go on from. What changes is that the player sees "out of memory" when that's the cause, and the log names everything else. **On `main`:** an exception out of any `MkwVkRenderer` function crashes with `0x20474343` and no line saying what it was. `tests/test_cxx_escape.py` shows each kind. Read first: `gpu/vk/cxx_escape.h`, `gpu/vk/renderer_guard.h`, `mkwvk_init` / `init_body` in `gpu/vk/vk_main.cpp`. See #1280 (closed, still seen on v3.0.0). See #947. ## Testing **Tested:** Linux, RTX 3090, NVIDIA 620.70.06, this PR @ 1a3655a9 as a pack over main's base, through the job queue: 1. A memory tour from `kokiri-start` through Kokiri Forest, Hyrule Field and the Market: in play, clean quit, no `came out of` or `could not be started` line, no crash. 2. The same with `HOS_TEST_ALLOC_REFUSE=renderer` (#1291's hook): `HOS_TEST_ALLOC_REFUSE: one C++ allocation of 1 PB`, then `a C++ allocation refused ... tried again for up to 5 s`, `the renderer's C++ allocations: still refused after 5 s`, the out-of-memory line and `FATAL: out of memory (RAM plus swap): the renderer's C++ allocations`: the game stopped itself with the notice, as on main, with the table wrapped and the replay thread started through `spawn_needed`; no 0x20474343. The game-free test (`tests/test_cxx_escape.py` with `tests/cxx_escape_test.cpp`) was not run on this PC (no test suites here); CI runs it. Only a compile check of the test file was made (`g++` and `clang++ -fsyntax-only -Wall -Wextra -Werror`, both clean). On `main` the test fails because `gpu/vk/cxx_escape.h` is missing and nothing guards the table: the reason for the bug. **Still to test:** - Windows, any GPU (llvm-mingw, SEH unwinding): an exception thrown under `nvn_present` or on the render thread is caught by the guard, not passed on as `0x20474343`. There's no switch that throws on demand, so the check is that a normal run plays as on `main`, with no `vk: a C++ exception came out of` line (@Beanstalk a normal 10-minute run on Windows NVIDIA and AMD, and a Steam Deck, comparing with `main`). - Windows at the commit limit (the #1280 reports): the game says out of memory, or names the exception in its FATAL line, not "An unexpected error" (by hand, by whoever can reproduce #388's growth on a 16 GB PC). ## Measurements Nothing to measure for speed. Each `MkwVkRenderer` call goes through one more function call with a table-based `try`, which costs nothing unless something throws. A normal run's fps against `main` is in the run plan, to confirm. ## Risks - **A path that used to throw through the host and somehow went on now stops the game.** Nothing in the host catches C++ exceptions, so before, these ended the game anyway. - **The log line is a little off.** A caught `std::bad_alloc` goes through the host's `hos_alloc_gave_up`, whose line says "still refused after 5 s". With more than 1 GB of commit left, it's that function's "refused for 5 s with N MB of commit left" fatal instead. - **Thread starts this PR leaves alone.** Start-up preparation, shots, perf summary, bug report, menu, FSR 4 builds, capture helpers, imports, frame generation's display, the reaper and the jobs' workers are not converted. Their bodies still aren't guarded, except the jobs' workers and the recorder, which catch their own, and a failed start inside a guarded caller is now caught there. They can follow in a later PR, as can `mkwvk_close_asked`, which `tests/test_quit_not_crash.py` reads statement by statement. - **To undo it,** revert the PR. Nothing changes in saves, `longshot_options.ini` or the caches. ## Checklist ### Ready for review - [x] One topic, on current `main`; the title is a Conventional Commit. (Rebased onto `main`: 499fcda2 test, 1a3655a9 fix.) - [x] Tested on my own PC, in the game for a change the game runs, as Testing says. - [ ] `mise run test` passes (any failure that also fails on `main` is named). (Not run on this PC: CI runs it. The tests that cut changed code were updated by reading them.) - [x] Measured against `main`, or Measurements says why there is nothing to measure. - [x] No game files, build output or hand-edited generated files; `third_party/` untouched; nothing GPL-incompatible. ### When it applies - [x] A fix: a game-free test that fails on `main` and passes here. (It fails on `main` by its source check; whether it passes here is for CI.) - [x] A check under `tests/` that reads changed source is updated, not weakened. These follow the new text: `test_renderer_new_handler` and `test_mesh_mode_switch` read `init_body`; `test_present_acquire_ahead` reads the presenter's `spawn_needed` line; `test_texture_staging_publication` reads the `fork_join`. The `test_guest_holes` and `test_reserved_core` fixtures include `cxx_escape.h`. - [ ] Host code builds on Windows (llvm-mingw) and Linux, with UTF-8 paths. (Renderer code only; the main session's build shows it.) - [x] Docs updated (`docs/dev/reference/threads.md`, "A C++ exception out of a thread"). - [x] AI assistance recorded with `Assisted-by:` / `Co-Authored-By:` trailers. ### Ready to merge - [ ] Everything in Still to test has run, its results in Tested. - [ ] Tier 1 hardware it can affect is tested (docs/dev/standards.md, "Hardware tiers"). - [ ] Approved in review, every comment answered. - [ ] CI passes on the head; on current `main`, measured again or a comment says why the numbers still hold. 🤖 Generated with [Claude Code](https://claude.com/claude-code)
tests/test_cxx_escape.py builds tests/cxx_escape_test.cpp with the real MkwVkRenderer table behind
guard_renderer and the real thread helpers, and checks by text that every table member, mkwvk_init,
the replay's threads and the texture decode's helper threads go through them. Fails on main: nothing
there catches an exception before the host's C, so it ends the game with 0x20474343, unsaid (#1280, #947).

Co-Authored-By: Claude Opus 5.5 <[email protected]>
fix(vk): a C++ exception out of the renderer is said and stops the game as out of memory or a FATAL line, not 0x20474343
Some checks failed
CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Has been skipped
CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Has been skipped
CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Has been skipped
Third-party parts / The same as upstream (pull_request) Has been skipped
Website / Website on Cloudflare (pull_request) Has been skipped
CI / Conventional Commit title (pull_request) Successful in 9s
Website / Website on Cloudflare (pull_request_target) Has been skipped
CI / Workflow lint (actionlint) (pull_request) Successful in 20s
Guide screenshots / Guide images and local UI source review (pull_request) Failing after 15s
CI / Website build and checks (pull_request) Failing after 24s
Third-party parts / As their notes say (pull_request) Successful in 22s
PR description / Description follows the template (pull_request_target) Failing after 5s
beanstalk/approval planning the tests
1a3655a93e
Every function the renderer gives the host (renderer_guard.h) and mkwvk_init catch a std::exception;
so do the bodies of the render thread, the presenter and the order watch (spawn_needed). escaped
(cxx_escape.h) logs where it came out, its type and what(); std::bad_alloc or a thread that could not
be started goes to the host's out-of-memory line and notice, anything else to the FATAL line (exit 3).
Never catch (...): foreign exceptions stay the crash record's. The texture decode's helper threads go
through fork_join: a helper refused at the commit limit leaves its part to the caller, where the
vector of threads used to be destroyed with the others running (std::terminate). The tests that cut
the changed code follow it; threads.md says it.

Fixes #1280 (its v3.0.0 reports), part of #947.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Author
Owner

Tested in game: Linux, RTX 3090, NVIDIA 620.70.06, through the job queue: memory tour Kokiri/Hyrule Field/Market, in play, clean quit, no new lines; with HOS_TEST_ALLOC_REFUSE=renderer the out-of-memory path ran as on main and stopped with the notice, no 0x20474343. Windows SEH behaviour needs the Test Farm.

Tested in game: Linux, RTX 3090, NVIDIA 620.70.06, through the job queue: memory tour Kokiri/Hyrule Field/Market, in play, clean quit, no new lines; with HOS_TEST_ALLOC_REFUSE=renderer the out-of-memory path ran as on main and stopped with the notice, no 0x20474343. Windows SEH behaviour needs the Test Farm.
Author
Owner

@Beanstalk Windows NVIDIA, Windows AMD and a Steam Deck: a normal start from kokiri-start, play a few minutes through two or three areas, then quit. Expect it to play and close as on main, with no 'vk: a C++ exception came out of' line and no crash (this checks the new guards on Windows, where llvm-mingw's SEH unwinding is used).

@Beanstalk Windows NVIDIA, Windows AMD and a Steam Deck: a normal start from kokiri-start, play a few minutes through two or three areas, then quit. Expect it to play and close as on main, with no 'vk: a C++ exception came out of' line and no crash (this checks the new guards on Windows, where llvm-mingw's SEH unwinding is used).
Member

Beanstalk test farm · planning the tests

Commit 1a3655a · approved by HumanBean (untrusted) · dashboard

Planning the tests for 1a3655a…

<!-- beanstalk:sticky --> ### Beanstalk test farm · planning the tests Commit `1a3655a` · approved by HumanBean (untrusted) · [dashboard](https://beanstalk.magicbean.dev/prs/2170) Planning the tests for `1a3655a`…
HumanBean changed title from WIP: fix(vk): a C++ exception out of the renderer is said and stops the game as out of memory or a FATAL line, not 0x20474343 to fix(vk): a C++ exception out of the renderer is said and stops the game as out of memory or a FATAL line, not 0x20474343 2026-11-22 00:15:06 +00:00
HumanBean deleted branch fix/renderer-cxx-escape 2026-11-22 00:15:16 +00:00
Sign in to join this conversation.
No reviewers
No milestone
No project
No assignees
2 participants
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
MagicBeanDev/Longshot!2170
No description provided.