MagicBeanDev/Longshot

Watch

Fork 99

fix(build): a romfs without the game's heap definition is refused too, also from a .nup and for an update #863

Merged

HumanBean merged 1 commit from llun/Longshot:fix/romfs-heapdef-check into main

2026-11-15 18:18:43 +00:00

llun commented

2026-11-15 18:06:45 +00:00

Member

Type: fix · Risk: low · Tested on: Windows 11, RTX 5080 (the builder; the crash reproduced in the game) · Needs: none

Why

Some players' games crash the moment they start, every time, with "Longshot closed unexpectedly during the start" and "An unexpected error". Their game was built from game files missing the romfs's System folder, which holds the file that tells the game how big to make its memory heaps. Without it the game sets its heaps up with no room in them, and the first file it loads crashes it. Rebuilding from the same files gives the same crash, and nothing tells the player that their dump is the problem.

#534 refuses a chosen romfs folder without Pack/ZsDic.pack.zs or System/Resource. That catches a folder with no System at all, but not one that has System/Resource without the heap definition. It also doesn't check a romfs unpacked from a .nup, or the one an update builds from.

See #776, #689 (the same crash: guest lr 0x80c4afcc, "write 0x100000000000, guest 0"), and a v3.0.0-rc.4 player's report with the same backtrace.

How to reproduce

Seen on: #776 (Windows 11, Ryzen 5 8600G, nightly-189), #689 (Windows 11, GTX 1050, nightly-171), a v3.0.0-rc.4 player (Windows 11, i9-13900H); reproduced on Windows 11, RTX 5080, main 25726217
Settings and switches: defaults
Steps: build the game from a romfs folder without System/HeapSize/Product.Nin_Ounce_NVN2.HeapDef.byml.zs (System/Resource still there), then start it.
Expected: the builder refuses the folder before it builds and says which file is missing.
Actual: the build runs to the end. The game then crashes at boot with exception 0xc0000005 (write 0x100000000000, guest 0), guest backtrace lr 0x80c4afcc 0x83bb705c 0x83b7b47c 0x83b7a838 0x83b79e88 .... A romfs with the whole System folder missing crashes with the same backtrace and memory-touch addresses as #689's log.

What changed

The heap definition joins the files a romfs must hold, and the check now runs wherever the build takes its romfs from: a chosen folder, a .nup's fresh unpack, a .nup read before, and an update. A romfs that fails is refused before the build, with the file it lacks.

build (nx2/make_game.py): ROMFS_NEEDED lists Pack/ZsDic.pack.zs, System/Resource and the heap definition. The game opens System/HeapSize/Product.Nin_Ounce_NVN2.HeapDef.byml.zs, then Default.; either one passes. romfs_missing() names the first thing missing, and check_romfs_folder() (#534's check for a chosen folder, same message) uses it.
build (nx2/make_game.py): check_unpacked_romfs() gives the same check for the other three sources, with "Your .nup file ... does not hold all of the game's files (no ...): a game built from it could not start. Dump the game again and choose the complete dump in Longshot Builder."
- A fresh unpack is checked inside the extraction's try, so a refused one is removed and the folder's earlier files and saves stay.
- A .nup read before, and an update (--update), check the game folder's own romfs.
tests (tests/test_make_game_cache.py): the fixture's old game folder and its .nup extractions get stand-ins for the start files. New cases:
- a chosen folder without the heap definition is refused;
- one with only the Default heap definition reaches the lift;
- a .nup whose unpack lacks the start files is refused, with the earlier files and saves kept and no .game left;
- a .nup read before, and an update, whose romfs lacks the heap definition are refused.

Measurements

Verdict: pass. The builder refuses a romfs without the heap definition at once, where main builds it into a game that crashes at boot; the new checks fail on main and pass here; a real, complete romfs still passes.

Hardware: Windows 11 Pro, RTX 5080, Builder tools v2.1.0-rc.3-44 (llvm-mingw clang 23.1.5)
Scene: none, the builder (no game run for the measurement)
Method: 2026-11-15. nx2/make_game.py --romfs <copy> --exefs <game folder's .longshot_exefs> --out <scratch> --no-prepare-shaders, on a hard-linked copy of a complete romfs (186,583 files) with only System/HeapSize/Product.Nin_Ounce_NVN2.HeapDef.byml.zs left out; main's build was stopped after 90 s. The checks: tests/test_make_game_cache.py and tests/run_all.py.
How to reproduce: the command above, in a checkout of each commit
Baseline: the builder rows ran on main @ 1c91dafb against this PR @ b7324bd8, before the rebases. Since then main brought in #561 (a test), #857 (the exe's name and icon, in a part of nx2/make_game.py this PR doesn't touch) and #346 (the renderer); the romfs check and where it runs are the same. The test rows ran again on the rebased commits.

	Before: main @ `1c91dafb` (tests: `1310ecc6`)	After: this PR @ b7324bd8 (tests: `be4de235`)
builder, romfs without the heap definition	builds (`@@stage read`, game folder made; stopped at 90 s)	refused at once, no game folder
builder, romfs without `System`	refused (#534)	refused, the same message
`romfs_missing()` on a complete romfs unpacked from the `.nup`		`None`: passes
`test_make_game_cache.py`: the three new refusal cases	fail (the build reaches the lift)	pass
`tests/run_all.py`	496 passed, 20 failed, 80 skipped (`test_longshot_ctl` failed once, then passed twice: timing)	497 passed, 19 failed, 80 skipped: main's other 19

After:  @@error The romfs folder C:/.../romfs-hard-no-heapdef does not hold the game's files (no System/HeapSize/Product.Nin_Ounce_NVN2.HeapDef.byml.zs): choose the extracted RomFS.
Before: @@stage read Reading your game files   (then the build went on)

Flags and settings

None.

Testing

Tested:

Windows 11, RTX 5080: the builder on a real romfs without the heap definition (refused here, built on main), and on one without System (refused on both).
The new check functions on a complete romfs that the builder unpacked from a .nup: it passes.
In the game (main 25726217), the crash itself: a romfs without only the heap definition, and one without the whole System folder, both crash at boot with the players' backtrace. The whole romfs reaches the intro.
Game-free: tests/run_all.py on Windows. On this PC tests/test_make_game_cache.py fails on main too: the work folder's lock is left open on Windows (#804), and with no RX_VK_PGO the renderer profile's clang probe hits the test's no-process guard. With RX_VK_PGO=off, every assertion passes here (23 tests), and the errors left are the lock cleanup alone (main has 15 of them, this PR 18, three of them the new tests').

Not tested:

A real .nup unpacked through the new check: a 25 GB extraction. The unpacked result was checked instead (above), and the path itself is covered by the test.
Linux: test_make_game_cache.py runs fully on the Linux checks.

Must be tested: None.

Risks

What could break: a dump that holds neither heap definition, but that the game would start from, is now refused. The supported version opens Product. and then Default., and crashes without both, as reproduced above.

Turning it off: revert the commit.

Compatibility: nothing changes for a game that's already built. A new build, a .nup read again or an update, from game files without the heap definition, is refused with the missing file named instead of building a game that can't start.

Issues

See #776
See #689

Checklist

Everyone

One topic.
The title is a Conventional Commit (type(scope): description) of the status line's type.
Why gives the reason plainly and links the issues that fit (the tracker searched).
Measurements has a verdict, or one line saying why there is nothing to measure.
Testing names what was tested, what wasn't, and what must still be tested, where and how.
Rebased on current main.
python -I -B tests/run_all.py passes (any failure that also fails on main is named). Main's own 19 here: test_appimage_player_build, test_bake_portability, test_cpp_preflight, test_crash_dialog, test_env_on, test_fpx_sticky, test_lift_cache, test_lift_cleanup_error, test_linux_preflight, test_make_game_cache, test_make_game_prepare, test_missing_nup, test_oot_state_tap, test_options_budget_once, test_readback_late, test_spawn_emit, test_tested_configurations, test_vk_pgo, test_walk_key.
No game files, dumps, build output or generated files.
third_party/ untouched (a new folder with a PROVENANCE.md instead).
The license is OK: GPL-3.0-or-later, nothing incompatible and nothing decompiled.

When it applies

AI assistance recorded with Assisted-by: / Co-Authored-By: trailers.
A game-free test added for the fix, failing on main and passing here.
A check under tests/ that reads changed source updated in this PR, not weakened.

🤖 Generated with Claude Code

**Type:** fix · **Risk:** low · **Tested on:** Windows 11, RTX 5080 (the builder; the crash reproduced in the game) · **Needs:** none ## Why Some players' games crash the moment they start, every time, with "Longshot closed unexpectedly during the start" and "An unexpected error". Their game was built from game files missing the romfs's `System` folder, which holds the file that tells the game how big to make its memory heaps. Without it the game sets its heaps up with no room in them, and the first file it loads crashes it. Rebuilding from the same files gives the same crash, and nothing tells the player that their dump is the problem. #534 refuses a chosen romfs folder without `Pack/ZsDic.pack.zs` or `System/Resource`. That catches a folder with no `System` at all, but not one that has `System/Resource` without the heap definition. It also doesn't check a romfs unpacked from a `.nup`, or the one an update builds from. See #776, #689 (the same crash: guest lr `0x80c4afcc`, "write 0x100000000000, guest 0"), and a v3.0.0-rc.4 player's report with the same backtrace. ## How to reproduce **Seen on:** #776 (Windows 11, Ryzen 5 8600G, nightly-189), #689 (Windows 11, GTX 1050, nightly-171), a v3.0.0-rc.4 player (Windows 11, i9-13900H); reproduced on Windows 11, RTX 5080, main 25726217 **Settings and switches:** defaults **Steps:** build the game from a romfs folder without `System/HeapSize/Product.Nin_Ounce_NVN2.HeapDef.byml.zs` (`System/Resource` still there), then start it. **Expected:** the builder refuses the folder before it builds and says which file is missing. **Actual:** the build runs to the end. The game then crashes at boot with `exception 0xc0000005 (write 0x100000000000, guest 0)`, guest backtrace `lr 0x80c4afcc 0x83bb705c 0x83b7b47c 0x83b7a838 0x83b79e88 ...`. A romfs with the whole `System` folder missing crashes with the same backtrace and memory-touch addresses as #689's log. ## What changed The heap definition joins the files a romfs must hold, and the check now runs wherever the build takes its romfs from: a chosen folder, a `.nup`'s fresh unpack, a `.nup` read before, and an update. A romfs that fails is refused before the build, with the file it lacks. - build (`nx2/make_game.py`): `ROMFS_NEEDED` lists `Pack/ZsDic.pack.zs`, `System/Resource` and the heap definition. The game opens `System/HeapSize/Product.Nin_Ounce_NVN2.HeapDef.byml.zs`, then `Default.`; either one passes. `romfs_missing()` names the first thing missing, and `check_romfs_folder()` (#534's check for a chosen folder, same message) uses it. - build (`nx2/make_game.py`): `check_unpacked_romfs()` gives the same check for the other three sources, with "Your .nup file ... does not hold all of the game's files (no ...): a game built from it could not start. Dump the game again and choose the complete dump in Longshot Builder." - A fresh unpack is checked inside the extraction's `try`, so a refused one is removed and the folder's earlier files and saves stay. - A `.nup` read before, and an update (`--update`), check the game folder's own `romfs`. - tests (`tests/test_make_game_cache.py`): the fixture's old game folder and its `.nup` extractions get stand-ins for the start files. New cases: - a chosen folder without the heap definition is refused; - one with only the Default heap definition reaches the lift; - a `.nup` whose unpack lacks the start files is refused, with the earlier files and saves kept and no `.game` left; - a `.nup` read before, and an update, whose romfs lacks the heap definition are refused. ## Measurements **Verdict:** pass. The builder refuses a romfs without the heap definition at once, where main builds it into a game that crashes at boot; the new checks fail on main and pass here; a real, complete romfs still passes. Hardware: Windows 11 Pro, RTX 5080, Builder tools v2.1.0-rc.3-44 (llvm-mingw clang 23.1.5) Scene: none, the builder (no game run for the measurement) Method: 2026-11-15. `nx2/make_game.py --romfs <copy> --exefs <game folder's .longshot_exefs> --out <scratch> --no-prepare-shaders`, on a hard-linked copy of a complete romfs (186,583 files) with only `System/HeapSize/Product.Nin_Ounce_NVN2.HeapDef.byml.zs` left out; main's build was stopped after 90 s. The checks: `tests/test_make_game_cache.py` and `tests/run_all.py`. How to reproduce: the command above, in a checkout of each commit Baseline: the builder rows ran on main @ 1c91dafb against this PR @ b7324bd8, before the rebases. Since then main brought in #561 (a test), #857 (the exe's name and icon, in a part of `nx2/make_game.py` this PR doesn't touch) and #346 (the renderer); the romfs check and where it runs are the same. The test rows ran again on the rebased commits. | | Before: main @ 1c91dafb (tests: 1310ecc6) | After: this PR @ b7324bd8 (tests: be4de235) | |---|---|---| | builder, romfs without the heap definition | builds (`@@stage read`, game folder made; stopped at 90 s) | refused at once, no game folder | | builder, romfs without `System` | refused (#534) | refused, the same message | | `romfs_missing()` on a complete romfs unpacked from the `.nup` | | `None`: passes | | `test_make_game_cache.py`: the three new refusal cases | fail (the build reaches the lift) | pass | | `tests/run_all.py` | 496 passed, 20 failed, 80 skipped (`test_longshot_ctl` failed once, then passed twice: timing) | 497 passed, 19 failed, 80 skipped: main's other 19 | ``` After: @@error The romfs folder C:/.../romfs-hard-no-heapdef does not hold the game's files (no System/HeapSize/Product.Nin_Ounce_NVN2.HeapDef.byml.zs): choose the extracted RomFS. Before: @@stage read Reading your game files (then the build went on) ``` ### Flags and settings None. ## Testing **Tested:** - Windows 11, RTX 5080: the builder on a real romfs without the heap definition (refused here, built on main), and on one without `System` (refused on both). - The new check functions on a complete romfs that the builder unpacked from a `.nup`: it passes. - In the game (main 25726217), the crash itself: a romfs without only the heap definition, and one without the whole `System` folder, both crash at boot with the players' backtrace. The whole romfs reaches the intro. - Game-free: `tests/run_all.py` on Windows. On this PC `tests/test_make_game_cache.py` fails on main too: the work folder's lock is left open on Windows (#804), and with no `RX_VK_PGO` the renderer profile's clang probe hits the test's no-process guard. With `RX_VK_PGO=off`, every assertion passes here (23 tests), and the errors left are the lock cleanup alone (main has 15 of them, this PR 18, three of them the new tests'). **Not tested:** - A real `.nup` unpacked through the new check: a 25 GB extraction. The unpacked result was checked instead (above), and the path itself is covered by the test. - Linux: `test_make_game_cache.py` runs fully on the Linux checks. **Must be tested:** None. ## Risks **What could break:** a dump that holds neither heap definition, but that the game would start from, is now refused. The supported version opens `Product.` and then `Default.`, and crashes without both, as reproduced above. **Turning it off:** revert the commit. **Compatibility:** nothing changes for a game that's already built. A new build, a `.nup` read again or an update, from game files without the heap definition, is refused with the missing file named instead of building a game that can't start. ## Issues See #776 See #689 ## Checklist ### Everyone - [x] One topic. - [x] The title is a Conventional Commit (`type(scope): description`) of the status line's type. - [x] Why gives the reason plainly and links the issues that fit (the tracker searched). - [x] Measurements has a verdict, or one line saying why there is nothing to measure. - [x] Testing names what was tested, what wasn't, and what must still be tested, where and how. - [x] Rebased on current `main`. - [x] `python -I -B tests/run_all.py` passes (any failure that also fails on `main` is named). Main's own 19 here: test_appimage_player_build, test_bake_portability, test_cpp_preflight, test_crash_dialog, test_env_on, test_fpx_sticky, test_lift_cache, test_lift_cleanup_error, test_linux_preflight, test_make_game_cache, test_make_game_prepare, test_missing_nup, test_oot_state_tap, test_options_budget_once, test_readback_late, test_spawn_emit, test_tested_configurations, test_vk_pgo, test_walk_key. - [x] No game files, dumps, build output or generated files. - [x] `third_party/` untouched (a new folder with a `PROVENANCE.md` instead). - [x] The license is OK: GPL-3.0-or-later, nothing incompatible and nothing decompiled. ### When it applies - [x] AI assistance recorded with `Assisted-by:` / `Co-Authored-By:` trailers. - [x] A game-free test added for the fix, failing on `main` and passing here. - [x] A check under `tests/` that reads changed source updated in this PR, not weakened. 🤖 Generated with [Claude Code](https://claude.com/claude-code)

llun added 1 commit

2026-11-15 18:06:47 +00:00

fix(build): a romfs without the game's heap definition is refused too, also from a .nup and for an update

CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Has been skipped

Details

CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Has been skipped

Details

CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Has been skipped

Details

Third-party parts / The same as upstream (pull_request) Has been skipped

Details

Website / Website on Cloudflare (pull_request_target) Has been skipped

Details

PR description / Description follows the template (pull_request_target) Has been cancelled

Details

CI / Conventional Commit title (pull_request) Has been cancelled

Details

CI / Workflow lint (actionlint) (pull_request) Has been cancelled

Details

Guide screenshots / Guide images and local UI source review (pull_request) Has been cancelled

Details

Third-party parts / As their notes say (pull_request) Has been cancelled

Details

7085be4422

A romfs folder with no System folder built, and the game crashed at boot every time with "write 0x100000000000,
guest 0" (issues 689 and 776, a v3.0.0-rc.4 player): without System/HeapSize's heap definition the game sets its
heaps up without their sizes. #534 refuses a chosen folder without Pack/ZsDic.pack.zs or System/Resource; the heap
definition joins that list (Product. or Default., the names the game opens), and the same check now runs on a .nup's
unpacked romfs (before it replaces the folder's) and on the romfs a .nup read before or an update builds from.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

llun changed title from ~~WIP: fix(build): a romfs without the game's heap definition is refused too, also from a .nup and for an update~~ to fix(build): a romfs without the game's heap definition is refused too, also from a .nup and for an update

2026-11-15 18:07:03 +00:00

BeanIssues added the

kind

bug

area/install

labels

2026-11-15 18:07:16 +00:00

llun force-pushed fix/romfs-heapdef-check from 7085be4422

CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Has been skipped

Details

CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Has been skipped

Details

CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Has been skipped

Details

Third-party parts / The same as upstream (pull_request) Has been skipped

Details

Website / Website on Cloudflare (pull_request_target) Has been skipped

Details

PR description / Description follows the template (pull_request_target) Has been cancelled

Details

CI / Conventional Commit title (pull_request) Has been cancelled

Details

CI / Workflow lint (actionlint) (pull_request) Has been cancelled

Details

Guide screenshots / Guide images and local UI source review (pull_request) Has been cancelled

Details

Third-party parts / As their notes say (pull_request) Has been cancelled

Details

to be4de2351d

CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Has been skipped

Details

CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Has been skipped

Details

CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Has been skipped

Details

Third-party parts / The same as upstream (pull_request) Has been skipped

Details

PR description / Description follows the template (pull_request_target) Successful in 5s

Details

CI / Conventional Commit title (pull_request) Successful in 4s

Details

CI / Workflow lint (actionlint) (pull_request) Successful in 7s

Details

Guide screenshots / Guide images and local UI source review (pull_request) Failing after 5s

Details

Third-party parts / As their notes say (pull_request) Successful in 7s

Details

2026-11-15 18:12:08 +00:00

Compare

llun commented

2026-11-15 18:12:10 +00:00

Author

Member

Rebased onto main 1310ecc6 (#561, #857, #346; #857 touches make_game.py elsewhere, the romfs check is the same). Suite again on both: 497 passed / 19 failed here, the same 19 as main's (main also had one timing failure of test_longshot_ctl, which passed on rerun); test_make_game_cache's assertions all pass.

Rebased onto main 1310ecc6 (#561, #857, #346; #857 touches make_game.py elsewhere, the romfs check is the same). Suite again on both: 497 passed / 19 failed here, the same 19 as main's (main also had one timing failure of test_longshot_ctl, which passed on rerun); test_make_game_cache's assertions all pass.