MagicBeanDev/Longshot

Watch

Fork 99

fix(build): a romfs folder without the game's files is refused before the build #534

Merged

HumanBean merged 1 commit from jarynclouatre/Longshot:fix/romfs-without-game-files into main

2026-11-15 17:46:55 +00:00

jarynclouatre commented

2026-11-14 18:12:00 +00:00

Contributor

Type: fix · Risk: low · Tested on: Linux x86-64 (game-free checks only) · Needs: none

Why

Issue #467: a romfs folder that held another tool's packed copy (an index.json and one 25 GB romfs.bin) was accepted. The build ran for 15 minutes, filled 27 GB, and the game stopped at boot ("indirect call to an address with no function"), since its system files were not there.

nx2/make_game.py only checks that the chosen romfs and exefs are folders (parse_args for --game, main for --romfs with --exefs), then links the romfs into the game folder as it is.

How to reproduce

Seen on: Issue #467; OS, GPU and driver not recorded.
Settings and switches: --game, or --romfs with --exefs, using folders.
Steps: Choose a romfs folder holding another tool's packed copy, an index.json and one 25 GB romfs.bin, and build.
Expected: The folder is refused before the build, with a message to choose the extracted RomFS.
Actual: The folder was accepted; the build ran for 15 minutes, filled 27 GB, and the game stopped at boot ("indirect call to an address with no function") because its system files were missing.

What changed

The builder now checks that a chosen romfs folder holds the files it needs before making the game folder. A packed copy is refused with a message to extract it first.

build (nx2/make_game.py): check_romfs_folder looks for what the build itself reads from the romfs, Pack/ZsDic.pack.zs (prompt_assets.py, shader_census.py) and System/Resource (prompt_assets.py's size table). It runs in main before the game folder is made, for --game and for --romfs with --exefs; not for --nup (which has its own "did not contain a romfs folder") or --update. A folder without them fails with "The romfs folder does not hold the game's files (no Pack/ZsDic.pack.zs): choose the extracted RomFS."; one holding a romfs.bin also says it is a packed copy to extract first. It looks through long_path, as the copy does, so a dump whose deepest paths pass 260 characters on Windows is still found. No file count or hash: a good dump with mods is not refused.
tests (tests/test_make_game_cache.py): the packed copy, an empty folder and a folder with Pack/ZsDic.pack.zs but no System/Resource are each refused with their sentence, before the game folder exists and before the exefs is copied; a folder with both reaches the lift and is linked in; the check looks through long_path. The harness lets nx2/unpack.py through for a chosen exefs (its stand-in module is already unpacked), and the reparse case's source romfs gets the two stand-ins so it still reaches the check it tests.

Measurements

Verdict: the new checks fail on main and pass here; the suite has no new failures.

this PR: fix/romfs-without-game-files @ 0f83aaae (the suite runs below were on 5101c7ae; 0f83aaae changes only the test's dictionary stand-in, which now starts as zstd, so a later check of the chosen folder's .zs files reads it as whole)
Hardware: none used (game-free checks on a Linux x86-64 PC, Python 3.14, gcc 15.2)
Scene: none, the game was not run
Method: 2026-11-14. python3 -I -B tests/run_all.py, two runs each, interleaved (main, PR, main, PR). The new checks also ran against main's nx2/make_game.py (LONGSHOT_CACHE_TEST_SOURCE, in a full checkout of main).
How to reproduce: the commands above in a checkout of each commit

	Before: main @ `543846eb`	After: this PR @ `5101c7ae`
`tests/run_all.py` (run 1, run 2)	485 passed, 2 failed, 84 skipped (both)	485 passed, 2 failed, 84 skipped (both)
failing checks	`test_bake_portability.py`, `test_walk_key.py`	the same two
`tests/test_make_game_cache.py`, new refusal case	the three folders build on to the lift	passes

test_bake_portability.py fails here for a missing Python module (zstandard). test_walk_key.py's fixture doesn't link on main (undefined reference to hos_oot_autoload_input).

Flags and settings

None.

Testing

Tested: the game-free suite on a Linux PC, as in Measurements.

Not tested: a real dump or a real packed copy, the AppImage, and the launcher's folder picker (in its own repository); the message shows wherever make_game's error lines show.

Must be tested: None.

Risks

What could break: A romfs folder without Pack/ZsDic.pack.zs or System/Resource is now refused where it was built before; every supported dump has both, since the build reads them.

Turning it off: Reverting the commit takes the check out.

Compatibility: Nothing changes for a game already built; a packed copy chosen as the romfs is refused with a message before the build starts.

Issues

Fixes #467

Checklist

Everyone

One topic.
The title is a Conventional Commit (type(scope): description) of the status line's type.
Why gives the reason plainly and links the issues that fit (the tracker searched).
Measurements has a verdict, or one line saying why there is nothing to measure.
Testing names what was tested, what wasn't, and what must still be tested, where and how.
Rebased on current main.
python -I -B tests/run_all.py passes (any failure that also fails on main is named).
No game files, dumps, build output or generated files.
third_party/ untouched (a new folder with a PROVENANCE.md instead).
The license is OK: GPL-3.0-or-later, nothing incompatible and nothing decompiled.

When it applies

Numbers from the same PC, scene and settings, at least 2 interleaved runs each, the median reported.
A game-free test added for the fix, failing on main and passing here.
A check under tests/ that reads changed source updated in this PR, not weakened.

**Type:** fix · **Risk:** low · **Tested on:** Linux x86-64 (game-free checks only) · **Needs:** none ## Why Issue #467: a romfs folder that held another tool's packed copy (an `index.json` and one 25 GB `romfs.bin`) was accepted. The build ran for 15 minutes, filled 27 GB, and the game stopped at boot ("indirect call to an address with no function"), since its system files were not there. `nx2/make_game.py` only checks that the chosen romfs and exefs are folders (`parse_args` for `--game`, `main` for `--romfs` with `--exefs`), then links the romfs into the game folder as it is. ## How to reproduce **Seen on:** Issue #467; OS, GPU and driver not recorded. **Settings and switches:** `--game`, or `--romfs` with `--exefs`, using folders. **Steps:** Choose a romfs folder holding another tool's packed copy, an `index.json` and one 25 GB `romfs.bin`, and build. **Expected:** The folder is refused before the build, with a message to choose the extracted RomFS. **Actual:** The folder was accepted; the build ran for 15 minutes, filled 27 GB, and the game stopped at boot ("indirect call to an address with no function") because its system files were missing. ## What changed The builder now checks that a chosen romfs folder holds the files it needs before making the game folder. A packed copy is refused with a message to extract it first. - build (`nx2/make_game.py`): `check_romfs_folder` looks for what the build itself reads from the romfs, `Pack/ZsDic.pack.zs` (`prompt_assets.py`, `shader_census.py`) and `System/Resource` (`prompt_assets.py`'s size table). It runs in `main` before the game folder is made, for `--game` and for `--romfs` with `--exefs`; not for `--nup` (which has its own "did not contain a romfs folder") or `--update`. A folder without them fails with "The romfs folder <path> does not hold the game's files (no Pack/ZsDic.pack.zs): choose the extracted RomFS."; one holding a `romfs.bin` also says it is a packed copy to extract first. It looks through `long_path`, as the copy does, so a dump whose deepest paths pass 260 characters on Windows is still found. No file count or hash: a good dump with mods is not refused. - tests (`tests/test_make_game_cache.py`): the packed copy, an empty folder and a folder with `Pack/ZsDic.pack.zs` but no `System/Resource` are each refused with their sentence, before the game folder exists and before the exefs is copied; a folder with both reaches the lift and is linked in; the check looks through `long_path`. The harness lets `nx2/unpack.py` through for a chosen exefs (its stand-in module is already unpacked), and the reparse case's source romfs gets the two stand-ins so it still reaches the check it tests. ## Measurements **Verdict:** the new checks fail on main and pass here; the suite has no new failures. this PR: fix/romfs-without-game-files @ 0f83aaae (the suite runs below were on 5101c7ae; 0f83aaae changes only the test's dictionary stand-in, which now starts as zstd, so a later check of the chosen folder's .zs files reads it as whole) Hardware: none used (game-free checks on a Linux x86-64 PC, Python 3.14, gcc 15.2) Scene: none, the game was not run Method: 2026-11-14. `python3 -I -B tests/run_all.py`, two runs each, interleaved (main, PR, main, PR). The new checks also ran against main's `nx2/make_game.py` (`LONGSHOT_CACHE_TEST_SOURCE`, in a full checkout of main). How to reproduce: the commands above in a checkout of each commit | | Before: main @ 543846eb | After: this PR @ 5101c7ae | |---|---|---| | `tests/run_all.py` (run 1, run 2) | 485 passed, 2 failed, 84 skipped (both) | 485 passed, 2 failed, 84 skipped (both) | | failing checks | `test_bake_portability.py`, `test_walk_key.py` | the same two | | `tests/test_make_game_cache.py`, new refusal case | the three folders build on to the lift | passes | `test_bake_portability.py` fails here for a missing Python module (zstandard). `test_walk_key.py`'s fixture doesn't link on main (`undefined reference to hos_oot_autoload_input`). ### Flags and settings None. ## Testing **Tested:** the game-free suite on a Linux PC, as in Measurements. **Not tested:** a real dump or a real packed copy, the AppImage, and the launcher's folder picker (in its own repository); the message shows wherever make_game's error lines show. **Must be tested:** None. ## Risks **What could break:** A romfs folder without `Pack/ZsDic.pack.zs` or `System/Resource` is now refused where it was built before; every supported dump has both, since the build reads them. **Turning it off:** Reverting the commit takes the check out. **Compatibility:** Nothing changes for a game already built; a packed copy chosen as the romfs is refused with a message before the build starts. ## Issues Fixes #467 ## Checklist ### Everyone - [x] One topic. - [x] The title is a Conventional Commit (`type(scope): description`) of the status line's type. - [x] Why gives the reason plainly and links the issues that fit (the tracker searched). - [x] Measurements has a verdict, or one line saying why there is nothing to measure. - [x] Testing names what was tested, what wasn't, and what must still be tested, where and how. - [x] Rebased on current `main`. - [x] `python -I -B tests/run_all.py` passes (any failure that also fails on `main` is named). - [x] No game files, dumps, build output or generated files. - [x] `third_party/` untouched (a new folder with a `PROVENANCE.md` instead). - [x] The license is OK: GPL-3.0-or-later, nothing incompatible and nothing decompiled. ### When it applies - [x] Numbers from the same PC, scene and settings, at least 2 interleaved runs each, the median reported. - [x] A game-free test added for the fix, failing on `main` and passing here. - [x] A check under `tests/` that reads changed source updated in this PR, not weakened.

jarynclouatre added 1 commit

2026-11-14 18:12:01 +00:00

fix(build): a romfs folder without the game's files is refused before the build

PR description / Description follows the template (pull_request_target) Failing after 5s

Details

Website / Website on Cloudflare (pull_request_target) Has been skipped

Details

CI / Conventional Commit title (pull_request) Has been cancelled

Details

CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Has been cancelled

Details

CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Has been cancelled

Details

CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Has been cancelled

Details

CI / Workflow lint (actionlint) (pull_request) Has been cancelled

Details

Third-party parts / As their notes say (pull_request) Has been cancelled

Details

Third-party parts / The same as upstream (pull_request) Has been cancelled

Details

5101c7aeb1

make_game.py only checked that the chosen romfs and exefs were folders.
A folder holding another tool's packed copy (index.json and one
romfs.bin, issue 467) was linked in as it was: the build ran for 15
minutes and the game stopped at boot without its system files.

check_romfs_folder looks for what the build itself reads from the
romfs, Pack/ZsDic.pack.zs and System/Resource, before the game folder
is made, for --game and for --romfs with --exefs. A folder with a
romfs.bin is told that it is a packed copy to extract first.

jarynclouatre changed title from ~~WIP: fix(build): a romfs folder without the game's files is refused before the build~~ to fix(build): a romfs folder without the game's files is refused before the build

2026-11-14 18:13:10 +00:00

mooki added the

labels

2026-11-14 18:15:33 +00:00

jarynclouatre force-pushed fix/romfs-without-game-files from 5101c7aeb1

PR description / Description follows the template (pull_request_target) Failing after 5s

Details

Website / Website on Cloudflare (pull_request_target) Has been skipped

Details

CI / Conventional Commit title (pull_request) Has been cancelled

Details

CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Has been cancelled

Details

CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Has been cancelled

Details

CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Has been cancelled

Details

CI / Workflow lint (actionlint) (pull_request) Has been cancelled

Details

Third-party parts / As their notes say (pull_request) Has been cancelled

Details

Third-party parts / The same as upstream (pull_request) Has been cancelled

Details

to 0f83aaaeb4

CI / Conventional Commit title (pull_request) Blocked by required conditions

Details

CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Blocked by required conditions

Details

CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Blocked by required conditions

Details

CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Blocked by required conditions

Details

CI / Workflow lint (actionlint) (pull_request) Blocked by required conditions

Details

Third-party parts / As their notes say (pull_request) Blocked by required conditions

Details

Third-party parts / The same as upstream (pull_request) Blocked by required conditions

Details

Website / Website on Cloudflare (pull_request_target) Has been skipped

Details

PR description / Description follows the template (pull_request_target) Successful in 9s

Details

beanstalk/approval Test farm not requested

2026-11-14 18:15:51 +00:00

Compare

mooki added the

platform/linux

platform/windows

labels

2026-11-14 18:25:03 +00:00

jarynclouatre referenced this pull request

2026-11-14 19:03:35 +00:00

fix(build): a game built from a romfs folder gets its damaged textures replaced, as one from a .nup does #546

futondestroyer approved these changes

2026-11-14 21:13:05 +00:00

futondestroyer left a comment

Reviewed exact head 0f83aaaeb4f48978765f516d4cf7692a22053e5d; no correctness issue found in the complete two-file diff.

Independent Windows 11 build 26300 / Python 3.14.7 check: tests/test_make_game_cache.py runs 20 cases, OK with one existing platform-dependent skip. Running that same new fixture against unchanged main 543846eb gives four errors: the helper is absent, and the packed/empty/incomplete folder cases reach the fixture's lift boundary instead of being refused. On the PR those folders exit before the output or extraction folders are created, while an authored valid layout reaches the lift boundary. All files in this check are authored stand-ins; no game build or actual dump was used.

The guard runs before output-folder creation, uses the existing long-path helper, and checks two paths actually consumed by the prompt/shader readers. NUP and update handling retain their previous route. The added mocks support the new folder-input cases; existing cache/security assertions remain. This approves the bounded folder-layout validation change, not comprehensive dump integrity validation or a fresh full-suite/game build. The long-path routing case passes; no separate native >260-character fixture was run in this review.

Reviewed exact head `0f83aaaeb4f48978765f516d4cf7692a22053e5d`; no correctness issue found in the complete two-file diff. Independent Windows 11 build 26300 / Python 3.14.7 check: `tests/test_make_game_cache.py` runs 20 cases, OK with one existing platform-dependent skip. Running that same new fixture against unchanged main `543846eb` gives four errors: the helper is absent, and the packed/empty/incomplete folder cases reach the fixture's lift boundary instead of being refused. On the PR those folders exit before the output or extraction folders are created, while an authored valid layout reaches the lift boundary. All files in this check are authored stand-ins; no game build or actual dump was used. The guard runs before output-folder creation, uses the existing long-path helper, and checks two paths actually consumed by the prompt/shader readers. NUP and update handling retain their previous route. The added mocks support the new folder-input cases; existing cache/security assertions remain. This approves the bounded folder-layout validation change, not comprehensive dump integrity validation or a fresh full-suite/game build. The long-path routing case passes; no separate native >260-character fixture was run in this review.

malon referenced this pull request

2026-11-15 15:11:03 +00:00

Game: Keeps on crashing in death mountain #817

llun commented

2026-11-15 17:42:04 +00:00

Member

This would also stop two boot crashes: issue #689 and a v3.0.0-rc.4 player's report both crash at boot with "write 0x100000000000, guest 0" because their romfs had no System folder. Removing only System/HeapSize/Product.Nin_Ounce_NVN2.HeapDef.byml.zs from a whole romfs reproduces it (RTX 5080), so a dump that has System/Resource but no heap definition still passes this check. Could you add the heap definition to the list (the game opens Product. then Default.Nin_Ounce_NVN2.HeapDef.byml.zs), and run the check after a .nup is extracted too? Then this fixes #689 as well.

This would also stop two boot crashes: issue #689 and a v3.0.0-rc.4 player's report both crash at boot with "write 0x100000000000, guest 0" because their romfs had no System folder. Removing only `System/HeapSize/Product.Nin_Ounce_NVN2.HeapDef.byml.zs` from a whole romfs reproduces it (RTX 5080), so a dump that has `System/Resource` but no heap definition still passes this check. Could you add the heap definition to the list (the game opens `Product.` then `Default.Nin_Ounce_NVN2.HeapDef.byml.zs`), and run the check after a `.nup` is extracted too? Then this fixes #689 as well.