Feedback on the Modding API Spec from the code mod work (#134 and its stack) #1097

Open
opened 2026-11-16 09:00:53 +00:00 by snaildev99 · 1 comment
Contributor

What this is

Feedback on the Modding API Spec (as of its 2026-11-16 revision with section 5.5), asked for by its author. It is only about the part this contributor's work touches: mods made for a console or an emulator, loaded as they are (5.4, 5.5, 6.1, 8.4, and the unverified list in 19). Nothing here is about the script API, the UI toolkit or the portal. Every point comes from something built and run in the game on #134 and the pull requests stacked on it; where it is only read or reasoned, it says so.

1. Section 5.5, "Not loaded: a whole replacement main, subsdk": subsdk can be loaded, and is a different case from main

The spec marks this line as an assumption. A replaced main is a different game executable, and the reason given holds for it. A mod's subsdk* is not a replacement: it is one more module beside the game's, and the lift takes it like the game's own modules.

  • #984 builds a mod's exefs/subsdk* in with the game: laid out after the game's last module (no game address moves), relocated and bound as the game's modules are, translated, its start routine called after theirs. The Always-On Rupee Counter (a module plus an .ips whose stubs call into it) plays.
  • Modules that install their hooks at start-up (exlaunch and similar: they map the game's code writable and write branches into it) cannot run that start-up in a translated game. The branch mods/modules-stack on the fork runs the start-up once at build time in a processor emulator against the image, takes the bytes it wrote as it takes a patch file's, and leaves the start-up out of the game. Ragdoll Button plays that way. An independent review went through it for what a hostile module can do to the build (time and memory bounds, user mode in the emulator, refusals named).
  • What does not work, by construction: a module that hooks later or by a setting read at run time, one that needs services the host lacks (sockets), a module that loads further code while the game runs.

Suggested change: split the line. A whole main (and main.npdm): not loaded, as written. subsdk*: the unsafe tier, the same warning and the same rebuild as ExeFS patches, with the Mods tab saying per module "built in", "waits for a build" or "left out, and why" (#940 has the note for the case where it is not loaded at all).

2. Section 8.4, switching a code mod: there is a half state between "switched" and "rebuilt", and it can be closed

A mod with patches has files that follow the switch at the next start and code that follows it at the next build. In between the game runs half a mod. Both directions were seen in the game: Off after a build left the code in and drew a menu without its layout; On before a build served the files and the mod's menu did nothing. A tester raised it on #134.

  • #1036 makes the switch real: the build keeps a second version of each function a mod's patch touches (the game's own), a line at the function's top picks one by a byte the host sets at the start, and the host writes the On mods' bytes into the image before any game code runs. Off, restart: the game's own code. On, restart: the mod. No build. A 7.7 KB patch with a code cave needs 58 functions twice (0.13 % more lifted code, about 5 s more lift). It works a function at a time, so a real code cave needs no special rule. Reviewed independently; played twice.
  • It only helps for mods that were in the folder at the build. A mod added or edited afterwards still needs one.
  • Section 5.5 says a mod with exefs/ is off when found and warns first. That fits: the warning gates whether the build takes the mod's code at all; once it has, the switch is free.

Suggested change to 8.4: "Enabling or disabling one offers Rebuild for mods" becomes: a code mod the last build took is switched at the next start; one it did not take (new, edited, or never accepted) is held back whole, files included, until Rebuild for mods. Two settings are yours (they are one constant each in #1036): whether the build takes the code of mods that are Off (so they can be switched on later) or only those On; and how many mods changing one function are switchable separately (each combination is a version: two mods, four versions; the cap is two, the rest fall back to a build).

3. Section 8.4, "Two mods writing the same bytes ... neither is applied until the player picks one"

Two things from real mods:

  • The usual clash is not two mods changing the same instruction. It is two mods putting their own new code in the same free space. Mod authors all look for padding after the game's code and find the same place: three public HUD mods put stubs at main+0x3C46700, inside the range a fourth mod's code occupies. Picking one does not help the player who wants both, and order cannot fix it either. The fix that would is on the format's side: a mod declares how much space it needs and the build places it (for .ips that needs relocating the mod's branches into its cave, which the build can do for direct branches, since it knows what the patch wrote). Not built; raised here because the spec is where such a rule would live.
  • For a true clash, #961 keeps the list's rule (the lower mod wins, as for files), names both mods and the place in the log and on the Mods tab, and makes the lower mod win every byte of the place (also where it writes the game's own byte back). "Neither until the player picks" needs a picker; the list's order is one already. Either is fine; they should not differ between files and patches without a reason.

4. Section 8.4, "A patch must not break the engine"

Checkable today only for the hooks that have addresses (nx2/oot_wide.py, oot_fov.py, oot_jitter.py): a branch for that exists (a note when a patch writes there, the hook then left out). The roughly forty text patches of nx2/oot_patches.py have no address list, so a mod patching one of those functions is not detected, and with #1036 the engine's edit and the mod's version have to be kept apart by hand (it is, with a test). Step 1's site table makes this rule enforceable; until then the spec's sentence promises more than the build can know.

5. A rule the spec lacks: hooks written for a console that return with a branch

Many mods enter their hook with bl written over a game instruction and go back with b <the next instruction>, because their own calls used up the link register. On a console that branch is a return. Lifted, bl is a C call and the b a goto into the rest of the caller, inside the callee: the game crashes a few calls later. It took down a real mod. #960 rewrites that branch as a return before the lift, only where the routine was all written by the patch and is entered from that one call; otherwise it leaves it and says so in the file's note. Worth one bullet in 8.4, since it is the first thing an emulator mod's author hits here.

6. Section 6.1, "Resource size table: never shipped. Computed from the final merged files", and section 19's "the size rule per file type"

What the merge of #134 and #942 learned, all on this game:

  • A file's entry is not a function of its size alone. Per-type factors measured on the game's table are in host/restbl.c. For parameter files (BYML) the entry follows the number of list elements: fits over 5,772 files of two types give a constant, about one entry byte a file byte, and 60 to 290 bytes per element. Growing an entry in proportion to the file was not enough in the game (a load failed on a null object); 8 entry bytes per byte grown is, at the price of about twice the heap such a file needs.
  • Every file inside an actor pack has an entry of its own, and the game enforces it (a grown member with its old entry hangs a loading screen). So "computed from the final merged files" has to walk the packs' members, not only loose files.
  • A mod's own table is worth keeping as a floor for an emulator mod (its author measured it on the real thing): take the larger of it and the computed value, as the merge does. "Never shipped" is right for packages.
  • The first start after new mod files is slow (seconds, reading every served file's header) and fast afterwards; the computed table wants its cache keyed on the files, not made at every start.

7. Section 6.1, archives by member path, and section 19's "the game loads a pack written by a host tool"

  • Verified in the game: it loads actor packs, the layout archive and text archives written by the host, as a zstd frame of raw blocks without the pack dictionary (#1089, played on Windows).
  • #1089 merges an archive several mods ship, member by member against the game's copy, at the start, cached by content hash. Things that mattered and are easy to get wrong: the game reads members where they lie, so each member must keep the alignment its own archive gave it; the layout archive holds some names several times (they are taken together, in order); nodes sorted by hash; a member name is untrusted text (one with a line break crashed an early version through its own log); a bound on the result's size (a 2 KB file could ask for 314 MB).
  • The spec's "applied to every pack that carries the member" goes further than #1089 (which merges the archives mods ship, one path at a time). The member lists and the dictionary handling carry over.

8. Section 11.7 and section 19, "the game's UI layout format"

Mapped, for what a mod needs: a mod on this stack adds its own screen to the game's menu with its own layouts, panes, pane animations and texts, written by a tool, not edited by hand. Known pitfalls: the layout archive stores every layout four times and all four must change; panes must be stored in pre-order; an animation on a pane's alpha does not reach the children of a child pane that has its own. If a write-up would help 11.7, say so and it will be added to the docs.

9. Section 5.4, mods.json: where the build finds it

Built on a branch for #134 (the list as mods.json beside the settings file, unknown keys kept, the old file read once, a file that is not a list never overwritten). One gap the spec does not cover: nx2/make_game.py knows the game folder, not the user folder, and it must read the same list as the game, or a code mod is built for one order and served in another. The branch lets the build ask, in order, HOS_OPTIONS_FILE, a new make_game.py --user <folder>, and a one-line note the game leaves in the game folder when the list is elsewhere. If the launcher always passes --user, the note can go. Which do you want?

10. Small points

  • 5.5 discovery: console packs also carry patches as atmosphere/exefs_patches/<name>/*.ips (the files right in the folder, no exefs/). A branch reads that layout too.
  • 5.5, "the warning ... shown again when its patches change": #134 already records each built patch file's hash in exefs_patches.txt, which is what that check needs.
  • 8.4, the lift key: keyed on the patches' list with the mods' names, a renamed mod folder translates the game again; keyed on the bytes the patches apply, it does not (a branch has this). The same care is needed when the lift key stops hashing every OOT_ variable (Hookshot's main did that): the patches' hash has to be named in it, or a changed patch silently keeps the old translation.
  • Cheats: #134 runs Atmosphere's data opcodes each frame and refuses cheats that write code; those belong with ExeFS patches in the unsafe tier.

Where each thing is

Topic Where Played in the game
The base: list, Mods tab, merged size table, ExeFS patches, cheats #134 Windows, Linux
A note for a mod's code module; size entries inside packs #940, #942 Windows
The returning branch; two mods on one place #960, #961 Windows
Code modules built in #984, branch mods/modules-stack Windows
The switch without a build #1036 Windows
Archives merged by member #1089 Windows
### What this is Feedback on the [Modding API Spec](https://git.magicbean.dev/MagicBeanDev/Longshot/wiki/Modding-API-Spec) (as of its 2026-11-16 revision with section 5.5), asked for by its author. It is only about the part this contributor's work touches: mods made for a console or an emulator, loaded as they are (5.4, 5.5, 6.1, 8.4, and the unverified list in 19). Nothing here is about the script API, the UI toolkit or the portal. Every point comes from something built and run in the game on #134 and the pull requests stacked on it; where it is only read or reasoned, it says so. ### 1. Section 5.5, "Not loaded: a whole replacement main, subsdk": `subsdk` can be loaded, and is a different case from `main` The spec marks this line as an assumption. A replaced `main` is a different game executable, and the reason given holds for it. A mod's `subsdk*` is not a replacement: it is one more module beside the game's, and the lift takes it like the game's own modules. - **#984** builds a mod's `exefs/subsdk*` in with the game: laid out after the game's last module (no game address moves), relocated and bound as the game's modules are, translated, its start routine called after theirs. The Always-On Rupee Counter (a module plus an `.ips` whose stubs call into it) plays. - **Modules that install their hooks at start-up** (exlaunch and similar: they map the game's code writable and write branches into it) cannot run that start-up in a translated game. The branch `mods/modules-stack` on the fork runs the start-up once at build time in a processor emulator against the image, takes the bytes it wrote as it takes a patch file's, and leaves the start-up out of the game. Ragdoll Button plays that way. An independent review went through it for what a hostile module can do to the build (time and memory bounds, user mode in the emulator, refusals named). - What does not work, by construction: a module that hooks later or by a setting read at run time, one that needs services the host lacks (sockets), a module that loads further code while the game runs. **Suggested change:** split the line. A whole `main` (and `main.npdm`): not loaded, as written. `subsdk*`: the unsafe tier, the same warning and the same rebuild as ExeFS patches, with the Mods tab saying per module "built in", "waits for a build" or "left out, and why" (#940 has the note for the case where it is not loaded at all). ### 2. Section 8.4, switching a code mod: there is a half state between "switched" and "rebuilt", and it can be closed A mod with patches has files that follow the switch at the next start and code that follows it at the next build. In between the game runs half a mod. Both directions were seen in the game: Off after a build left the code in and drew a menu without its layout; On before a build served the files and the mod's menu did nothing. A tester raised it on #134. - **#1036** makes the switch real: the build keeps a second version of each function a mod's patch touches (the game's own), a line at the function's top picks one by a byte the host sets at the start, and the host writes the On mods' bytes into the image before any game code runs. Off, restart: the game's own code. On, restart: the mod. No build. A 7.7 KB patch with a code cave needs 58 functions twice (0.13 % more lifted code, about 5 s more lift). It works a function at a time, so a real code cave needs no special rule. Reviewed independently; played twice. - It only helps for mods that were in the folder at the build. A mod added or edited afterwards still needs one. - Section 5.5 says a mod with `exefs/` is off when found and warns first. That fits: the warning gates whether the build takes the mod's code at all; once it has, the switch is free. **Suggested change to 8.4:** "Enabling or disabling one offers Rebuild for mods" becomes: a code mod the last build took is switched at the next start; one it did not take (new, edited, or never accepted) is held back **whole**, files included, until Rebuild for mods. Two settings are yours (they are one constant each in #1036): whether the build takes the code of mods that are Off (so they can be switched on later) or only those On; and how many mods changing one function are switchable separately (each combination is a version: two mods, four versions; the cap is two, the rest fall back to a build). ### 3. Section 8.4, "Two mods writing the same bytes ... neither is applied until the player picks one" Two things from real mods: - **The usual clash is not two mods changing the same instruction. It is two mods putting their own new code in the same free space.** Mod authors all look for padding after the game's code and find the same place: three public HUD mods put stubs at `main+0x3C46700`, inside the range a fourth mod's code occupies. Picking one does not help the player who wants both, and order cannot fix it either. The fix that would is on the format's side: a mod declares how much space it needs and the build places it (for `.ips` that needs relocating the mod's branches into its cave, which the build can do for direct branches, since it knows what the patch wrote). Not built; raised here because the spec is where such a rule would live. - For a true clash, #961 keeps the list's rule (the lower mod wins, as for files), names both mods and the place in the log and on the Mods tab, and makes the lower mod win every byte of the place (also where it writes the game's own byte back). "Neither until the player picks" needs a picker; the list's order is one already. Either is fine; they should not differ between files and patches without a reason. ### 4. Section 8.4, "A patch must not break the engine" Checkable today only for the hooks that have addresses (`nx2/oot_wide.py`, `oot_fov.py`, `oot_jitter.py`): a branch for that exists (a note when a patch writes there, the hook then left out). The roughly forty text patches of `nx2/oot_patches.py` have no address list, so a mod patching one of those functions is not detected, and with #1036 the engine's edit and the mod's version have to be kept apart by hand (it is, with a test). Step 1's site table makes this rule enforceable; until then the spec's sentence promises more than the build can know. ### 5. A rule the spec lacks: hooks written for a console that return with a branch Many mods enter their hook with `bl` written over a game instruction and go back with `b <the next instruction>`, because their own calls used up the link register. On a console that branch is a return. Lifted, `bl` is a C call and the `b` a goto into the rest of the caller, inside the callee: the game crashes a few calls later. It took down a real mod. **#960** rewrites that branch as a return before the lift, only where the routine was all written by the patch and is entered from that one call; otherwise it leaves it and says so in the file's note. Worth one bullet in 8.4, since it is the first thing an emulator mod's author hits here. ### 6. Section 6.1, "Resource size table: never shipped. Computed from the final merged files", and section 19's "the size rule per file type" What the merge of #134 and #942 learned, all on this game: - A file's entry is not a function of its size alone. Per-type factors measured on the game's table are in `host/restbl.c`. For parameter files (BYML) the entry follows the number of list elements: fits over 5,772 files of two types give a constant, about one entry byte a file byte, and 60 to 290 bytes per element. Growing an entry in proportion to the file was **not enough in the game** (a load failed on a null object); 8 entry bytes per byte grown is, at the price of about twice the heap such a file needs. - **Every file inside an actor pack has an entry of its own, and the game enforces it** (a grown member with its old entry hangs a loading screen). So "computed from the final merged files" has to walk the packs' members, not only loose files. - A mod's own table is worth keeping as a **floor** for an emulator mod (its author measured it on the real thing): take the larger of it and the computed value, as the merge does. "Never shipped" is right for packages. - The first start after new mod files is slow (seconds, reading every served file's header) and fast afterwards; the computed table wants its cache keyed on the files, not made at every start. ### 7. Section 6.1, archives by member path, and section 19's "the game loads a pack written by a host tool" - **Verified in the game:** it loads actor packs, the layout archive and text archives written by the host, as a zstd frame of raw blocks **without** the pack dictionary (#1089, played on Windows). - #1089 merges an archive several mods ship, member by member against the game's copy, at the start, cached by content hash. Things that mattered and are easy to get wrong: the game reads members where they lie, so each member must keep the alignment its own archive gave it; the layout archive holds some names several times (they are taken together, in order); nodes sorted by hash; a member name is untrusted text (one with a line break crashed an early version through its own log); a bound on the result's size (a 2 KB file could ask for 314 MB). - The spec's "applied to every pack that carries the member" goes further than #1089 (which merges the archives mods ship, one path at a time). The member lists and the dictionary handling carry over. ### 8. Section 11.7 and section 19, "the game's UI layout format" Mapped, for what a mod needs: a mod on this stack adds its own screen to the game's menu with its own layouts, panes, pane animations and texts, written by a tool, not edited by hand. Known pitfalls: the layout archive stores every layout four times and all four must change; panes must be stored in pre-order; an animation on a pane's alpha does not reach the children of a child pane that has its own. If a write-up would help 11.7, say so and it will be added to the docs. ### 9. Section 5.4, `mods.json`: where the build finds it Built on a branch for #134 (the list as `mods.json` beside the settings file, unknown keys kept, the old file read once, a file that is not a list never overwritten). One gap the spec does not cover: `nx2/make_game.py` knows the game folder, not the user folder, and it must read the same list as the game, or a code mod is built for one order and served in another. The branch lets the build ask, in order, `HOS_OPTIONS_FILE`, a new `make_game.py --user <folder>`, and a one-line note the game leaves in the game folder when the list is elsewhere. If the launcher always passes `--user`, the note can go. Which do you want? ### 10. Small points - 5.5 discovery: console packs also carry patches as `atmosphere/exefs_patches/<name>/*.ips` (the files right in the folder, no `exefs/`). A branch reads that layout too. - 5.5, "the warning ... shown again when its patches change": #134 already records each built patch file's hash in `exefs_patches.txt`, which is what that check needs. - 8.4, the lift key: keyed on the patches' list with the mods' names, a renamed mod folder translates the game again; keyed on the bytes the patches apply, it does not (a branch has this). The same care is needed when the lift key stops hashing every `OOT_` variable (Hookshot's main did that): the patches' hash has to be named in it, or a changed patch silently keeps the old translation. - Cheats: #134 runs Atmosphere's data opcodes each frame and refuses cheats that write code; those belong with ExeFS patches in the unsafe tier. ### Where each thing is | Topic | Where | Played in the game | |---|---|---| | The base: list, Mods tab, merged size table, ExeFS patches, cheats | #134 | Windows, Linux | | A note for a mod's code module; size entries inside packs | #940, #942 | Windows | | The returning branch; two mods on one place | #960, #961 | Windows | | Code modules built in | #984, branch `mods/modules-stack` | Windows | | The switch without a build | #1036 | Windows | | Archives merged by member | #1089 | Windows |
Author
Contributor

Addendum: a census of real code mods, and point 9 is on #134

Since this was posted, nine public code mods for this game were put through the stack's own build-time checks (game-free; three of them have been played on the stack, the rest not). What that adds to the points above:

  • Point 1 (subsdk). Six of the nine ship a subsdk* module: four are a module plus a small .ips whose stubs call into it, two install their hooks at start-up (exlaunch). Three are patches only. None replaces main, none is a plugin loader, none uses the network. So "subsdk: not loaded" would leave out two thirds of the code mods that exist for this game today. None of the nine holds an instruction the translator lacks.
  • A third way a module hooks, which the stack missed until now. One mod (save profiles) writes no code at all: at start-up it points 20 slots of the game's import table (file, controller and graphics-loader functions) at its own functions. The lift binds a call through a PLT stub to the slot's value when the game is built, so in a translated game that rewrite comes too late. The mod was listed as built in and would have done nothing, with no note. A branch now tries such a start-up at build time and builds the slots in, as it does for the kind that writes code (a game-free test; in the built image all 20 calls lead to the mod's code; not played yet). For the spec, whatever it says about modules: the import table is fixed at build time, and a mod that changes it has to be seen by the build.
  • Point 3 (clashes), in numbers. With one larger mod in the list, five of the nine overlap it, every one of them in the padding after the game's code (the stubs of four module mods and one patch). No two of the nine change the same game instruction. That is the "same free space" clash, not the "same bytes of the game" one.
  • A host gap one mod meets: nn::oe::RestartProgram only logs. A mod that restarts the game after a choice works if the player restarts by hand.
  • What no mod at hand needs: hooks installed later than start-up, SDK calls inside a start-up, sockets. (A co-op mod in development will need sockets; today a module's socket calls would return zero and it would never connect, so a build note naming such stubbed imports is the cheap first step.)

Point 9 is done on #134: the list is mods.json at head 3fbc327a, reviewed independently, its first three commits played (comment there). The question for you stands: the one-line note the game leaves for the build, or the launcher always passing --user.

### Addendum: a census of real code mods, and point 9 is on #134 Since this was posted, nine public code mods for this game were put through the stack's own build-time checks (game-free; three of them have been played on the stack, the rest not). What that adds to the points above: - **Point 1 (`subsdk`).** Six of the nine ship a `subsdk*` module: four are a module plus a small `.ips` whose stubs call into it, two install their hooks at start-up (exlaunch). Three are patches only. None replaces `main`, none is a plugin loader, none uses the network. So "subsdk: not loaded" would leave out two thirds of the code mods that exist for this game today. None of the nine holds an instruction the translator lacks. - **A third way a module hooks, which the stack missed until now.** One mod (save profiles) writes no code at all: at start-up it points 20 slots of the game's import table (file, controller and graphics-loader functions) at its own functions. The lift binds a call through a PLT stub to the slot's value when the game is built, so in a translated game that rewrite comes too late. The mod was listed as built in and would have done nothing, with no note. A branch now tries such a start-up at build time and builds the slots in, as it does for the kind that writes code (a game-free test; in the built image all 20 calls lead to the mod's code; **not played yet**). For the spec, whatever it says about modules: the import table is fixed at build time, and a mod that changes it has to be seen by the build. - **Point 3 (clashes), in numbers.** With one larger mod in the list, five of the nine overlap it, every one of them in the padding after the game's code (the stubs of four module mods and one patch). No two of the nine change the same game instruction. That is the "same free space" clash, not the "same bytes of the game" one. - **A host gap one mod meets:** `nn::oe::RestartProgram` only logs. A mod that restarts the game after a choice works if the player restarts by hand. - **What no mod at hand needs:** hooks installed later than start-up, SDK calls inside a start-up, sockets. (A co-op mod in development will need sockets; today a module's socket calls would return zero and it would never connect, so a build note naming such stubbed imports is the cheap first step.) **Point 9 is done on #134:** the list is `mods.json` at head 3fbc327a, reviewed independently, its first three commits played ([comment there](https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/134#issuecomment-34804)). The question for you stands: the one-line note the game leaves for the build, or the launcher always passing `--user`.
Sign in to join this conversation.
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
MagicBeanDev/Longshot#1097
No description provided.