WIP: feat(mods): a Mods tab, merged resource size tables, ExeFS patches and Atmosphere cheats (HumanBean's draft from Hookshot, with the lift fix) #134
No reviewers
Labels
No labels
area/accessibility
area/audio
area/input
area/install
area/menu
area/renderer
area/saves
area/upscaling
good first issue
gpu/amd
gpu/intel
gpu/nvidia
help wanted
kind
bug
kind
chore
kind
docs
kind
feature
kind
performance
kind
question
needs measurements
platform/linux
platform/steam-deck
platform/windows
priority
critical
priority
high
priority
low
status
blocked
status
confirmed
status
duplicate
status
invalid
status
needs-info
status
wontfix
testing wanted
No milestone
No project
No assignees
2 participants
Notifications
Due date
No due date set.
Dependencies
No dependencies set
Reference
MagicBeanDev/Longshot!134
Loading…
Reference in a new issue
No description provided.
Delete branch "snaildev99/Longshot:feat/mods-exefs-patches"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Why
Mods that change the game's code (ExeFS patches:
.ips,.pchtxt) do nothing on Longshot, two mods that each ship a resource size table cannot load together, and there is no place in the game to see or order the mods. HumanBean's draft for all of this was on Hookshot (https://github.com/NotRexed/Hookshot/pull/2478) and never came over; he asked for it to be brought here.What changed
HumanBean's two commits are ported as they were, under his name; the changes of the port and of the review follow them.
host/mods.creads the mods' order and switches fromlongshot_mods.txt, serves one size table merged from every enabled mod's (host/restbl.c,host/zs.c, written tomodcache/rsizetable.zs), and gives the menu its list (hos_mod_info,hos_mod_edit,hos_mods_changed).nx2/exefs_patches.pyreadsmods/<name>/exefs/*.ipsand*.pchtxtwhen the game is built,nx2/image.pywrites them into main before it is relocated and lifted (a write into the padding after the code makes it code),nx2/make_game.pyputs them in the lift's key and writesexefs_patches.txt.host/cheats.c, Atmosphere cheats of the mods that are on, data cheats switched in the menu.gpu/vk/vk_menu.cpp(draw_mods,mods_tick), shown when there are mods;MkwVkHostgains five functions (gpu/vk/vk_api.h,host/rx_backend.c).draw_tune_bar).What the port changed in HumanBean's commits (7 files conflicted with main):
gpu/vk/vk_api.h,host/rx_backend.c: the five functions come after main'sscreen_size;MKWVK_API_VERSIONis 22 (the draft's was 18, main is at 21), andcan_mods()asks for 22.gpu/vk/vk_menu.cpp: the Mods block sits beforedraw_options;tab_titles(at file scope on main) gains "Mods"; the row handling ofmenu_tickis main's, inside the draft'selsefor the tabs that are not Mods.host/mods.c: main'shos_mods_file(an exact path) with the draft's rule for the size table in it.nx2/build.py:zs.cjoinsZSTD_HOST.tests/test_mods_size_table.py:zs.cbeside the others.README.md: the draft's Mods section with Longshot's names; main's note on tables for several game versions kept. The Mods tab's file islongshot_mods.txt.tests/test_mods_menu.py: version 22 and the new order of the host's functions.What the review changed (the last commit):
host/mods.chos_mods_file: no table for prompts.c when the merged one is served.host/mods.cmod_text,exefs_status: a mod that is off says "Its ExeFS patches are not applied while it is off", in the tab and in the log, instead of asking for a build.host/mods.c: "Longshot" in the Mods tab's texts and inlongshot_mods.txt's header.host/mods.cfit_sizes,game_case: a mod file whose path is in another case than the game's takes the game's entry. The game's case is looked up only when the mod's own case misses the table, so a mod whose paths match costs no extra reads at the start.gpu/vk/vk_menu.cppmenu_tick: a Restart or Quit waiting for its second press is cleared on the Mods tab.nx2/exefs_patches.pyclassify: bytes into a relocated word are left out as before, the file's note inexefs_patches.txtnow says how many, and the comment says what happens to DT_RELR words (tests/test_exefs_patches.py, the README).Flags and settings
longshot_mods.txt(game folder)mods/<name>/exefs/*.ips,*.pchtxtmods/<name>/cheats/<build id>.txtMeasurements
main:
39647e1a(2026-11-13)this PR: feat/mods-exefs-patches @
ca32fd3bHardware: RTX 3090, Windows 11
Scene: a child save in Kokiri Forest and Hyrule Field, 2048x1152, 60 fps
Method: one start and about twelve minutes of play by the mod's author, with the mod of the build lines below as the only mod; then a second start; the numbers are
longshot_log.txtlines. The merge's time is from the two starts of this PR's build (the first right after the build, the second with the files read before). main has no merge to time against it, and without amodsfolderhos_mods_initreturns before any of it, so there is no cost when no mods are installed (read in the code, not timed)How to reproduce: a mod with ExeFS patches in
mods,nx2/make_game.pyinto that game folder, start the game39647e1aca32fd3bexefsfolder is left outThe build and the run are of 3c23346a, the same changes before the rebases onto
39647e1a(main's commits since touch release and CI files only; the docs commit came after), Windows 11, llvm-mingw:(The second build, after the review's changes:
mods.candvk_menu.cppcompiled again, no warning in either. The first, of the port itself, took 15 min 18 s.)The run's log:
Testing
Tested:
python -I -B tests/run_all.pyon Windows 11: 462 passed, 70 skipped, 10 failed. The same 10 fail on main on this PC:test_builder_details_hitbox,test_builder_dlss_update,test_portable_archive,test_portable_builder_home,test_portable_builder_startup(no .NET SDK here),test_fpx_sticky,test_spawn_emit(they build the translator; this toolchain copy has no C++ standard headers),test_check_third_party,test_release,test_setup_runtime.Not tested:
Needs other testers:
Screenshots
this PR @ 3c23346a (the build of the run above), Windows 11, RTX 3090, from the mod's author:
The mod's Characters page in the game's own menu with King Zora picked, the game in a window (its title bar says Longshot). The page's fourth tab, the picked character's model and its view all come from the mod's ExeFS code patches, so this is those patches at work in a Longshot build. On main the patches are left out: nothing to show beside it.
The Insert menu (F1) of the same run: the Mods tab, with the mod listed and On.
Risk and rollback
The Mods tab adds a tab to the Insert menu's strip only when there are mods. With mods, the size table the game gets is the merged one, where before it was the last mod's own. A build with ExeFS patches lifts patched code: without such patches the lift and its key are as before. Reverting the commits is enough.
Reviewer notes
draw_tune_bar, becausetests/test_menu_tune.pyasks forif (tune_mode) draw_tune_bar(k);as it is. The other way would be to leave the Mods tab out of the strip while the bar is up. Say if you prefer that.apply(); I can do that here or as a follow-up.Issues
See #42
Checklist
For everyone
type(scope): description).main.python -I -B tests/run_all.pyrun and passing (any failure that also fails onmainis named above).third_party/untouched (a new folder with aPROVENANCE.mdinstead).If applicable
Assisted-by:/Co-Authored-By:trailers.MKWVK_API_VERSIONbumped (gpu/vk/vk_api.h'sMkwVkHostorMkwVkRendererchanged).mainand passing here.tests/that reads changed source updated in this PR, not weakened.🤖 Generated with Claude Code
Code review only: I read the diff and the code around it, and did not build or run anything.
The port is careful. The API bump to 22 with
can_mods()checking it, the posix shims thatzs.candmods.cnow need, the exefs patches going into the lift key through the environment, and the checks that read source being updated all look right. Some things need to change before this can merge:hos_mods_filestill gives prompts.c the last mod's own table, which goes against its own comment and the description (inline).exefs_patches.txt, so the Mods tab tells the player to rebuild to apply patches that the build will always skip (inline).longshot_mods.txt(inline).cf0dbb34). The lift commit here (e20dc232) is the same change and becomes empty after a rebase. The "Rebased on current main" box is ticked, but the PR still sits on1e5fcc20, and the Reviewer notes about merging 131 first are out of date.There are also smaller points inline: RELR words in exefs_patches.py, the case of mod paths in the size table keys, and the Restart/Quit confirmation on the Mods tab. The size table is now merged at every start with mods (decoding the game's table and reading the header of every mod
.zs), so please put that startup time in the Measurements along with the in-game run.@ -5105,1 +5247,3 @@changed = true;Row& r = rows[tab_row_to_global(cur_tab, selected)];const bool confirm_selected = &r == &rows[kQuitRow] || &r == &rows[kRestartRow];if (pending && (pending != &r || !mine)) pending = nullptr, changed = true;Nit: now that this sits in the
elsebranch for the tabs that aren't Mods, a pending Restart/Quit confirmation is no longer cleared when the player goes to the Mods tab. It stays armed for when they come back to that row. Clearingpendingwhenevercur_tab == TAB_MODSwould keep main's behaviour.@ -167,3 +286,3 @@return;}qsort(mods, (size_t)nmods, sizeof *mods, mod_cmp);fputs("# Hookshot's mods: their order and switches, as the Mods tab sets them (Insert menu). One line a mod, top of the\n"This header goes into the player's
longshot_mods.txt: it should say Longshot's mods, not Hookshot's.@ -180,0 +350,4 @@while (line && line != built && line[-1] != '\n') line = strstr(line + 1, want);const char *kind = line ? line + strlen(want) : "";if (!line) {m->exefs_waiting++;exefs_statusalso runs over mods that are off, butwrite_listonly writes the files of mods that are on. So every ExeFS file of an off mod lands here asexefs_waiting, andmod_texttells the player to "build the game again (Hookshot Builder) to apply" patches that the build will skip. Count waiting files only for enabled mods. For an off mod, either say nothing or say that its patches are not applied while it is off.@ -180,0 +454,4 @@const Entry *e = &table[i];if (!e->key || e->is_dir) continue;char key[MAX_PATH], game[MAX_PATH], host[MAX_PATH];restbl_key(e->rel, key, sizeof key);Suggestion: the key uses the mod file's own path case (
e->relcomes fromfd.cFileName). RESTBL keys are CRC32s of the game's case-sensitive paths, so a mod that shipsui/...where the game hasUI/...gets an entry under the wrong hash. On Windows,game_filestill finds the game file, sorestbl_get(base, key)misses and thiscontinues as if the file were streamed. Taking the case from the game's own path (or from the shipped table's key) would make the raise work for those mods.@ -202,2 +609,4 @@snprintf(key, sizeof key, "%s", rel);lower_slash(key);if (sizes_key[0] && !strcmp(key, sizes_key)) {if (!sizes_fallback[0]) return 0;This returns the last mod's own table even when
sizes_served(the merged table) is what gets served. The comment above and the PR description both say "none when the merged one is served". prompts.cbuild()then checks and merges into that mod table. If prompts can't merge it (a frame without a content size, or a table not in the game's format),build()returns 0 and the player gets the game's own icons in texts, even though the served merged table already has the prompts' entries. Suggested fix:if (sizes_served[0] || !sizes_fallback[0]) return 0;.@ -238,0 +669,4 @@k = 0, d = m->note;if (m->overridden) ADD("%d of its files are taken by a mod lower in the list. ", m->overridden);if (m->exefs_waiting)ADD("%d ExeFS patch file%s %s not in the game yet: build the game again (Hookshot Builder) to apply %s. ", m->exefs_waiting,The player sees this in Longshot's Mods tab: "Hookshot Builder" should be "Longshot Builder". The same goes for "Nothing Hookshot uses." a few lines up (668).
@ -0,0 +237,4 @@if word is not None and self.bound[word] == "JUMP_SLOT":return "unsupported", "it writes a function pointer the dynamic linker binds at main+%X" % word, {}if word is not None:continue # (an explicit-addend relocation's word: the dynamic linker writes over it, as on the console. A DT_RELRnso2.py turns DT_RELR words into
RELATIVErecords whoseaddendis read from the file before the patch, so they are inself.boundand their patch bytes are dropped here. On the console, the RELR addend is the (patched) word, so the patch takes effect there. The comment says the opposite of what happens. Either apply RELR words, which needs the addend recomputed afterapply(), or fix the comment and treat them like the explicit-addend ones, with a note in exefs_patches.txt saying so.You cooking 🔥
1faee42e07841d80ed5dThanks for the review. Rebased onto main @
0eb3a3c6(the lift commit dropped out, as #131 is in), and the last commit (841d80ed) takes each point:hos_mods_filenow returns 0 then (sizes_served[0] || !sizes_fallback[0]), as you suggested. I checked thatfit_sizesstarts the merged table fromprompts/rsizetable.zs, so the prompts' entries are in what is served.longshot_mods.txt.pendingis cleared there.game_casegives the path as the game's folders spell it, andfit_sizesasks for it only when the mod's own case misses the table, so a mod whose paths match the game's does no extra reads at the start. Windows only: where the file system tells the cases apart the path is left as it is.exefs_patches.txt(and the README says so). Applying them instead needs the addends read again afterapply(): say if you want that here.Built on Windows after these changes (
mods.candvk_menu.cppcompile without a warning; the build's lines are in the description). The full run_all is 462 passed with the same 10 failures as main on this PC. Not run in the game yet: that is next, with the merge's start-up time from the log.841d80ed5dca32fd3b1aRun in the game (Windows 11, RTX 3090, a child save, the build of the review's changes): the Mods tab lists the mod and behaves; the mod's code patches all work (its menu page, picking a character, the swap in the field, the cloth and the voice hooks); the size table's merge took 111 ms at the start (1 table merged, 90 entries; 15 raised); 60 fps. The log's lines are in the description. Also: a line on longshot_mods.txt in docs/development.md, rebased onto main @
39647e1a, and a test request for Linux (T-3AW7). Still to come: a picture of the Mods tab, and what the description lists under Not tested (switching and moving mods on the tab, a cheat file, a pchtxt mod).A picture from the run is in the description's Screenshots section.
Two pictures from the run are in the description now: the mod's page in the game's menu (its code patches at work), and the Insert menu's Mods tab with the mod listed and On.
A second start of the same build: the size table's merge took 56 ms (111 ms on the first start, right after the build). Both lines are in the description. What keeps this WIP: the Linux build and run (T-3AW7 is open for it), which I cannot do here.
A correction for whoever takes T-3AW7: for its second step, do not use the mod's public v0.1.2 zip. Its patch file enters one routine with
bland comes back with a plainb, which the console and the emulators take as a return and the lift cannot (the rest of the calling function runs twice): it crashes at the first character picked. That is a fault of the mod, fixed in its source (one byte, theblis ab) and in the build I ran here; a v0.1.3 of the mod with it is on its way, and I will link it here. The first step (the Mods tab with any RomFS mod) is unaffected.A mod package for testing this, for T-3AW7 and anyone who wants to see ExeFS patches at work:
https://github.com/snaildev99/OoT-Character-Select/releases/tag/v0.1.3 (
OoT-Character-Select-v0.1.3-lite.zip, 171 MB, SHA-256ff60cd9bd4f7c5ca1cdbfbb2242fd8fd2dfa08e221e856dd1c2c65d64dd2d5e1)It is the mod of the measurements above: one mod folder with RomFS files, a resource size table and one
.ipsfile of six code patches, for the game's 1.1.1 build.OoT Character Selectfrom the zip'smodsfolder into the game folder'smods.nx2/make_game.py). Expected in the build's output:exefs: code OoT Character Select/exefs/E0F3CCFAF50065AE4EA01EB5AEB13B1A45DA3348.ips (4935 bytes)andexefs: main's code reaches 0x3c47000 now (a code cave).longshot_log.txt:ExeFS ... built into the gameand themods: resource size table: 1 tables of mods mergedline; the Insert menu shows the Mods tab with the mod On.Take v0.1.3, not v0.1.2: the older package has a routine that is entered with
bland goes back with a branch, which the lift cannot run (the function's tail runs twice), and it crashes at the first pick. That was the mod's fault and is fixed in v0.1.3; it is the one byte between the two packages.Still open here: the checklist's Linux box, which is why this stays WIP. Nothing changed on the branch since
ca32fd3b.View command line instructions
Checkout
From your project repository, check out a new branch and test the changes.