WIP: feat(mods): two mods' ExeFS patches that write one place differently are named, and the lower one wins every byte #961
No reviewers
Labels
No labels
area/accessibility
area/audio
area/input
area/install
area/menu
area/renderer
area/saves
area/tooling
area/upscaling
beanstalk: approved
beanstalk: failed
beanstalk: needs approval
beanstalk: running
beanstalk: tested
ci: approved
good first issue
gpu/amd
gpu/igpu
gpu/intel
gpu/nvidia
help wanted
kind
bug
kind
chore
kind
docs
kind
feature
kind
performance
kind
question
needs measurements
platform/linux
platform/macos
platform/steam-deck
platform/windows
preview
priority
critical
priority
high
priority
low
regression
risk
high
risk
low
risk
medium
status
blocked
status
confirmed
status
duplicate
status
invalid
status
needs-info
status
wontfix
ci: approved
No milestone
No project
No assignees
1 participant
Notifications
Due date
No due date set.
Dependencies
No dependencies set
Reference
MagicBeanDev/Longshot!961
Loading…
Reference in a new issue
No description provided.
Delete branch "snaildev99/Longshot:mods/exefs-clashes"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Type: feat · Risk: low · Tested on: Windows 11, RTX 3090 (in the game, and the game-free checks) · Needs: Linux, any GPU: one game-free test
Why
When two code mods change the same place in the game's code, #134 uses the bytes of the one lower in the list and
says nothing: a player has no way to know that one of their mods is partly overwritten, which is the first thing to
know when a mod "does nothing" or the game crashes. Real case: two HUD mods (Rupee Counter, HUD of Time) put a small
stub into spare space that a character mod's patch also uses.
There was also a fault: where the lower mod writes the game's own byte back over the higher mod's change, that byte
was dropped (the build keeps only the bytes that differ from the game), so the place held a mix of both mods' code.
Emulators write every byte of each patch in order, so there the lower mod's code is used whole.
What changed
The build names every place two patch files write differently, in its log and in both files' notes, and the Mods tab
shows those notes. The lower file's bytes are used whole.
nx2/exefs_patches.py:overlaps()(called bycollect()): over the files that apply, in the list's order, eachbyte one file writes differently from a file above it; the game's own bytes the lower file writes there go back into
its writes (a file that was "empty" becomes code or data when it puts bytes back). Notes like "50 of its bytes are
taken by Rupee Counter lower in the list (main+3C46740-3C46773)" and "... written differently by Characters higher in
the list; this one's are used (...)", and one log line per pair.
parse_file()isclassify_file()'s parse, shared.host/mods.c: the Mods tab showed a patch file's note only when the file cannot be applied; it now also shows thefirst note of a file built in (
exefs_said,kind_note(), one line inmod_text).kind_note()also drops atrailing
\r.apply()closes the JSON it reads.Measurements
Verdict: an offline check with real mods; nothing measured in the game (collect() took 1.3 s with three mods).
With the game's code on build e0f3ccfa, the list Character Select v0.2.1, Rupee Counter v0.9, HUD of Time v1.3 (their
.ipsonly; collect() only, no build):The Rupee Counter's applied bytes grow from 50 to 58 and HUD of Time's from 60 to 67: the game's own bytes they write
over Character Select's, which #134 dropped.
Flags and settings
None
Testing
Tested:
tests/test_exefs_overlaps.py(10 cases, among them the game's own byte written back, which fails on #134)and
tests/run_all.pyon Windows: 539 passed, 25 failed; the same 25 fail on #134's head here.tests/test_mods_sizes.py(a built file's note on the Mods tab) is Linux only; it was run on Windows through a copythat builds it over llvm-mingw's Win32 instead of the posix layer: passes.
In the game, Windows 11, RTX 3090 (driver 610.88), a build of #134's head with this branch (and the other follow-ups) merged in, four mods with patch files, two of them made for the test and writing main+3C46200 (unused padding) differently: the build's log says
... write 4 bytes differently (main+3C46200-3C46203): the lower one's are used, clash-test-b, and the Mods tab's note of the higher one says its 4 bytes are taken by the lower one, the lower one's that this one's are used, each with the place. The other two mods' rows read as before and the game plays.Not tested:
tests/test_mods_sizes.pyon Linux; two real mods that clash, played (the pair at hand overwrite each other's code, so one of them cannot work: the note is what this adds).Must be tested:
python -I -B tests/test_mods_sizes.py, expecting its three PASS lines (Test Farm: @Beanstalk linux, tests/test_mods_sizes.py).Risks
What could break: a note longer than the Mods tab's field is cut (160 characters). A build with mods that clash
now uses the lower mod's bytes whole where it used a mix; a mod that "worked" only by that mix would change.
Turning it off: revert the commits.
Compatibility: nothing for players without clashing mods; with clashing mods the lift's key changes once (the
applied bytes change).
Checklist
Everyone
type(scope): description) of the status line's type.main. (Stacked on #134.)python -I -B tests/run_all.pypasses (any failure that also fails onmainis named).third_party/untouched (a new folder with aPROVENANCE.mdinstead).When it applies
Assisted-by:/Co-Authored-By:trailers.mainand passing here.test copy above; no full build.)
Where two patch files write the same byte, the one lower in the list won, and nobody was told. collect() now names every such place (overlaps): in the build's log, and in both files' notes in exefs_patches.txt ("120 of its bytes are taken by HUD lower in the list (main+...)" / "... written differently by Characters higher in the list; this one's are used"), where the Mods tab can show it. The lower file now also wins where it writes the game's own byte back over a higher file's change. Main.classify leaves out the bytes a file writes that equal the game's, so such a byte was not applied and the higher file's byte stayed: a mix of two mods' code that neither wrote. Those bytes are put back into the lower file's writes, and a file that was "empty" (every byte the game's) becomes code or data when it puts bytes back. Modelled on a real case: two HUD mods put a stub inside the range a character mod's patch writes. parse_file() is classify_file()'s parse, shared. apply() closes the JSON it reads. Co-Authored-By: Claude Opus 5.5 <[email protected]>Tested in game: Windows 11, RTX 3090 (driver 610.88), a build of #134's head
5192b46fwith this branch merged in, four mods with patch files, two of them writing main+3C46200 differently, the F1 menu's Mods tab: the build's log names the 4 bytes and the mod whose bytes are used, and each of the two mods' notes on the tab names the other and the place; the game plays with the other mods as before. This branch is stacked on #134 (it depends on it), so the diff shows #134's commits too until that merges.1615b98ce26e6c5231ceMoved onto #134's new head
fafc7ab2(it was rebased onto main @343a1dfa): head6e6c5231. No conflict, no change in this branch's own files; its tests pass. Not built or played at this head.6e6c5231cedc5998f247Moved onto #134's new head
fb2a419c(rebased onto main @e4d035e9; the Mods tab's functions are API version 25 there, after main's audio outputs): headdc5998f2. No conflict, no change in this branch's own files; its tests pass at this head. Not built or played at it.dc5998f247387174a042Moved onto #134's new head
3fbc327a(the list of mods ismods.jsonthere now: #134 (comment)). Head387174a0.Two small conflicts in
nx2/exefs_patches.py(the lines abovecollect, and the module docstring where both changed neighbouring lines); both sides kept. One commit on top:tests/test_exefs_overlaps.pywritesmods.json, and #134'stests/test_mods_list.pystubsparse_filefor its run (it givescollectno game image).tests/run_all.pyon Windows: nothing fails that #134's head passes. Not built or played at this head; the Linux halves of the changed fixtures were read, not run.387174a04262dd499199Moved onto #134's new head
2b52cd1b(#134 was rebased onto main @8a4aa282: the Mods tab's functions are API version 26, and the patches' hash is named in the lift's key after main's #22: #134 (comment)). Head62dd4991.No conflict; the commits are unchanged. It sets nothing of its own for the lift.
tests/run_all.pyon Windows: nothing fails that #134's head passes. Not built or played at this head.62dd4991994d5f28fc5cMoved onto #134's new head
c5cead8d(#134 was rebased onto main @ac778751and brings main's new players' mods guide in line with it: #134 (comment)). Head4d5f28fc.No conflict; the commits are unchanged.
Nothing fails in
tests/run_all.pyon Windows that #134's head passes. Not built or played at this head.View command line instructions
Checkout
From your project repository, check out a new branch and test the changes.