forked from MagicBeanDev/Longshot
fix(mods): a module's own address-taken functions are lift entries #1
Loading…
Reference in a new issue
No description provided.
Delete branch "rainy/Longshot-module-gaps:fix/mods-taken-functions"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Type: fix · Risk: low · Tested on: Linux (Omarchy, kernel 7.2.15; Intel Core Ultra 7 270K Plus; RTX 4090, driver 615.78.08) · Needs: a run on Windows, any GPU
TL;DR
A mod's code module without unwind tables can hand the address of one of its own functions to other code. The build never made those functions, so the first call through such an address stopped the game. This PR finds those functions by the instructions that take their address and builds them. A new game-free test fails without the fix and passes here. The full check suite on this branch has the same 38 failures as
mods/module-gaps, and one more pass. It also ran in the game on Linux, in a full patched build on an earlier base of the same module-gaps commits.Breakdown
tests/test_exefs_module_taken.pypasses here and fails with the fix reverted.tests/test_exefs_modules.pystill passes.python -I -B tests/run_all.py --jobs 8has the same 38 failures asmods/module-gapsand one more pass. It also ran in the game on Linux, in a full patched build on an earlier base of the same module-gaps commits.Why
I found this while running a code mod, sloperinaGZ 0.0.8 (https://github.com/PracticeROM/sloperina/releases/tag/v0.0.8), on this branch. Its
subsdk9has no .eh_frame. It hands its menu callbacks to its menu code with ADRP+ADD, and 132 of those targets had no function in the build. Opening its settings menu called one at +0x28400, and the game stopped witha64_icall: an indirect call to an address with no function. Its frame advance icon hit the same crash at +0x43e80, where the compiler hoisted the ADRP out of a loop and the ADD came 62 instructions later.The game's own address-taken functions all have .eh_frame entries, so main never needed this. A mod built without unwind tables is ordinary, so any such mod with callbacks hits it. ADRP and ADD are the A64 way to form a code address, per the Arm Architecture Reference Manual's ADRP and ADD (immediate) pages.
This PR targets snaildev99's
mods/module-gapsat2756bd26, not main, becausenx2/exefs_modules.pyexists only in that stack (#984). I searched the tracker for eh_frame, ADRP and taken: no issue or PR covers it.How to reproduce
Seen on: Linux, RTX 4090, driver 615.78.08. The crash does not depend on the GPU.
Settings and switches: defaults, with sloperinaGZ 0.0.8 in the mods folder
Steps: build the game with the mod on
mods/module-gaps, start it, and open the mod's settings menu.Expected: the menu opens.
Actual: the game stops with
a64_icall: an indirect call to an address with no function.What changed
entries()innx2/exefs_modules.pyadds a mod module's own address-taken functions when the module has no .eh_frame.taken_functions(module)innx2/exefs_modules.pyscans the module's code for ADR, and for ADRP followed by ADD #imm on the same register. It looks for the ADD up to the function's RET, at most 1024 instructions, and stops where the register is set again. A target counts only where a function starts: 16-byte aligned, after padding, a RET, a B or a BR.entries()adds these for each mod module without .eh_frame and logs how many it found.tests/test_exefs_module_taken.pybuilds a made-up module and checks which targets are taken and which are not, including an ADD 61 instructions after its ADRP and an ADD after a RET.On sloperinaGZ 0.0.8 the scan finds 280 functions. Every RELATIVE relocation into its code was an entry already.
Measurements
Nothing to measure. The scan runs once per mod module at build time, and the game's modules skip it.
Flags and settings
None
Testing
Tested: game-free tests on Linux with gcc, on this branch at
58c66d01:python -I -B tests/test_exefs_module_taken.pypasses.nx2/exefs_modules.pyreverted tomods/module-gaps, it fails withAttributeError: module 'exefs_modules' has no attribute 'taken_functions'.the taken functions: ['0x10', '0x100', '0x380', '0x80', '0xc0']. It misses the hoisted ADRP's target, 0x300, and takes 0x380, whose ADD comes after a RET.python -I -B tests/test_exefs_modules.pypasses here and onmods/module-gaps.python -I -B tests/run_all.py --jobs 8on Linux (Omarchy, kernel 7.2.15; Intel Core Ultra 7 270K Plus; RTX 4090, driver 615.78.08).mods/module-gaps@2756bd26: 686 passed, 38 failed, 97 skipped. this branch @58c66d01: 687 passed, 38 failed, 97 skipped, 0 new failures compared by test name. The 38 failures also fail onmods/module-gapsat2756bd26. They are:Tests that also fail on mods/module-gaps @
2756bd26in this environmenttests/test_bug_report_defaults.pytests/test_capture_read_once.pytests/test_cpu_report.pytests/test_crash_record.pytests/test_crt_options.pytests/test_damage_debug_menu.pytests/test_dev_packs.pytests/test_display_startup_failure.pytests/test_dlss_feature_size.pytests/test_failed_compile_keeps_objects.pytests/test_guest_imported.pytests/test_high_fps_fixes.pytests/test_host_guest_writes.pytests/test_hud_layout_row.pytests/test_invalidate_quick.pytests/test_invalidate_vcore.pytests/test_libc_hooks.pytests/test_link_path_comma.pytests/test_linux_preflight.pytests/test_linux_pressure.pytests/test_linux_update.pytests/test_log_open_failure.pytests/test_missing_nup.pytests/test_mods_size_table.pytests/test_pending_guest_import.pytests/test_pipeline_play_cache.pytests/test_prepare_cache_save.pytests/test_present_thread.pytests/test_profiling_mode.pytests/test_psychov35_fsr3_fg.pytests/test_readback_timeout_source.pytests/test_run_all_changed.pytests/test_slow_frames.pytests/test_song_note_cue.pytests/test_switch_table.pytests/test_upscaler_rows.pytests/test_walk_key.pytests/test_vbytes.py(timed out at 600 s)In the game, on Linux with an RTX 4090, this fix ran as part of a full patched build: all the fixes together, on an earlier base of the same module-gaps commits (
ed1a8513). The settings menu and frame advance stops were not seen again, and everything tried in the mod worked. It was not run alone in the game on this exact branch.Not tested: Windows (no build or run there) and other GPUs.
Must be tested:
Testing with sloperinaGZ
This branch alone isn't enough to use sloperinaGZ in the game. The in-game run above also had these fixes applied:
A branch with all of them applied together: https://git.magicbean.dev/rainy/Longshot/src/branch/longslop-rc8
Risks
What could break: a data word that looks like an ADRP+ADD pair could add an extra entry inside a mod's module. That costs build time and nothing else. The game's modules are not scanned.
Turning it off: revert the commit
Compatibility: a build with a mod module without .eh_frame lifts again, because the lift's entries change. Nothing else changes.
Checklist
Everyone
type(scope): description) of the status line's type.can't be run by hand is in Testing's Not testable by hand, with why.
in Risks.
main. This PR targets snaildev99'smods/module-gaps, not main.python -I -B tests/run_all.pypasses (any failure that also fails onmainis named). The failures named below also fail onmods/module-gapsat2756bd26, which is this PR's base.third_party/untouched (a new folder with aPROVENANCE.mdinstead).When it applies
Assisted-by:/Co-Authored-By:trailers.mainand passing here.tests/that reads changed source updated in this PR, not weakened.Thank you for this, and for running a real mod on the branch: that is exactly the kind of case it had not met. I'd like to take it, with one change to the scan first.
Windows run (Windows 11, the builder's clang):
tests/test_exefs_module_taken.pypasses at58c66d01and fails with the fix reverted;tests/test_exefs_modules.pystill passes. It merges with your other two without conflict. No run in the game on Windows yet (I have not built with sloperinaGZ).What I'd change before merging (medium): the forward search for the ADD (
nx2/exefs_modules.py:189) stops only when the register is set again by an ADR or ADRP. A probe against this code took three targets it should not:ldr x8, [x0]ormov x8, x0had rewritten it;br(adr x1, .L; add x1, x1, w0, sxtb #2; br x1; .L:) and so passes "after a BR" whenever it is 16-aligned;b) and the search runs on to the next RET.By the lifter's own model a wrong entry is still correct code (control reaching it tail-calls it), so this is extra code and a call where there was a fall-through, not a crash. But a jump-table base becoming an entry can land in a hot loop. Suggested: stop at any instruction that writes the register (as
code_constantsdoes for MOVZ), stop at an unconditionalbas well, and accept "after BR" only forbr x16/br x17. The three cases above as "not taken" in the test would pin it.Small: the description says the search stops where the register is set again and at the function's RET; both are a little wider in the code than that. And "costs build time and nothing else" should mention the possible tail call at run time.
Checked and fine: it applies only to a mod's module without
.eh_frame; the lift's key covers both this file and the module's bytes, so no stale lift is reused.Two practical notes for all three:
mods/module-gapsstays where it is (2756bd26) until these are in, so your branches keep their base. The branches under it are moving onto v3.0.0's main today; I will rebase this one after merging.Assisted-by:trailer).If you'd rather not do the follow-up yourself, say so and I will put it on top of your commit.
Thanks for the careful review. The three cases are real gaps. Stopping at any write to the register is the right rule, since the scan only cares where the value came from, and limiting "after BR" to
br x16/br x17keeps it to PLT-style stubs. The jump-table case is the one I'd most want pinned in the test, since it's the one that could hit a hot loop. You're also right that the description undersold the run-time cost. Please go ahead and add your follow-ups as your own commits on top of ours, your way. For credit, my commits should be authored asrainy, with anAssisted-by: Grok Bottrailer.View command line instructions
Checkout
From your project repository, check out a new branch and test the changes.Merge
Merge the changes and update on Forgejo.Warning: The "Autodetect manual merge" setting is not enabled for this repository, you will have to mark this pull request as manually merged afterwards.