WIP: feat(host): feature modules: the engine's features a folder each, with a manifest of what they hook and an audit #1822
No reviewers
Labels
No labels
area/accessibility
area/audio
area/input
area/install
area/menu
area/renderer
area/saves
area/tooling
area/upscaling
beanstalk: approved
beanstalk: failed
beanstalk: needs approval
beanstalk: running
beanstalk: tested
ci: approved
good first issue
gpu/amd
gpu/igpu
gpu/intel
gpu/nvidia
help wanted
kind
bug
kind
chore
kind
docs
kind
feature
kind
performance
kind
question
needs measurements
platform/linux
platform/macos
platform/steam-deck
platform/windows
preview
priority
critical
priority
high
priority
low
regression
risk
high
risk
low
risk
medium
status
blocked
status
confirmed
status
duplicate
status
invalid
status
needs-info
status
wontfix
ci: approved
No milestone
No project
No assignees
1 participant
Notifications
Due date
No due date set.
Dependencies
No dependencies set
Reference
MagicBeanDev/Longshot!1822
Loading…
Reference in a new issue
No description provided.
Delete branch "dyst/Longshot:feat/feature-modules"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Type: feat · Risk: low · Tested on: Windows 11, RTX 3080, Ryzen 9 3900X · Needs: a run on Linux
TL;DR
The engine's game-code features become modules: one folder each under
host/features/, holding the feature's code and a short manifest of the hook sites it attaches to. A feature's pull request then changes its own folder, so two features never conflict in git, and what can still collide, two features hooking the same place, is found mechanically: an audit, run by every build and by a game-free test, refuses features that share a site with no order or a function with no acknowledgement, and prints a map of what hooks what. Text speed and Text size are the first two modules; nothing a player sees changes.Breakdown
host/and one 1,800-line patch file every feature edits; the site table of #1821 was one file every new site would append to; nothing said when two features hook the same function.tests/run_all.pyagainst main; a cold review; a run in the game.Why
The maintainers agreed on the approach: the engine's features are built as the native, compiled tier of the Modding API's mods (the Spec's section 8.3 declares a compiled mod's sites the same way: a signature, and a place for each game build), each in its own folder, so the team can move the roughly forty text patches of
nx2/oot_patches.pyonto sites one feature at a time, in parallel, without conflicts, and with every overlap between features found by a check rather than by luck. This is the infrastructure for the plan's milestone M2, piece 3.Stacked on #1821 (the hook compiler), and through it on #1587 and #1560.
What changed
Features are folders with manifests, the build compiles them, sites are a file each, and an audit keeps the features from colliding.
host/features/<name>/(new): a feature's C andfeature.toml: a summary, and a[[hook]]per site (the site, the handler,afterfor the order on a shared site,sharesfor the features acknowledged in that site's function). Underhost/so the Builder's source package, the PR checker's game paths and the checks that scanhost/take them in.nx2/features.py(new): reads the manifests and audits them without the game, by the functions the site files declare. It stops on a manifest that is not valid, an unknown site, a handler its folder does not define (comments aside) or two features define, two features on one site with no order between them (or orders that go round), two features with sites in one function that neither acknowledges there (an acknowledgement holds in its own function and game build only), what the lift reads fromhost/*.citself in a feature's C, two files with one object name. It warns of a hooked function the text patches still name, and prints the hook map.python -I nx2/features.py auditruns it.nx2/build.py: the features' C compiled with the host's;features.c, the handlers' table, written from the manifests in their order (only when it changes, as a new file: a pack's work folder holds hard links); the audit stops the build.host/hook_sites.c,host/hook_sites.h: the handlers attached from that table before main (HOS_SITEis gone: a manifest is the only way in); one line at start when features hook sites the game build lacks.nx2/sites/<name>.toml: a file per site, with a table per game build (the address, and the function the lift checks it is in); a site's C names its own byte and number (hos_site_<name>), so a new site changes no other site's function.nx2/hooks.pyreads them; two sites at one place are refused.host/features/text-speed, its option read moved out ofhost/nn_vi.c) and Text size (host/features/text-size,host/oot_text_size.cmoved there), both ontextStep, Text size after Text speed as they ran before;host/oot_text_step.cis gone.docs/development.md: a Feature modules section (a feature's folder, a hook site, the audit, adding a feature).tests/test_features.py(new);tests/test_hook_sites.py,test_text_speed.py,test_text_size.pyfor the new places;test_env_on.py(HOS_TEXT_SPEED is a number),test_frame_rate_cap_step.py, andtest_host_guest_writes.pyandtest_msvc_oldnames.py, which now read the features' C too.Measurements
Verdict: nothing to measure in frames: the translated game is byte for byte the same as #1821's but for the site's own function (
f_80b1e140: the site's C naming its byte) and the sites' table; Text speed now reads its option in its handler, as Text size always did.Baseline: the hook compiler's PR (#1821), which this PR is stacked on: main differs from it by that PR, #1587 and #1560, each compared on its own.
Hardware: Windows 11 Pro N (build 22631), RTX 3080, Ryzen 9 3900X, 64 GB
Scene: none: the translation, game-free (the in-game run is in Testing)
Method:
scripts/lift_parity.py lift(#1798) of each commit, rc.8's ExeFS, the Builder's Python and llvm-mingw, thencompareHow to reproduce:
python -I scripts/lift_parity.py lift --checkout <a checkout>for #1821's head and for this branch, thenpython -I scripts/lift_parity.py compare <one> <the other>a64_sites.c, the chunk cost estimatesf_80b1e140(the site's C naming its own byte)After the rebase onto main
0de25c3e(which brought #134's ExeFS patches, #1740's text patch and #1774's menu modules), lifted again:725cc27a(byte for byte main 0de25c3e's)957a28b8a64_sites.cnew, the chunk cost estimatesf_80b1e140(the site, its locals kept, its own byte)Flags and settings
None (
HOS_TEXT_SPEEDandHOS_TEXT_SIZEread as before, now in their features' folders).Testing
Tested:
tests/test_features.py(new): manifests (16 kinds of mistake, each named); the audit over made-up features and sites (orders and chains of them, cycles, unknown names, acknowledgements per function and per game build, the lift's markers, the text patches' warning, the hook map); object names; the handlers' table compiled withhost/hook_sites.cand run; the tree's own features audited clean. Withtests/test_hook_sites.py(the site files, two sites at one place, a site checked against its declared function's own code, a64_sites.c in the lift's order) andtests/test_text_speed.py(an emptyHOS_TEXT_SPEEDtoo): twenty deliberate breaks of the code, each caught.A cold review of the branch by a second agent (code read, nothing run); its findings fixed in 67b0cbe2.
The whole translation compared byte for byte (#1798), as in Measurements.
python -I -B tests/run_all.py(the Builder's llvm-mingw first onPATH,LONGSHOT_MINGWset), on main0de25c3e: 707 passed / 45 failed at957a28b8. The one failing here only,tests/test_a64_helper_build_flags.py(#756's: its stand-in lift folder lacked #1821'sa64_sites.c), is fixed in #1821 and passes; the other 44 fail on main0de25c3etoo, rerun there.A pack of this branch over main 47430e81's base (#1729's header fix applied for the build only): the features' C and the handlers' table compile with the host's, and the game's link map holds them.
In the game (Windows 11, RTX 3080, Ryzen 9 3900X), by the person I work for, at 95fbe486: the repository's
kokiri-startsave, Kokiri Forest's signs and dialogue. Text speed: Normal types the letters out, the fastest setting shows each page at once. Text size: larger, as before, including #1779's first showing at the default size, which main has too.Rebased onto main
0de25c3e(with #1821): the conflict wastests/test_text_speed.py's reader of the menu rows (#1774'smenu_text()); the checks of the files it touches pass; the comparisons andrun_allabove are on it.Not tested: Linux. In the game, the review's fixes after 95fbe486 (67b0cbe2): their translation differs in
f_80b1e140alone, and their host code is covered by the compiled runs oftests/test_features.pyandtests/test_hook_sites.py.Must be tested:
build: 2 features, 2 hooks, and Text speed and Text size in a dialogue as on Windows (by hand, or the Test Farm on a Linux machine).Risks
What could break: Text speed and Text size, whose code moved (the in-game run covers them); a feature folder a build stops on, by design, with the reason.
Turning it off: revert; a feature is a folder, and taking one out leaves its site dormant.
Compatibility: none for players; a new feature goes in
host/features/, asdocs/development.mdsays.Checklist
Everyone
type(scope): description) of the status line's type.can't be run by hand is in Testing's Not testable by hand, with why.
in Risks.
main.python -I -B tests/run_all.pypasses (any failure that also fails onmainis named).third_party/untouched (a new folder with aPROVENANCE.mdinstead).When it applies
Assisted-by:/Co-Authored-By:trailers.tests/that reads changed source updated in this PR, not weakened.🤖 Generated with Claude Code
Tested in game: Windows 11 Pro N 22631, RTX 3080 (NVIDIA 621.14), Ryzen 9 3900X, by the person I work for: a pack of this branch at 95fbe486 over main 47430e81's base, #1729's header fix applied for the build only. The repository's kokiri-start save, Kokiri Forest's signs and dialogue: Text speed (now host/features/text-speed) at Normal types the letters out, at its fastest shows each page at once; Text size (host/features/text-size) larger as before, including #1779's first showing at the default size, which main has too. Not run in the game since: the review's fixes (67b0cbe2: per-site bytes, A64_NOESC, every handler of an after site) and the rebase onto
0de25c3e.957a28b8758aefb31cd9View command line instructions
Manual merge helper
Use this merge commit message when completing the merge manually.
Checkout
From your project repository, check out a new branch and test the changes.