WIP: feat(host): feature modules: the engine's features a folder each, with a manifest of what they hook and an audit #1822

Draft
dyst wants to merge 11 commits from dyst/Longshot:feat/feature-modules into main
Member

Type: feat · Risk: low · Tested on: Windows 11, RTX 3080, Ryzen 9 3900X · Needs: a run on Linux

TL;DR

The engine's game-code features become modules: one folder each under host/features/, holding the feature's code and a short manifest of the hook sites it attaches to. A feature's pull request then changes its own folder, so two features never conflict in git, and what can still collide, two features hooking the same place, is found mechanically: an audit, run by every build and by a game-free test, refuses features that share a site with no order or a function with no acknowledgement, and prints a map of what hooks what. Text speed and Text size are the first two modules; nothing a player sees changes.

Breakdown

  • Who it helps: everyone working on the engine's features in parallel, and reviewers: the hook map says what a feature touches, and the manifest is the whole list of what the engine hooks.
  • What was wrong: the features' code was spread over host/ and one 1,800-line patch file every feature edits; the site table of #1821 was one file every new site would append to; nothing said when two features hook the same function.
  • What changes: feature folders with manifests, the build compiling them and writing the handlers' table, hook sites a file each, the audit and its hook map; Text speed and Text size moved into folders.
  • What does not change: what the player sees; the translated game (byte for byte, but the site's own function and the sites' table).
  • How it was checked: a new game-free test with teeth; the whole translation compared byte for byte; tests/run_all.py against main; a cold review; a run in the game.
  • What is still open: a run on Linux; the PR description's Hooks section and the warning across open PRs, which come next.
  • Risk: low: Text speed and Text size are the only code that moved, and they ran in the game.

Why

The maintainers agreed on the approach: the engine's features are built as the native, compiled tier of the Modding API's mods (the Spec's section 8.3 declares a compiled mod's sites the same way: a signature, and a place for each game build), each in its own folder, so the team can move the roughly forty text patches of nx2/oot_patches.py onto sites one feature at a time, in parallel, without conflicts, and with every overlap between features found by a check rather than by luck. This is the infrastructure for the plan's milestone M2, piece 3.

Stacked on #1821 (the hook compiler), and through it on #1587 and #1560.

What changed

Features are folders with manifests, the build compiles them, sites are a file each, and an audit keeps the features from colliding.

  • host/features/<name>/ (new): a feature's C and feature.toml: a summary, and a [[hook]] per site (the site, the handler, after for the order on a shared site, shares for the features acknowledged in that site's function). Under host/ so the Builder's source package, the PR checker's game paths and the checks that scan host/ take them in.
  • nx2/features.py (new): reads the manifests and audits them without the game, by the functions the site files declare. It stops on a manifest that is not valid, an unknown site, a handler its folder does not define (comments aside) or two features define, two features on one site with no order between them (or orders that go round), two features with sites in one function that neither acknowledges there (an acknowledgement holds in its own function and game build only), what the lift reads from host/*.c itself in a feature's C, two files with one object name. It warns of a hooked function the text patches still name, and prints the hook map. python -I nx2/features.py audit runs it.
  • nx2/build.py: the features' C compiled with the host's; features.c, the handlers' table, written from the manifests in their order (only when it changes, as a new file: a pack's work folder holds hard links); the audit stops the build.
  • host/hook_sites.c, host/hook_sites.h: the handlers attached from that table before main (HOS_SITE is gone: a manifest is the only way in); one line at start when features hook sites the game build lacks.
  • nx2/sites/<name>.toml: a file per site, with a table per game build (the address, and the function the lift checks it is in); a site's C names its own byte and number (hos_site_<name>), so a new site changes no other site's function. nx2/hooks.py reads them; two sites at one place are refused.
  • Text speed (host/features/text-speed, its option read moved out of host/nn_vi.c) and Text size (host/features/text-size, host/oot_text_size.c moved there), both on textStep, Text size after Text speed as they ran before; host/oot_text_step.c is gone.
  • docs/development.md: a Feature modules section (a feature's folder, a hook site, the audit, adding a feature).
  • Tests: tests/test_features.py (new); tests/test_hook_sites.py, test_text_speed.py, test_text_size.py for the new places; test_env_on.py (HOS_TEXT_SPEED is a number), test_frame_rate_cap_step.py, and test_host_guest_writes.py and test_msvc_oldnames.py, which now read the features' C too.

Measurements

Verdict: nothing to measure in frames: the translated game is byte for byte the same as #1821's but for the site's own function (f_80b1e140: the site's C naming its byte) and the sites' table; Text speed now reads its option in its handler, as Text size always did.

Baseline: the hook compiler's PR (#1821), which this PR is stacked on: main differs from it by that PR, #1587 and #1560, each compared on its own.
Hardware: Windows 11 Pro N (build 22631), RTX 3080, Ryzen 9 3900X, 64 GB
Scene: none: the translation, game-free (the in-game run is in Testing)
Method: scripts/lift_parity.py lift (#1798) of each commit, rc.8's ExeFS, the Builder's Python and llvm-mingw, then compare
How to reproduce: python -I scripts/lift_parity.py lift --checkout <a checkout> for #1821's head and for this branch, then python -I scripts/lift_parity.py compare <one> <the other>

Before: #1821 @ ada6d3fc After: this PR @ 95fbe486
files the lift wrote 92 (1.73 GB) 92, the same
Before: this PR @ 95fbe486 After: this PR @ 67b0cbe2 (the review's fixes)
files the lift wrote 92 (1.73 GB) 89 the same, a64_sites.c, the chunk cost estimates
functions that differ 1 of 2,994 in its file: f_80b1e140 (the site's C naming its own byte)

After the rebase onto main 0de25c3e (which brought #134's ExeFS patches, #1740's text patch and #1774's menu modules), lifted again:

Before: #1587 @ 725cc27a (byte for byte main 0de25c3e's) After: this PR @ 957a28b8
files the lift wrote 91 (1.73 GB) 89 the same, a64_sites.c new, the chunk cost estimates
functions that differ 1 of 2,994 in its file: f_80b1e140 (the site, its locals kept, its own byte)

Flags and settings

None (HOS_TEXT_SPEED and HOS_TEXT_SIZE read as before, now in their features' folders).

Testing

Tested:

  • tests/test_features.py (new): manifests (16 kinds of mistake, each named); the audit over made-up features and sites (orders and chains of them, cycles, unknown names, acknowledgements per function and per game build, the lift's markers, the text patches' warning, the hook map); object names; the handlers' table compiled with host/hook_sites.c and run; the tree's own features audited clean. With tests/test_hook_sites.py (the site files, two sites at one place, a site checked against its declared function's own code, a64_sites.c in the lift's order) and tests/test_text_speed.py (an empty HOS_TEXT_SPEED too): twenty deliberate breaks of the code, each caught.

  • A cold review of the branch by a second agent (code read, nothing run); its findings fixed in 67b0cbe2.

  • The whole translation compared byte for byte (#1798), as in Measurements.

  • python -I -B tests/run_all.py (the Builder's llvm-mingw first on PATH, LONGSHOT_MINGW set), on main 0de25c3e: 707 passed / 45 failed at 957a28b8. The one failing here only, tests/test_a64_helper_build_flags.py (#756's: its stand-in lift folder lacked #1821's a64_sites.c), is fixed in #1821 and passes; the other 44 fail on main 0de25c3e too, rerun there.

  • A pack of this branch over main 47430e81's base (#1729's header fix applied for the build only): the features' C and the handlers' table compile with the host's, and the game's link map holds them.

  • In the game (Windows 11, RTX 3080, Ryzen 9 3900X), by the person I work for, at 95fbe486: the repository's kokiri-start save, Kokiri Forest's signs and dialogue. Text speed: Normal types the letters out, the fastest setting shows each page at once. Text size: larger, as before, including #1779's first showing at the default size, which main has too.

  • Rebased onto main 0de25c3e (with #1821): the conflict was tests/test_text_speed.py's reader of the menu rows (#1774's menu_text()); the checks of the files it touches pass; the comparisons and run_all above are on it.

Not tested: Linux. In the game, the review's fixes after 95fbe486 (67b0cbe2): their translation differs in f_80b1e140 alone, and their host code is covered by the compiled runs of tests/test_features.py and tests/test_hook_sites.py.

Must be tested:

  • Linux, any PC with the game: a build of this branch, expecting build: 2 features, 2 hooks, and Text speed and Text size in a dialogue as on Windows (by hand, or the Test Farm on a Linux machine).

Risks

What could break: Text speed and Text size, whose code moved (the in-game run covers them); a feature folder a build stops on, by design, with the reason.

Turning it off: revert; a feature is a folder, and taking one out leaves its site dormant.

Compatibility: none for players; a new feature goes in host/features/, as docs/development.md says.

Checklist

Everyone

  • One topic.
  • The title is a Conventional Commit (type(scope): description) of the status line's type.
  • Why gives the reason plainly and links the issues that fit (the tracker searched).
  • Tested on my own PC where it runs (in the game, for a change the game runs), as Testing's Tested says; what
    can't be run by hand is in Testing's Not testable by hand, with why.
  • Measurements has a verdict, or one line saying why there is nothing to measure.
  • Nothing gets worse that I know of: any cost (speed, latency, memory, a setup that worked before) is measured and
    in Risks.
  • Testing names what was tested, what wasn't, and what must still be tested, where and how.
  • Rebased on current main.
  • python -I -B tests/run_all.py passes (any failure that also fails on main is named).
  • No game files, dumps, build output or generated files.
  • third_party/ untouched (a new folder with a PROVENANCE.md instead).
  • The license is OK: GPL-3.0-or-later, nothing incompatible and nothing decompiled.

When it applies

  • AI assistance recorded with Assisted-by: / Co-Authored-By: trailers.
  • A check under tests/ that reads changed source updated in this PR, not weakened.
  • Host code builds on both Windows (llvm-mingw) and Linux, with UTF-8 paths. (Windows built and run; Linux not built here: Must be tested)
  • Docs updated.

🤖 Generated with Claude Code

**Type:** feat · **Risk:** low · **Tested on:** Windows 11, RTX 3080, Ryzen 9 3900X · **Needs:** a run on Linux ## TL;DR The engine's game-code features become modules: one folder each under `host/features/`, holding the feature's code and a short manifest of the hook sites it attaches to. A feature's pull request then changes its own folder, so two features never conflict in git, and what can still collide, two features hooking the same place, is found mechanically: an audit, run by every build and by a game-free test, refuses features that share a site with no order or a function with no acknowledgement, and prints a map of what hooks what. Text speed and Text size are the first two modules; nothing a player sees changes. ## Breakdown - **Who it helps:** everyone working on the engine's features in parallel, and reviewers: the hook map says what a feature touches, and the manifest is the whole list of what the engine hooks. - **What was wrong:** the features' code was spread over `host/` and one 1,800-line patch file every feature edits; the site table of #1821 was one file every new site would append to; nothing said when two features hook the same function. - **What changes:** feature folders with manifests, the build compiling them and writing the handlers' table, hook sites a file each, the audit and its hook map; Text speed and Text size moved into folders. - **What does not change:** what the player sees; the translated game (byte for byte, but the site's own function and the sites' table). - **How it was checked:** a new game-free test with teeth; the whole translation compared byte for byte; `tests/run_all.py` against main; a cold review; a run in the game. - **What is still open:** a run on Linux; the PR description's Hooks section and the warning across open PRs, which come next. - **Risk:** low: Text speed and Text size are the only code that moved, and they ran in the game. ## Why The maintainers agreed on the approach: the engine's features are built as the native, compiled tier of the Modding API's mods (the Spec's section 8.3 declares a compiled mod's sites the same way: a signature, and a place for each game build), each in its own folder, so the team can move the roughly forty text patches of `nx2/oot_patches.py` onto sites one feature at a time, in parallel, without conflicts, and with every overlap between features found by a check rather than by luck. This is the infrastructure for the plan's milestone M2, piece 3. Stacked on #1821 (the hook compiler), and through it on #1587 and #1560. ## What changed Features are folders with manifests, the build compiles them, sites are a file each, and an audit keeps the features from colliding. - `host/features/<name>/` (new): a feature's C and `feature.toml`: a summary, and a `[[hook]]` per site (the site, the handler, `after` for the order on a shared site, `shares` for the features acknowledged in that site's function). Under `host/` so the Builder's source package, the PR checker's game paths and the checks that scan `host/` take them in. - `nx2/features.py` (new): reads the manifests and audits them without the game, by the functions the site files declare. It stops on a manifest that is not valid, an unknown site, a handler its folder does not define (comments aside) or two features define, two features on one site with no order between them (or orders that go round), two features with sites in one function that neither acknowledges there (an acknowledgement holds in its own function and game build only), what the lift reads from `host/*.c` itself in a feature's C, two files with one object name. It warns of a hooked function the text patches still name, and prints the hook map. `python -I nx2/features.py audit` runs it. - `nx2/build.py`: the features' C compiled with the host's; `features.c`, the handlers' table, written from the manifests in their order (only when it changes, as a new file: a pack's work folder holds hard links); the audit stops the build. - `host/hook_sites.c`, `host/hook_sites.h`: the handlers attached from that table before main (`HOS_SITE` is gone: a manifest is the only way in); one line at start when features hook sites the game build lacks. - `nx2/sites/<name>.toml`: a file per site, with a table per game build (the address, and the function the lift checks it is in); a site's C names its own byte and number (`hos_site_<name>`), so a new site changes no other site's function. `nx2/hooks.py` reads them; two sites at one place are refused. - Text speed (`host/features/text-speed`, its option read moved out of `host/nn_vi.c`) and Text size (`host/features/text-size`, `host/oot_text_size.c` moved there), both on `textStep`, Text size after Text speed as they ran before; `host/oot_text_step.c` is gone. - `docs/development.md`: a Feature modules section (a feature's folder, a hook site, the audit, adding a feature). - Tests: `tests/test_features.py` (new); `tests/test_hook_sites.py`, `test_text_speed.py`, `test_text_size.py` for the new places; `test_env_on.py` (HOS_TEXT_SPEED is a number), `test_frame_rate_cap_step.py`, and `test_host_guest_writes.py` and `test_msvc_oldnames.py`, which now read the features' C too. ## Measurements **Verdict:** nothing to measure in frames: the translated game is byte for byte the same as #1821's but for the site's own function (`f_80b1e140`: the site's C naming its byte) and the sites' table; Text speed now reads its option in its handler, as Text size always did. Baseline: the hook compiler's PR (#1821), which this PR is stacked on: main differs from it by that PR, #1587 and #1560, each compared on its own. Hardware: Windows 11 Pro N (build 22631), RTX 3080, Ryzen 9 3900X, 64 GB Scene: none: the translation, game-free (the in-game run is in Testing) Method: `scripts/lift_parity.py lift` (#1798) of each commit, rc.8's ExeFS, the Builder's Python and llvm-mingw, then `compare` How to reproduce: `python -I scripts/lift_parity.py lift --checkout <a checkout>` for #1821's head and for this branch, then `python -I scripts/lift_parity.py compare <one> <the other>` | | Before: #1821 @ ada6d3fc | After: this PR @ 95fbe486 | |---|---|---| | files the lift wrote | 92 (1.73 GB) | 92, the same | | | Before: this PR @ 95fbe486 | After: this PR @ 67b0cbe2 (the review's fixes) | |---|---|---| | files the lift wrote | 92 (1.73 GB) | 89 the same, `a64_sites.c`, the chunk cost estimates | | functions that differ | | 1 of 2,994 in its file: `f_80b1e140` (the site's C naming its own byte) | After the rebase onto main 0de25c3e (which brought #134's ExeFS patches, #1740's text patch and #1774's menu modules), lifted again: | | Before: #1587 @ 725cc27a (byte for byte main 0de25c3e's) | After: this PR @ 957a28b8 | |---|---|---| | files the lift wrote | 91 (1.73 GB) | 89 the same, `a64_sites.c` new, the chunk cost estimates | | functions that differ | | 1 of 2,994 in its file: `f_80b1e140` (the site, its locals kept, its own byte) | ### Flags and settings None (`HOS_TEXT_SPEED` and `HOS_TEXT_SIZE` read as before, now in their features' folders). ## Testing **Tested:** - `tests/test_features.py` (new): manifests (16 kinds of mistake, each named); the audit over made-up features and sites (orders and chains of them, cycles, unknown names, acknowledgements per function and per game build, the lift's markers, the text patches' warning, the hook map); object names; the handlers' table compiled with `host/hook_sites.c` and run; the tree's own features audited clean. With `tests/test_hook_sites.py` (the site files, two sites at one place, a site checked against its declared function's own code, a64_sites.c in the lift's order) and `tests/test_text_speed.py` (an empty `HOS_TEXT_SPEED` too): twenty deliberate breaks of the code, each caught. - A cold review of the branch by a second agent (code read, nothing run); its findings fixed in 67b0cbe2. - The whole translation compared byte for byte (#1798), as in Measurements. - `python -I -B tests/run_all.py` (the Builder's llvm-mingw first on `PATH`, `LONGSHOT_MINGW` set), on main 0de25c3e: 707 passed / 45 failed at 957a28b8. The one failing here only, `tests/test_a64_helper_build_flags.py` (#756's: its stand-in lift folder lacked #1821's `a64_sites.c`), is fixed in #1821 and passes; the other 44 fail on main 0de25c3e too, rerun there. - A pack of this branch over main 47430e81's base (#1729's header fix applied for the build only): the features' C and the handlers' table compile with the host's, and the game's link map holds them. - In the game (Windows 11, RTX 3080, Ryzen 9 3900X), by the person I work for, at 95fbe486: the repository's `kokiri-start` save, Kokiri Forest's signs and dialogue. Text speed: Normal types the letters out, the fastest setting shows each page at once. Text size: larger, as before, including #1779's first showing at the default size, which main has too. - Rebased onto main 0de25c3e (with #1821): the conflict was `tests/test_text_speed.py`'s reader of the menu rows (#1774's `menu_text()`); the checks of the files it touches pass; the comparisons and `run_all` above are on it. **Not tested:** Linux. In the game, the review's fixes after 95fbe486 (67b0cbe2): their translation differs in `f_80b1e140` alone, and their host code is covered by the compiled runs of `tests/test_features.py` and `tests/test_hook_sites.py`. **Must be tested:** - Linux, any PC with the game: a build of this branch, expecting `build: 2 features, 2 hooks`, and Text speed and Text size in a dialogue as on Windows (by hand, or the Test Farm on a Linux machine). ## Risks **What could break:** Text speed and Text size, whose code moved (the in-game run covers them); a feature folder a build stops on, by design, with the reason. **Turning it off:** revert; a feature is a folder, and taking one out leaves its site dormant. **Compatibility:** none for players; a new feature goes in `host/features/`, as `docs/development.md` says. ## Checklist ### Everyone - [x] One topic. - [x] The title is a Conventional Commit (`type(scope): description`) of the status line's type. - [x] Why gives the reason plainly and links the issues that fit (the tracker searched). - [x] Tested on my own PC where it runs (in the game, for a change the game runs), as Testing's Tested says; what can't be run by hand is in Testing's Not testable by hand, with why. - [x] Measurements has a verdict, or one line saying why there is nothing to measure. - [x] Nothing gets worse that I know of: any cost (speed, latency, memory, a setup that worked before) is measured and in Risks. - [x] Testing names what was tested, what wasn't, and what must still be tested, where and how. - [x] Rebased on current `main`. - [x] `python -I -B tests/run_all.py` passes (any failure that also fails on `main` is named). - [x] No game files, dumps, build output or generated files. - [x] `third_party/` untouched (a new folder with a `PROVENANCE.md` instead). - [x] The license is OK: GPL-3.0-or-later, nothing incompatible and nothing decompiled. ### When it applies - [x] AI assistance recorded with `Assisted-by:` / `Co-Authored-By:` trailers. - [x] A check under `tests/` that reads changed source updated in this PR, not weakened. - [ ] Host code builds on both Windows (llvm-mingw) and Linux, with UTF-8 paths. (Windows built and run; Linux not built here: Must be tested) - [x] Docs updated. 🤖 Generated with [Claude Code](https://claude.com/claude-code)
dyst added 11 commits 2026-11-20 00:08:22 +00:00
Longshot adds to a64-kit's Lifter.emit_function in four places: fastfp's runs of float
instructions, lift_fixes' entry label and goto, entry_hooks' call and poll's interrupt
polls. Each replaced or wrapped the method when nx2/lift.py imported it (fastfp with a
151-line copy), so what the lift wrote depended on the order of those imports.

nx2/emit.py now holds the one copy of the kit's method (its added lines tagged
"# (emit)", pinned at the kit's tree as before) and the points the modules register
into: a run gatherer (fastfp), start hooks (lift_fixes' label), head lines (entry_hooks'
call, then lift_fixes' goto) and passes over the finished text (poll). The order of
what they add is written there, not left to install order. nx2/chunks.py's new-process
check compares the registered hooks too.

The lifted C is byte-identical: a standalone lift of main (79d4a143) and of this branch,
same inputs and environment as make_game.py's, wrote the same 106 files (1.69 GB).
tests/test_fastfp_override.py becomes tests/test_lift_emit.py: the same pins, and the
hooks checked against the wrappers they replaced, in either registration order.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
refactor(lift): guest locals register into emit_function's hooks, in place of two wrappers
Some checks failed
CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Has been skipped
CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Has been skipped
CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Has been skipped
Third-party parts / The same as upstream (pull_request) Has been skipped
Website / Website on Cloudflare (pull_request) Has been skipped
CI / Conventional Commit title (pull_request) Successful in 12s
PR description / Description follows the template (pull_request_target) Successful in 16s
CI / Workflow lint (actionlint) (pull_request) Successful in 19s
Guide screenshots / Guide images and local UI source review (pull_request) Failing after 15s
Third-party parts / As their notes say (pull_request) Successful in 18s
4287bd5b34
nx2/guest_locals.py wrapped Lifter.emit_function twice (the context-flag texts for a function in no_local, and which
function localize_regs was for, through two globals) and replaced the kit's localize_regs. With #1560's one emit_function
those become two of its points:

- texts: called with (lifter, entry, texts) before a function is written; guest_locals' plain gives a function in
  no_local the context-flag texts.
- localize (set_localize): one hook, told the function being written, where the copied method makes registers and flags
  locals; guest_locals' localize keeps the context for a function in no_local or one the game's patches would change.

emit.py's copy now reads local_regs and localize_regs through two functions of its own, asked when it runs, not the names
it imported: guest_locals sets the kit's defaults after emit is imported, and a rebase onto it had to patch emit's names
to keep the locals on. emit.fingerprint() lists both hooks, which the lift's new processes compare.

A lift of the game with this commit is byte-identical to main's (868f6be8): 92 files, 1.63 GB. tests/test_lift_emit.py
checks the two hooks against the kit's own method; making emit's local_regs answer as the imported one did, or a localize
hook that ignores no_local, fails it.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
nx2/lift_fixes.py starts such a function with a goto to its entry right after the head,
and with A64_LOCAL_REGS or A64_LOCAL_FLAGS the kit declares the register locals there too,
loaded from the context. The goto came first and jumped over the declarations, so the
function ran on locals that were never loaded. A made-up function emitted as the lift does
shows it; this fails here and on main.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
fix(lift): with local registers, a function with code below its entry loads them before its entry
Some checks failed
CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Has been skipped
CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Has been skipped
CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Has been skipped
Third-party parts / The same as upstream (pull_request) Has been skipped
Website / Website on Cloudflare (pull_request) Has been skipped
CI / Conventional Commit title (pull_request) Successful in 11s
PR description / Description follows the template (pull_request_target) Successful in 16s
Guide screenshots / Guide images and local UI source review (pull_request) Failing after 18s
CI / Workflow lint (actionlint) (pull_request) Successful in 22s
Third-party parts / As their notes say (pull_request) Successful in 25s
725cc27ae7
nx2/emit.py put its head lines (entry_hooks' call, lift_fixes' goto to the entry) into
a function's finished text, after the kit's localize_regs had declared the register
locals right after the head, so the goto came first and jumped over the loads: with
A64_LOCAL_REGS or A64_LOCAL_FLAGS, each function with code below its entry (26 in a lift
of OoT) ran on locals never loaded from the context. The head lines now go into the
function's lines before localize_regs, which then declares the locals ahead of them.
A PLT stub, written before that point, still gets them in its text.

Without local registers (the default) the C is unchanged; tests/test_lift_entry_local_regs.py
passes now.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
The hook compiler of the Modding API Spec (section 8.1; the plan's M2, piece 1). nx2/hooks.py reads the supported game
build's site table, finds each site by its signature (instruction words with masks; exactly one match in its module or
the lift stops, every bad site named), and compiles it in: an "after" site as C spliced after its instruction
(Lifter.after), an "entry" site as a head line that returns when its call says the function is done and otherwise reads
the function's locals again. A site costs a byte load and a predicted branch while no handler wants it.

nx2/emit.py now splices Lifter.after past an instruction that is not an op but falls through (a conditional branch, a
call, an SVC); the kit splices only after ops. The entry sites' lines reach the emit's new processes as
nx2/chunks.py's EMIT_TABLES site_entries. nx2/make_game.py's lift key takes .toml files in nx2/.

The table of OoT 1.1.1 is present and empty: the translation is unchanged.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Co-Authored-By: Claude Opus 5.5 <[email protected]>
The first feature moved from nx2/oot_patches.py's text patches to a hook site (the Modding API Plan, M2 piece 2).
textStep, in OoT 1.1.1's table, is the letters' block of f_80b1e140 found by four words (the branches' offsets masked),
where the patch was. host/oot_text_step.c's handler does what the patch did: s0 times hos_text_scale when it is not
1, then hos_oot_text_pane with x19.

The host's side of the sites: the lift writes a64_sites.c (the sites' names, in order, and a byte each, hos_sites),
build.py compiles it, and host/hook_sites.c attaches handlers by name (HOS_SITE at start-up) and calls them in order
(hos_site_call) until one says the function is done.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
An after site went into Lifter.after, whose C makes its function keep every register in the context
(nx2/guest_locals.py keep_spliced): a site slowed the code it was put in. Sites now have their own table,
Lifter.site_after, and their C carries the context marks of the instruction they follow: inside the
branch a set byte takes, the locals changed since the last sync are written back and the flags stored
before the handler, and every local is read again after it. A dormant site is a load of its byte and a
branch, and the function keeps its locals.

Each site is counted where the emit writes it (the counters come back from the worker processes), and
the lift stops when a site was written into no function: its table not sent to the emit, or its place
in no function's code. The lift says how many sites are in functions with locals, and names the others.

Tested: tests/test_hook_sites.py (the emitted text; a stand-in run gatherer; the emitted function
compiled and run, dormant and set; a patch and a site on one instruction; the written check), with
teeth for seven breaks; test_spawn_emit, test_lift_emit, test_menu_hooks, test_entry_below_locals,
test_text_speed, test_text_size pass.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
fix(lift): the hook site call is A64_NOESC, an after site runs every handler, BLRAA is no branch away
Some checks failed
CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Has been skipped
CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Has been skipped
CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Has been skipped
Third-party parts / The same as upstream (pull_request) Has been skipped
Website / Website on Cloudflare (pull_request) Has been skipped
CI / Conventional Commit title (pull_request) Successful in 11s
PR description / Description follows the template (pull_request_target) Successful in 14s
CI / Workflow lint (actionlint) (pull_request) Successful in 17s
Guide screenshots / Guide images and local UI source review (pull_request) Failing after 18s
Third-party parts / As their notes say (pull_request) Successful in 21s
Website / Website on Cloudflare (pull_request_target) Has been skipped
4cdd901d75
From a cold review of this branch. The site's call of the host is declared with A64_NOESC, as every call lifted code
makes: without it the call captured the context in the whole function, its dormant path too, and the compiler reloaded
the context's registers after every guest store (a64-kit's a64.h). An after site runs every handler; only an entry
site's stop at the first that says it did the function's work (a64_site_entry, in a64_sites.c). never_falls no longer
refuses BLRAA and BLRAB, which return. A signature is found by an exact anchor word (bytes.find), not by a regex tried
at every byte, so many sites resolve in the time of one. A lift with more sites than host/hook_sites.c holds stops.
The host writes a handler's user before its function; host/hook_sites.h says what a handler may rely on (not the
flags; not keeping c) and that attaching is for the program's start.

Tests: the NOESC declaration, the branches an after site may follow, a signature of masked words alone, too many sites,
an after site running every handler and an entry site stopping at the first done.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
The engine's features move into host/features/<name>/, one folder each: the feature's host code and feature.toml, which
says what it hooks (a site and a handler per hook, `after` for the order on a shared site, `shares` for the features that
hook the same functions, acknowledged). A feature's pull request changes its own folder, so two never conflict in git.
The build compiles the folders' C with the host's and writes features.c, the table host/hook_sites.c attaches the
handlers from in their declared order; a manifest is the only way a handler is attached (HOS_SITE's constructors are
gone), so the manifests list everything the engine hooks.

Hook sites are a file each, nx2/sites/<name>.toml, with a table per game build giving the site's address and the
function it is in (the shape of the Modding API Spec's section 8.3); a build's sites are numbered in name order. The lift
now stops when a site is not in the function its file declares.

nx2/features.py audits the features without the game, by those declared functions, in every build and in
tests/test_features.py: it stops on a manifest that is not valid, an unknown site, a handler the folder does not define
or two features define, two features on one site with no order between them (or orders that go round), and two features
with sites in one function that neither acknowledges; it warns of a hooked function the text patches still name, and
prints the hook map (function, site, features in order). `python -I nx2/features.py audit` runs it by hand.

The first two features: Text speed (host/features/text-speed, its option read moved out of host/nn_vi.c) and Text size
(host/features/text-size, host/oot_text_size.c moved there), both on textStep; host/oot_text_step.c is gone. Feature
folders sit under host/, so the Builder's package, the PR checker's game paths and the checks that scan host/ take them
in; the two host checks that read only host/*.c (unmarked guest writes, MSVC's old names) read the features' too.
docs/development.md gets a Feature modules section.

Tested: tests/test_features.py (new), test_hook_sites, test_text_speed, test_text_size, test_env_on,
test_frame_rate_cap_step, test_msvc_oldnames, test_switch_table, test_doc_anchors pass; test_host_guest_writes fails
as on main.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
fix(host): the cold reviews' findings on the hook sites and the feature modules
Some checks failed
CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Has been skipped
CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Has been skipped
CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Has been skipped
Third-party parts / The same as upstream (pull_request) Has been skipped
Website / Website on Cloudflare (pull_request) Has been skipped
CI / Conventional Commit title (pull_request) Successful in 10s
PR description / Description follows the template (pull_request_target) Successful in 14s
Guide screenshots / Guide images and local UI source review (pull_request) Failing after 15s
CI / Workflow lint (actionlint) (pull_request) Successful in 16s
Third-party parts / As their notes say (pull_request) Successful in 17s
Website / Website on Cloudflare (pull_request_target) Has been skipped
957a28b875
The site call is declared with A64_NOESC, as every call lifted code makes: without it the call captured the context in
the whole function, its dormant path too, and the compiler reloaded the context's registers after every guest store
(a64-kit's a64.h). A site's C names its own byte and number (hos_site_<name>, hos_site_id_<name>, in a64_sites.c), so a
new site changes no other site's function. An after site runs every handler (only an entry site's stop at the first that
says it did the function's work: a64_site_entry). The lift checks a site against its declared function's own code, not
against the nearest start below it (a setjmp's return starts a piece there), refuses two sites at one place and more
sites than the host's table holds; never_falls no longer refuses BLRAA and BLRAB, which return. A signature is found by
an exact anchor word (bytes.find), not a regex at every byte.

The audit: an acknowledgement holds in the function where it is made (shares and after on a hook), so a pair that meets
in a second function, or in another game build, is found again; two features on one site are held by the order rule,
through any chain. A feature's C may not hold what the lift reads from host/*.c itself (ENTRY_HOOK_SYM, HLE functions),
which the lift does not read in feature folders; a handler named in a comment does not count; dotfiles and a file
browser's files are left alone; feature names have no -- or trailing -, and two files with one object name stop the
build (object_name moves to nx2/features.py). The host says once at start when features hook sites the lift lacks, and
writes a handler's user before its function. Text size includes vk_api.h by its path from its folder; Text speed and
size read their switch once without a torn read. host/hook_sites.h and docs/development.md say what a handler may rely
on and what the audit cannot see.

Tests: a site's NOESC, its own instruction's marks, a64_sites.c in the lift's order, the branches an after site may
follow, a masked signature, two sites at one place, too many sites, a piece inside a function, an after site running
every handler and an entry site stopping; the audit's chains, per-function and per-build acknowledgements, the lift's
markers, comments, OS files, names, object names, features.c kept off hard links; HOS_TEXT_SPEED set empty. Twenty
breaks of the code, each caught; test_features no longer leaves its folders in TEMP.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Author
Member

Tested in game: Windows 11 Pro N 22631, RTX 3080 (NVIDIA 621.14), Ryzen 9 3900X, by the person I work for: a pack of this branch at 95fbe486 over main 47430e81's base, #1729's header fix applied for the build only. The repository's kokiri-start save, Kokiri Forest's signs and dialogue: Text speed (now host/features/text-speed) at Normal types the letters out, at its fastest shows each page at once; Text size (host/features/text-size) larger as before, including #1779's first showing at the default size, which main has too. Not run in the game since: the review's fixes (67b0cbe2: per-site bytes, A64_NOESC, every handler of an after site) and the rebase onto 0de25c3e.

Tested in game: Windows 11 Pro N 22631, RTX 3080 (NVIDIA 621.14), Ryzen 9 3900X, by the person I work for: a pack of this branch at 95fbe486 over main 47430e81's base, #1729's header fix applied for the build only. The repository's kokiri-start save, Kokiri Forest's signs and dialogue: Text speed (now host/features/text-speed) at Normal types the letters out, at its fastest shows each page at once; Text size (host/features/text-size) larger as before, including #1779's first showing at the default size, which main has too. Not run in the game since: the review's fixes (67b0cbe2: per-site bytes, A64_NOESC, every handler of an after site) and the rebase onto 0de25c3e.
dyst force-pushed feat/feature-modules from 957a28b875
Some checks failed
CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Has been skipped
CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Has been skipped
CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Has been skipped
Third-party parts / The same as upstream (pull_request) Has been skipped
Website / Website on Cloudflare (pull_request) Has been skipped
CI / Conventional Commit title (pull_request) Successful in 10s
PR description / Description follows the template (pull_request_target) Successful in 14s
Guide screenshots / Guide images and local UI source review (pull_request) Failing after 15s
CI / Workflow lint (actionlint) (pull_request) Successful in 16s
Third-party parts / As their notes say (pull_request) Successful in 17s
Website / Website on Cloudflare (pull_request_target) Has been skipped
to 8aefb31cd9
Some checks failed
CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Has been skipped
CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Has been skipped
CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Has been skipped
Third-party parts / The same as upstream (pull_request) Has been skipped
Website / Website on Cloudflare (pull_request) Has been skipped
CI / Conventional Commit title (pull_request) Successful in 24s
Guide screenshots / Guide images and local UI source review (pull_request) Failing after 32s
CI / Workflow lint (actionlint) (pull_request) Successful in 34s
Third-party parts / As their notes say (pull_request) Successful in 35s
PR description / Description follows the template (pull_request_target) Successful in 7s
2026-11-20 00:49:21 +00:00
Compare
Some checks failed
CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Has been skipped
CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Has been skipped
CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Has been skipped
Third-party parts / The same as upstream (pull_request) Has been skipped
Website / Website on Cloudflare (pull_request) Has been skipped
CI / Conventional Commit title (pull_request) Successful in 24s
Guide screenshots / Guide images and local UI source review (pull_request) Failing after 32s
CI / Workflow lint (actionlint) (pull_request) Successful in 34s
Third-party parts / As their notes say (pull_request) Successful in 35s
PR description / Description follows the template (pull_request_target) Successful in 7s
This pull request has changes conflicting with the target branch.
  • docs/design.md
  • docs/development.md
  • nx2/fastfp.py
  • nx2/lift.py
  • tests/test_fastfp_override.py
View command line instructions

Manual merge helper

Use this merge commit message when completing the merge manually.

Checkout

From your project repository, check out a new branch and test the changes.
git fetch -u 'feat/feature-modules:dyst-feat/feature-modules'
git switch 'dyst-feat/feature-modules'
Sign in to join this conversation.
No reviewers
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
MagicBeanDev/Longshot!1822
No description provided.