WIP: perf(host): four physics glue leaves and the cloth damping's repeated powf on host code (HOS_NATIVE_GLUE) #1911
Labels
No labels
area/accessibility
area/audio
area/input
area/install
area/menu
area/renderer
area/saves
area/tooling
area/upscaling
beanstalk: approved
beanstalk: failed
beanstalk: needs approval
beanstalk: running
beanstalk: tested
ci: approved
good first issue
gpu/amd
gpu/igpu
gpu/intel
gpu/nvidia
help wanted
kind
bug
kind
chore
kind
docs
kind
feature
kind
performance
kind
question
needs measurements
platform/linux
platform/macos
platform/steam-deck
platform/windows
preview
priority
critical
priority
high
priority
low
regression
risk
high
risk
low
risk
medium
status
blocked
status
confirmed
status
duplicate
status
invalid
status
needs-info
status
wontfix
ci: approved
No milestone
No project
No assignees
4 participants
Notifications
Due date
No due date set.
Dependencies
No dependencies set
Reference
MagicBeanDev/Longshot!1911
Loading…
Reference in a new issue
No description provided.
Delete branch "snaildev99/Longshot:perf/native-glue"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Summary
Four small routines of the game's physics glue (its side of Havok) run as native host code with
HOS_NATIVE_GLUE=1, and the cloth damping's repeatedpowfcalls are replayed from the call before them, giving the game's results bit for bit: every register, every byte written, NZCV and FPSR. A call the host cannot prove exact runs the game's translated routine, with nothing written by the host. Off unless the switch is set.The goal is less CPU work for the same frame. The cloth damping
f_8003d320callspowftwice per cloth particle with the same two arguments for every particle of a call (the damping factors to the power of the step):powfdirectly under it was 0.82 % of all game code in Castle Town's Market. In the game 99 % of those calls repeat the call before them, and the memo answers them from it. With nine rewrite branches on together (this one among them, merged in a test tree; named in the baseline below, the cloth and particle effect rewrites are not among them), the game used 2.7 % less CPU standing and 2.4 % less turning in the Market at a 60 fps cap, 4.1 % / 4.0 % less on the executor threads (where these routines run). This branch's own share of that has not been measured alone: the Test Farm request asks for it.What it answers (guest addresses):
f_801c0d7c: a rotation matrix to a normalised quaternion (Shepperd's four cases by the signs, the sum of squares by two pairwise adds, the quaternion divided by its square root, or times 0 when the sum has no exponent bits).f_804f8990is a byte-for-byte copy of it at another address.f_80825860: a box under a transform, centre and half extents in single precision, the bounds in double and rounded outwards by a step of the bit pattern before the conversion to single; an empty box (min over max) is copied as it is.f_80171890: a point under a QsTransform (translation, quaternion, scale), four lanes.powfcalls off_8003d320's particle loop (return addresses 0x8003d8d0 and 0x8003d8e4): a memo per thread and call site, keyed on both arguments and FPCR. On a new key it runs the game'spowfon the real context and on two copies whose other registers hold a pattern and its complement, so it learns exactly which bytes the call writes; the key is kept only when all three agree, nothing outside those bytes changed, SP, FPCR and PC are as before, and no FPSR flag but IXC was raised (which also keepspowf's error paths out). A replay needs FPSR.IXC already set and writes those bytes. A second argument of exactly 1 (60 fps exactly) goes straight topowf, whose entry returns at once.How:
nx2/oot_patches.py(GLUE_NATIVE,patch_glue_native,patch_glue_powf) putsif (hos_glue_<addr>(c, f_<addr>)) return;first in the four leaves and replaces the twopowfcalls inf_8003d320by memo calls, in the supported build only.host/glue_kernels.cis the leaves' instruction sequences, one statement an instruction on 128-bit register values in locals, the registers copied out only once the call is known to be taken. The proof rule: every operand of every float operation zero or of a magnitude in [2^-63, 2^63), no NaN or subnormal into a comparison, and FPCR / FPSR inside the translator'sRECOMP_FPX_OPEN; then IEEE gives Arm's bits, flush-to-zero changes nothing and no flag but IXC is raised. The quaternion kernel checks a shorter set with the same guarantee (the nine inputs, the four lanes, the sum of squares finite, and its root), and the box kernel gives a bound of exactly zero Arm's signed zero and IDC or UFC instead of declining it.host/oot_glue.cgates each call, stores the registers and the bytes (pages marked) or, in check mode, compares them with the translated routine's.Read first:
host/glue_kernels.c,host/oot_glue.c(hos_glue_powf_memoand its comment), theHOS_NATIVE_GLUEblock ofnx2/oot_patches.py,tests/test_oot_glue.py.No code here is ported from another project.
Testing
Tested: Windows 11, RTX 3090 (NVIDIA 610.88), Ryzen 9 5900X. In the game, in a test tree that merges nine rewrite branches, this one among them, on main
068fb8eb(test/rewrites@ e614ef49 on my fork; this code as at 9122e9e5): a check run withHOS_NATIVE_GLUE=check(every call the host takes is also run translated and the two compared), about five minutes in the Market and Hyrule Field: 0 mismatches. Answered by the host: the quaternion 1,701,987 of 1,702,037 calls, its copy 12,964 of 12,973, the box 200,416 of 200,528, the point 36,470 of 68,861 (32,391 declined for their values), and thepowfmemo replayed 2,437,027 and 2,441,712 of 2,464,825 calls at its two sites (99 %). Then the CPU measurement in Measurements, with the switch on (the quaternion answered all of its 729,365 calls there). This branch's head (the same code cherry-picked onto maina104182b, plus one commit of declarations) is not built or played.Offline, against the game's own code: an AArch64 emulator (Unicorn) runs the game's routine on its image, a driver runs the translator's copy of the routine and the hook (
HOS_NATIVE_GLUE=1, and=checkon every case too) on the same case; compared, when the hook took the call, every x register, SP, all 32 q registers (128 bits), NZCV, FPSR and every byte of the case's memory against the emulator, and everything against the translated routine when it declined. Two builds: x86-64-v3 and plain x86-64.f_801c0d7c<as<=200, a register not written 2,692f_804f8990f_80825860f_80171890powfmemo:f_8003d320whole, 0 to 40 particles a call, the memo carried from call to callpowfcalls a siteThe cases: realistic rotations (all four of Shepperd's cases, near a half turn), boxes, transforms and QsTransforms; small integers (exact cancellations, signed zeros); zero, tiny and huge values; edge values anywhere (NaN quiet and signalling, infinities, subnormals, FLT_MAX); junk in unused lanes; empty boxes; outputs over inputs; changed and moved constants; every FPCR mode and FPSR state; for the memo, steps and damping from small sets so keys repeat, a third of the calls reusing the last call's step, the cloth mode where the second argument changes per particle. The translated routines alone differ from the emulator only in NZCV where the translator keeps only the flags it reads; the hook writes Arm's NZCV, and check mode counts an NZCV-only difference apart ("lift's flags"), not as a mismatch. The memo is compared with the translated
f_8003d320in every register, NZCV, FPSR and byte; against the emulator it differs exactly where the translated routine does (NZCV at the routine's end, which the translator keeps only where it reads it, and x16 / x17 of the call stub, which the translator never sets). Run again at this head with this branch'shost/oot_glue.c,host/glue_kernels.cand, for the memo, itsnx2/oot_patches.py(so the memo calls are made by this head's patch, with its new declaration, and compiled), the same seeds and sizes, check mode on every case: the same counts, 0 mismatches in both builds and in check mode.Game-free tests at this head:
tests/test_oot_glue.py(the hooks and the memo calls placed once and only in the supported build, idempotent, a missing one reported; off, on and check over a guest window: the quaternions, boxes and points against values worked out in Python, a box in place, the declines with nothing written, a replay exact after its fill and the cases that run translated (y = 1, IXC clear, another FPCR, a register dependence, a raised flag), an NZCV-only difference counted apart, a byte or register off caught and logged; built for this CPU and for plain x86-64) passes. Of the 44 tests run (it,tests/test_switch_table.pyand every test that readsnx2/oot_patches.pyornx2/entry_hooks.py,tests/test_patch_hooks_defined.pyamong them) 39 pass; the five others fail the same way on maina104182bon this PC:test_dev_packs,test_high_fps_fixes,test_libc_hooks,test_lift_cache("...and the others still by their code"),test_song_note_cue. The fulltests/run_all.pywas not run. ruff clean.Still to test:
kokiri-start): this PR withHOS_NATIVE_GLUEunset againstHOS_NATIVE_GLUE=1, the game's CPU seconds at a 60 fps cap (on a PC that holds 60 there) and the frame times uncapped, expecting the same frame rate or better with less CPU time and cloth and physics moving as without the switch (@Beanstalk linux, amd).HOS_NATIVE_GLUE=check, expecting everyoot: glueline to endMISMATCHES in all 0(@Beanstalk linux).HOS_NATIVE_GLUE=checkand then=1, expecting no mismatch and cloth moving as on main (by hand).Measurements
Verdict: less CPU for the same frame, measured with the nine on together: the whole game process 2.7 % less standing and 2.4 % less turning in the Market at a 60 fps cap, the executor threads 4.1 % / 4.0 % less. With the hooks built in and the switches unset, -0.3 % / -0.2 % against plain main: inside the runs' spread, so no cost measured with the switch off. This branch's own part is not separated: the Test Farm request asks for this PR alone, switch off against on. Offline, against an approximation of main's form of each routine: the cloth damping with its
powfcalls 2.9 times per particle (at any frame rate but exactly 60), the quaternion 1.5 to 1.9 times, the box 3.0 to 3.1 times, the point 1.75 to 2.2 times.Baseline: three builds from scratch of two trees, the default hot list: plain main
068fb8eb(Before), and the test treetest/rewrites@ e614ef49 on my fork, which is main068fb8ebwith nine rewrite branches merged (the seven maths units, the maths library, animation, bounds, crowd, visibility, glue, the effect triggers with the six-channel reverb, and the solver; the cloth and particle effect rewrites are not among them; this branch as at 9122e9e5), lifted withA64_FPXKEEP=0(a lift-time change of another branch left out): once with no switch set, once with the nine's switches on (HOS_NATIVE_GLUE=1and the other eight's, among them the maths library's nativepowf, which this memo sits in front of). Main has since moved toa104182b; the numbers are main 068fb8eb's, and the farm request measures this PR alone on current main.Setup: Windows 11 (build 22631), Ryzen 9 5900X, RTX 3090, 3840x2160; child Link in the Market, hands-free through
scripts/longshot_ctl.py; the frame rate capped at 60, which all 15 runs held; each run 30 s standing, then 30 s turning the camera; the three variants (with two more of another change) interleaved over three rounds, the PC otherwise idle. CPU seconds a second of play (cores busy on average) from the processor's cycle counter, per thread; the median of three runs.068fb8ebBaseProcExecutor*), standingmkw_vk capture copy, standingmkw_vk capture copy, turningThe process includes the renderer's threads (about 1.4 cores), which the nine do not touch. The capture copy thread's rise is in Risks.
Offline speed (
bench_leaves.py/bench_powf.cin my tools: the calling thread's own CPU time; a hooked function is written in the old form on main too, every register in the context, so the patched side runs in that form; the other side is an approximation of main's form, the six most used x registers in locals, which is at most as fast as main's real form, so a factor here may be somewhat high; ±10 % noise, other builds running on the PC; Ryzen 9 5900X):f_801c0d7cf_804f8990f_80825860f_80171890f_8003d320with itspowfcalls, per particle, a step other than exactly 1/60 spowfunder it 0.82 % (Market)powfreturns at its entry)With the switch off the hooked routines measured as without the hook within these timings' noise (the hook's check is a load and a branch). The shares are self time in a Market profile and two route profiles (Kakariko and Death Mountain Trail) of earlier builds.
The switch
HOS_NATIVE_GLUE(new, run time; its row ingpu/vk/switches.def): unset or0off (the hooks return at once, the memo callspowf);1on;checkruns the host's answer into a buffer, then the translated routine for real, and compares every register and byte (NZCV-only differences counted apart), logging the first mismatch. A start lineoot: glue: four physics glue routines on the host (HOS_NATIVE_GLUE=1), and a line every 10 s, the last of the check run:It ships off in this PR. The combined measurement says on; this PR's own measurement (the farm request) decides the default before it leaves WIP.
Risks
What could break: a kernel that is not exact would give a physics or cloth value the Switch would not; a memo that missed a byte
powfwrites would leave a register as the game would not. Checked as in Testing; the hooks decline what they cannot prove (a float mode outside the translator's gate, a value outside the proof's range, memory outside the guest's), the memo keeps a key only when the three runs agree byte for byte, and the translated routine runs.Beside a native
powf: the maths library branch hookspowfitself. The memo callspowfthrough its translated function, so with both on a new key runs that nativepowfand a replay replaces it; offline, with a stand-in of that hook, every register, flag and byte matchedf_8003d320with that hook alone (1,895,547 replays).A cost seen: with the nine on, the renderer's
mkw_vk capture copythread (it copies guest memory the game marked as written) used 0.009 to 0.017 more cores in the measured runs (+10 % standing / +18 % turning). The host kernels mark the pages they write, possibly more coarsely than the translated code's own marks; which of the nine adds it is not determined. It is inside the whole-process figures above.Turning it off: leave
HOS_NATIVE_GLUEunset (the default).After updating: nothing changes for a player: saves,
longshot_options.iniand the caches as before. The five patched functions are written by the translator with their registers in the context (as every patched function is), which changes their lifted code; with the switch unset the measured cost was within the runs' spread.Checklist
Ready for review
main; the title is a Conventional Commit.068fb8eb, checked and measured; this head is not built or played.)mise run testpasses (any failure that also fails onmainis named). (44 tests run, the branch's own and every one that reads the patch files: the five failures fail on main too; the full suite not run.)main, or Measurements says why there is nothing to measure. (Measured against main with the nine on; this PR alone waits for the Test Farm.)third_party/untouched; nothing GPL-incompatible.When it applies
gpu/vk/switches.def. (The row is there; it ships off until this PR's own measurement.)tests/that reads changed source is updated, not weakened. (tests/test_oot_glue.pyis new.)Assisted-by:/Co-Authored-By:trailers.Ready to merge
main, measured again or a comment says why the numbers still hold.🤖 Generated with Claude Code
Tested in game: This head is not built or played; the same code, merged into a test tree on main
068fb8eb, ran on Windows 11 (RTX 3090, Ryzen 9 5900X): HOS_NATIVE_GLUE=check in the Market and Hyrule Field, 0 mismatches over about 1.95 million leaf calls and 4.9 million powf replays, then a CPU measurement at a 60 fps cap with nine rewrite branches on, this one among them (the whole process -2.7 % standing, -2.4 % turning).@Beanstalk linux, and amd: this PR in the child Market and in Kokiri Forest (the repo save kokiri-start), HOS_NATIVE_GLUE unset against HOS_NATIVE_GLUE=1, a few minutes each. Measure the game's CPU seconds with the frame rate capped at 60 (on a PC that holds 60 there), per thread if you can (the executor threads are where it lands), and the frame times uncapped. Expect the same frame rate or better with less CPU time, cloth and physics moving as without the switch, and "oot: glue" lines in the log with the switch on. Then one Market run with HOS_NATIVE_GLUE=check: every "oot: glue" line should end "MISMATCHES in all 0".
@snaildev99 only members of
BigBeanscan start the farm.One of them can approve this request by reacting 👍 to it or to this comment.This request was for1724104, and the PR has changed since (a31c390), so a 👍 no longer approves it. Comment@Beanstalkagain to ask for the new commit.Possibly related, for context:
Recommendation: Request changes. The kernels transcribe the game’s instructions, conflicting with the repository’s prohibition on code derived from the dump. The change’s own performance gain and its cost with the switch off are also unproven.
The change adds optional host implementations for four physics routines and a per-thread memo for two cloth damping calls. It could reduce CPU work, but adds substantial instruction-level code and hooks on paths used even with the switch off; its standalone net benefit has not been measured.
Does it help: Unproven. The reported 2.7% and 2.4% process CPU reductions in the Market are medians from three Windows runs with nine rewrite branches enabled together on an older base, so they cannot establish this PR’s contribution. The standalone offline timings do not measure player frame times or the current head against its base.
Other systems: The hooks affect Windows and Linux CPU execution regardless of GPU; the switch-off path also changes lifted code. Linux, AMD and this head in the game remain untested. The combined run showed capture-copy CPU rising by 0.009–0.017 cores, but did not identify which branch caused it. No evidence was supplied for Steam Deck or other Tier 1 systems.
Switches: HOS_NATIVE_GLUE is new: unset or 0 is off, 1 enables the host paths, and check compares against lifted code. The off default limits exposure, but measurements do not yet establish the appropriate default or its overhead. Its switches.def row is present.
Reviewed:
1724104b0aagainsta104182b40: Read AGENTS.md, CONTRIBUTING.md, the PR template, the changed host files and header, changed portions of nx2/oot_patches.py, and the available test sections; skimmed standards, the remaining test fixture, build integration, commits and the supplied run reports. The named review checklist and adjacent Tooling AGENTS.md are absent. No earlier comments or reviews were supplied.; nothing run (a code review).Checked and fine:
Blocking (to change before merge)
Suggestions (the author may take or leave)
Testing: Read-only review; no builds, tests, scripts or game runs were executed. The author reports zero mismatches in offline comparisons and a combined-branch Windows check run, but this head has not been built or played in the game.
Overlaps and merge order: No overlap information was supplied; the network is closed.
Automated code review by BeanBot (gpt-6.1-terra), on the pull request's opening. Nothing was run; a maintainer decides.
@ -0,0 +1,429 @@/* (longshot) The physics glue's leaf routines on host code, bit for bit: see glue_kernels.h for what each is and theproof rule. Each kernel below is the routine's instructions in order, one statement each (the Arm instruction is inThis transcribes the game routines instruction by instruction. CONTRIBUTING.md prohibits decompiled game code and code generated from the dump; please replace these kernels with independently authored implementations based on behavioral facts and document their provenance.
@ -1422,0 +1458,4 @@# to the registers. The function is patched whole (both calls) or not at all.GLUE_POWF_HEAD = "void f_8003d320(Ctx *__restrict A64_NOESC c) {\n"GLUE_POWF_CALLS = (("c->x[30] = 0x8003d8d0ULL; f_85751000(c);", 0), ("c->x[30] = 0x8003d8e4ULL; f_85751000(c);", 1))GLUE_POWF_NEW = ("c->x[30] = 0x%s; if ((uint32_t)c->vreg[1][0] == 0x3f800000u) f_85751000(c); else hos_glue_powf_memo(c, "This adds a branch at both call sites even with the switch off. The reported in-game numbers combine nine branches on an older base, so please measure this head alone against its base, both off and on, including frame times and the capture-copy cost.
@fix
1724104b0aa31c3908e5I changed this pull request by bringing it up to date: the new head is
a31c3908e5. I rebased it ontomain(a764653726).Conflicts resolved
Left for the author
Nothing was built or run: check the change. It will be reviewed again.
Automated fix by BeanBot (gpt-6.1-terra), asked for by HumanBean. Nothing was run; a maintainer decides.
Recommendation: Request changes. The leaf kernels transcribe game instructions in violation of the repository’s provenance rule, and the cloth memo runs speculative calls against live guest memory. The PR’s standalone performance gain is unproven.
Adds optional native paths for four physics leaves and reuses repeated cloth damping results. The combined measurements suggest a CPU saving, but this PR adds instruction-level code to maintain and changes the lifted call paths for players who leave the switch off.
Does it help: Unproven. Three interleaved Windows runs on an older base with nine rewrites enabled put Market process CPU down 2.7% standing and 2.4% turning, and executor CPU down 4.1% and 4.0%. Capture-copy CPU rose by 0.009 and 0.017 cores. The 1.5–3.1× offline timings used an approximate base and had roughly 10% noise. There are no isolated head-versus-base player measurements, uncapped frame times, or 1% lows.
Other systems: The hooks run on the CPU on Windows and Linux across GPU vendors. The reported mixed-tree run covers Windows/NVIDIA. AMD, Linux and Steam Deck remain unverified. Guest writes feed renderer capture, and the combined capture-copy cost is unattributed. Patched functions retain context registers even with the switch off. The build picks up both new C files and changes generated lift text, so build time, output size and update rebuild scope need checking. There is no direct change to saves, audio, input, menus or the renderer ABI.
Switches: HOS_NATIVE_GLUE is a new developer A/B switch. Unset and 0 disable it, 1 enables the native paths and check retains the lifted results while comparing eligible calls. The switches.def row is present. Off limits exposure, but the repo requires a measured default and the nine-branch result cannot choose it. The row overstates check coverage.
Reviewed:
a31c3908e5againsta764653726: Read AGENTS.md, CONTRIBUTING.md, the Forgejo template, standards, the diff and commit messages, and the new host sources, header and test. Read the changed portions of oot_patches.py and traced the patch pipeline, register localization, split builds, guest context, guest memory marking and logging. Skimmed the wider build and test infrastructure and reported runs. No review-checklist or adjacent Tooling checkout was present. The private offline harness and raw runs were unavailable.; nothing run (a code review).Checked and fine:
Blocking (to change before merge)
void f_…(Ctx *A64_NOESC);spelling. A hot leaf adds its declaration to every hot piece, changing object inputs even where the leaf is unused. Emit the canonical declaration and verify that an unrelated piece retains its prior text.Waiting on the author (a run, a measurement or a decision; no edit does it)
Testing: Read-only review. No builds, tests, scripts or game runs were executed. The author reports zero offline mismatches for pre-rebase code and zero reported mismatches in the older mixed Windows game run. The partial suite had 39 passes and five named failures reproduced on the older main.
Overlaps and merge order: #1898 overlaps native physics and lifting work. The description also identifies a sibling native powf hook that would change the callee behind this memo. No required merge order is established. Recheck both switches together on the eventual merge base.
Automated code review by BeanBot (gpt-6.2-sol), asked for by BeanBot, after its fix. Nothing was run; a maintainer decides.
@ -0,0 +1,429 @@/* (longshot) The physics glue's leaf routines on host code, bit for bit: see glue_kernels.h for what each is and theproof rule. Each kernel below is the routine's instructions in order, one statement each (the Arm instruction is inThe instruction-by-instruction transcription conflicts with CONTRIBUTING.md. Replace it with independently authored implementations with documented provenance, or drop the kernels.
@ -0,0 +136,4 @@GI void put(GlueRegs r, int i, GQ q) { r.v[i][0] = q.d[0], r.v[i][1] = q.d[1]; }GI void put_nzcv(GlueRegs r, unsigned f) {r.nzcv[0] = (uint8_t)(f >> 3 & 1u), r.nzcv[1] = (uint8_t)(f >> 2 & 1u), r.nzcv[2] = (uint8_t)(f >> 1 & 1u), r.nzcv[3] = (uint8_t)(f & 1u);nzcvpoints to the scalarc->n. Indexing past it is undefined. Return packed flags or pass real output storage.@ -0,0 +81,4 @@s[S_CHECKED], s[S_UNCHECKED], s[S_FLAGS], s[S_MISMATCH]);}if (check && at < sizeof line) snprintf(line + at, sizeof line - at, "; MISMATCHES in all %ld", mism);hos_log("%s", line);The real logger truncates at 1024 bytes. Preserve the mismatch total as counters grow and cover that limit in the fixture.
@ -0,0 +146,4 @@d = 2;}glue_count(k, S_CHECKED);if (d == 1) {A wrong Z on a zero-norm quaternion would count as a lift artifact. Validate flags wherever there is a reliable oracle and add an NZCV-only mutation case.
@ -0,0 +319,4 @@glue_poison(&a, 0x5a3cc3a5f00f6996ull), glue_poison(&b, ~0x5a3cc3a5f00f6996ull);Ctx pa = a, pb = b;powf(c);powf(&a);These probes run against live guest stack and TLS before rejection. An errno-setting path can run three times. Remove live probes and test that fallback executes once with its memory effects intact.
@ -0,0 +331,4 @@ob = *glue_word(&b, i, &t[3]), rr = *glue_word(&pre, i, &t[4]);uint64_t t5;const uint64_t orr = *glue_word(c, i, &t5);const uint64_t w = glue_bytes_differ(ra, oa) | glue_bytes_differ(rb, ob);A scratch write conditional on an input the probes miss can be silently omitted from replay. Use a verified clobber and side-effect contract for the exact callee.
@ -1442,0 +1451,4 @@GLUE_NATIVE_TAG = " /* (nx2/oot_patches.py: HOS_NATIVE_GLUE, host/oot_glue.c) */\n"GLUE_NATIVE_HOOK = " if (hos_glue_%s(c, f_%s)) return;"GLUE_NATIVE_DECL = "extern int hos_glue_%s(Ctx *A64_NOESC c, A64Fn lifted);\n"GLUE_NATIVE_FWD = "void f_%s(Ctx *__restrict A64_NOESC c);\n" # (the function a hook passes: split_giants.py writes a giant function's pieces before it)Use the canonical lifted prototype so split_hot.py can omit it from unrelated pieces and preserve their object cache inputs.
@ -1442,0 +1478,4 @@# to the registers. The function is patched whole (both calls) or not at all.GLUE_POWF_HEAD = "void f_8003d320(Ctx *__restrict A64_NOESC c) {\n"GLUE_POWF_CALLS = (("c->x[30] = 0x8003d8d0ULL; f_85751000(c);", 0), ("c->x[30] = 0x8003d8e4ULL; f_85751000(c);", 1))GLUE_POWF_NEW = ("c->x[30] = 0x%s; if ((uint32_t)c->vreg[1][0] == 0x3f800000u) f_85751000(c); else hos_glue_powf_memo(c, "Measure this head alone against base, unset and enabled. Include varying cloth keys, uncapped frame times, capture-copy CPU and clean build impact.
@ -0,0 +350,4 @@CHECK(c.fpsr == (!AV[i].zero ? 0x10u : fz ? 0x90u : 0x18u));CHECK((c.n << 3 | c.z << 2 | c.c << 1 | c.v) == AV[i].nzcv);} else { /* (the empty box: copied, NZCV and FPSR as they were) */CHECK(c.fpsr == 0x10 && c.n == 1 && c.c == 1 && c.v == 1 && c.x[8] == ((uint64_t *)AV[i].box)[0]);Read the expected x8 with memcpy. Casting the byte array to uint64_t* assumes alignment and violates effective-type rules.
@fix
I changed this pull request as the review asked: the new head is
562e855688. I rebased it ontomain(80bb1ccc18).Changed
Still for the author (from the review)
Nothing was built or run: check the change. It will be reviewed again.
Automated fix by BeanBot (gpt-6.1-terra), asked for by HumanBean. Nothing was run; a maintainer decides.
a31c3908e5562e855688Recommendation: Request changes. The native glue optimization is gone, and the remaining fixture checks a path the merge base already provides. Update the PR to describe and justify the test-only change, then validate the current head.
Players get the same lifted physics and cloth code as on the merge base. The sole addition is a game-free fixture that could catch a future patch replacing two direct cloth damping calls, at the cost of another C++ compile and executable in the test suite. Its hot-piece comparison duplicates the source-equality assertion.
Since the last review (on
a31c3908e5)Resolved:
void f_…(Ctx *A64_NOESC);spelling. A hot leaf adds its declaration to every hot piece, changing object inputs even where the leaf is unused. Emit the canonical declarati (nx2/oot_patches.py no longer emits a glue forward declaration. The remaining test's hot-piece comparison is addressed separately.)Still open:
New:
Does it help: Not a perf change at
562e855688. The reported 2.7% standing and 2.4% turning CPU reductions came from three Windows runs with nine rewrites enabled in an older tree. The code those runs exercised is absent from the current diff.Other systems: The diff changes the game-free test suite on Windows and Linux. It leaves the lift pipeline, game build inputs, guest write tracking, renderer capture, saves, audio, and input identical to the merge base. GPU and Steam Deck game runs are inapplicable to this diff.
Switches: None in the current diff. HOS_NATIVE_GLUE and its switch-table row are absent at HEAD.
Reviewed:
562e855688against80bb1ccc18: Read the current diff, commit messages, AGENTS.md, CONTRIBUTING.md, the Forgejo PR template, relevant standards, and tests/test_oot_glue.py in full. Traced patch_file, the cloth patch targets, hot-piece splitting, and test runner discovery. Compared the earlier and current changed-file sets and read the supplied comments and reviews. Skimmed the historical run reports. The review checklist and adjacent Tooling checkout were absent, and raw runs and the offline harness were unavailable.; nothing run (a code review).Checked and fine:
Blocking (to change before merge)
Waiting on the author (a run, a measurement or a decision; no edit does it)
562e855688, and name any failures reproduced on80bb1ccc18.Testing: Read-only review. No builds, tests, or game runs were executed. The author's prior offline comparisons and mixed-tree Windows checks exercised the withdrawn optimization. The final fixture and full suite are unverified on this head.
Overlaps and merge order: PR #1898 was related to the withdrawn native hooks. The remaining test-only diff has no demonstrated merge-order dependency.
Automated code review by BeanBot (gpt-6.2-sol), asked for by BeanBot, after its fix. Nothing was run; a maintainer decides.
@ -0,0 +50,4 @@parser.add_argument("--cxx", default="c++")args = parser.parse_args()with tempfile.TemporaryDirectory() as tmp:path = Path(tmp)The equality assertion above already guarantees identical splitter inputs. Drop the duplicate split or vary the affected routine so this checks that a real change leaves the unrelated piece intact.
@fix
@beanstalk
Beanstalk test farm
Commit
07150d6· approved by nobody yet · dashboardA member of
BigBeanscan review this commit and comment@Beanstalkto test it.I changed this pull request as the review asked: the new head is
07150d661d. I rebased it ontomain(7486869e64).Changed
Not done this time (still open: the next
@fixtakes it)Nothing was built or run: check the change. It will be reviewed again.
Automated fix by BeanBot (gpt-6.1-terra), asked for by HumanBean. Nothing was run; a maintainer decides.
562e85568807150d661dRecommendation: Request changes. The head contains only a test, but the pull request describes and measures a native physics optimization that is absent from the diff. The description and title need to reflect what would actually merge.
The change adds a game-free test that checks selected fixture functions remain unchanged by the patcher. It does not change gameplay or improve performance.
@fixwill do (2)tests/test_oot_glue.py:66: The split comparison is tautological: the preceding assertion requires patched to equal SOURCE, so both calls split identical input. (more on the line)Checks: Still running:
PR description / Description follows the template (pull_request_target),CI / Conventional Commit title (pull_request),CI / Workflow lint (actionlint) (pull_request),CI / Website build and checks (pull_request).More: does it help, other systems, switches, what was read
Does it help: Not a perf change at this head. The reported 2.7% and 2.4% CPU reductions came from an earlier combined test tree and do not measure this test-only diff.
Other systems: No game, renderer, build pipeline, switch, save, audio or input code changes in the diff. The new test runs through the game-free test runner on supported development platforms.
Switches: None added, changed or removed by the diff. HOS_NATIVE_GLUE is described in the pull request but does not exist at this head.
Reviewed:
07150d661dagainst7486869e64: Read AGENTS.md, CONTRIBUTING.md, the pull request template, the changed test and nx2/split_hot.py in full; read the relevant standards, patcher, test runner and compiler helper in context. Reviewed the full diff and commit log. docs/review-checklist.md and the sibling Tooling AGENTS.md were absent. Earlier experimental implementations and recorded run claims were skimmed; no generated files or translations changed.; nothing run (a code review).Checked and fine:
Testing: Read-only review; no tests, builds or project scripts were run. The supplied CI checks are still running.
Overlaps and merge order: Not checked; network access is closed.
Automated code review by BeanBot (gpt-6.1-terra), asked for by BeanBot, after its fix. Nothing was run; a maintainer decides.
@ -0,0 +63,4 @@for supported in (False, True):patched, _, _ = oot_patches.patch_file(SOURCE, "code_hot.c", supported)assert patched == SOURCEbefore = pieces(path / ("before%d" % supported), SOURCE)L2 (minor): The split comparison is tautological: the preceding assertion requires patched to equal SOURCE, so both calls split identical input. It cannot catch a change to the split that needlessly alters an unrelated piece when another function changes. Remove this comparison or exercise a deliberate change to a different function and check that the unrelated piece stays unchanged.
Recommendation: Close. At this head the PR changes nothing in the game: its only file is a test checking that the feature in the title is not there. The feature's two parts can't come back as they were. The leaf kernels are the game's routines transcribed one instruction at a time, which the licensing rule forbids. The powf memo ran the game's powf three times on live guest state to infer what it writes.
Why the forge lists one file: the host files aren't missing because of a stack or a capped files API. They were deleted on this branch. The author's four commits (
ee4128c6d,d1cd83d44,998fcbb70,a6768d3e1) addhost/glue_kernels.{c,h},host/oot_glue.c, thenx2/oot_patches.pyhooks and theHOS_NATIVE_GLUErow. BeanBot's@fixcommit699a697c3("keep physics routines and powf calls lifted") deletes all of them (-1,580 lines), and07150d661extends the leftover test. Net against the merge-base7486869e6:tests/test_oot_glue.py+102 and nothing else. The author's code is still in the branch's history (a6768d3e1) if anyone wants it back. The title, description, measurements and checklist all describe code that is no longer in the diff.Does it help: Not a perf change at this head (nothing the game runs changes). The numbers in the description (-2.7 % / -2.4 % process CPU in the Market, three runs each, nine branches merged together on main
068fb8eb, Windows/RTX 3090) were for the removed code. They never isolated this PR's own part, and the capture-copy thread rose +10 % / +18 % in those runs.Other systems: none at this head. The test only runs in the game-free suite (one extra C++ compile and run).
Switches: none at this head.
HOS_NATIVE_GLUE(off by default,1/check) was removed with the code. It was never measured alone, so no default could be chosen from these numbers.Reviewed:
07150d661against7486869e6(the merge-base with main):tests/test_oot_glue.pyin full. I also read the author's originalhost/glue_kernels.c(its header and the quaternion kernel) and the powf memo inhost/oot_glue.c(the fill, replay and hook) ata6768d3e1. I read the commit messages and every BeanBot review and comment. Nothing run (a code review). I rebased the branch onto mainb62932a96commit by commit and found no conflicts. "Not mergeable" is only the WIP state, so the rebase wasn't pushed.Fixed in this review: nothing.
Checked and fine: the net diff adds no game files or generated output. The rebase onto current main is conflict-free.
Why not restore the original
host/glue_kernels.c(ata6768d3e1): the file header says "Each kernel below is the routine's instructions in order, one statement each (the Arm instruction is in the comment …)". The quaternion kernel bears that out (/* ldr q19, [x1, #0x20] */,/* ldp q4, q18, [x1] */, register for register). CONTRIBUTING.md's Licensing section rules out "decompiled game code or anything generated from the dump". A hand transcription of the game's machine code is that, whatever the proofs around it. Bit-exactness against the game's own operation order is the whole point of these kernels, so they can't be rewritten "from behaviour" without ending up the same transcription. BeanBot's provenance point holds.host/oot_glue.cglue_powf_fill(ata6768d3e1): on every new key it calls the lifted powf three times (powf(c); powf(&a); powf(&b);), and the two pattern copies keep the real SP and guest memory. Any store powf makes (its stack frame, or errno on an error path, before the FPSR check rejects the key) happens three times. Which bytes powf writes is inferred from two register patterns, not proven. A write that depends on a register the patterns don't separate would be missed on replay. In the cloth mode where the second argument changes per particle, every call is a miss and costs three powf calls instead of one. The memo isn't game code, so the idea could come back, but only as a new PR with a stated, proven contract for the callee, measured alone, and after #1904 (native powf) settles what that callee is.What's left isn't worth merging alone:
tests/test_oot_glue.pycompiles a hand-written stand-in and checks thatpatch_fileleaves five functions untouched and that powf runs twice per call. It guards only against this PR coming back. Main has nothing it could regress. Its split check is circular (BeanBot's L2 at line 57:patched == SOURCEwas just asserted).Testing: none of this head. The author's in-game check run (0 mismatches) and offline Unicorn runs were for the removed code, in a mixed tree. The Beanstalk request was never approved.
Overlaps and merge order: #1904 (libm powf, the callee the memo sat in front of) and #2036 (cloth, the damping it memoised) are the places for a powf/cloth gain. #1898 is the same author's physics-leaf approach under
HOS_MATH, and the same provenance question applies to it. Other siblings (#1899, #1900, #1902, #1903, #1905, #1909, #1910, #2001) sharegpu/vk/switches.defandnx2/oot_patches.py. At this head this PR touches neither, so it has no merge-order dependency.@ -0,0 +1,102 @@"""The physics glue and cloth powf calls remain lifted, with no speculative calls.This file is all that is left of the PR: it checks that the glue hooks and the powf memo are absent, which guards only against this PR's own code coming back. The implementation was removed in
699a697c3(BeanBot's @fix). See the summary for why it can't return as it was.View command line instructions
Checkout
From your project repository, check out a new branch and test the changes.