WIP: perf(host): the effect emitter matrix, the sounds' angle fraction and the audio reverb on host code, bit for bit (HOS_NATIVE_FXTRIGGERS, HOS_NATIVE_REVERB, on by default) #1903
Labels
No labels
area/accessibility
area/audio
area/input
area/install
area/menu
area/renderer
area/saves
area/tooling
area/upscaling
beanstalk: approved
beanstalk: failed
beanstalk: needs approval
beanstalk: running
beanstalk: tested
ci: approved
good first issue
gpu/amd
gpu/igpu
gpu/intel
gpu/nvidia
help wanted
kind
bug
kind
chore
kind
docs
kind
feature
kind
performance
kind
question
needs measurements
platform/linux
platform/macos
platform/steam-deck
platform/windows
preview
priority
critical
priority
high
priority
low
regression
risk
high
risk
low
risk
medium
status
blocked
status
confirmed
status
duplicate
status
invalid
status
needs-info
status
wontfix
ci: approved
No milestone
No project
No assignees
4 participants
Notifications
Due date
No due date set.
Dependencies
No dependencies set
Reference
MagicBeanDev/Longshot!1903
Loading…
Reference in a new issue
No description provided.
Delete branch "snaildev99/Longshot:perf/native-fxtriggers"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Summary
Three of the game's effect and audio routines run on host code with the game's exact results, behind two new switches, both on by default (
=0gives the lifted code):HOS_NATIVE_FXTRIGGERS: the effect link's "set the emitter's matrix" (f_8012ba70, 88 instructions, a leaf: copies a matrix and makes its three axes unit length with their lengths) and a sound's angle against an axis as a fraction between two limits (f_806fe5f0, 338 instructions, a leaf of the per-sound update: one factor of a sound's volume by direction). Both only compute: they call nothing, take no lock and touch no queue, so they cannot change which effect or sound starts, or when. In the Market profile they hold 0.040 % and 0.019 % of the game's code time.HOS_NATIVE_REVERB, a switch of its own so that it can be listened to alone: the audio engine's reverb (f_8372a5a0, 610 instructions, on the audio library's low-priority worker, one queued buffer a call) for six channels, and for two and four channels, where the game calls a one-sample routine (f_8372af30) for each sample. 1.14 % of the game's code time in the Market and 0.59 % on a Kakariko and trail route, off the frame (the audio worker). The profiles saw only the six-channel path (1,359 samples in f_8372a5a0, none in f_8372af30).The goal is less CPU work for the same results. A call the host cannot prove exact runs the lifted routine with nothing written. The two leaves answer under the lift's own float gate (FPCR 0 or FZ alone, FPSR.IXC set) when every value is in a class where the PC's IEEE arithmetic gives Arm's bits (the matrix: every lane zero or of magnitude in [2^-31, 2^31); the angle: every float loaded and made zero or in [2^-63, 2^63), a hardware FMA). The reverb answers with FPCR 0 and FPSR.IXC set, every float loaded finite, no invalid operation or overflow on the host; it saves only the bytes the routine writes (about 19 KB for 240 samples) and puts them back on a decline.
How:
nx2/oot_patches.py(patch_fxtriggers, the supported build only) puts a gate first in each lifted function, one load and an untaken branch when off; each gated function is declared at file scope (FXT_FWD), becausenx2/split_giants.pymay put a gate in a piece written beforef_<addr>.host/oot_fxtriggers.canswers withhost/fxt_kernels.c(FRSQRTE from the Arm pseudocode's table, FRSQRTS exactly in binary64, the angle routine's instructions in order on a copy of its registers).host/oot_reverb.canswers withhost/reverb_kernel.c: one SSE4.1 / FMA operation per instruction in the game's order, every parameter read at the same point of each sample as the game reads it (nothing hoisted, so a setting changed mid-buffer is taken up at the sample the game takes it up at), the integer part (the slew, the taps' places and wraps) as the game's register arithmetic. For two and four channels it runs every sample but the last itself, then the translated f_8372af30 for the last sample, so the registers at the end are that routine's own.Read first:
host/oot_fxtriggers.c,host/fxt_kernels.c,host/oot_reverb.c,host/reverb_kernel.c,nx2/oot_patches.py(patch_fxtriggers),tests/test_native_fxtriggers.py,tests/test_native_reverb.py, the four rows ingpu/vk/switches.def.Testing
Tested: Windows 11, RTX 3090 (NVIDIA 610.88), Ryzen 9 5900X. In the game, with the effect switch's code (at 952d42d0) merged with other rewrite branches into a test tree on main
068fb8eb,HOS_NATIVE_FXTRIGGERS=check, the Market and Hyrule Field for about five minutes: the emitter matrix answered 59,009 of 65,536 calls on the host with noDIFFERENTline. The angle fraction has not been checked in the game yet. The reverb has not been checked in the game and has not been listened to in the game: it was on (HOS_NATIVE_REVERB=1, the six-channel version at 85e4a89a) in the CPU measurement below, where its start line is in every log, but no counter was logged, so whether it answered any call there is not known. The two- and four-channel extension and this head (the rebase onto current main and the upkeep commit) are not built or played.Exactness offline (Unicorn runs the game's own routine on the executable; a driver runs the real host files):
sqrtf; 0 blocks executed outside the routine. Eight broken versions are caught (102 to 2,301 calls of 4,000 each).A64_NOESC), so its harness was not re-run.Game-free checks at this head:
tests/test_native_fxtriggers.py(24 PASS lines: both gates once each, first in their functions, the supported build only; both kernels against models written apart from them on 120,000 calls each; the declines; the hooks off, on, check and on another build; the 4 GB line; page marks; the build without FMA) andtests/test_native_reverb.py(the gate; the kernel through the hook against an independent model on six sequences of 400 consecutive six-channel buffers, 270,386 samples, and 2,720 two- and four-channel buffers at guest bases 0, 64 GB and near 2^39, every byte after every call; 17 kinds of decline with every byte put back; check mode with a difference in a sample, a delay line and a register; the build without SSE4.1 / FMA) pass, and so dotests/test_switch_table.py,tests/test_patch_hooks_defined.pyand 36 of the other 41 tests that readnx2/oot_patches.pyornx2/entry_hooks.py. The other five fail here and on maina104182balike:test_dev_packs,test_high_fps_fixes,test_libc_hooks,test_lift_cache,test_song_note_cue.tests/run_all.pywas not run.Still to test:
HOS_NATIVE_FXTRIGGERS=0andHOS_NATIVE_REVERB=0against both unset (on; and each alone if there is time), and main against this branch with them unset; the game's CPU seconds with the frame rate capped at 60, and the frame times uncapped. Expecting less CPU time with the switches on, the same frame rate or better, no cost with them=0, the picture and the sound as without them (@Beanstalk linux).HOS_NATIVE_REVERB=checkwithHOS_NATIVE_FXTRIGGERS=0, indoors, in a cave and in the Market, expectingreverb:count lines with calls taken and noDIFFERENT("parameters changed, not compared" is not a fault; every call declined for the float mode would mean the audio worker runs with flush to zero and the kernel is never used); then=1withHOS_NATIVE_REVERB_LOG=1, listening after short loud sounds: the tail dies away at the same length and colour as with the switch unset, with no click, hiss or cut into silence, and no jump walking from one room to another while a tail sounds (by hand).HOS_NATIVE_FXTRIGGERS=check, expecting count lines for both routines and noDIFFERENT; then=1: effects that follow a moving owner (torch flames, dust at Link's feet, splashes, sword sparks) in the same place, direction and size, and a directional sound fading in and out at the same places when walking around it (by hand).Measurements
Verdict: less CPU work, measured together with the other rewrite branches, not yet alone, and not for the reverb. In a test tree that merges nine rewrite branches (named in the baseline below; the cloth and particle effect rewrites are not among them), at a 60 fps cap in the Market, the nine on used 2.7 % less CPU standing and 2.4 % less turning (the whole process) than main, and the hooks in with every switch off cost nothing measurable (-0.3 % / -0.2 %). The audio library's worker, where the reverb runs, used 0.038 to 0.039 cores in every variant: no saving from the reverb is claimed. The Test Farm request asks for this branch's own number.
Baseline: main
068fb8eb(#1802) against the test treetest/rewrites@ e614ef49 (main068fb8ebwith nine rewrite branches: the seven maths units, the maths library, animation, bounds, crowd, visibility, glue, the effect triggers with the six-channel reverb (this branch at 85e4a89a), and the solver; the cloth and particle effect rewrites, the two- and four-channel reverb and later pieces of other branches are not among them; this PR's two- and four-channel commit was not in it), lifted withA64_FPXKEEP=0so that the lift is main's apart from the hooks. Not this PR alone: the run-time switches were all on together. Main has moved 28 commits since068fb8eb; this branch was not measured on current main.Setup: Windows 11, Ryzen 9 5900X, RTX 3090, 3840x2160; the child Market, hands-free; frame rate capped at 60 (all 15 runs held 60); each run 30 s standing, then 30 s turning the camera; five variants interleaved over three rounds, the PC otherwise idle. CPU seconds per second of play (cores busy on average), from the processor's cycle counter; the median of three runs (the three in brackets for the process).
068fb8ebThe off figures are inside the runs' spread. The process includes the renderer's threads (about 1.4 cores standing), which the rewrites do not touch.
Offline (CPU time, not the game; Ryzen 9 5900X with other jobs running, absolute times 15 to 50 % apart between runs, the ratios within a run closer). "Main's form" for the two leaves and the six-channel reverb is the lift with registers as C locals that main writes for these functions, approximated offline from the stock lift (not main's lifter); for the two- and four-channel reverb it is main's real lift (
nx2/lift.py --standaloneon agit archiveof068fb8eb). A gated function is written in the older context form; "switch off" is that form plus the gate.For these routines the old context form and main's form measure the same within the noise: their time is in the float operations, not the registers. The reverb's declines: before any work (FPCR, channel count) the lifted routine's time; after the whole buffer (a result past s32 in its last sample, which ordinary audio does not reach) about +8 % on that buffer; check mode +8 to +12 % over the lifted routine. The switch-off figures of the leaves and the six-channel reverb were measured before the leaves' hooks were declared with
A64_NOESC(the upkeep commit); the game measurement above found no off cost at the process's scale.The switches
HOS_NATIVE_FXTRIGGERS(new, run time,kill, on by default):0off; unset or1on (host code where exact, the rest lifted);checkruns both, compares memory, registers and new FPSR flags, logsfxtriggers: f_<addr> DIFFERENT from the lifted routine: ...and keeps the lifted result.HOS_NATIVE_FXTRIGGERS_LOG=1(always in check mode) logsfxtriggers: <routine> <n> calls: taken <k>, lifted for ...every 65,536 calls.HOS_NATIVE_REVERB(new, run time,kill, on by default):0off; unset or1on;checkruns the kernel, puts the saved bytes back, runs the lifted routine and compares every byte the routine writes, the registers and FPSR:reverb: f_8372a5a0 DIFFERENT from the lifted routine: ...; a parameter changed by another thread between the two runs is counted "parameters changed, not compared", neither a difference nor a pass.HOS_NATIVE_REVERB_LOG=1logsreverb: f_8372a5a0 <n> calls: taken <k>, ...every 4,096 calls, also with=1(that is how a run without check mode shows whether the reverb answers). Another game build leaves both off with a log line. Both ship off: neither has been measured alone yet.Risks
What could break: the effect matrix places and sizes particle effects (a wrong one would show as an effect out of place); the angle fraction scales a sound's volume by direction (a step or click when moving around a source); the reverb shapes what is heard after sounds (a click, hiss or wrong tail). Offline every answered call is exact, after every call of long sequences for the reverb; in the game only the emitter matrix has been compared so far. The reverb takes no lock beyond the one its worker already holds, and one virtual method of the effect class can rewrite the mode and the two matrices on a live object without a lock (as the lifted routine reads them, the kernel reads each at the same point of each sample; a setting landing a few samples earlier or later is possible, as it is between two runs of the lifted routine). The reverb answers only with FPCR 0: the game's job worker threads were seen with FZ set in every call of the maths units; the audio worker's FPCR has not been observed, and if FZ is set there every reverb call declines (no gain, no harm). Open: in the CPU measurement the renderer's
mkw_vk capture copythread used 10 to 18 % more CPU (0.009 to 0.017 cores) in both variants with the nine on and in neither without; the native kernels mark what they write (HOS_WRITTEN, as these hooks do), possibly more coarsely than the lifted code; which rewrite causes it is not determined.It ships on: docs/dev/standards.md has a switch ship at its value from the first push, never off because it is new, and the measurement with the rewrite branches together says on.
=0on each gives main's lifted code. This PR's own measurement (the farm request) can still turn it off before it leaves WIP.Turning it off:
HOS_NATIVE_FXTRIGGERS=0,HOS_NATIVE_REVERB=0.After updating: the routines run on the host by default, with the game's exact results; saves,
longshot_options.iniand the caches do not change.Checklist
Ready for review
main; the title is a Conventional Commit.mise run testpasses (any failure that also fails onmainis named). (The 40 tests that read the changed files pass, the five failures are main's; the whole suite not run.)main, or Measurements says why there is nothing to measure. (Measured with nine rewrite branches merged; this branch alone not yet; the reverb's saving not seen.)third_party/untouched; nothing GPL-incompatible.When it applies
gpu/vk/switches.def. (The rows are there; on by default,=0gives main's lifted code; this PR's own measurement can still turn it off.)tests/that reads changed source is updated, not weakened. (tests/test_native_fxtriggers.pyandtests/test_native_reverb.py, new; their "no other function was gated" slices find the functions by their head lines.)Assisted-by:/Co-Authored-By:trailers. (Every commit buta2728964, the two- and four-channel extension, which has no trailer.)Ready to merge
main, measured again or a comment says why the numbers still hold.🤖 Generated with Claude Code
FXT_DECL declares hos_fxt_<addr>(Ctx *A64_NOESC c, A64Fn lifted), as REVERB_DECL already did: without it the compiler takes the lifted function's context for escaped at the gate's call, which costs the gated function with the switch off. Each gated function (f_8012ba70, f_806fe5f0, f_8372a5a0) is declared at file scope (FXT_FWD): nx2/split_giants.py writes a giant function's pieces before f_<addr>, so a gate in the first piece must find it declared. Only the declarations in the lifted code change; the host code does not. tests/test_native_fxtriggers.py and tests/test_native_reverb.py found the functions that bound their "no other function was gated" slices by "void f_<addr>(", which now first finds the declaration at the file's head; they look for the head line "void f_<addr>(Ctx *__restrict A64_NOESC c) {". The reverb test's unused import of re is gone. Co-Authored-By: Claude Opus 5.5 <[email protected]>Tested in game: In a test tree of nine merged rewrite branches on main
068fb8eb(Windows, RTX 3090), check mode in the Market and Hyrule Field: the emitter matrix answered 59,009 of 65,536 calls with no difference from the lifted routine; the angle fraction and the reverb have not been checked in the game and the reverb has not been listened to (it was on in a 15-run CPU measurement at a 60 fps cap, every run held 60, with no counter logged); this head itself is not built or played.@Beanstalk linux: this branch with HOS_NATIVE_FXTRIGGERS and HOS_NATIVE_REVERB unset against both =1 (the same build, with HOS_NATIVE_FXTRIGGERS_LOG=1 and HOS_NATIVE_REVERB_LOG=1 on the =1 side), and main against this branch with them unset, each in the child Market for a few minutes and in Hyrule Field by day for a few minutes. Measure the game's CPU seconds with the frame rate capped at 60 (on a PC that holds 60 there), standing and turning the camera, per thread if you can (the reverb runs on aal::LowPrioWorkerThread), and the frame times uncapped. Expect less CPU time with the switches at 1, the same frame rate or better, no extra CPU time with them unset against main, the picture as without them, and "fxtriggers: ... calls: taken" and "reverb: f_8372a5a0 ... calls:" lines in the log; please report those lines (they say how many calls the host answered).
@snaildev99 only members of
BigBeanscan start the farm.One of them can approve this request by reacting 👍 to it or to this comment.This request was ford263d0d, and the PR has changed since (0bf0056), so a 👍 no longer approves it. Comment@Beanstalkagain to ask for the new commit.Possibly related, for context:
Recommendation: Request changes. The default-off reverb gate has a recorded 7.6–16.3% cost in the two- and four-channel routines, and the emitter kernel has unsynchronized table initialization. The change also has no demonstrated in-game gain on its own.
This adds optional native implementations for effect placement, directional sound and reverb, with a lifted fallback and comparison modes. The potential CPU saving is not yet established in the game; the current implementation adds substantial instruction-level complexity and a measured cost even when reverb is off.
Does it help: Unproven. Three interleaved Windows runs of nine rewrite branches together showed 2.7% less process CPU while standing and 2.4% while turning, but do not isolate this PR; the audio worker showed no measurable saving and reverb call counts were not recorded. Offline, switch-off two- and four-channel reverb cost 7.6–16.3% more per sample than main's lift. This head has not been built or played.
Other systems: The changes affect particle placement, directional sound, reverb and guest-memory page marks. The emitter matrix was checked in a merged Windows test tree, but the angle and reverb were not checked in game or listened to. Linux, AMD, Steam Deck, frame pacing, the reported capture-copy CPU increase and the current head remain unverified. The PR reports extensive offline comparisons, including fallback and sequential reverb state, but those do not establish in-game behavior or performance.
Switches: HOS_NATIVE_FXTRIGGERS and HOS_NATIVE_REVERB are new runtime attempt switches, both off by default; each supports 1 and check. HOS_NATIVE_FXTRIGGERS_LOG and HOS_NATIVE_REVERB_LOG are new diagnostic switches, both off by default. All four have table rows. The attempt defaults have not been selected by the required platform measurements; off also does not avoid the measured reverb gate cost.
Reviewed:
d263d0deeeagainsta104182b40: Read AGENTS.md, CONTRIBUTING.md, the PR template, standards, the changes to switches and patch wiring, and the host hooks and kernels; skimmed the new test models and recorded results. Some long tool outputs were truncated, so the full changed code and tests were not reliably reviewed. docs/review-checklist.md and the adjacent Tooling/AGENTS.md were absent. No generated files or translations changed. The supplied comments and absence of earlier reviews were considered.; nothing run (a code review).Checked and fine:
Blocking (to change before merge)
Suggestions (the author may take or leave)
Testing: No code, scripts, builds or tests were run for this review. The PR reports passing targeted game-free checks and offline comparisons, with five tests also failing on main; the full suite and current-head game runs were not reported.
Overlaps and merge order: BeanBot identified Longshot PR #1644 as touching game reverb with a different scope. Its current diff and merge order could not be checked from the supplied material.
Automated code review by BeanBot (gpt-6.1-terra), on the pull request's opening. Nothing was run; a maintainer decides.
@ -0,0 +42,4 @@const uint32_t mid = (lo + hi) / 2;if ((uint64_t)a * mid * mid < (UINT64_C(1) << 28)) lo = mid; else hi = mid;}rsq_tab[idx] = (uint8_t)(((lo + 1) / 2) & 255);Two first callers can write this table concurrently. Writing identical bytes is still a C data race; please use a constant table or synchronized single-writer initialization and cover concurrent entry.
@ -1226,0 +1233,4 @@# The game's reverb (0x8372a5a0, the audio engine's low-priority worker) that host/oot_reverb.c answers natively, sample# for sample (HOS_NATIVE_REVERB, a switch of its own, off by default; host/reverb_kernel.c): the same kind of gateREVERB_GATES = ("8372a5a0",)REVERB_GATE = (" if (__builtin_expect(hos_reverb_on_%s, 0) && hos_reverb_%s(c, f_%s, f_8372af30)) return; "This gate changes the lifted form even with the switch off. The reported two- and four-channel measurements show a 7.6–16.3% per-sample regression in that default; please preserve the fast disabled path and measure it in game.
@fix
I changed this pull request as the review asked: the new head is
0bf0056778. I rebased it ontomain(a764653726).Conflicts resolved
Changed
Nothing was built or run: check the change. It will be reviewed again.
Automated fix by BeanBot (gpt-6.1-terra), asked for by HumanBean. Nothing was run; a maintainer decides.
d263d0deee0bf0056778Recommendation: Request changes. Native reverb skips the lifted loop’s virtual-core checkpoints and can overwrite a pending scheduler request. The effect hooks also race during first-call setup, and the PR’s own in-game CPU gain and default-off cost are still unverified.
Adds optional CPU shortcuts for particle emitter placement, directional sound volume and reverb, with lifted fallbacks and comparison modes. The reported offline results show substantial routine-level speedups and broad numerical coverage. The tradeoff is three instruction-level implementations plus reverb rollback, with no isolated player-facing gain yet.
Does it help: Unproven. Three interleaved Windows runs of a nine-branch tree on older main showed 2.7% less process CPU standing and 2.4% less turning in the capped Market. The audio worker stayed at 0.038–0.039 cores, and the run recorded no reverb take counts. The older offline two- and four-channel paths cost 7.6–16.3% more per sample with reverb disabled. The current wrapper preserves eligibility for register locals, but there are no post-fix results. Compare
0bf0056778alone againsta764653726with both switches off and each on, including take counts, thread CPU, average FPS, 1% lows and median and p99 frame times.Other systems: Emitter placement, directional volume and audio tails depend on the changed paths. Native reverb also changes virtual-core scheduling and guest-memory write tracking. The nine-branch run saw capture-copy CPU rise 10–18% (0.009–0.017 cores), with attribution unresolved. Windows NVIDIA emitter evidence comes from an older merged tree. Current-head Windows and Linux builds, AMD and Steam Deck coverage, audio pacing and clean-build time and size are missing. The diff adds no save, settings-menu, input or shader changes. The lifter patch is included in the existing lift-key mechanism.
Switches: HOS_NATIVE_FXTRIGGERS and HOS_NATIVE_REVERB are new runtime attempt switches, off by default, with 1 and check modes. Off is the prudent value given the open correctness issues, but the project requires measured defaults and the default path still changes lifted code. HOS_NATIVE_FXTRIGGERS_LOG and HOS_NATIVE_REVERB_LOG are new diagnostics, off by default and implicit in check mode. That default is appropriate. All four have reference rows.
Reviewed:
0bf0056778againsta764653726: Read the full changed host kernels, hooks and headers, both test fixtures and drivers, and the changed switch and patch sections. Read AGENTS.md, CONTRIBUTING.md, the Forgejo template and developer standards. Traced the surrounding lifter, local-register, guest-memory, write-mark and virtual-core code. Read the supplied comments and stale review. Skimmed the profile entries and reported run summaries. Raw run logs and generated game output were unavailable. No local docs/review-checklist.md or adjacent Tooling/AGENTS.md was present.; nothing run (a code review).Checked and fine:
Blocking (to change before merge)
Waiting on the author (a run, a measurement or a decision; no edit does it)
Testing: Read-only review. No build, fixture or game run was executed. The supplied offline comparisons and mutation checks predate the current head, and the fixtures use stand-ins instead of compiling the generated entry paths. The author reports five failures also present on the previously tested main commit. The full suite and current-head CI are unverified.
Overlaps and merge order: PR #1644 is related audio work. The checkout already has a separate I3DL2 submix implementation in host/nn_audio.c, and this diff changes the guest reverb path. No required merge order was established.
Automated code review by BeanBot (gpt-6.2-sol), asked for by BeanBot, after its fix. Nothing was run; a maintainer decides.
@ -0,0 +119,4 @@}int hos_fxt_8012ba70(Ctx *c, A64Fn lifted) {if (fxt_mode < 0) fxt_start();Reading fxt_mode can bypass the acquire while another worker publishes it. Use atomic gate loads and synchronized configuration publication in both hooks.
@ -0,0 +192,4 @@int ok = m.table && m.mono;for (int i = 0; i < 6; i++) m.vec[i] = rv_range(vec_at[i], 16), ok = ok && m.vec[i];if (!ok) return rv_count(WHY_RANGE), 0;if (rv_save_n < pl.bytes) {These allocations outlive the thread that owns their pointers. Give the buffers teardown ownership and allocate the second buffer only for check mode.
@ -0,0 +293,4 @@}const uint8_t *k = rv_save[1];int dm = 0, first = -1;for (int i = 0; i < pl.n - 1; i++) {This stops before the saved frame segment. A frame-only mismatch counts as a pass. Compare defined frame bytes and test that case.
@ -0,0 +315,4 @@const int on[3] = {1, o[0x300] && o[0x2a1] != 1, o[0x450] && o[0x2a1] != 1};for (int i = 0; i < pl.n - 1; i++) if (on[pl.s[i].stage]) HOS_WRITTEN(pl.s[i].p, pl.s[i].n);}*c = got;Whole-context assignment can clear a concurrent virtual-core poll request. Preserve the live scheduler field and restore the lifted loop’s checkpoints on the native path.
@ -0,0 +49,4 @@the length in w17 and length - 1 in w18), then the next line's target into w4 */#define TAPS(V, POFF, TOFF) do { \x3 = ld64(o + (POFF)); \V = LDS(H(x3 + (SX(x4) << 2))); \A valid write place does not bound this tap. An excessive slewed distance can leave a negative index after wrapping. Validate the tap range before loading.
@ -0,0 +128,4 @@st32(H(x12), 0), st32(H(x16), 0);}v6 = I2F(ld32(H(x10)));x17 = ld32(o + 0x2a4);The live mode load has no defined synchronization with the documented setter. Establish a protocol and cover a change during the buffer.
@ -0,0 +482,4 @@if (_mm_getcsr() & 0x0d) { status = REVERB_INVALID; break; }}}const unsigned flags = _mm_getcsr();The fallback depends on FP exception side effects. Enforce their ordering inside the MXCSR window and test optimized code with otherwise-dead arithmetic.
@ -1246,0 +1284,4 @@at = _function(out, head)end = at[1]body = out[start:end].replace(head, "static __attribute__((noinline)) void reverb_lifted_%s(Ctx *__restrict A64_NOESC c) {\n" % addr, 1)wrapper = "\n" + head + gate + " reverb_lifted_%s(c);\n}\n" % addrMeasure this wrapper against current main with reverb unset. The earlier two- and four-channel default-off measurements showed 7.6–16.3% extra time per sample.
@ -0,0 +680,4 @@(tmp / f).write_text((ROOT / "host" / f).read_text(encoding="utf-8"), encoding="utf-8")simd = ["-msse4.1", "-mfma"]if subprocess.run([cc, "-x", "c"] + simd + ["-fsyntax-only", str(tmp / "reverb_kernel.c")], capture_output=True).returncode:simd = [] # (not an x86 host: the kernel is not built, the hook declines)Compiler acceptance does not establish CPU support. Guard execution of both new FMA fixtures on supported pre-FMA machines.
@fix
@Beanstalk
Beanstalk test farm · an earlier run: commit
16a54b2(the newer results are in a comment below)Commit
16a54b2· approved by HumanBean (reviewed) · dashboard⚠️ Note: Check mode retains the lifted result and adds comparison overhead, so its performance is not representative of native mode; periodic call counts may not appear during these short runs.
⚠️ Note: The farm cannot listen for audio artifacts or establish which reverb channel paths were exercised; check-mode logs also do not cover every possible state or scheduler race.
Started at 07:34 UTC.
Estimate: done in about 2 h 45 min, from the queue and the testers online now.
anyWhat was tested
What will be tested · AI plan (
gpt-6.1-terra)The plan compares main and the PR in the Market and Hyrule Field, with the new routines off and in combinations of native and check modes. It measures frame time, process CPU, load time and images, and fails on crashes, material regressions or logged native-versus-lifted differences. Gameplay runs take about 50 minutes per tester, plus timed lift builds.
1orcheck(PR only); HOS_NATIVE_FXTRIGGERS1orcheck(PR only).market-entrance, then move the stick, measure market for 20 s (after 5 s warm-up), take the picture market to compare with main, teleport to Hyrule Field, measure field for 20 s (after 5 s warm-up), take the picture field to compare with main; 1 run on each side.market-entrance, then move the stick, measure market for 20 s (after 5 s warm-up), take the picture market to compare with main, teleport to Hyrule Field, measure field for 20 s (after 5 s warm-up), take the picture field to compare with main; 1 run on each side for each of 4 cells (4 game starts a run).Every scenario and matrix cell
HOS_NATIVE_REVERB=1 (PR only), HOS_NATIVE_FXTRIGGERS=1 (PR only)HOS_NATIVE_REVERB=1 (PR only), HOS_NATIVE_FXTRIGGERS=check (PR only)HOS_NATIVE_REVERB=check (PR only), HOS_NATIVE_FXTRIGGERS=1 (PR only)HOS_NATIVE_REVERB=check (PR only), HOS_NATIVE_FXTRIGGERS=check (PR only)I changed this pull request where an edit could (the rest needs runs, measurements or decisions): the new head is
20694421db. I rebased it ontomain(7486869e64).Conflicts resolved
tests/test_native_fxtriggers.py,tests/test_native_reverb.pyChanged
Not done this time (still open: the next
@fixtakes it)Left for the author
28941d0796: The author must add the appropriate Assisted-by or Co-Authored-By trailer to that commit by rewriting it. This task prohibits changing git history.Still for the author (no edit does these)
28941d0796has no AI-assistance trailer, although the PR says it received AI assistance. Add the required Assisted-by or Co-Authored-By trailer to that commit.Nothing was built or run: check the change. It will be reviewed again.
Automated fix by BeanBot (gpt-6.1-terra), asked for by HumanBean. Nothing was run; a maintainer decides.
16a54b23b720694421dbBeanstalk test farm
Commit
e3024b8· approved by nobody yet · dashboardA member of
BigBeanscan review this commit and comment@Beanstalkto test it.Recommendation: Request changes. The new bitmap test fails against the current build script, and the tests include sampled lifted game instructions prohibited by the repository rules. The performance gain and in-game correctness of this head also remain unproven.
Adds opt-in host implementations for two effect and sound calculations and for reverb. The offline comparisons are extensive, but the added code and maintenance cost are not yet justified by a measured gain for this branch.
@fixwill do (5)tests/test_mark_bitmap_host.py:61: This test requires MARK_BITMAP as an unconditional top-level term in every host compile job, but build.py now supplies it through a conditional expression for selected files. (more on the line)tests/test_native_fxtriggers.py:59: The LIFTED fixtures contain sampled lifted instructions from the game; the reverb fixture contains another such instruction. (more on the line)host/oot_reverb.c:218: Reuse bounded scratch storage with a thread-lifetime cleanup, or otherwise remove the per-buffer allocation.For the author (2) (a run, a measurement or a decision)
host/oot_reverb.c:218: The final commit replaces reusable scratch storage with a malloc and free on every accepted audio buffer, after the reported measurements. (more on the line)More: does it help, other systems, switches, what was read
Does it help: Unproven. The reported 2.7% and 2.4% process CPU reductions compare main with nine rewrite branches enabled together, not this PR alone; the audio worker remained at 0.038–0.039 cores. The offline speedups do not establish an in-game gain, and this head was not played.
Other systems: The hooks affect effect placement, directional audio, the audio worker and GPU page tracking. The bitmap build-flag change also affects existing host hooks with both new switches off. Windows testing covered only an earlier merged tree; Linux, Steam Deck, AMD and the current head remain untested.
Switches: HOS_NATIVE_FXTRIGGERS and HOS_NATIVE_REVERB default off; their LOG switches also default off. All four have table rows. The safe performance default cannot yet be chosen from the available measurements. RX_MARK_BITMAP is unchanged in name and default, but its flag now reaches selected host files.
Reviewed:
20694421dbagainst7486869e64: Read the repository rules, standards, PR template, commits, diff, host hooks, kernel interfaces, build and patch changes, and relevant page-marking code. Skimmed portions of the long kernel implementations and test fixtures; no generated files or recorded runs were added. The network was unavailable, so overlapping PRs could not be checked.; nothing run (a code review).Checked and fine:
Testing: Read-only review; no project code, scripts, builds or tests were run. The test failure is determined from the test's AST check and the build expression.
Overlaps and merge order: Unknown; the network was unavailable.
Automated code review by BeanBot (gpt-6.1-terra), asked for by BeanBot, after its fix. Nothing was run; a maintainer decides.
@ -0,0 +215,4 @@int ok = m.table && m.mono;for (int i = 0; i < 6; i++) m.vec[i] = rv_range(vec_at[i], 16), ok = ok && m.vec[i];if (!ok) return rv_count(WHY_RANGE), 0;uint8_t *save = (uint8_t *)malloc(pl.bytes * (rv_mode == RV_CHECK ? 2 : 1));L4 (minor): The final commit replaces reusable scratch storage with a malloc and free on every accepted audio buffer, after the reported measurements. Measure this head against the base and switches-off build, including audio-worker time and frame pacing, and establish whether the reverb actually takes calls.
@ -0,0 +58,4 @@tup = call.args[0] if call.args else Noneif not isinstance(tup, ast.Tuple) or len(tup.elts) < 4:bad.append("line %d: a host job that is not (cc, source, object, flags, ...)" % call.lineno)elif not takes_mark_bitmap(tup.elts[3]):L1: This test requires MARK_BITMAP as an unconditional top-level term in every host compile job, but build.py now supplies it through a conditional expression for selected files. The new test therefore reports a failure on this head. Update the test to verify the intended selected-file policy and catch omissions, or restore an unconditional flag.
@ -0,0 +56,4 @@LIFTED = ('#include "a64.h"\n'"void f_8012ba70(Ctx *__restrict A64_NOESC c) {\n"" c->x[11]=recomp_load64(c,c->x[1] + 8);\n"L2: The LIFTED fixtures contain sampled lifted instructions from the game; the reverb fixture contains another such instruction. CONTRIBUTING.md prohibits committing output made from the game, including lifted code. Replace these bodies with arbitrary synthetic statements that exercise the patch without reproducing game output.
WIP: perf(host): the effect emitter matrix, the sounds' angle fraction and the audio reverb on host code, bit for bit (HOS_NATIVE_FXTRIGGERS, HOS_NATIVE_REVERB, off by default)to WIP: perf(host): the effect emitter matrix, the sounds' angle fraction and the audio reverb on host code, bit for bit (HOS_NATIVE_FXTRIGGERS, HOS_NATIVE_REVERB, on by default)HOS_NATIVE_FXTRIGGERSandHOS_NATIVE_REVERBare now on by default (e3024b87, on top of BeanBot's20694421), as docs/dev/standards.md asks: a switch ships at its value from the first push, not off because it is new.HOS_NATIVE_FXTRIGGERS=0/HOS_NATIVE_REVERB=0gives main's lifted code. The row's kind iskill, unset reads as on (env_on_or), and the test runs it unset and=1(on),=0and=off(off) andcheck; it fails when the old read is put back. Title and description updated to match.Recommendation: Request changes. The two kernels are the game's routines carried over one Arm instruction at a time, with the disassembly's mnemonics and the game's code addresses in the comments. CONTRIBUTING.md's Licensing section forbids decompiled game code, so neither kernel can merge as written. There are also three things to fix in the code itself: the reverb hook loses virtual-core requests, the PR's own bitmap test fails at this head, and the gate fixtures are lines of the lifted game.
What it does: the emitter matrix, a sound's direction factor and the audio worker's reverb run as host code with the game's results. It has no measured gain of its own yet, and the reverb's gain looks like zero.
Does it help: Unproven. The only in-game number (-2.7 % process CPU standing, -2.4 % turning, three runs each, Windows / RTX 3090, 60 fps cap) comes from nine rewrite branches together, on main
068fb8eband not this head. The audio worker, where the reverb runs, stayed at 0.038 to 0.039 cores in every variant, and noreverb:taken count was logged. Note that 1.14 % of the game's code time would be more than that whole thread's 0.039 cores, so it's quite possible every reverb call declined, for example on FPCR. The offline speed-ups (4 to 13 times per call) don't show a gain in the game.Other systems:
*c = got, below).A64_MARK_BITMAP, so the existing hooks'recomp_store*(hos_main.c, hos_wide_ui.c, oot_heap_retry.c, oot_shadow_fit.c, oot_text_size.c) mark only GPU pages, as the lifted stores already do. This applies with both switches off too.Switches:
HOS_NATIVE_FXTRIGGERS(kill, on by default) andHOS_NATIVE_REVERB(kill, on by default): shipping on matches docs/dev/standards.md (never off because new). The gain is Unproven, so I left the defaults alone. The Test Farm run of this branch alone decides them before WIP comes off.HOS_NATIVE_FXTRIGGERS_LOG/HOS_NATIVE_REVERB_LOG(diag, off): right.e3024b87.Reviewed:
e3024b87eagainst7486869e6(its base). Read in full: host/oot_fxtriggers.c, host/fxt_kernels.c/.h, host/oot_reverb.c, host/reverb_kernel.c/.h, the nx2/oot_patches.py and nx2/build.py hunks, gpu/vk/switches.def, tests/test_mark_bitmap_host.py, tests/run_all.py. Read in part: the two native tests' fixtures and models. Nothing was run; this is a code review. The commits rebase onto mainb62932a96commit by commit with no conflict.Fixed in this review: nothing pushed: the push to snaildev99's fork was refused here. The three fixes are below as suggestions, and the rebase onto
b62932a96is clean.Checked and fine:
started's acquire/release coversfxt_mode,fxt_log,rv_modeandrv_log, and the gates are atomic loads. BeanBot's race is resolved.rsq_tabuses single-writer initialisation with release/acquire.H()isA64_MEM + aasA64_PTR), so this is no new fault.-ffast-mathand contraction guards.Blocking (to change before merge)
Licensing: the kernels are transcriptions of the game's code. CONTRIBUTING.md's Licensing section allows the host's replacements to be written only from facts about how the game behaves. These files are written statement for statement from the disassembly:
host/reverb_kernel.c:1-2says so itself: "The game's instructions in their order; the comments name the game's addresses".host/reverb_kernel.h:30: "one host vector operation an instruction".host/reverb_kernel.c:50-66(TAPS) and:98-125keep the game's register allocation (x3, x4, x5, x17, x18, v6, v7, v16..v24) and branch targets (0x8372a620,0x8372ad80,0x8372ad9c).host/reverb_kernel.c:233:SUBS w2, w2, #2; CSEL w18, w18, w2, lt.host/fxt_kernels.c:195: "the routine's instructions in their order on a copy of its registers".host/fxt_kernels.c:244-313: the cofactor chain with the game's register numbers, at806fe5f0and806fe6cc.host/fxt_kernels.c:291(FMSUB) and:339(FCMP s3, #0.0; FCCMP s1, s4, #0, eq; CSET w9, eq).host/fxt_kernels.c:155-158(FMUL .4s,EXT #8,FADD .2s,FADDP) and thex_fromtable at:142.host/oot_reverb.c:260-296rebuilds the game's exact register and frame state (x8..x19, the saved-pair layout).This is the same problem as on 1911. What would pass: kernels written from what the routines do (normalise three axes, the cone fraction, a feedback-delay reverb), in your own words and citing public sources, that return only the results the callers use. Keep the bit-for-bit check against the lifted routine in check mode, but drop the register-for-register reproduction.
host/oot_reverb.c:344(and:304in check mode):*c = got/*c = entrywrite the whole context back as it was at entry. That includespoll, whichvc_wake_holder(host/hos_thread.c:1902) sets on another thread to ask this thread to give up its virtual core. A request posted while a buffer runs is lost, and the waiter stalls until the worker's next SVC. Fix: copy back only x, vreg, NZCV and FPSR (suggested below).nx2/build.py:1898: BeanBot's20694421narrowedMARK_BITMAPto seven named host files.tests/test_mark_bitmap_host.py(this PR's) requires it unconditionally and fails at this head. A new hook usingrecomp_store*would also silently mark every page. Every sibling (1898, 1899, 1900, 1902, 1904, 1909, 1910) has the unconditionalMARK_BITMAP +. Put that back.tests/test_native_fxtriggers.py:59,64andtests/test_native_reverb.py:48,691: the fixture bodies are the routines' first lifted lines (recomp_load32(c,c->x[1] + 36)is the kernel's first load,mtx + 0x24). "The lifted C" is on the banned list. Use stand-in statements (for examplec->x[9]=c->x[2] ^ 5;). The tests only check where the gate goes.The description: the Switches paragraph says "Both ship off" (see Switches). The checklist says one commit lacks a trailer, but
a2728964/ 7ee2adf3 still has none.Suggestions
b62932a96but couldn't push):regs_to(Ctx *c, const Ctx *from)that copiesx,vreg,n/z/c/vandfpsr, used in place of both whole-struct assignments;MARK_BITMAP +with the "Every host file" comment;host/oot_reverb.c:221: the per-buffermalloc/freecame after the measurements. A per-thread buffer that grows and is freed at thread exit would avoid a heap call a buffer. Minor next to the kernel.host/reverb_kernel.c:95-96,485: the declines rely on MXCSR flags from intrinsics. Clang and GCC don't model FP exceptions without-ffp-exception-behavior=strict/#pragma STDC FENV_ACCESS ON, so in principle an operation could move past the_mm_getcsr. Worth the pragma for this file once rewritten.pl.n - 1). That's harmless for the result (the frame is below SP), but say so in the comment.Testing: no run at this head. The emitter matrix was checked in the game in an older merged tree (Windows, RTX 3090). The angle fraction and the reverb have not been checked or listened to in the game. Missing: this branch alone against main with each switch
=0and unset (with the_LOGcounts), the reverb listening check, and Linux. The Test Farm request expired with the head (the sticky comment wants a BigBean@Beanstalkone3024b8).Overlaps and merge order:
MARK_BITMAPhunk andtests/test_mark_bitmap_host.py, so whichever merges first carries it and the rest drop it on rebase. Each also adds rows to gpu/vk/switches.def and hooks in nx2/oot_patches.py (textual neighbours only).@ -0,0 +192,4 @@M(7, -S(1));W(24, S(16) / S(5));W(5, S(4) * S(24));W(4, -(S(6) * S(2)));Licensing (blocking): "the routine's instructions in their order on a copy of its registers". Lines 244-313 and 339 reproduce the disassembly (register numbers, FMSUB, FCMP/FCCMP/CSET). Write it from what it computes (the inverse, the angle, the cone fraction) instead.
@ -0,0 +341,4 @@for (int i = 0; i < pl.n - 1; i++)if (on[pl.s[i].stage]) reverb_written(pl.s[i].p, pl.s[i].n);}*c = got;This writes back the whole Ctx from entry,
pollincluded. A virtual-core request (vc_wake_holder) posted while the buffer ran is lost. Copy back only x, vreg, NZCV and FPSR. Line 304 (check mode) has the same problem.@ -0,0 +1,520 @@/* (longshot) HOS_NATIVE_REVERB's kernel: see reverb_kernel.h. The game's instructions in their order; the commentsLicensing (blocking): by its own comment this file is the game's routine instruction by instruction, with the game's registers and code addresses. CONTRIBUTING.md forbids decompiled game code. Rewrite it from behaviour (a feedback-delay reverb), in your own words, and keep the bit-for-bit check in check mode.
@ -0,0 +230,4 @@x2 = S(x2) < 1 ? W(x17) : W(x2);x3 = x1 + (SX(x2) << 2);{const int lt = S(x2) < 2; /* SUBS w2, w2, #2; CSEL w18, w18, w2, lt */Licensing:
SUBS w2, w2, #2; CSEL w18, w18, w2, ltis the disassembly carried over as-is.@ -1894,6 +1895,8 @@ def main():(["-I" + os.path.join(FFMPEG, "include")] if FFMPEG else []) +(FFMPEG_LX[0] if LINUX and FFMPEG_LX else []) +([stamp_def] if os.path.basename(s) == "version.c" else []) +(MARK_BITMAP if os.path.basename(s) in ("oot_fxtriggers.c", "oot_reverb.c", "hos_main.c",tests/test_mark_bitmap_host.pyfails with this conditional, and a new hook usingrecomp_store*would miss the bitmap. Put back the unconditionalMARK_BITMAP +, as every sibling branch has it.@ -0,0 +61,4 @@"void f_8012bbd0(Ctx *__restrict A64_NOESC c) {\n""}\n""void f_806fe5f0(Ctx *__restrict A64_NOESC c) {\n"" c->x[8]=recomp_load32(c,c->x[1] + 36);\n"This and line 59 (and test_native_reverb.py:48) are the routines' own first lifted lines. Use stand-in statements: the test only checks where the gate goes.
View command line instructions
Checkout
From your project repository, check out a new branch and test the changes.