WIP: perf(host): the Havok cloth's solver passes, collisions and skinning on host code, bit for bit the game's (HOS_NATIVE_CLOTH) #2036
No reviewers
Labels
No labels
area/accessibility
area/audio
area/input
area/install
area/menu
area/renderer
area/saves
area/tooling
area/upscaling
beanstalk: approved
beanstalk: failed
beanstalk: needs approval
beanstalk: running
beanstalk: tested
ci: approved
good first issue
gpu/amd
gpu/igpu
gpu/intel
gpu/nvidia
help wanted
kind
bug
kind
chore
kind
docs
kind
feature
kind
performance
kind
question
needs measurements
platform/linux
platform/macos
platform/steam-deck
platform/windows
preview
priority
critical
priority
high
priority
low
regression
risk
high
risk
low
risk
medium
status
blocked
status
confirmed
status
duplicate
status
invalid
status
needs-info
status
wontfix
ci: approved
No milestone
No project
No assignees
5 participants
Notifications
Due date
No due date set.
Dependencies
No dependencies set
Reference
MagicBeanDev/Longshot!2036
Loading…
Reference in a new issue
No description provided.
Delete branch "snaildev99/Longshot:perf/native-cloth-pr"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Summary
Fourteen pieces of the game's Havok cloth (the townspeople's clothes, Link's cap) run as native host code with
HOS_NATIVE_CLOTH=1, giving the game's results bit for bit: every byte written, FPSR, NZCV where the callers read it, and every register the routine leaves. A call or a record the host cannot prove exact runs the game's translated routine, with nothing written by the host first. Off unless the switch is set.The goal is less CPU time for the same frame. Cloth is about 11.8 % of game code in the child Market and 10.6 % on Death Mountain Trail (sampling profiles), and runs on the update workers while all six are busy. In the game, on one PC, the same build with the switch off against on: cloth's thread time 7.74 -> 2.83 ms a frame (-63.4 %), the whole process's CPU 103.31 -> 98.64 ms a frame (-4.5 %), every "on" run below every "off" run (Measurements). The frame rate stayed at about 60 in play either way: the expected result is less CPU time, and more frames only where the CPU is the limit.
The pieces and the guest routines they answer (addresses only; what each routine does is read from its instructions):
f_81c7f880f_81c86720f_81c7d2e0f_81c7bd00f_81c88940f_81b38800f_81b5a3c0f_81cdff40f_81c66a60f_81b8c000f_81b8c000f_81c1a240f_81db9a40f_81ce6500What stays translated:
f_81b3a558, the sphere collision's tail, the capsule collision's modes 1 and 2 (no call in play: every capsule and sphere call counted in the Market was mode 0), Object Space Deform's types 4-7 and its other loops (0 % and 0.1 % of its work in play), the passes' timer scopes, and everything above the solve.How:
nx2/oot_patches.py(patch_cloth_native, in the supported build only) puts a hook first in the four leaves and, in the other nine, a hook at the loop's entry label and one at its exit label. Each hook answers the records it can prove and hands the rest to the translated loop at the record where it stopped, with the state the game would have there; a function whose places are not all found once is left unpatched, with a WARNING.host/oot_cloth.cholds the hooks, the gate (the translator's ownRECOMP_FPX_OPEN: FPCR at its default but FZ, FPSR.IXC already set; memory throughhos_guest_range; declines for stores that could reach what a routine reads once), the page marks (HOS_WRITTEN) and the counts. The kernels arehost/cloth_*.cpp, C++ without its runtime (HOST_CXXinnx2/build.py: no exceptions, no RTTI, linked as C). Each one is the routine's float operations in the game's order under a class rule: where every operand is zero or in a range in which no result can be tiny, infinite or NaN, IEEE gives the Arm's bits and SSE runs it; the unused fourth lane of each vector (which the game fills with NaNs and huge values) is done by the Arm's own rules (host/cloth_wlane.hpp: the NaN it picks, default NaN, FZ's flush, the flags); and Prepare Matrices, the Verlet loop, the particle boxes, the cull and the two skinning loops do any value outside the class lane by lane with the Arm's operations, so they take every value. FRSQRTE is Arm's 8-bit table, FRSQRTS fused. The vector kernels need SSE4.1; FMA only speeds them up; without SSE4.1 a scalar kernel gives the same bits.Nothing made from the game is in the source tree. Two pieces run on tables read from the routines' instructions: the capsule and sphere collisions' record of 16 particles (its stores and its whole exit state, as tables and straight-line code) and where Prepare Matrices leaves each value. The build makes them:
nx2/cloth_gen.pyandnx2/cloth_sym.py(one block innx2/build.py) read the image the player's own lift wrote (<work>/sa/image/image.bin) with capstone, as the lifter does, and write<gen>/cloth/(cloth_lastrec_tables.h,cloth_lastrec_code.inc,cloth_pm_tables.h,cloth_gen.h), whichhost/includes. Before a piece is used the build checks that the reader knows every instruction on the paths it follows, that the last record's tables compose (the full path's symbolic run against the header's table and each group's, for all 1,411 contact sets of one or two groups and 2,000 random sets of three or four), that Prepare Matrices computes the very graph the kernel computes, and that the made files are the ones proven offline (by their SHA-256: a digest names the files and says nothing of the game). A check that fails turns that piece off (its hooks decline, the translated code runs), and the build prints why:cloth_gen: WARNING: ...: that piece runs lifted; the game's start line says so too. The cost: about 28.5 s on the first build (27 s of it the composition proof), then 0.3 s (a stamp: the image's digest, the two scripts,host/cloth_matrices.cpp, capstone's and Python's versions). The guest addresses are the only game-specific values in the tree; the unit test that needs one of the image's words reads it from the image when it runs (tests/cloth_tables.py,LONGSHOT_CLOTH_IMAGE, skipped without an image).It stacks with #1384 (the cloth's time step below 60 fps): that patch changes the simulate setup
f_81b3aa48, which this does not hook; the pieces sit below it and get the time step as an input, as the game's routines do.Read first:
host/oot_cloth.c(its head comment: every piece, the switch's modes, what check mode compares),host/cloth_bend.cpp(the pattern of every kernel),host/cloth_wlane.hpp, theCLOTH_NATIVEblock ofnx2/oot_patches.py,nx2/cloth_gen.py's docstring,tests/test_oot_cloth.py.Nothing here is ported from another project.
Testing
Tested: Windows 11, RTX 3090, Ryzen 9 5900X. In the game, a build of this code at 01855e95 (on main
068fb8eb, its build-time tables made from the image as above), the child Market (standing, then turning the camera), withHOS_NATIVE_CLOTH=check: every call the host takes also runs translated, the two are compared byte for byte and register for register, the translated result is kept. Every one of the fourteen pieces was taken by the host on every call (two bending calls in part), and every one endedMISMATCHES 0. The last count line of each piece (one update worker's totals):f_81c7f880f_81c86720f_81c7d2e0f_81c7bd00f_81c88940f_81b38800f_81b5a3c0f_81cdff40f_81c66a60f_81b8c000, recordsf_81c1a240f_81db9a40f_81ce6500No call fell to the translated code for its float mode (
FPCR/FPSR 0on every line). Earlier builds of this branch ran the same check as pieces were added (five passes, seven, nine), each withMISMATCHES 0, and with the switch at 1 the cloth looked as without it. The earlier head (the same thirteen commits cherry-picked onto mainfd7dc40d, and one commit of declarations) was not built or played; none of its host files differed from 01855e95. The current head also changes check-mode copies, diagnostics and test compiler selection; it has not been built or played.Offline, against the game's own code: an AArch64 emulator (Unicorn) runs the game's routine on its image, a driver runs the translated routine with the hook on the same case, and everything is compared: every x register, SP, all 32 q registers, NZCV, FPSR and every byte of every region (the particles and a sentinel past them, records, objects, the stack). Each in four builds: x86-64-v3 (SSE4.1 and FMA), its scalar kernel, x86-64-v2 (SSE4.1, no FMA), x86-64; and check mode on every batch. Every deliberately broken copy was caught.
The cases: realistic cloth (grids, folded and stretched sheets, skinned cloth, cloth around a capsule, about half in contact), half-integer grids (exact cancellations, signed zeros), degenerate geometry (coincident and collinear particles, zero-length links and normals), edge values in any lane (NaN payloads quiet and signalling, infinities, subnormals, FLT_MAX, the class's bounds), values that FZ flushes, scaled cloth, the fourth lane clean, as the game has it and garbage, aliasing (records, objects or the stack among the particles written), every FPCR mode and FPSR state the gate takes and others, empty and negative counts, and for the two collisions every one of the 15 contact patterns of each group of four. Also: 8,501 calls captured in the game that an earlier form declined, replayed: 99.996 % of their 1,096,630 records now native, all equal. The three giant functions (
f_81b8c000,f_81c1a240,f_81db9a40) also ran as main's build compiles them, cut bynx2/split_giants.py(158, 158 and 97 pieces): 4,000 calls each, all equal. Op by op: FRSQRTE on every positive normal float (2.1 billion) against the emulator, and Mesh Bone Deform's operations over 1.1 billion lane results, equal. A call the host takes only in part hands the translated loop the state of the record where it stopped; those hand-offs are in every suite above.The declarations commit changes only declarations in the generated code (the hooks take
Ctx *A64_NOESC c, and the two leaves a hook is passed are declared at file scope); the emulator harnesses call the host routines and do not compile those declarations, so they were not re-run for it. The thirteen patched functions with the new declarations compile clean (clang -fsyntax-only -Wall), and no hook keeps the context pointer past its call. The current head also changes host and test code; these results predate that change.Game-free tests at the earlier head:
tests/test_oot_cloth.py(the hooks placed once and only in the supported build, a function missing a place left alone, a second run changes nothing, the counts and check mode's lines),tests/test_cloth_kernels.py(the five passes against the translated loops, FRSQRTE's table),tests/test_cloth_loops.py(boxes and cull),tests/test_cloth_step8.py(Prepare Matrices, Verlet),tests/test_cloth_step9.pyandtests/test_cloth_step10.py(the two collisions, with tables made bynx2/cloth_gen.pyfrom the image when one is set),tests/test_cloth_step12.py(the two skinning loops), the hook tests also with the guest window at 64 GB and near the top of the 39-bit window. Of the 51 checks run (these seven,tests/test_oot_cloth_60.py,tests/test_oot_cloth_time.py,tests/test_switch_table.py,tests/test_patch_hooks_defined.pyand every test that readsnx2/oot_patches.pyornx2/entry_hooks.py) 46 pass; the five others fail the same way on mainfd7dc40don this PC:test_dev_packs,test_high_fps_fixes,test_libc_hooks,test_lift_cache,test_song_note_cue. The fulltests/run_all.pywas not run at that head. ruff was clean. These checks have not been reported for the current head.Still to test:
HOS_NATIVE_CLOTH=0against the switch unset (on, the default; the same build), and main against this PR withHOS_NATIVE_CLOTH=0, interleaved runs (at least 10 pairs of each comparison, the same route and length, the first run after a build discarded): the game's CPU time at a 60 fps cap and the frame times uncapped, over play only (no menu frames), the median and the spread of each side; expecting less CPU time with the switch at 1, the frame rate the same or better, the cloth as without it, and no extra CPU time withHOS_NATIVE_CLOTH=0against main (@Beanstalk linux, amd).HOS_NATIVE_CLOTH=check, expecting everyMISMATCHEScount in theoot: clothlines to be 0 and nooot: cloth: MISMATCHline (@Beanstalk linux).HOS_NATIVE_CLOTH=checkand then=1, expecting no difference and the cloth as on main (by hand).Measurements
Verdict: less CPU time in the game: with the switch on, cloth's thread time a frame -63.4 % and the whole process's CPU a frame -4.5 %, every on run below every off run, on one PC. The frame rate held about 60 in play either way (mean 60.0 -> 61.4, mixed run to run): this PC is not CPU-bound there, so the gain is CPU time, and frames rise only where the CPU is the limit. Not yet measured against main: the Test Farm request asks for main against this PR and for other hardware.
Baseline: the same build of this code (01855e95 on main
068fb8eb, plain, no diagnostics), withHOS_NATIVE_CLOTHoff (Before; unset was off at that commit) and=1(After); this measures what the switch saves, not this PR against main.Setup: Windows 11 (22631), Ryzen 9 5900X, RTX 3090, 3840x2160; child Link in the Market, driven hands-free through
scripts/longshot_ctl.py, standing 10 s then turning the camera 15 s; not capped (about 60 fps in play in every run); three off and three on runs, interleaved (a pair, then off, on, off, on), Discord and tray apps closed. CPU from the processor's cycle counter over the play window only (the title and file-select frames, which draw at hundreds of fps, left out); cloth's thread time from the sampling profiler (HOS_PROFILE=1): a busy sample with a Havok cloth function in its stack, the host kernels included. The mean of three runs, [the runs] in order.The medians give the same picture: cloth 7.84 -> 2.79, busy 60.22 -> 55.58, process 102.36 -> 98.14. The busy threads' drop (4.21 ms a frame) is about cloth's own (4.91 ms). The whole-process figure is small against the runs' spread (the off runs alone range over 3.8 ms), so three pairs show its direction, not its size; the cloth figure is clear in every pair. The process includes the renderer's threads, which this does not touch.
Offline speed, one call (Ryzen 9 5900X; outputs compared equal first; on the game's data unless said):
The cost with the switch off (
=0): each hooked function loses main's register locals (a patched function keeps its registers in the context) and each hook is a load and a branch. Against main's own lift, offline: none for the particle boxes, Prepare Matrices and Verlet (the context form ran them 1-10 % faster), the cull none on the game's data (+10-22 % on clean data), the capsule collision +1-3 %, the sphere -2 to +1 %, Object Space Deform -2 to +4 %, Mesh Bone Deform -4 to +1 %. Not measured: the five passes against main's form, and the rest of the cull's giant function (its narrow phase is in the context form too). In the game the build with the switch off was not measured against main; the Test Farm request asks for it.The switch
HOS_NATIVE_CLOTH(new, run time; its row ingpu/vk/switches.def):0(oroff) runs everything translated, as on main (each hook returns at once); unset or1on, the default;checkruns the host version into copies where it can, runs the translated code for real and compares, logging the first difference (slow: for a look in the game); calls it cannot copy safely are not compared. A start line, then anoot: clothline at most every 10 s per group of pieces:Three diagnostics switches, off by default:
HOS_CLOTH_DECLINES=1(why records run translated, by reason),HOS_CLOTH_CAPTURE=<folder>(declined calls written out for an offline replay;HOS_CLOTH_CAPTURE_Ncaps them),HOS_CLOTH_COUNTS=1(the collisions' and the skinning's mix of cases, only in a build made withA64_CLOTH_COUNTS=1, which puts counting lines into three functions and so is never a build to measure).It is on by default: docs/dev/standards.md asks that a switch ship at its value from the first push, not off because it is new.
HOS_NATIVE_CLOTH=0runs all of it translated, as on main.Risks
What could break: a host version that is not exact would move cloth differently from the Switch. Checked as in Testing; the hooks decline what they cannot prove (a float mode outside the translator's gate, memory outside the guest's, stores that could reach what is read later, a value outside a kernel's class where the kernel has no lane-by-lane way) and the translated code runs from there.
The build: the first game build after this takes about 28.5 s more (the tables and their proof), later builds 0.3 s. An image the reader does not know (another game version, an ExeFS mod that patches one of these routines) or a capstone that decodes otherwise gives other tables: the build says which piece and why, and that piece runs translated whatever the switch. The tables were proven with the builder's capstone (the 5.0.9 package, which reports 5.0.7; the same DLL).
Turning it off:
HOS_NATIVE_CLOTH=0.After updating: nothing changes for a player: saves,
longshot_options.iniand the caches as before. The thirteen hooked functions are written by the translator with their registers in the context (as every patched function is), which changes their translated code; the offline cost with the switch off is above.Checklist
Ready for review
main; the title is a Conventional Commit.068fb8eb, no difference; this head is not built or played.)mise run testpasses (any failure that also fails onmainis named). (At the earlier head, 51 checks ran, the branch's own and every one that reads the patch files: the five failures failed on main too; the full suite was not run. Checks have not been reported for the current head.)main, or Measurements says why there is nothing to measure. (Measured switch off against on in one build; main against this PR waits for the Test Farm.)third_party/untouched; nothing GPL-incompatible.When it applies
gpu/vk/switches.def. (The row is there; on by default, as the standards ask.)tests/that reads changed source is updated, not weakened. (tests/test_oot_cloth.pyand six more are new.)Assisted-by:/Co-Authored-By:trailers.Ready to merge
main, measured again or a comment says why the numbers still hold.🤖 Generated with Claude Code
host/cloth_lastrec.{h,cpp}: a batch kernel's record of 16 particles from tables read out of the routine's instructions (cloth_lastrec_tables.h): its stores, and for the call's last record every register and frame word it leaves, so the record loop no longer has to finish lifted. The capsule (f_81b8c000) does its records but the last by its kernel as before and the last by the tables, then goes on to the loop's way out; the sphere (f_81c1a240) does every record by the tables, and a call without a tail leaves at the tail's head. Check mode compares the registers and the frame at the tail's head; HOS_CLOTH_COUNTS counts both. (Rewritten in step 11, 2026-11-20: the tables and code made from the game's instructions are not in this commit; the build makes them from the player's image, nx2/cloth_gen.py, from the step 11 commit on.) Co-Authored-By: Claude Opus 5.5 <[email protected]>The batch kernels' last-record tables and code (capsule f_81b8c000, sphere f_81c1a240) and Prepare Matrices' exit tables (f_81cdff40) are the routines' dataflow read from their instructions, so they are not in the source tree: the build reads them from the image the lift wrote (<work>/sa/image/image.bin) into <gen>/cloth, which host/ includes as "cloth/..." through -I gen (a folder of its own: the lift's fingerprint of <gen> and the game-code header key do not see it). - nx2/cloth_sym.py: the routines' blocks, jump tables and a lane-level symbolic executor (capstone, as nx2/lift.py decodes). - nx2/cloth_gen.py: the tables, the straight-line C++ and Prepare Matrices' register and frame maps, with the checks that turn a piece off (its hooks decline, the lifted code runs; the build prints why and goes on): the reader stops on an instruction or branch its model does not know; the composition proof (3,411 contact sets a routine, node for node against the full paths' symbolic run); Prepare Matrices' block and rest loop compute host/cloth_matrices.cpp's PM_DEF, OUT_NODE and SWAPPED; the made files are the ones proven exact offline (their SHA-256). Made again only when the image, the scripts, host/cloth_matrices.cpp or capstone change (a stamp beside them). The numbering no longer depends on Python's string hash: the files are the same in every run. - nx2/build.py: one block that runs it before the host files are compiled. - host/: cloth_lastrec.cpp, cloth_matrices.cpp and oot_cloth.c include cloth/cloth_gen.h; with no tables the last records, the sphere's records and Prepare Matrices decline ("no tables") and the "oot: cloth" line says why. - tests: tests/cloth_tables.py makes the files into a test's folder from LONGSHOT_CLOTH_IMAGE or <OOT_WORK>/sa/image; test_cloth_step8 and test_cloth_step10 say SKIP without an image; the others link the declining stand-ins. Co-Authored-By: Claude Opus 5.5 <[email protected]>- host/cloth_skin.{h,cpp}: the skinning's 16-vertex blocks by 4, 3, 2 or 1 bones (SSE4.1 and lane by lane, the Arm's bits: ways R / W / A as the Verlet kernel's); hooked at L_81dc0638 (the block's head), every block the host proves, the lifted loop from the first it does not (blocks of the other kinds), check mode's compare at L_81dc1270; HOS_CLOTH_COUNTS counts a native block as the lifted lines do. - host/cloth_bones.{h,cpp}: Mesh Bone Deform's bone loop (a bone's frame from a triangle of particles; FRSQRTE / FRSQRTS for any operand the Arm's way), hooked at L_81ce6624 through L_81ce685c. - No constant of the game's image in comments or tests any more: the updateAabbs start value, the solver's eps and constant pairs, the dispatch's mask (test_cloth_step10 reads it from the image when it runs); tests/cloth_tables.py image_words(). - tests/test_cloth_step12.py (new); the other cloth tests link the two kernels; test_oot_cloth.py the two splices and Object Space Deform hooked and counted together. Co-Authored-By: Claude Opus 5.5 <[email protected]>Tested in game: This head is not built or played; the same code at 01855e95 (on main
068fb8eb) ran on Windows 11 (RTX 3090, Ryzen 9 5900X) in the child Market with HOS_NATIVE_CLOTH=check: all fourteen pieces taken by the host on every call, MISMATCHES 0 in every one. Switch off against on in that build, three interleaved pairs in the Market: cloth's thread time -63.4 % and the process's CPU -4.5 % a frame, every on run below every off run, the frame rate about 60 either way.@Beanstalk linux, and amd: this PR with HOS_NATIVE_CLOTH unset against HOS_NATIVE_CLOTH=1 (the same build), and main against this PR with the switch unset, in the child Market and in Kakariko Village (townspeople in cloth). Measure the game's CPU seconds with the frame rate capped at 60 (on a PC that holds 60 there), standing and turning the camera, and the frame times uncapped. The frame rate may not rise, and that is not a failure: on our PC it stayed about 60 in play either way. The expected result is less CPU time; frames rise only where the CPU is the limit. So the pass is less CPU time with the switch at 1, the frame rate the same or better, the cloth looking as without the switch, and no extra CPU time with the switch unset against main.
The saving is small at the process's scale (-4.5 % of the process's CPU a frame here, about 4.7 ms of 103, against a few ms of spread between runs), so please: interleaved runs (off, on, off, on ...), at least 10 pairs of each comparison, the same scene, route and length in every run, nothing else running, the first run after a build discarded as warm-up, and the measuring window over play only (the menus draw at hundreds of fps and must not be in the frame count). Report the median and the spread (min-max or IQR) of each figure for each side, not one run each. If you can sample threads, cloth's own thread time a frame is the clearest figure (-63 % here) and needs far fewer runs: please add it where you can; the process's CPU is the main one. With the switch on the log has "oot: cloth" lines; please report them.
Then one run of each scene with HOS_NATIVE_CLOTH=check: every MISMATCHES count in the "oot: cloth" lines should be 0 (the lines end "MISMATCHES 0" or "MISMATCHES in all 0"), with no "oot: cloth: MISMATCH" line.
@snaildev99 only members of
BigBeanscan start the farm. One of them can approve this request by reacting 👍 to it or to this comment.Possibly related, for context:
@Beanstalk
Beanstalk test farm · an earlier run: commit
a362423(the newer results are in a comment below)Commit
a362423· approved by llun (reviewed) · dashboard⚠️ Note: The farm cannot run the requested 10 pairs per comparison, discard a first post-build run, or sample cloth thread time; two pairs are used, so the small CPU saving may be noisy.
⚠️ Note: Uncapped frame times and check-mode mismatch counts are omitted to keep the plan within the run limit; camera movement precedes each measured standing segment.
⚠️ Note: Main ignores the PR-only switch, so its two matrix cells exercise the same code; the CPU rule against main applies to both PR cells, not just switch-off.
Started at 07:04 UTC.
Estimate: done in about 2 h, from the queue and the testers online now.
linuxamdcloth-towns· frame time p99 (ms)HOS_NATIVE_CLOTH=0(reference)HOS_NATIVE_CLOTH=1linuxamdWhat was tested
What will be tested · AI plan (
gpt-6.1-terra)On Linux and AMD PCs, the plan compares native cloth off and on in the child Market and Kakariko Village, with main as a baseline. It measures capped-play CPU time and frame rate, compares pictures, and times lift builds. Per target, gameplay takes roughly 45 minutes; crashes, higher CPU time with native cloth, a frame-rate regression, or excessive visual differences fail the plan.
0or1(PR only).market-entrance(settings frame_rate_cap=0), then teleport to Market, move the stick, take the picture market to compare with main, measure market for 30 s (after 10 s warm-up), teleport to Kakariko Village, take the picture kakariko to compare with main, measure kakariko for 30 s (after 10 s warm-up); 2 runs on each side for each of 2 cells (2 game starts a run).@fix
Should be on by default
a362423c0f2d74bd71c6Recommendation: Approve. The native path shows a substantial cloth CPU saving on one earlier Windows build, but its gain against main and the default-off cost remain unmeasured. This head also needs the requested Windows, Linux and AMD validation. Nothing blocking is left: what remains is for the author before merging (below). Recommend approval; not approved on the forge.
What
@fixchangedI changed this pull request: the new head is
2d74bd71c6. I rebased it ontomain(b62932a960). I edited its description to match the change.Changed
Left for the author
Nothing was built or run: the checks run again on the new head.
The change replaces parts of the cloth solver with native code behind an off-by-default switch. It could reduce CPU time, at the cost of substantial host code, build-time table generation and possible overhead even when disabled.
@fixwill do (2)nx2/oot_patches.py:829: Avoid forcing affected functions into the context form when the native path is disabled, if the comparison shows a cost.Before merging, for the author (2) (a run, a measurement or a decision; it doesn't hold the recommendation)
nx2/oot_patches.py:829: The hooks change the compiled form of the affected functions even with the default switch off. The PR has not measured this head against main in play, including the five solver passes and the cull's remaining work, so the cost to every player using the default is unknown. Compare main and this head with the switch unset in the requested scenes, including frame-time tails.Checks: they run again on this head; a failure shows on the pull request.
Also noticed (1; not asked for, not blocking)
More: does it help, other systems, switches, what was read
Does it help: Unproven against main and this head. On an earlier Windows build, three interleaved switch-off/on pairs showed cloth thread time falling from 7.74 to 2.83 ms per frame; process CPU fell from 103.31 to 98.64 ms, close to the spread between runs.
Other systems: The hooks also change translated code with the switch off. Linux, AMD, frame-time tails and the current head's in-game behavior remain unverified; the description reports extensive earlier emulator and Windows checks.
Switches: HOS_NATIVE_CLOTH defaults off, with 1 enabling native cloth and check enabling comparisons. HOS_CLOTH_DECLINES, HOS_CLOTH_CAPTURE, HOS_CLOTH_CAPTURE_N and HOS_CLOTH_COUNTS provide diagnostics; A64_CLOTH_COUNTS enables build-time counting. The off default limits exposure while platform results are pending, but the hooks can still cost time.
Reviewed:
2d74bd71c6againstb62932a960: Read the current fix diff, the cloth table helper, the check-mode copy and scratch-buffer code, the relevant patch and switch initialization excerpts, and the supplied description and comments. Skimmed the test changes and repository rules; the full original diff, kernels, generated material and recorded runs were not read in full.; nothing run (a code review).Checked and fine:
Testing: No commands that build or run tests were executed. CI is reported as not yet run on this head; the reported game and automated results predate the latest fix.
Overlaps and merge order: The description reports that #1384 changes a separate cloth time-step setup and can stack with this change; merge order was not established.
Automated code review by BeanBot (gpt-6.1-terra), asked for by BeanBot, after its fix. Nothing was run; a maintainer decides.
Beanstalk test farm
Commit
6eda484· approved by nobody yet · dashboardA member of
BigBeanscan review this commit and comment@Beanstalkto test it.HOS_NATIVE_CLOTHis now on by default (6eda4844), as docs/dev/standards.md asks: a switch ships at its value from the first push, not off because it is new.HOS_NATIVE_CLOTH=0gives main's translated code. The row's kind iskill, unset reads as on (env_on_or), and the tests run it unset and=1(on),=0and=off(off), andcheck.Also in this push: the kernels' comments say what they compute instead of listing instructions (
4f854842, comments only: every file is the same with its comments stripped), and three test fixtures no longer clash with theCONST/OUTmacros ofwindows.h, whichhost/oot_cloth.cincludes since2d74bd71. Every cloth suite passes on Windows with clang 23.1.4. The description is updated to match.🤖 Generated with Claude Code
Recommendation: Request changes. The gain is real: cloth thread time drops 63 % on one PC. But the source tree carries the game's disassembly in comments, in five kernel files, and CONTRIBUTING.md's Licensing rules that out. Beyond that, the hooks rebuild the compiled routine's register, NZCV and mid-function state, register by register. That is the design #1900 was held on, and it needs a maintainer's ruling before this can merge in any form.
For a player, this moves most of the Havok cloth (townspeople's clothes, Link's cap) from the translated Arm code to native SSE code. The cloth looks the same and costs less CPU. The kernels are well built: an exactness class, lane 3 done by the Arm's rules, a fallback to the translated code, a check mode, and a heavy offline proof. The cost is about 10,800 lines of host and build code for 13 game routines, plus a second Arm reader and prover (
nx2/cloth_gen.py,nx2/cloth_sym.py) that runs on every player's build.Does it help: Yes for cloth CPU time, on one PC; Unproven against main and elsewhere. On Windows 11, Ryzen 9 5900X, RTX 3090, child Market, three interleaved off/on pairs in one build (01855e95, not this head):
Not measured: main against this PR (the switch-off build loses main's register locals in the 13 hooked functions); frame-time tails; Linux; AMD CPUs; a CPU-bound PC where frames could rise.
Other systems:
.cpphost files have never been built there, and the description says so.A64_FPXKEEP) speeds up the translated path for the same NaN w lanes, so this PR's gain over main shrinks once 1905 lands. Whichever merges second should re-measure and re-runHOS_NATIVE_CLOTH=check.Switches:
HOS_NATIVE_CLOTH(kill, enum): on at this head (env_on_or(…, 1),6eda4844);=0gives main's translated code;checkcompares. On is the right default per standards, given the cloth figure and the check-modeMISMATCHES 0. Two conditions: this head has to be built and run in check mode once, and built on Linux (both are in Still to test). Keep it on.HOS_CLOTH_DECLINES,HOS_CLOTH_CAPTURE,HOS_CLOTH_CAPTURE_NandHOS_CLOTH_COUNTS: diagnostics, off by default, with rows. These are fine.A64_CLOTH_COUNTS: a build-environment variable for a diagnostics build. These are fine too.host/oot_cloth.c:1and its start line say "fourteen". Make them agree.Reviewed:
6eda48445against mainb62932a96(the PR is on main's head, so no rebase was needed).nx2/oot_patches.py,nx2/build.py,gpu/vk/switches.def,host/hos.h,host/hos_relaunch.candtests/crash_record_test.chunks;host/cloth_why.h,cloth_wlane.hpp,cloth_common.hpp,cloth_bend.handcloth_bend.cpp;cloth_verlet.cpp.host/oot_cloth.c: the head and switch read, counts and log, declines, scratch, copy and compare; the bend, link, bend link, tether and anchor answers;label_hook; the skin walk and registers.cloth_aabb.cpp,cloth_cull.cpp,cloth_bones.cpp,cloth_skin.cpp;nx2/cloth_gen.py's docstring,make,build_stepandstamp_key; the tests' docstrings.cloth_capsule.cpp,cloth_lastrec.cpp,cloth_link.cpp,cloth_bend_link.cpp,cloth_anchor.cpp,cloth_tether.cpp,cloth_matrices.cpp,cloth_skin.cppandcloth_bones.cpp; the capsule, sphere, Prepare Matrices and bones parts ofoot_cloth.c;nx2/cloth_sym.py; most of the seven test files.Fixed in this review: nothing. The blockers are licensing and design calls, not nits.
Checked and fine:
env_on_orreads0/off/false/noas off; any value starting withcis check.-ffp-contract=offis inCPUfor every host file, and the files' own pragmas back it up.-ffast-mathis an#error.<gen>/cloth/*files are underhost_roots, so a change to the tables recompiles the objects that include them.cloth_genfailures (no image, an unknown image, a proof failing) turn the pieces off, and the build goes on._Thread_localhand-offs between a hook at a loop's entry and its exit cross no guest call, so a fiber switch cannot land between them.hos_cloth_endis stubbed intests/crash_record_test.c.Blocking (to change before merge)
Game disassembly in the source tree (CONTRIBUTING.md, Licensing: "no decompiled game code or anything generated from the dump"). Five kernel files open with the routines' instructions, copied from the image with their addresses, operands and register numbers:
host/cloth_aabb.cpp:3-11, e.g.81b389ac ldr q16,[x13]; ldr w15,[x11]; cbnz w15 -> 81b389c4; ldr s17,[x14]; fcmp s17,#0.0; b.eq -> 81b3898c;host/cloth_verlet.cpp:3-20, e.g.81c66ec8 ldr x8,[sp,#8]; add w9,w25,#3; cmp w25,#0; csel w11,w9,w25,lt; ...;host/cloth_cull.cpp:3-15, e.g.81b5a674 cbnz w21 -> 81b5a6a4; ldp q1,q2,[sp,#0x90]; ldr q0,[x9]; fcmge v1 = v0 >= v1; ...;host/cloth_bones.cpp:4-27, the whole bone loop, e.g.v22.2s = frsqrte(v17); fcmp s17, s18; v23.2s = v22 * v22; v17.2s = frsqrts(v17, v23); ...;host/cloth_skin.cpp:4-30, all four paths, e.g.str q4, [x7 = [x20+0x18], w6]; cmp x4, #16.4f854842("the comments say what they compute, not which game instructions") changed only 48 lines and left all five listings. The description's "Nothing made from the game is in the source tree" is therefore not true at this head.Fix: delete the listings. Describe each loop in your own words, as
cloth_bend.cpp:3-17mostly does. The mnemonics inhost/oot_cloth.cgo too::824,:952,:1015(subs x9, x9, #1 reaching 0),:1436,:1664, and thefcmp, fccmp, fccmpat:1190. The listings are in the branch's earlier commits too (9d630e6conwards), so the merge must not carry those commits: squash it, or rewrite the branch.The hooks rebuild the compiled routine's exit state register by register; this needs a maintainer's ruling, as on #1900. The routines' results are behaviour (the particles written). What else is reproduced here is the compiled code's register allocation and control state:
...Laststruct is named after the game's registers, e.g.cloth_bend.h:27uint32_t v4[4], v5[4], v6[4], v7[4], v16[4], v25[4], written toc->vreg[4..26]andc->x[8..15]atoot_cloth.c:805-824;cloth_verlet.cpp:41-53keep()puts each value into the register the game's compiler chose for it (put(2, p.n), put(7, p.cur)...);oot_cloth.c:824, 1295, 1436, 1569, 1664);nx2/oot_patches.py:830:goto L_81c7bd60/L_81c7be4c/L_81c7bf2c/L_81c7c020);oot_cloth.c:2221:0x81dc0664 + 4 * table[t]);nx2/cloth_gen.pywrites "straight-line C++ (a statement a node)" of two routines from the image at build time. It is not in the tree, but it is a second translator for those two routines.The kernels' arithmetic reads as written from the computation (
cloth_bend.cppandcloth_verlet.cpp's bodies are formulas, not one statement per instruction), so this is not #1911's case. The register state is a different matter. Beyond licensing, it is also the most fragile part, and much of it shouldn't be needed. For the four whole-function hooks (bend, anchor, Prepare Matrices; and the bones/skin loops if they were hooked whole), AAPCS64 makes x8-x15, v0-v7 and v16-v31 and the flags dead at the return. Only the return value, the callee-saved registers and memory are observable. Hooking whole functions and dropping the exit-register tables would remove most ofoot_cloth.c's state code, the...Laststructs and the composition proof. If a caller in the game does rely on a non-ABI register, check mode would show it. Then check mode would compare memory, FPSR and the callee-saved registers. A maintainer should say whether the current design can merge.+28.5 s on every player's first build, avoidable as it stands (
nx2/cloth_gen.py:1127,nx2/build.py:1876-1880). The composition proof (27 s) re-proves, on each player's PC, tables whose SHA-256 must already equal the ones proven offline (PROVEN). The safe shortcut is to pin a digest of the two routines' bytes (their address ranges inimage.bin) and skip the proof when they match. Identical bytes give the identical tables and full paths that were proven, so the build pays about 1 s. Otherwise, measure the build time on a slow PC (the Steam Deck, an older 4-core CPU, an HDD) and state it in Risks. It stacks onto the lift that every player runs.Suggestions
nx2/oot_patches.py:894: "Read from the image by OoT-Modding's longshot-dev/cloth-rewrite-tools/step8_sites.py" names a tool outside this repository as the source ofCLOTH_COUNT_SITES. Either say what the addresses are (the loops' heads) and how to find them, or drop the reference.switches.def:112says thirteen,oot_cloth.c:1and:116say fourteen.Questions
Testing:
MISMATCHES 0); the offline Unicorn comparison suites (large, with broken copies caught); the 3+3 measurement runs. All of these predate the current head, and the description says this head is neither built nor played.a362423was approved by llun; its results were not posted when this review was written, and it predates the current head.Overlaps and merge order:
gpu/vk/switches.defand hooks innx2/oot_patches.pyandnx2/build.py. Expect textual conflicts there; no logical ones were found.@ -105,6 +109,7 @@ SW(HOS_MOVIE_PACE, kill, bool, "on: a 60 FPS cap is raised for a movie's pictureSW(HOS_MUTEX_TRACE, diag, int, "0: off", "Logs mutex waits longer than <ms>, with the thread that held the mutex.")SW(HOS_MUTEX_WATCH, diag, string, "off", "Logs every operation on the mutex at that guest address (hex) and its condition variables' signals.")SW(HOS_MW_WAKE, kill, enum, "only changes to objects linked into a multi-wait wake WaitAny", "=all: every event, semaphore or queue change wakes every WaitAny, as before.")SW(HOS_NATIVE_CLOTH, kill, enum, "on: the cloth pieces on host code where it gives the game's bits exactly, the rest lifted", "=0 (or false, off, no): every call runs as lifted, as on main; 1 (or unset) runs thirteen pieces of the Havok cloth (the solver's bending, link, bend link, skinned anchor and tether passes, updateAabbs, the capsule cull, Prepare Matrices, the Verlet integration, the capsule and sphere collisions' mode 0, Object Space Deform's block loop, Mesh Bone Deform) on host code where it gives the game's bits exactly, the rest lifted; check runs both on every call and logs any difference (slow). An oot: cloth line every 10 s counts the calls (host/oot_cloth.c).")Nit: "thirteen pieces" here, "fourteen" in host/oot_cloth.c:1 and its start line. The default (on) is right: keep it.
@ -0,0 +1,181 @@/* Native kernel for the particle loop of f_81b38800, hclSimClothInstance::updateAabbs (see cloth_aabb.h).The loop, as the image has it (0x81b3893c .. 0x81b38a10; registers at its exit are what the hook hands back):81b3895c ldr x14,[x9,#0x40]; movi v6 = 0.5; mov v5 = v0; ldr x11,[x9,#0xe8]; mov v4 = v1; fmov v7 = 1.5;Blocking (Licensing): this is the game's disassembly, its addresses, mnemonics and registers, in the source tree. Describe the loop in your own words, as cloth_bend.cpp's head does, and delete the listing.
@ -0,0 +24,4 @@uint32_t ia, id, ib, ic; /* the record's four u16 slot indices (+0x18, +0x1a, +0x1c, +0x1e) */uint32_t rec[6]; /* its six floats (bits) */uint32_t ima, imc; /* the particle words at pdata + 16*ia + 4, + 16*ic + 4 */uint32_t v4[4], v5[4], v6[4], v7[4], v16[4], v25[4];Blocking 2: the struct is named after the game's registers (v4..v25) and rebuilds the compiled routine's exit state. A whole-function leaf only has to give AAPCS64's observable state (memory, the return value, callee-saved registers). Needs a maintainer's ruling, as on #1900.
@ -0,0 +3,4 @@The loop as the image has it (0x81ce666c..0x81ce684c; operand order as written; "2s" operations write lanes 0-1 andzero the upper 64 bits):ldrh w16, [x14], #4 (the pair's triangle offset); x16 += x12; q17 = [x10+0x20] (R2); w17 = [x16], q4 = P0;Blocking (Licensing): the whole bone loop as instructions (frsqrte/frsqrts/fcmp, register by register). Delete it and describe the computation.
@ -0,0 +1,156 @@/* Native kernel for the cull loop of f_81b5a3c0, the capsule collidable (see cloth_cull.h).The loop, as the image has it (registers at its exit, 0x81b5a75c, are what the hook hands back):81b5a638 cmp w22,#1; tbz w24,#0 -> 81b5a6c0Blocking (Licensing): the routine's instructions, copied from the image. Delete them.
@ -0,0 +3,4 @@The four paths as the image has them (the registers each leaves are its last vertex's; operand order as written):4 bones, 0x81dc0664 (loop 0x81dc0680, x14): x7 = x14 + 4i; d4 = [x27 + 8i]; x6 = x14 + 8i; d5 = [x7 + 0xa0]; x3 += 4;idx b0..b3 = [x6 + 0x20..0x26] -> x7, x19, x23, x6 (bone pointers); w19 = [x20+0x24]; p in v4; w = ucvtf * v1 in v5;Blocking (Licensing): all four paths as instructions with their register allocation. Delete them.
@ -0,0 +1,180 @@/* Native kernel for the Verlet loop of f_81c66a60, hclSimulateOperator::_integrateTask (see cloth_verlet.h).The loop, as the image has it (registers at its exit are what the hook hands back):81c66ec8 ldr x8,[sp,#8]; add w9,w25,#3; cmp w25,#0; csel w11,w9,w25,lt; ldr x9,[x22,#0x30]; ldr x10,[x8,#0x40];Blocking (Licensing): the routine's instructions, copied from the image. Delete them and describe the loop instead.
@ -0,0 +38,4 @@};/* the registers a particle leaves: `j` its place among the four of a four-wide pass, -1 a single one */inline void keep(ClothVerlet *a, int j, const Part &p) {Blocking 2:
keep()places each value in the register the game's compiler allocated it to. That is not behaviour.@ -0,0 +821,4 @@c->vreg[23][0] = c->vreg[23][1] = lo64(z->rec[0], z->rec[0]);c->vreg[24][0] = z->rec[1], c->vreg[24][1] = 0;c->vreg[26][0] = z->rec[3], c->vreg[26][1] = 0;c->n = 0, c->z = 1, c->c = 1, c->v = 0; /* subs x8, x8, #1 reaching 0 */Blocking: NZCV is rebuilt to match a game instruction (
subs x8, x8, #1): the compiled code's state, not the routine's behaviour. The mnemonic comment goes either way (also at :952, :1015, :1436, :1664).@ -0,0 +2218,4 @@} else {const uint8_t *jp = (const uint8_t *)hos_guest_range(x[0] + 2ull * t, 2);if (!jp) return *why = "memory: the type table", 0;const uint64_t target = 0x81dc0664ull + 4ull * skin_ld16(jp);Blocking 2: this computes the game's own jump-table branch target inside the routine and dispatches on it. The same design question as #1900.
@ -0,0 +1124,4 @@time.time() - t0))except Exception as e: # (whatever stops the reader: that piece is not made, the build goes on)why.append("the last records' tables (f_81b8c000, f_81c1a240): %s" % (e or type(e).__name__))if lr and do_prove:Blocking 3: this proof costs about 27 s on every player's first build, for tables already pinned to PROVEN's digest. Pin a digest of the two routines' bytes and skip the proof when they match, or measure the build on a slow PC and state the cost in Risks.
@ -814,0 +891,4 @@# HOS_CLOTH_COUNTS (host/oot_cloth.c, diagnostics): counting lines in the capsule and sphere batch kernels and Object Space# Deform, put in only when the build's environment has A64_CLOTH_COUNTS=1 (an A64_ variable: in the lift key, so a build with# it and one without each lift as they should). A function with them is written in the context form (touches() says it is# changed), slower than main's locals form: never in a build that is measured. Read from the image by OoT-Modding'sSuggestion: this names a tool outside this repository (OoT-Modding's step8_sites.py) as the addresses' source. Say what the sites are instead.
View command line instructions
Checkout
From your project repository, check out a new branch and test the changes.