perf(lift): fast float regions span guest stores and vector moves, so more of the game's float maths runs as plain SSE #1880

Merged
HumanBean merged 1 commit from HumanBean/Longshot:perf/fastfp-stores into main 2026-11-20 20:52:41 +00:00
Owner

Summary

On fast PCs the Market's frame is now set by the translated game code: each frame's critical path (HOS_CHAIN) is ~97% lifted code running on a chain across the main thread, the module workers and the job executors, with no GPU or renderer wait on it. Float maths is ~15% of the guest instructions in the hot code, and an op outside a fast float region pays its exactness checks one by one (RECOMP_FPX_OPEN, the integer checks, the exact fallback); only ~31% of the float ops of the functions play runs sat in fast regions (nx2/fastfp.py).

Two changes widen the regions:

  • Stores inside regions (A64_FASTFP_STORES, on): a run may hold whole, unconditional guest stores (recomp_store8..64, recomp_stp32/64, their _sp forms) when no guest load follows a store in it; the run is cut before such a load (ff_split). The fast copy computes each store's address and value into locals and makes the stores, in order and with their page marks, after a64_ff_end passed; on a fall-back the exact copy runs from the region's start with its stores in place, so memory sees the same accesses in the same order either way. Functions an oot_patches edit names keep their stores out (a patch anchored on a store's text must not miss the fast copy).
  • Vector moves join regions again (A64_FASTFP_VMOVES, on): #1576 rewrote LDR Q, DUP, EXT and TBL into a64_vld16, a64_vdup, A64_EXT16/8 and a64_tbl1, which aren't in FF_PURE, so since then they ended runs (~23,700 run ends beside float code in the functions play runs). They are integer SSE with no float flags; a64_vld16 counts as a load for the store rule.

Float ops in regions in the functions play runs: 31% -> 60% (49.8k -> 95.2k of 158.9k). With both switches =0 the lift is main's.

Read first: ff_regions, ff_split, ff_defer, ff_patched in nx2/fastfp.py; the tagged lines in emit_function's copy.

Testing

Tested: Linux, RTX 3090 (NVIDIA 620.x), Ryzen 9 7900X, 2560x1440: the game built from this branch (main's renderer, the base's work folder) against main's own build on the same base, Castle Town Market standing, walking and turning, three interleaved game starts each after warm caches: no crash, no Vulkan error. tests/test_fastfp_regions.py (extended: eligible and ineligible store forms, cuts, deferred-store shape and order, x29 _sp in both copies, check-mode comparisons; C at x86-64-v3 and v2: memory, store order and mark count match the reference on the fast path, each store happens once on every fall-back, 20,000 check-mode runs with 0 mismatches; each check fails when broken on purpose), tests/test_fastfp_override.py (extended) and tests/test_fastfp_fma.py pass.

Still to test:

  • Linux, any GPU: a game built with A64_FASTFP=check, the Market and Kokiri Forest for 10 minutes, expecting no "fast region differs" line (by hand: it needs a full game build, which ran out of memory on my PC)
  • windows+amd, windows+nvidia, windows+intel, linux+amd, linux+nvidia, linux+intel, steamdeck, amd+igpu: three areas against main, expecting equal or better fps (more while moving), identical pictures, no crash (@Beanstalk)
  • Linux and Windows: mise run test, test_vecfp.py, test_function_ends.py, test_light_passes.py (they call ff_region/ff_eligible) expecting them to pass (by hand: CI on this pull request)

Measurements

Verdict: a small gain while the game moves (+2.9% walking in the Market, better p99), even standing; the build takes ~6% longer.

Baseline: main @ 89c3d21e's own build (dev.py rt --empty on its base) against this PR's game built on the same base with the same renderer, so the two differ only in the lifted code.

Setup: Linux, RTX 3090, Ryzen 9 7900X, 2560x1440, Threaded rendering Auto, Castle Town Market; three interleaved game starts each (median, runs in brackets).

Market Before: main @ 89c3d21e After: this PR @ 1f667883
standing fps 89.7 (86, 90, 90) 89.5 (89, 89, 90)
walking fps 81.9 (78, 82, 83) 84.3 (83, 84, 85)
walking p99 ms 19.5 17.9
turning fps 101.2 (101, 100, 102) 102.2 (102, 90, 103)

Build time: a full game build of this branch took 10:11 (compile 7:27) against 9:36 for main's base on the same PC; the lifted C grows ~3% (+34 MB from the vector moves, +8 MB from the stores). Both switches ship on: never slower here, and the regions they add are checked to match the exact copy.

Risks

A wrongly deferred store would show as a wrong value in game memory (glitches, physics) on the fast path only: A64_FASTFP=check compares both copies including the stores, and A64_FASTFP_STORES=0 / A64_FASTFP_VMOVES=0 (lift time) restore main's lift. Other threads see a region's stores slightly later (after the region's register work), never with other values or in another order. The game build takes a little longer. Nothing changes in the player's files; the new lift is built when the game is built.

Checklist

Ready for review

  • One topic, on current main; the title is a Conventional Commit.
  • Tested on my own PC, in the game for a change the game runs, as Testing says.
  • mise run test passes (any failure that also fails on main is named).
  • Measured against main, or Measurements says why there is nothing to measure.
  • No game files, build output or hand-edited generated files; third_party/ untouched; nothing GPL-incompatible.

When it applies

  • A switch added or changed: it ships at the default the measurements chose, with its row in gpu/vk/switches.def.
  • A check under tests/ that reads changed source is updated, not weakened.
  • Docs updated.
  • AI assistance recorded with Assisted-by: / Co-Authored-By: trailers.

Ready to merge

  • Everything in Still to test has run, its results in Tested.
  • Tier 1 hardware it can affect is tested (docs/dev/standards.md, "Hardware tiers").
  • Approved in review, every comment answered.
  • CI passes on the head; on current main, measured again or a comment says why the numbers still hold.

🤖 Generated with Claude Code

https://claude.ai/code/session_017vw8e3aEPBsd7QfUkaatjP

## Summary On fast PCs the Market's frame is now set by the translated game code: each frame's critical path (HOS_CHAIN) is ~97% lifted code running on a chain across the main thread, the module workers and the job executors, with no GPU or renderer wait on it. Float maths is ~15% of the guest instructions in the hot code, and an op outside a fast float region pays its exactness checks one by one (`RECOMP_FPX_OPEN`, the integer checks, the exact fallback); only ~31% of the float ops of the functions play runs sat in fast regions (`nx2/fastfp.py`). Two changes widen the regions: - **Stores inside regions (`A64_FASTFP_STORES`, on):** a run may hold whole, unconditional guest stores (`recomp_store8..64`, `recomp_stp32/64`, their `_sp` forms) when no guest load follows a store in it; the run is cut before such a load (`ff_split`). The fast copy computes each store's address and value into locals and makes the stores, in order and with their page marks, after `a64_ff_end` passed; on a fall-back the exact copy runs from the region's start with its stores in place, so memory sees the same accesses in the same order either way. Functions an `oot_patches` edit names keep their stores out (a patch anchored on a store's text must not miss the fast copy). - **Vector moves join regions again (`A64_FASTFP_VMOVES`, on):** #1576 rewrote LDR Q, DUP, EXT and TBL into `a64_vld16`, `a64_vdup`, `A64_EXT16/8` and `a64_tbl1`, which aren't in `FF_PURE`, so since then they ended runs (~23,700 run ends beside float code in the functions play runs). They are integer SSE with no float flags; `a64_vld16` counts as a load for the store rule. Float ops in regions in the functions play runs: 31% -> 60% (49.8k -> 95.2k of 158.9k). With both switches `=0` the lift is main's. Read first: `ff_regions`, `ff_split`, `ff_defer`, `ff_patched` in `nx2/fastfp.py`; the tagged lines in `emit_function`'s copy. ## Testing **Tested:** Linux, RTX 3090 (NVIDIA 620.x), Ryzen 9 7900X, 2560x1440: the game built from this branch (main's renderer, the base's work folder) against main's own build on the same base, Castle Town Market standing, walking and turning, three interleaved game starts each after warm caches: no crash, no Vulkan error. `tests/test_fastfp_regions.py` (extended: eligible and ineligible store forms, cuts, deferred-store shape and order, x29 `_sp` in both copies, check-mode comparisons; C at x86-64-v3 and v2: memory, store order and mark count match the reference on the fast path, each store happens once on every fall-back, 20,000 check-mode runs with 0 mismatches; each check fails when broken on purpose), `tests/test_fastfp_override.py` (extended) and `tests/test_fastfp_fma.py` pass. **Still to test:** - Linux, any GPU: a game built with `A64_FASTFP=check`, the Market and Kokiri Forest for 10 minutes, expecting no "fast region differs" line (by hand: it needs a full game build, which ran out of memory on my PC) - windows+amd, windows+nvidia, windows+intel, linux+amd, linux+nvidia, linux+intel, steamdeck, amd+igpu: three areas against main, expecting equal or better fps (more while moving), identical pictures, no crash (@Beanstalk) - Linux and Windows: `mise run test`, `test_vecfp.py`, `test_function_ends.py`, `test_light_passes.py` (they call `ff_region`/`ff_eligible`) expecting them to pass (by hand: CI on this pull request) ## Measurements **Verdict:** a small gain while the game moves (+2.9% walking in the Market, better p99), even standing; the build takes ~6% longer. Baseline: main @ 89c3d21e's own build (`dev.py rt --empty` on its base) against this PR's game built on the same base with the same renderer, so the two differ only in the lifted code. Setup: Linux, RTX 3090, Ryzen 9 7900X, 2560x1440, Threaded rendering Auto, Castle Town Market; three interleaved game starts each (median, runs in brackets). | Market | Before: main @ 89c3d21e | After: this PR @ 1f667883 | |---|---|---| | standing fps | 89.7 (86, 90, 90) | 89.5 (89, 89, 90) | | walking fps | 81.9 (78, 82, 83) | 84.3 (83, 84, 85) | | walking p99 ms | 19.5 | 17.9 | | turning fps | 101.2 (101, 100, 102) | 102.2 (102, 90, 103) | Build time: a full game build of this branch took 10:11 (compile 7:27) against 9:36 for main's base on the same PC; the lifted C grows ~3% (+34 MB from the vector moves, +8 MB from the stores). Both switches ship on: never slower here, and the regions they add are checked to match the exact copy. ## Risks A wrongly deferred store would show as a wrong value in game memory (glitches, physics) on the fast path only: `A64_FASTFP=check` compares both copies including the stores, and `A64_FASTFP_STORES=0` / `A64_FASTFP_VMOVES=0` (lift time) restore main's lift. Other threads see a region's stores slightly later (after the region's register work), never with other values or in another order. The game build takes a little longer. Nothing changes in the player's files; the new lift is built when the game is built. ## Checklist ### Ready for review - [x] One topic, on current `main`; the title is a Conventional Commit. - [x] Tested on my own PC, in the game for a change the game runs, as Testing says. - [ ] `mise run test` passes (any failure that also fails on `main` is named). - [x] Measured against `main`, or Measurements says why there is nothing to measure. - [x] No game files, build output or hand-edited generated files; `third_party/` untouched; nothing GPL-incompatible. ### When it applies - [x] A switch added or changed: it ships at the default the measurements chose, with its row in `gpu/vk/switches.def`. - [x] A check under `tests/` that reads changed source is updated, not weakened. - [x] Docs updated. - [x] AI assistance recorded with `Assisted-by:` / `Co-Authored-By:` trailers. ### Ready to merge - [ ] Everything in Still to test has run, its results in Tested. - [ ] Tier 1 hardware it can affect is tested (docs/dev/standards.md, "Hardware tiers"). - [ ] Approved in review, every comment answered. - [ ] CI passes on the head; on current `main`, measured again or a comment says why the numbers still hold. 🤖 Generated with [Claude Code](https://claude.com/claude-code) https://claude.ai/code/session_017vw8e3aEPBsd7QfUkaatjP
perf(lift): fast float regions span guest stores, deferred past the region's MXCSR check (A64_FASTFP_STORES)
Some checks failed
CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Has been skipped
CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Has been skipped
CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Has been skipped
Third-party parts / The same as upstream (pull_request) Has been skipped
Website / Website on Cloudflare (pull_request) Has been skipped
Website / Website on Cloudflare (pull_request_target) Has been skipped
CI / Conventional Commit title (pull_request) Successful in 18s
CI / Workflow lint (actionlint) (pull_request) Successful in 29s
Guide screenshots / Guide images and local UI source review (pull_request) Failing after 29s
Third-party parts / As their notes say (pull_request) Successful in 30s
PR description / Description follows the template (pull_request_target) Failing after 7s
beanstalk/approval approved by HumanBean (reviewed)
beanstalk/any 0/1 pass, 0 running, 0 queued
CI / Website build and checks (pull_request) Failing after 5m3s
1f667883de
A run of float code may now hold guest stores (STR/STP, any base, writeback; not STLR, a conditional store or a text
whose load follows its store), cut before any guest load that follows a store, so every load of a region comes before
every store. The fast copy computes each store's address and values into locals and makes the stores, in order and with
the same helpers (the same page marks; x29 frame stores unmarked as frame_stores makes them), once a64_ff_end passed;
on a fall-back nothing has been stored and the exact copy runs as before. A64_FASTFP=check compares each store's
address and values as well as the registers. No stores join in functions holding an address nx2/oot_patches.py names.

Also nx2/fastvec.py's vector moves and shuffles (a64_vld16, a64_vdup, A64_EXT16, A64_EXT8, a64_tbl1, named since #1576
and absent from FF_PURE) join regions again (A64_FASTFP_VMOVES=0: not). On the lifted C of 89c3d21e's base, the float
ops inside regions in the functions play runs go from 31% to 60% (vector moves alone 56%, stores alone 36%).

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_017vw8e3aEPBsd7QfUkaatjP
Author
Owner

Tested in game: Linux, RTX 3090 (NVIDIA 620.x), Ryzen 9 7900X, 2560x1440: game built from this branch vs main's own build on the same base, Castle Town Market standing/walking/turning, three interleaved starts each; no crash or Vulkan error; walking +2.9%.

Tested in game: Linux, RTX 3090 (NVIDIA 620.x), Ryzen 9 7900X, 2560x1440: game built from this branch vs main's own build on the same base, Castle Town Market standing/walking/turning, three interleaved starts each; no crash or Vulkan error; walking +2.9%.
Author
Owner

@Beanstalk Please test this PR against main on every platform: windows+amd, windows+nvidia, windows+intel, linux+amd, linux+nvidia, linux+intel, the steamdeck, amd+igpu. Settings vsync=2 (Off) and frame_rate_cap=6 (Uncapped), Threaded rendering Auto. From the kokiri-start save: Kokiri Forest, Hyrule Field and Castle Town Market, in each 20 s standing, 15 s walking and 15 s turning the camera, two game starts a side; make sure each side has reached the area before measuring it. Measure fps, p99 frame time, stutters and crashes, and compare pictures with main. This changes the translated game code (float maths), so a crash, a hang or a visual or physics difference is the main thing to catch; the gain is expected mostly while moving, on the CPU side.

@Beanstalk Please test this PR against main on every platform: windows+amd, windows+nvidia, windows+intel, linux+amd, linux+nvidia, linux+intel, the steamdeck, amd+igpu. Settings vsync=2 (Off) and frame_rate_cap=6 (Uncapped), Threaded rendering Auto. From the `kokiri-start` save: Kokiri Forest, Hyrule Field and Castle Town Market, in each 20 s standing, 15 s walking and 15 s turning the camera, two game starts a side; make sure each side has reached the area before measuring it. Measure fps, p99 frame time, stutters and crashes, and compare pictures with main. This changes the translated game code (float maths), so a crash, a hang or a visual or physics difference is the main thing to catch; the gain is expected mostly while moving, on the CPU side.
Member

@HumanBean approval of 1f66788 recorded. This diff touches sensitive paths, so a second member of BigBeans must approve it too.

@HumanBean approval of `1f66788` recorded. This diff touches sensitive paths, so a second member of `BigBeans` must approve it too.
Member

Beanstalk test farm · 🔄 testing (0 of 1 report so far)

Commit 1f66788 · approved by HumanBean (reviewed) · dashboard

Done (started 18:18 UTC). Comment @Beanstalk to run it again.

Target Result Reports Frame time p99 (ms), main → PR VRAM peak (MB) Crashes
any ⏳ queued 0 of 1 — — —
What was tested

What will be tested · AI plan (gpt-6.1-terra)

The plan compares main and the pull request in Kokiri Forest, Hyrule Field and the Market, with movement, pictures, frame time and FPS measurements. One start per side keeps gameplay testing to roughly 15 minutes per tester; separate repeated lift-cache builds measure build-time changes. A crash, materially slower gameplay, a substantial picture difference or a build-time regression fails the plan.

  • Where: any (any PC, 1-2 reports).
  • three-areas: from the repository's save kokiri-start (settings vsync=2, frame_rate_cap=6), then teleport to Kokiri Forest, move the stick, take the picture kokiri to compare with main, measure kokiri for 15 s (after 3 s warm-up), teleport to Hyrule Field, take the picture field to compare with main, measure field for 15 s (after 3 s warm-up), teleport to Market, take the picture market to compare with main, measure market for 15 s (after 3 s warm-up); 1 run on each side.
  • Build times: main and the PR each built 2 times, alternating, each whole in a fresh folder: the lift again (no kept lift, main's compiled objects kept); every stage timed (shader preparation is not), and the lifted C compared.
  • Measured: crashes, frame times, stutters and hitches, draws and frame rate.
  • The PR fails if: it crashes; or the whole build's time is more than 20% above main; or frame rate is more than 10% below main; or a picture differs from main by more than 5%; or the lift's time is more than 20% above main; or the compile's time is more than 20% above main; or frame time (p99) is more than 15% above main.
  • Time: about 12 min of game runs per tester (2 game starts), plus the two builds and 4 timed builds (about 80 min).
<!-- beanstalk:sticky --> ### Beanstalk test farm · 🔄 testing (0 of 1 report so far) Commit `1f66788` · approved by HumanBean (reviewed) · [dashboard](https://beanstalk.magicbean.dev/prs/1880) Done (started 18:18 UTC). Comment `@Beanstalk` to run it again. | Target | Result | Reports | Frame time p99 (ms), main → PR | VRAM peak (MB) | Crashes | | --- | --- | --- | --- | --- | --- | | `any` | ⏳ queued | 0 of 1 | — | — | — | <details><summary>What was tested</summary> **What will be tested** · AI plan (`gpt-6.1-terra`) The plan compares main and the pull request in Kokiri Forest, Hyrule Field and the Market, with movement, pictures, frame time and FPS measurements. One start per side keeps gameplay testing to roughly 15 minutes per tester; separate repeated lift-cache builds measure build-time changes. A crash, materially slower gameplay, a substantial picture difference or a build-time regression fails the plan. - **Where:** any (any PC, 1-2 reports). - **three-areas:** from the repository's save `kokiri-start` (settings vsync=2, frame_rate_cap=6), then teleport to Kokiri Forest, move the stick, take the picture kokiri to compare with main, measure kokiri for 15 s (after 3 s warm-up), teleport to Hyrule Field, take the picture field to compare with main, measure field for 15 s (after 3 s warm-up), teleport to Market, take the picture market to compare with main, measure market for 15 s (after 3 s warm-up); 1 run on each side. - **Build times:** main and the PR each built 2 times, alternating, each whole in a fresh folder: the lift again (no kept lift, main's compiled objects kept); every stage timed (shader preparation is not), and the lifted C compared. - **Measured:** crashes, frame times, stutters and hitches, draws and frame rate. - **The PR fails if:** it crashes; or the whole build's time is more than 20% above main; or frame rate is more than 10% below main; or a picture differs from main by more than 5%; or the lift's time is more than 20% above main; or the compile's time is more than 20% above main; or frame time (p99) is more than 15% above main. - **Time:** about 12 min of game runs per tester (2 game starts), plus the two builds and 4 timed builds (about 80 min). </details>
Member

Possibly related, for context:

  • PR #1782 perf(lift): SSE fast paths for the float forms the translator does lane by lane (A64_VECFP) — Merged SSE fast-float paths; this extends their regions
  • issue #362 perf(build): a first build in under 10 minutes: what was measured, and a proposal (a clang-free cold tier) — Open build-time proposal overlaps lifter compile-time changes
Possibly related, for context: - PR #1782 perf(lift): SSE fast paths for the float forms the translator does lane by lane (A64_VECFP) — Merged SSE fast-float paths; this extends their regions - issue #362 perf(build): a first build in under 10 minutes: what was measured, and a proposal (a clang-free cold tier) — Open build-time proposal overlaps lifter compile-time changes
Author
Owner

@beanstalk

@beanstalk
Member

@HumanBean approval of 1f66788 recorded. This diff touches sensitive paths, so a second member of BigBeans must approve it too.

@HumanBean approval of `1f66788` recorded. This diff touches sensitive paths, so a second member of `BigBeans` must approve it too.
Author
Owner

@Beanstalk

@Beanstalk
HumanBean changed title from WIP: perf(lift): fast float regions span guest stores and vector moves, so more of the game's float maths runs as plain SSE to perf(lift): fast float regions span guest stores and vector moves, so more of the game's float maths runs as plain SSE 2026-11-20 20:52:34 +00:00
HumanBean deleted branch perf/fastfp-stores 2026-11-20 20:52:41 +00:00
Author
Owner

After the merge, for the record: the two items this PR's Testing still listed, run on the lab (Linux, RX 580, RADV Mesa 26.2.6, i7-9700K, 1080p).

Check mode: a game built from this branch with A64_FASTFP=check (both copies of every region run and compared, stores included), one session through Kokiri Forest, Hyrule Field and Castle Town Market standing and turning: 118 per-file reports, every one 0 mismatched when clean (about 4.2 million region runs a file); no crash, no Vulkan error.

fps on AMD: main @ 89c3d21e's own build against this branch's on the same base, at 50% render resolution (the RX 580 still 84-98% busy), two interleaved game starts each:

fps (p99 ms) main @ 89c3d21e this PR @ 1f667883
Market standing 39.1 (33.4) 39.2 (28.4)
Market turning 47.7 (37.8) 48.5 (37.0)
Hyrule Field standing 56.4 (20.0) 56.5 (20.0)
Hyrule Field turning 58.8 (25.3) 59.7 (25.6)
Kokiri Forest standing 57.8 (18.9) 57.7 (19.0)
Kokiri Forest turning 44.3 (44.5) 44.4 (47.6)

Even to slightly better: the GPU limits the frame there, so the CPU side's gain shows only where the CPU holds some of it (turning in the Market and the Field).

After the merge, for the record: the two items this PR's Testing still listed, run on the lab (Linux, RX 580, RADV Mesa 26.2.6, i7-9700K, 1080p). **Check mode:** a game built from this branch with `A64_FASTFP=check` (both copies of every region run and compared, stores included), one session through Kokiri Forest, Hyrule Field and Castle Town Market standing and turning: 118 per-file reports, every one `0 mismatched when clean` (about 4.2 million region runs a file); no crash, no Vulkan error. **fps on AMD:** main @ 89c3d21e's own build against this branch's on the same base, at 50% render resolution (the RX 580 still 84-98% busy), two interleaved game starts each: | fps (p99 ms) | main @ 89c3d21e | this PR @ 1f667883 | |---|---|---| | Market standing | 39.1 (33.4) | 39.2 (28.4) | | Market turning | 47.7 (37.8) | 48.5 (37.0) | | Hyrule Field standing | 56.4 (20.0) | 56.5 (20.0) | | Hyrule Field turning | 58.8 (25.3) | 59.7 (25.6) | | Kokiri Forest standing | 57.8 (18.9) | 57.7 (19.0) | | Kokiri Forest turning | 44.3 (44.5) | 44.4 (47.6) | Even to slightly better: the GPU limits the frame there, so the CPU side's gain shows only where the CPU holds some of it (turning in the Market and the Field).
Sign in to join this conversation.
No reviewers
No milestone
No project
No assignees
3 participants
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
MagicBeanDev/Longshot!1880
No description provided.