Every guest memory barrier becomes a full fence, and no test pins the ordering the lifter writes #1507

Open
opened 2026-11-18 01:42:35 +00:00 by mooki · 1 comment
Member

Type: performance · Area: lifter (guest memory ordering) · Related: #1493

TL;DR

The lifter writes a full fence for every ARM memory barrier. 919 of the 3,823 in the game (dmb ishld and dmb ishst) need only a compiler barrier on x86-64. Acquire and release loads and stores, the atomic update forms and the exclusive pair are already ordered in the lifted C, so the reordering hazard first suspected in the research is not there, but no test pins any of it. I propose a game-free test of what the lifter writes for each of these, then the compiler-only lowering, and a count of executed barriers to decide whether the change is worth more than the test.

Breakdown

  • Who it affects: everyone in principle; only CPU-bound PCs could see a gain, and none is claimed. Anyone who changes the lifter (#1469's locals, a new lifter) is affected by the missing test.
  • What is wrong: every DMB and DSB becomes one out-of-line call that runs a sequentially consistent fence, whatever the barrier's kind. x86-64 already orders load to load, load to store and store to store, so the load-only (870) and store-only (49) kinds need no hardware fence. And nothing in tests/ checks what ordering the lifter gives acquire/release accesses, atomics, exclusives or barriers.
  • What we propose: a test of the emitted C and its compiled order, per instruction class; then a compiler-only barrier for the load-only and store-only kinds (a real fence stays for the full kinds, DSB and ISB, and on any non-x86 host); a count of executed barriers by kind before any speed claim.
  • What does not change: how LDAR, STLR, LDAPR, the LSE forms, CAS and the exclusives are lowered; full barriers; the game's behaviour on x86-64.
  • What we do not claim: a gain. No measurement of the fences' cost exists.
  • How we would prove it: the test fails today for the load-only and store-only barriers and passes for the rest; then paired timing after a null pair, only if the count says these barriers run often enough to matter.

Evidence

Code on forge/main at 310e7c9dfa. The research report (translation, section 2.5) said LDAR and STLR become plain accesses. That is wrong on main: nx2/insn.py replaces the translator's text for them.

How each class is lowered now:

  • nx2/lift.py:51 installs nx2/insn.py before the lift. nx2/insn.py:244-245 sends LDAR, STLR, LDLAR and STLLR (all sizes) to ldar_stlr (:226-239). LDAR is __atomic_load_n(..., __ATOMIC_SEQ_CST). STLR is __atomic_store_n(..., __ATOMIC_SEQ_CST) followed by the page mark. The docstring (:227-232) says why: the compiler moving other accesses across them, and ARM's order of a release store before an acquire load. insn.emit on such words gives exactly this C (run to check). The translator's own plain version, third_party/a64-kit/lift/suyu/arm64_to_c.h:1632-1641 and :1730-1741, is never reached for them.
  • LDAPR (byte, half, word, doubleword): nx2/insn.py:67-68, __ATOMIC_ACQUIRE. LSE updates and SWP: :69-86, CAS: :89-94, CASP: :97-111, all __ATOMIC_SEQ_CST.
  • LDXR, LDAXR, STXR, STLXR (5 sites in the game): out-of-line recomp_ldxr and recomp_stxr (third_party/a64-kit/rt/a64_rt.c:270-280), sequentially consistent load and compare-and-swap (:241-268).
  • Barriers: arm64_to_c.h:1108 matches the whole DSB/DMB group by its fixed bits and writes recomp_barrier();. It never reads the option field. :1102 is CLREX, :1104-1107 is ISB. rt/a64_rt.c:236: void recomp_barrier(void) { __atomic_thread_fence(__ATOMIC_SEQ_CST); }, an out-of-line call. It is passed nothing, so registers stay in host registers across it (nx2/rtgen.py:19-26).

How many, counted from the relocated guest code (research, translation-data/mnemonics.json, capstone):

  • 3,823 barrier words: dmb ish 2,892; dmb ishld 870; dmb ishst 49; dsb sy 4; dsb ish 2; isb 3; one each of dmb oshst, dmb st, dsb st. So 919 are load-only or store-only. The two other store-only forms would make 921.
  • The base lift has 3,801 recomp_barrier() calls, in 1,922 of 294,522 functions (census-base.tsv).
  • By hot tier, from the same image and its function table (a script that prints only totals): the 100 hottest functions hold 85 barriers (44 load-only, 41 full); the 3,680 hot-listed functions 815 (289 load-only, 9 store-only, 517 full); the 849 functions of the shipped hot list 308 (154 load-only, 154 full). These are sites, not executions.

What a barrier costs: unknown. rt/a64.h:162-165 records one experiment, inlining the barrier, LDXR, STXR and CLREX in the castle: 69.5 to 68.4 fps, not kept. It compares a call with inline code, not a fence with none.

Tests that touch these today: tests/test_mark_groups.py:74-77 and :115 use recomp_barrier();, an atomic and the STLR form as places where a mark run ends. nx2/test_insn.py checks values on one thread under Unicorn, which cannot see ordering. A git grep finds no test that names what ordering LDAR, STLR, LDAPR or a DMB gets.

#1469 turns registers and flags into C locals, written back around every call and every sequentially consistent atomic. It reports one heap-corruption crash (Windows exception 0xc0000374) in about 41 runs of local-register builds and none in about 30 of main, with no cause yet. This issue does not offer one. The instruction classes above are already ordered, so we have no ordering explanation for it. What is missing is a check of whether a lift variant (locals on, split pieces, a higher level) changes the emitted order.

Proposed change

  1. tests/test_lift_ordering.py (below), written first. Its rows for load-only and store-only barriers fail until step 2. Every other row passes today and pins it.
  2. In nx2/insn.py emit(): match DMB, read the option's type bits. Load-only and store-only types write a macro, defined by nx2/rtgen.py into the runtime header next to the marks (:55-65). On x86-64 it is __atomic_signal_fence(__ATOMIC_SEQ_CST), which emits no instruction. On any other host it is the real fence. Full types, DSB and ISB keep recomp_barrier(). The vendored kit is not edited, as for #1423.
  3. A diagnostic build that counts executed barriers by kind in Kokiri Forest and Hyrule Field, to bound the gain before anything is timed.
  4. One caution to settle first: x86 orders only ordinary stores; non-temporal stores are weaker. host/ has none, and the only ones in gpu/ are the opt-in capture copy into the renderer's own buffer (gpu/vk/vk_memory.cpp:1762-1768), which ends with a store fence. The C library's large memcpy may use them; not checked here.

How we would test it

  • Game-free, built like tests/test_sdiv_overflow.py: the translator at -O0 in a temporary folder, nx2/insn.py installed as nx2/lift.py does, hand-encoded words through a64emit.translate, compiled at -O0, -O1 and -O2. Skipped when there is no compiler.
  • Text, per class: LDAR, STLR and LDLAR/STLLR in four sizes and with XZR give a sequentially consistent builtin (STLR then the mark); LDAPR gives acquire; the LSE forms, SWP, CAS give sequentially consistent updates; LDXR, LDAXR, STXR, STLXR give the runtime calls; DMB full types, DSB and ISB give recomp_barrier(); DMB load-only and store-only types give the compiler-only macro. The last row fails today.
  • Compiled order at -O2, by reading the assembly: a publish pattern (data stores, then STLR; reader LDAR, then data loads) keeps every data access on its side of the flag access. With a store-only barrier between two stores, the two keep their order and the code has no fence and no call. With a full barrier a fence or the call sits between them.
  • Each variant the lifter has: the default, locals on (A64_LOCAL_REGS=6, A64_LOCAL_FLAGS=1) and split pieces, through the real function emitter as tests/test_split_giants.py does.
  • Optional litmus run, x86-64 only: store buffering (STLR then LDAR on two threads; both reads zero must never happen) and message passing under the lowered barriers. A pass proves nothing alone.
  • Game: not needed for the test. For the change, only if the count justifies it: Kokiri Forest and Hyrule Field, lifted-code CPU a frame (HOS_PROFILE=1, HOS_THREAD_TIMES), main against main first, then interleaved pairs of main and the change, on a CPU-bound PC. The lifted C of about 1,900 functions changes, so one new lift is needed.

#1493 (renderer rewrite, parallel CPU track). #1469 (open: registers and flags as C locals; the one crash). #1142 (open: a test that the hot pieces' split and -O1 against -O2 compute the same). #1140 (open: pins the compile flags that keep float results and pointer punning safe at a higher level). #1423 and #361 (merged and closed: SDIV in nx2/insn.py; its test is this one's template). A search of issues and pull requests for LDAR, STLR, LDAPR, DMB and mfence finds none.

New on this forge; not opened on GitHub.

**Type:** performance · **Area:** lifter (guest memory ordering) · **Related:** #1493 ## TL;DR The lifter writes a full fence for every ARM memory barrier. 919 of the 3,823 in the game (`dmb ishld` and `dmb ishst`) need only a compiler barrier on x86-64. Acquire and release loads and stores, the atomic update forms and the exclusive pair are already ordered in the lifted C, so the reordering hazard first suspected in the research is not there, but no test pins any of it. I propose a game-free test of what the lifter writes for each of these, then the compiler-only lowering, and a count of executed barriers to decide whether the change is worth more than the test. ## Breakdown - **Who it affects:** everyone in principle; only CPU-bound PCs could see a gain, and none is claimed. Anyone who changes the lifter (#1469's locals, a new lifter) is affected by the missing test. - **What is wrong:** every DMB and DSB becomes one out-of-line call that runs a sequentially consistent fence, whatever the barrier's kind. x86-64 already orders load to load, load to store and store to store, so the load-only (870) and store-only (49) kinds need no hardware fence. And nothing in `tests/` checks what ordering the lifter gives acquire/release accesses, atomics, exclusives or barriers. - **What we propose:** a test of the emitted C and its compiled order, per instruction class; then a compiler-only barrier for the load-only and store-only kinds (a real fence stays for the full kinds, DSB and ISB, and on any non-x86 host); a count of executed barriers by kind before any speed claim. - **What does not change:** how LDAR, STLR, LDAPR, the LSE forms, CAS and the exclusives are lowered; full barriers; the game's behaviour on x86-64. - **What we do not claim:** a gain. No measurement of the fences' cost exists. - **How we would prove it:** the test fails today for the load-only and store-only barriers and passes for the rest; then paired timing after a null pair, only if the count says these barriers run often enough to matter. ## Evidence Code on `forge/main` at `310e7c9dfa`. The research report (translation, section 2.5) said LDAR and STLR become plain accesses. That is wrong on main: `nx2/insn.py` replaces the translator's text for them. How each class is lowered now: - `nx2/lift.py:51` installs `nx2/insn.py` before the lift. `nx2/insn.py:244-245` sends LDAR, STLR, LDLAR and STLLR (all sizes) to `ldar_stlr` (`:226-239`). LDAR is `__atomic_load_n(..., __ATOMIC_SEQ_CST)`. STLR is `__atomic_store_n(..., __ATOMIC_SEQ_CST)` followed by the page mark. The docstring (`:227-232`) says why: the compiler moving other accesses across them, and ARM's order of a release store before an acquire load. `insn.emit` on such words gives exactly this C (run to check). The translator's own plain version, `third_party/a64-kit/lift/suyu/arm64_to_c.h:1632-1641` and `:1730-1741`, is never reached for them. - LDAPR (byte, half, word, doubleword): `nx2/insn.py:67-68`, `__ATOMIC_ACQUIRE`. LSE updates and `SWP`: `:69-86`, `CAS`: `:89-94`, `CASP`: `:97-111`, all `__ATOMIC_SEQ_CST`. - LDXR, LDAXR, STXR, STLXR (5 sites in the game): out-of-line `recomp_ldxr` and `recomp_stxr` (`third_party/a64-kit/rt/a64_rt.c:270-280`), sequentially consistent load and compare-and-swap (`:241-268`). - Barriers: `arm64_to_c.h:1108` matches the whole DSB/DMB group by its fixed bits and writes `recomp_barrier();`. It never reads the option field. `:1102` is CLREX, `:1104-1107` is ISB. `rt/a64_rt.c:236`: `void recomp_barrier(void) { __atomic_thread_fence(__ATOMIC_SEQ_CST); }`, an out-of-line call. It is passed nothing, so registers stay in host registers across it (`nx2/rtgen.py:19-26`). How many, counted from the relocated guest code (research, `translation-data/mnemonics.json`, capstone): - 3,823 barrier words: `dmb ish` 2,892; `dmb ishld` 870; `dmb ishst` 49; `dsb sy` 4; `dsb ish` 2; `isb` 3; one each of `dmb oshst`, `dmb st`, `dsb st`. So 919 are load-only or store-only. The two other store-only forms would make 921. - The base lift has 3,801 `recomp_barrier()` calls, in 1,922 of 294,522 functions (`census-base.tsv`). - By hot tier, from the same image and its function table (a script that prints only totals): the 100 hottest functions hold 85 barriers (44 load-only, 41 full); the 3,680 hot-listed functions 815 (289 load-only, 9 store-only, 517 full); the 849 functions of the shipped hot list 308 (154 load-only, 154 full). These are sites, not executions. What a barrier costs: unknown. `rt/a64.h:162-165` records one experiment, inlining the barrier, LDXR, STXR and CLREX in the castle: 69.5 to 68.4 fps, not kept. It compares a call with inline code, not a fence with none. Tests that touch these today: `tests/test_mark_groups.py:74-77` and `:115` use `recomp_barrier();`, an atomic and the STLR form as places where a mark run ends. `nx2/test_insn.py` checks values on one thread under Unicorn, which cannot see ordering. A `git grep` finds no test that names what ordering LDAR, STLR, LDAPR or a DMB gets. #1469 turns registers and flags into C locals, written back around every call and every sequentially consistent atomic. It reports one heap-corruption crash (Windows exception 0xc0000374) in about 41 runs of local-register builds and none in about 30 of main, with no cause yet. This issue does not offer one. The instruction classes above are already ordered, so we have no ordering explanation for it. What is missing is a check of whether a lift variant (locals on, split pieces, a higher level) changes the emitted order. ## Proposed change 1. `tests/test_lift_ordering.py` (below), written first. Its rows for load-only and store-only barriers fail until step 2. Every other row passes today and pins it. 2. In `nx2/insn.py` `emit()`: match DMB, read the option's type bits. Load-only and store-only types write a macro, defined by `nx2/rtgen.py` into the runtime header next to the marks (`:55-65`). On x86-64 it is `__atomic_signal_fence(__ATOMIC_SEQ_CST)`, which emits no instruction. On any other host it is the real fence. Full types, DSB and ISB keep `recomp_barrier()`. The vendored kit is not edited, as for #1423. 3. A diagnostic build that counts executed barriers by kind in Kokiri Forest and Hyrule Field, to bound the gain before anything is timed. 4. One caution to settle first: x86 orders only ordinary stores; non-temporal stores are weaker. `host/` has none, and the only ones in `gpu/` are the opt-in capture copy into the renderer's own buffer (`gpu/vk/vk_memory.cpp:1762-1768`), which ends with a store fence. The C library's large `memcpy` may use them; not checked here. ## How we would test it - Game-free, built like `tests/test_sdiv_overflow.py`: the translator at `-O0` in a temporary folder, `nx2/insn.py` installed as `nx2/lift.py` does, hand-encoded words through `a64emit.translate`, compiled at `-O0`, `-O1` and `-O2`. Skipped when there is no compiler. - Text, per class: LDAR, STLR and LDLAR/STLLR in four sizes and with XZR give a sequentially consistent builtin (STLR then the mark); LDAPR gives acquire; the LSE forms, SWP, CAS give sequentially consistent updates; LDXR, LDAXR, STXR, STLXR give the runtime calls; DMB full types, DSB and ISB give `recomp_barrier()`; DMB load-only and store-only types give the compiler-only macro. The last row fails today. - Compiled order at `-O2`, by reading the assembly: a publish pattern (data stores, then STLR; reader LDAR, then data loads) keeps every data access on its side of the flag access. With a store-only barrier between two stores, the two keep their order and the code has no fence and no call. With a full barrier a fence or the call sits between them. - Each variant the lifter has: the default, locals on (`A64_LOCAL_REGS=6`, `A64_LOCAL_FLAGS=1`) and split pieces, through the real function emitter as `tests/test_split_giants.py` does. - Optional litmus run, x86-64 only: store buffering (STLR then LDAR on two threads; both reads zero must never happen) and message passing under the lowered barriers. A pass proves nothing alone. - Game: not needed for the test. For the change, only if the count justifies it: Kokiri Forest and Hyrule Field, lifted-code CPU a frame (`HOS_PROFILE=1`, `HOS_THREAD_TIMES`), main against main first, then interleaved pairs of main and the change, on a CPU-bound PC. The lifted C of about 1,900 functions changes, so one new lift is needed. ## Related #1493 (renderer rewrite, parallel CPU track). #1469 (open: registers and flags as C locals; the one crash). #1142 (open: a test that the hot pieces' split and `-O1` against `-O2` compute the same). #1140 (open: pins the compile flags that keep float results and pointer punning safe at a higher level). #1423 and #361 (merged and closed: SDIV in `nx2/insn.py`; its test is this one's template). A search of issues and pull requests for LDAR, STLR, LDAPR, DMB and mfence finds none. New on this forge; not opened on GitHub.
Collaborator

Possibly related, for context:

  • PR #1140 test: the lifted code's flags stay semantics-safe when its optimization level rises — Related test coverage for lifted code ordering and optimization levels
  • issue #432 Idea: guest-memory fixes from the macOS port on Windows and Linux too, once timed (dirty-page atomics, zeroed pages marked, fatal allocation failures) — Related guest-memory correctness and performance changes
Possibly related, for context: - PR #1140 test: the lifted code's flags stay semantics-safe when its optimization level rises — Related test coverage for lifted code ordering and optimization levels - issue #432 Idea: guest-memory fixes from the macOS port on Windows and Linux too, once timed (dirty-page atomics, zeroed pages marked, fatal allocation failures) — Related guest-memory correctness and performance changes
Sign in to join this conversation.
No milestone
No project
No assignees
2 participants
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
MagicBeanDev/Longshot#1507
No description provided.