Every guest memory barrier becomes a full fence, and no test pins the ordering the lifter writes #1507
Labels
No labels
area/accessibility
area/audio
area/input
area/install
area/menu
area/renderer
area/saves
area/tooling
area/upscaling
beanstalk: approved
beanstalk: failed
beanstalk: needs approval
beanstalk: running
beanstalk: tested
ci: approved
good first issue
gpu/amd
gpu/igpu
gpu/intel
gpu/nvidia
help wanted
kind
bug
kind
chore
kind
docs
kind
feature
kind
performance
kind
question
needs measurements
platform/linux
platform/macos
platform/steam-deck
platform/windows
preview
priority
critical
priority
high
priority
low
regression
risk
high
risk
low
risk
medium
status
blocked
status
confirmed
status
duplicate
status
invalid
status
needs-info
status
wontfix
ci: approved
No milestone
No project
No assignees
2 participants
Notifications
Due date
No due date set.
Dependencies
No dependencies set
Reference
MagicBeanDev/Longshot#1507
Loading…
Reference in a new issue
No description provided.
Delete branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Type: performance · Area: lifter (guest memory ordering) · Related: #1493
TL;DR
The lifter writes a full fence for every ARM memory barrier. 919 of the 3,823 in the game (
dmb ishldanddmb ishst) need only a compiler barrier on x86-64. Acquire and release loads and stores, the atomic update forms and the exclusive pair are already ordered in the lifted C, so the reordering hazard first suspected in the research is not there, but no test pins any of it. I propose a game-free test of what the lifter writes for each of these, then the compiler-only lowering, and a count of executed barriers to decide whether the change is worth more than the test.Breakdown
tests/checks what ordering the lifter gives acquire/release accesses, atomics, exclusives or barriers.Evidence
Code on
forge/mainat310e7c9dfa. The research report (translation, section 2.5) said LDAR and STLR become plain accesses. That is wrong on main:nx2/insn.pyreplaces the translator's text for them.How each class is lowered now:
nx2/lift.py:51installsnx2/insn.pybefore the lift.nx2/insn.py:244-245sends LDAR, STLR, LDLAR and STLLR (all sizes) toldar_stlr(:226-239). LDAR is__atomic_load_n(..., __ATOMIC_SEQ_CST). STLR is__atomic_store_n(..., __ATOMIC_SEQ_CST)followed by the page mark. The docstring (:227-232) says why: the compiler moving other accesses across them, and ARM's order of a release store before an acquire load.insn.emiton such words gives exactly this C (run to check). The translator's own plain version,third_party/a64-kit/lift/suyu/arm64_to_c.h:1632-1641and:1730-1741, is never reached for them.nx2/insn.py:67-68,__ATOMIC_ACQUIRE. LSE updates andSWP::69-86,CAS::89-94,CASP::97-111, all__ATOMIC_SEQ_CST.recomp_ldxrandrecomp_stxr(third_party/a64-kit/rt/a64_rt.c:270-280), sequentially consistent load and compare-and-swap (:241-268).arm64_to_c.h:1108matches the whole DSB/DMB group by its fixed bits and writesrecomp_barrier();. It never reads the option field.:1102is CLREX,:1104-1107is ISB.rt/a64_rt.c:236:void recomp_barrier(void) { __atomic_thread_fence(__ATOMIC_SEQ_CST); }, an out-of-line call. It is passed nothing, so registers stay in host registers across it (nx2/rtgen.py:19-26).How many, counted from the relocated guest code (research,
translation-data/mnemonics.json, capstone):dmb ish2,892;dmb ishld870;dmb ishst49;dsb sy4;dsb ish2;isb3; one each ofdmb oshst,dmb st,dsb st. So 919 are load-only or store-only. The two other store-only forms would make 921.recomp_barrier()calls, in 1,922 of 294,522 functions (census-base.tsv).What a barrier costs: unknown.
rt/a64.h:162-165records one experiment, inlining the barrier, LDXR, STXR and CLREX in the castle: 69.5 to 68.4 fps, not kept. It compares a call with inline code, not a fence with none.Tests that touch these today:
tests/test_mark_groups.py:74-77and:115userecomp_barrier();, an atomic and the STLR form as places where a mark run ends.nx2/test_insn.pychecks values on one thread under Unicorn, which cannot see ordering. Agit grepfinds no test that names what ordering LDAR, STLR, LDAPR or a DMB gets.#1469 turns registers and flags into C locals, written back around every call and every sequentially consistent atomic. It reports one heap-corruption crash (Windows exception 0xc0000374) in about 41 runs of local-register builds and none in about 30 of main, with no cause yet. This issue does not offer one. The instruction classes above are already ordered, so we have no ordering explanation for it. What is missing is a check of whether a lift variant (locals on, split pieces, a higher level) changes the emitted order.
Proposed change
tests/test_lift_ordering.py(below), written first. Its rows for load-only and store-only barriers fail until step 2. Every other row passes today and pins it.nx2/insn.pyemit(): match DMB, read the option's type bits. Load-only and store-only types write a macro, defined bynx2/rtgen.pyinto the runtime header next to the marks (:55-65). On x86-64 it is__atomic_signal_fence(__ATOMIC_SEQ_CST), which emits no instruction. On any other host it is the real fence. Full types, DSB and ISB keeprecomp_barrier(). The vendored kit is not edited, as for #1423.host/has none, and the only ones ingpu/are the opt-in capture copy into the renderer's own buffer (gpu/vk/vk_memory.cpp:1762-1768), which ends with a store fence. The C library's largememcpymay use them; not checked here.How we would test it
tests/test_sdiv_overflow.py: the translator at-O0in a temporary folder,nx2/insn.pyinstalled asnx2/lift.pydoes, hand-encoded words througha64emit.translate, compiled at-O0,-O1and-O2. Skipped when there is no compiler.recomp_barrier(); DMB load-only and store-only types give the compiler-only macro. The last row fails today.-O2, by reading the assembly: a publish pattern (data stores, then STLR; reader LDAR, then data loads) keeps every data access on its side of the flag access. With a store-only barrier between two stores, the two keep their order and the code has no fence and no call. With a full barrier a fence or the call sits between them.A64_LOCAL_REGS=6,A64_LOCAL_FLAGS=1) and split pieces, through the real function emitter astests/test_split_giants.pydoes.HOS_PROFILE=1,HOS_THREAD_TIMES), main against main first, then interleaved pairs of main and the change, on a CPU-bound PC. The lifted C of about 1,900 functions changes, so one new lift is needed.Related
#1493 (renderer rewrite, parallel CPU track). #1469 (open: registers and flags as C locals; the one crash). #1142 (open: a test that the hot pieces' split and
-O1against-O2compute the same). #1140 (open: pins the compile flags that keep float results and pointer punning safe at a higher level). #1423 and #361 (merged and closed: SDIV innx2/insn.py; its test is this one's template). A search of issues and pull requests for LDAR, STLR, LDAPR, DMB and mfence finds none.New on this forge; not opened on GitHub.
Possibly related, for context: