WIP: perf(hle): GetCurrentThread, GetTlsValue, SetTlsValue and a thread's own GetThreadPriority answered at their call sites in lifted code (A64_OS_INLINE) #1956

Draft
siantm wants to merge 3 commits from siantm/Longshot:perf/os-inline into main
Contributor

Summary

nn::os::GetCurrentThread, GetTlsValue, SetTlsValue and GetThreadPriority are HLE functions
(host/hos_thread.c) that each read or write one field of the calling thread's record. Called from lifted code, each
pays the whole way out for it: the caller's guest registers written to the context and read back after the call, the
hooked definition's tail call, HOS_HLE_STATS's test and the host thread-local cur. GetTlsValue has the most call
sites of any HLE function in the game.

This PR answers them where lifted code calls them (lift time, A64_OS_INLINE, on by default):

  • nx2/guest_os.py rewrites each direct call, the likely arm of a devirtualised call, and each tail call of their hooked
    definitions into the field access itself, through the context's thread pointer (Ctx.thread, the HosThread
    running the context; a fiber's context takes its thread's at each switch). GetThreadPriority is answered so only for
    the calling thread's own ThreadType (the HLE function's own fast case, thread_arg); about another thread it calls
    the hooked definition as before, as a TAILCALL where the call was a tail call. Calls through a pointer and the
    emitted bodies of PLT stubs (a function pointer can enter one) still reach the HLE functions; a function that calls
    none of the hooked definitions (its refs) is not scanned.
  • The record's head is a type, HosThreadHead (guest_tt, tls_slots[32], priority), declared in the generated
    a64.h (nx2/rtgen.py), which the host includes too. HosThread starts with those fields; host/hos_thread.c
    asserts their offsets and sizes against HosThreadHead, and the lifted code reaches them as HosThreadHead
    (tls_slots indexed as its array).
  • A TLS slot (the virtual-core scheduler writes a running thread's core cell, vc_sync_tls) and the priority
    (ChangeThreadPriority, svcSetThreadPriority) are written by other threads, so every access to them is atomic
    (relaxed): in the lifted code, the GetTlsValue/SetTlsValue HLE functions, vc_pref_core, vc_sync_tls,
    hos_thread_priority (an exchange when it sets) and every scheduler and diagnostic read of the priority
    (hos_thread_prio). Left as on main (not this PR's fields): core_tls_slot, the slot index, is still a volatile
    read outside its lock in vc_pref_core, and guest_tt is written by hos_thread_adopt_guest without atomics.
  • nx2/lift.py installs it at its module level and configures it in the standalone lift from the hooks;
    nx2/chunks.py sends its table to the emit's new processes. A64_OS_INLINE=0 lifts as before; the switch is in the
    lift key. Its row is in docs/dev/reference/build.md.

On by default because it is bit-exact by test: the four HLE functions, cut from hos_thread.c, against the inlined C
over two threads' records give the same registers and the same records, and the real emit's output (one process and
spawned ones) holds that C at each site and compiles. HOS_HLE_STATS=1 no longer counts the inlined calls: in a default
lift its counts of the four are partial, which scripts/perf.py cpu --calls says in its report and
docs/dev/reference/build.md and profiling.md say (lift with A64_OS_INLINE=0 for full counts).

From 4d98f819 (its lifter and HosThread half) and bb27910b on exp/x86-asm-sync.

Read first: nx2/guest_os.py; THREAD_HEAD in nx2/rtgen.py; the HosThread head and its assert in
host/hos_thread.c; tests/test_guest_os.py and test_os_inline in tests/test_spawn_emit.py.

Testing

Tested: Windows 11, i7-9700, RTX 4060 (NVIDIA 610.88), llvm-mingw clang 23.1.5. These are game-free checks only,
plus a micro-benchmark run at an earlier form of the change. This change has not run in the game on this PC yet; the
run is queued (Still to test). Each check ran in a hidden console of its own, in one job of the PC's job queue
(fb-1956-2), on this branch rebased onto main @ f1f2dc5b (with #1949), head 2a1ad8f2:

  • python -I -B tests/test_guest_os.py (no --cc: LONGSHOT_MINGW's clang): exit 0, its lines:
    PASS: 20000 calls: the inlined GetCurrentThread, GetTlsValue, SetTlsValue and GetThreadPriority (at a call and at a tail call) as the HLE functions
    PASS: the lift's rewrite inlines the hooked calls and tail calls only (another thread's priority: a call, a tail call at a tail site, kept so in a giant's pieces), and A64_OS_INLINE=0 keeps them
    PASS: the lift installs and configures the rewrite, and its table reaches the emit's new processes
    
    The same passed with CC set, in the run before the rebase (fb-1956-1, the same fix on main @ 6e1521a7). At the
    test commit (before the fix) it fails: "FAIL: GetThreadPriority reads its own thread's and calls the hooked
    definition for another's" (fb-1956-1).
  • python -I -B tests/test_spawn_emit.py --cxx clang++ (llvm-mingw's): both tests pass ("Ran 2 tests ... OK"),
    test_os_inline among them. At the test commit 7df10ab4 it fails as it should: "AssertionError: 9 != 8 : every call
    of a hooked definition outside a PLT stub is a site, and only those", because the PLT stub's body was rewritten.
  • python -I -B tests/test_invalidate_vcore.py alone, twice at 2a1ad8f2 and once on main @ f1f2dc5b: all three
    passed (every case, immediate included). Before this round, HostLiftPRs had also run it twice alone at b922eadf,
    and both passed. Its two failures in the parallel --changed runs were timeouts: at b922eadf, the immediate case
    at 10 s; at this round's first head (fb-1956-1), its g++ compile.
  • python -I -B tests/run_all.py --strict --changed origin/main at 2a1ad8f2: exit 1, "81 passed, 15 failed, 3
    skipped". Skipped: nx2/test_leaf.py, which needs the game's modules, and tests/test_posix_thread_priority.py
    and tests/test_tracy_funcs.py, which are Linux only. All 15 failures also fail on main @ 6e1521a7 with the same
    messages; they were run there by name in fb-1956-1, and main's one commit since then (f1f2dc5b, #1870) touches
    nx2/oot_patches.py and its own test. The 15: test_crash_record, test_dev_packs, test_host_guest_writes,
    test_libc_hooks, test_lift_cache, test_linux_preflight, test_make_game_ngx, test_mods_list,
    test_oot_shadow_fit, test_perf_report, test_profiling_mode, test_sdiv_overflow, test_slow_frames,
    test_vcore_escape and test_vcore_mask0. The last two fail compiling their own harness (hos_fps_timing,
    env_on_or) on main too.

Still to test:

  • Windows 11, this PC (i7-9700, RTX 4060 (NVIDIA 610.88)): warmed and interleaved runs of main @ f1f2dc5b (the merge
    base), this PR with A64_OS_INLINE=0 (off) and this PR with it unset (on). Castle Town's market, a heavy scene and
    a fiber-heavy one, 60 s each, with the same save, settings and native resolution. Report average fps, 1% lows, and
    median and p99 frame times. Each build is a lift of its own. Expecting the same game, no crash, and the on build at
    least as fast as the other two (by hand, queued).
  • Windows and Linux: a clean lift and end-to-end build of this PR against the merge base: lift and build times,
    generated C and linked sizes. The first build of this PR compiles every lifted chunk again, because the generated
    a64.h changed (by hand).
  • Linux: a standalone lift and build, and python3 -I -B tests/test_guest_os.py and tests/test_spawn_emit.py,
    expecting the lift's line "nn::os field reads at their call sites" and the tests to pass (by hand).
  • Linux and Windows with other CPUs and GPUs (an AMD CPU, fewer cores), and the Steam Deck: Castle Town's market for two
    minutes, this PR against main (the PR's build lifts the game again), expecting the same game, no crash and frame
    times as main's or better (@Beanstalk play Castle Town's market for two minutes on PCs with AMD CPUs and on the
    Steam Deck, this PR against main, comparing frame times and checking for crashes)

Measurements

Verdict: micro-benchmark only, not measured in play yet: answered at its call site, GetTlsValue took 1.13 ns a call
against 2.56 ns through the HLE function, and a thread's own GetThreadPriority 1.14 ns against 2.96 ns (the second run:
2.56 -> 1.13 and 2.95 -> 1.13). No play measurement exists yet. These figures are of the inlined C as it was at
974297f5 (plain loads through uint64_t offsets). The micro-benchmark has not been run again at 2a1ad8f2, where the C
reads through HosThreadHead with relaxed atomics.

Baseline: the before side is not a main build but the call as lifted code makes it today, built in the same test at
974297f5 (main @ 51f5f9da with this change; the same lines as 73b63e17 after the rebase onto fc4a6769 and b922eadf after the one onto 2e1f6c32): a call of the
hooked definition, which calls the HLE function cut from hos_thread.c behind HOS_HLE_STATS's test, the context in
memory. The after side is the inlined C the lift emits instead.

Setup: Windows 11, i7-9700, llvm-mingw clang 23.1.5, -std=gnu11 -O2 (the test's own build). python -I -B tests/test_guest_os.py --cc clang --bench: a loop of 20,000,000 calls, best of 5, the test run twice in one
timing-cpu job of the PC's job queue.

Before, the call: this PR @ 974297f5 After, inlined: this PR @ 974297f5
GetTlsValue, ns a call (best of 5; run 1 / run 2) 2.56 / 2.56 1.13 / 1.13
a thread's own GetThreadPriority, ns a call (best of 5; run 1 / run 2) 2.96 / 2.95 1.14 / 1.13
bench: GetTlsValue: 2.56 ns a call, 1.13 ns inlined; own GetThreadPriority: 2.96 ns a call, 1.14 ns inlined (a loop of 20,000,000, best of 5)
bench: GetTlsValue: 2.56 ns a call, 1.13 ns inlined; own GetThreadPriority: 2.95 ns a call, 1.13 ns inlined (a loop of 20,000,000, best of 5)

Risks

A lift change: a build of this PR lifts the game again (and again with A64_OS_INLINE=0), and as the generated a64.h
gains HosThreadHead its first build compiles every lifted chunk again, with the switch on or off. The inlined C reads
the record through Ctx.thread; a context whose thread pointer were not the running thread's would read another
thread's TLS. Every guest context gets it in setup_guest and a fiber's at each switch, as cur does; the layout is
asserted against HosThreadHead. HOS_HLE_STATS=1 counts fewer calls (partial for the four, said in the report).
Turn it off with A64_OS_INLINE=0 at build time; a revert is enough. Players notice nothing (no saves, options or
caches change).

Checklist

Ready for review

  • One topic, on current main; the title is a Conventional Commit.
  • Tested on my own PC, in the game for a change the game runs, as Testing says.
  • mise run test passes (any failure that also fails on main is named).
  • Measured against main, or Measurements says why there is nothing to measure.
  • No game files, build output or hand-edited generated files; third_party/ untouched; nothing GPL-incompatible.

When it applies

  • A check under tests/ that reads changed source is updated, not weakened.
  • Host code builds on Windows (llvm-mingw) and Linux, with UTF-8 paths.
  • Docs updated.
  • AI assistance recorded with Assisted-by: / Co-Authored-By: trailers.

Ready to merge

  • Everything in Still to test has run, its results in Tested.
  • Tier 1 hardware it can affect is tested (docs/dev/standards.md, "Hardware tiers").
  • Approved in review, every comment answered.
  • CI passes on the head; on current main, measured again or a comment says why the numbers still hold.
## Summary `nn::os::GetCurrentThread`, `GetTlsValue`, `SetTlsValue` and `GetThreadPriority` are HLE functions (`host/hos_thread.c`) that each read or write one field of the calling thread's record. Called from lifted code, each pays the whole way out for it: the caller's guest registers written to the context and read back after the call, the hooked definition's tail call, `HOS_HLE_STATS`'s test and the host thread-local `cur`. `GetTlsValue` has the most call sites of any HLE function in the game. This PR answers them where lifted code calls them (lift time, `A64_OS_INLINE`, on by default): - `nx2/guest_os.py` rewrites each direct call, the likely arm of a devirtualised call, and each tail call of their hooked definitions into the field access itself, through the context's thread pointer (`Ctx.thread`, the `HosThread` running the context; a fiber's context takes its thread's at each switch). `GetThreadPriority` is answered so only for the calling thread's own `ThreadType` (the HLE function's own fast case, `thread_arg`); about another thread it calls the hooked definition as before, as a `TAILCALL` where the call was a tail call. Calls through a pointer and the emitted bodies of PLT stubs (a function pointer can enter one) still reach the HLE functions; a function that calls none of the hooked definitions (its refs) is not scanned. - The record's head is a type, `HosThreadHead` (`guest_tt`, `tls_slots[32]`, `priority`), declared in the generated `a64.h` (`nx2/rtgen.py`), which the host includes too. `HosThread` starts with those fields; `host/hos_thread.c` asserts their offsets and sizes against `HosThreadHead`, and the lifted code reaches them as `HosThreadHead` (`tls_slots` indexed as its array). - A TLS slot (the virtual-core scheduler writes a running thread's core cell, `vc_sync_tls`) and the priority (`ChangeThreadPriority`, `svcSetThreadPriority`) are written by other threads, so every access to them is atomic (relaxed): in the lifted code, the `GetTlsValue`/`SetTlsValue` HLE functions, `vc_pref_core`, `vc_sync_tls`, `hos_thread_priority` (an exchange when it sets) and every scheduler and diagnostic read of the priority (`hos_thread_prio`). Left as on `main` (not this PR's fields): `core_tls_slot`, the slot index, is still a `volatile` read outside its lock in `vc_pref_core`, and `guest_tt` is written by `hos_thread_adopt_guest` without atomics. - `nx2/lift.py` installs it at its module level and configures it in the standalone lift from the hooks; `nx2/chunks.py` sends its table to the emit's new processes. `A64_OS_INLINE=0` lifts as before; the switch is in the lift key. Its row is in `docs/dev/reference/build.md`. On by default because it is bit-exact by test: the four HLE functions, cut from `hos_thread.c`, against the inlined C over two threads' records give the same registers and the same records, and the real emit's output (one process and spawned ones) holds that C at each site and compiles. `HOS_HLE_STATS=1` no longer counts the inlined calls: in a default lift its counts of the four are partial, which `scripts/perf.py cpu --calls` says in its report and `docs/dev/reference/build.md` and `profiling.md` say (lift with `A64_OS_INLINE=0` for full counts). From 4d98f819 (its lifter and `HosThread` half) and bb27910b on `exp/x86-asm-sync`. Read first: `nx2/guest_os.py`; `THREAD_HEAD` in `nx2/rtgen.py`; the `HosThread` head and its assert in `host/hos_thread.c`; `tests/test_guest_os.py` and `test_os_inline` in `tests/test_spawn_emit.py`. ## Testing **Tested:** Windows 11, i7-9700, RTX 4060 (NVIDIA 610.88), llvm-mingw clang 23.1.5. These are game-free checks only, plus a micro-benchmark run at an earlier form of the change. This change has not run in the game on this PC yet; the run is queued (Still to test). Each check ran in a hidden console of its own, in one job of the PC's job queue (fb-1956-2), on this branch rebased onto `main` @ f1f2dc5b (with #1949), head 2a1ad8f2: - `python -I -B tests/test_guest_os.py` (no `--cc`: `LONGSHOT_MINGW`'s clang): exit 0, its lines: ``` PASS: 20000 calls: the inlined GetCurrentThread, GetTlsValue, SetTlsValue and GetThreadPriority (at a call and at a tail call) as the HLE functions PASS: the lift's rewrite inlines the hooked calls and tail calls only (another thread's priority: a call, a tail call at a tail site, kept so in a giant's pieces), and A64_OS_INLINE=0 keeps them PASS: the lift installs and configures the rewrite, and its table reaches the emit's new processes ``` The same passed with `CC` set, in the run before the rebase (fb-1956-1, the same fix on `main` @ 6e1521a7). At the test commit (before the fix) it fails: "FAIL: GetThreadPriority reads its own thread's and calls the hooked definition for another's" (fb-1956-1). - `python -I -B tests/test_spawn_emit.py --cxx clang++` (llvm-mingw's): both tests pass ("Ran 2 tests ... OK"), `test_os_inline` among them. At the test commit 7df10ab4 it fails as it should: "AssertionError: 9 != 8 : every call of a hooked definition outside a PLT stub is a site, and only those", because the PLT stub's body was rewritten. - `python -I -B tests/test_invalidate_vcore.py` alone, twice at 2a1ad8f2 and once on `main` @ f1f2dc5b: all three passed (every case, `immediate` included). Before this round, HostLiftPRs had also run it twice alone at b922eadf, and both passed. Its two failures in the parallel `--changed` runs were timeouts: at b922eadf, the `immediate` case at 10 s; at this round's first head (fb-1956-1), its g++ compile. - `python -I -B tests/run_all.py --strict --changed origin/main` at 2a1ad8f2: exit 1, "81 passed, 15 failed, 3 skipped". Skipped: `nx2/test_leaf.py`, which needs the game's modules, and `tests/test_posix_thread_priority.py` and `tests/test_tracy_funcs.py`, which are Linux only. All 15 failures also fail on `main` @ 6e1521a7 with the same messages; they were run there by name in fb-1956-1, and `main`'s one commit since then (f1f2dc5b, #1870) touches `nx2/oot_patches.py` and its own test. The 15: `test_crash_record`, `test_dev_packs`, `test_host_guest_writes`, `test_libc_hooks`, `test_lift_cache`, `test_linux_preflight`, `test_make_game_ngx`, `test_mods_list`, `test_oot_shadow_fit`, `test_perf_report`, `test_profiling_mode`, `test_sdiv_overflow`, `test_slow_frames`, `test_vcore_escape` and `test_vcore_mask0`. The last two fail compiling their own harness (`hos_fps_timing`, `env_on_or`) on `main` too. **Still to test:** - Windows 11, this PC (i7-9700, RTX 4060 (NVIDIA 610.88)): warmed and interleaved runs of `main` @ f1f2dc5b (the merge base), this PR with `A64_OS_INLINE=0` (off) and this PR with it unset (on). Castle Town's market, a heavy scene and a fiber-heavy one, 60 s each, with the same save, settings and native resolution. Report average fps, 1% lows, and median and p99 frame times. Each build is a lift of its own. Expecting the same game, no crash, and the on build at least as fast as the other two (by hand, queued). - Windows and Linux: a clean lift and end-to-end build of this PR against the merge base: lift and build times, generated C and linked sizes. The first build of this PR compiles every lifted chunk again, because the generated `a64.h` changed (by hand). - Linux: a standalone lift and build, and `python3 -I -B tests/test_guest_os.py` and `tests/test_spawn_emit.py`, expecting the lift's line "nn::os field reads at their call sites" and the tests to pass (by hand). - Linux and Windows with other CPUs and GPUs (an AMD CPU, fewer cores), and the Steam Deck: Castle Town's market for two minutes, this PR against `main` (the PR's build lifts the game again), expecting the same game, no crash and frame times as `main`'s or better (@Beanstalk play Castle Town's market for two minutes on PCs with AMD CPUs and on the Steam Deck, this PR against main, comparing frame times and checking for crashes) ## Measurements **Verdict:** micro-benchmark only, not measured in play yet: answered at its call site, GetTlsValue took 1.13 ns a call against 2.56 ns through the HLE function, and a thread's own GetThreadPriority 1.14 ns against 2.96 ns (the second run: 2.56 -> 1.13 and 2.95 -> 1.13). No play measurement exists yet. These figures are of the inlined C as it was at 974297f5 (plain loads through `uint64_t` offsets). The micro-benchmark has not been run again at 2a1ad8f2, where the C reads through `HosThreadHead` with relaxed atomics. Baseline: the before side is not a `main` build but the call as lifted code makes it today, built in the same test at 974297f5 (`main` @ 51f5f9da with this change; the same lines as 73b63e17 after the rebase onto fc4a6769 and b922eadf after the one onto 2e1f6c32): a call of the hooked definition, which calls the HLE function cut from `hos_thread.c` behind `HOS_HLE_STATS`'s test, the context in memory. The after side is the inlined C the lift emits instead. Setup: Windows 11, i7-9700, llvm-mingw clang 23.1.5, `-std=gnu11 -O2` (the test's own build). `python -I -B tests/test_guest_os.py --cc clang --bench`: a loop of 20,000,000 calls, best of 5, the test run twice in one `timing-cpu` job of the PC's job queue. | | Before, the call: this PR @ 974297f5 | After, inlined: this PR @ 974297f5 | |---|---|---| | GetTlsValue, ns a call (best of 5; run 1 / run 2) | 2.56 / 2.56 | 1.13 / 1.13 | | a thread's own GetThreadPriority, ns a call (best of 5; run 1 / run 2) | 2.96 / 2.95 | 1.14 / 1.13 | ``` bench: GetTlsValue: 2.56 ns a call, 1.13 ns inlined; own GetThreadPriority: 2.96 ns a call, 1.14 ns inlined (a loop of 20,000,000, best of 5) bench: GetTlsValue: 2.56 ns a call, 1.13 ns inlined; own GetThreadPriority: 2.95 ns a call, 1.13 ns inlined (a loop of 20,000,000, best of 5) ``` ## Risks A lift change: a build of this PR lifts the game again (and again with `A64_OS_INLINE=0`), and as the generated `a64.h` gains `HosThreadHead` its first build compiles every lifted chunk again, with the switch on or off. The inlined C reads the record through `Ctx.thread`; a context whose thread pointer were not the running thread's would read another thread's TLS. Every guest context gets it in `setup_guest` and a fiber's at each switch, as `cur` does; the layout is asserted against `HosThreadHead`. `HOS_HLE_STATS=1` counts fewer calls (partial for the four, said in the report). Turn it off with `A64_OS_INLINE=0` at build time; a revert is enough. Players notice nothing (no saves, options or caches change). ## Checklist ### Ready for review - [x] One topic, on current `main`; the title is a Conventional Commit. - [ ] Tested on my own PC, in the game for a change the game runs, as Testing says. - [ ] `mise run test` passes (any failure that also fails on `main` is named). - [x] Measured against `main`, or Measurements says why there is nothing to measure. - [x] No game files, build output or hand-edited generated files; `third_party/` untouched; nothing GPL-incompatible. ### When it applies - [x] A check under `tests/` that reads changed source is updated, not weakened. - [ ] Host code builds on Windows (llvm-mingw) and Linux, with UTF-8 paths. - [x] Docs updated. - [x] AI assistance recorded with `Assisted-by:` / `Co-Authored-By:` trailers. ### Ready to merge - [ ] Everything in Still to test has run, its results in Tested. - [ ] Tier 1 hardware it can affect is tested (docs/dev/standards.md, "Hardware tiers"). - [ ] Approved in review, every comment answered. - [ ] CI passes on the head; on current `main`, measured again or a comment says why the numbers still hold.
perf(hle): GetCurrentThread, GetTlsValue, SetTlsValue and a thread's own GetThreadPriority answered at their call sites in lifted code (A64_OS_INLINE)
Some checks failed
PR description / Description follows the template (pull_request_target) Successful in 8s
Website / Website on Cloudflare (pull_request_target) Has been skipped
CI / Conventional Commit title (pull_request) Has been cancelled
CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Has been cancelled
CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Has been cancelled
CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Has been cancelled
CI / Workflow lint (actionlint) (pull_request) Has been cancelled
CI / Website build and checks (pull_request) Has been cancelled
Guide screenshots / Guide images and local UI source review (pull_request) Has been cancelled
Third-party parts / As their notes say (pull_request) Has been cancelled
Third-party parts / The same as upstream (pull_request) Has been cancelled
Website / Website on Cloudflare (pull_request) Has been cancelled
b922eadf4a
Each of the four is an HLE function that reads or writes one field of the calling thread's record. Called from lifted
code it pays the way out: the caller's registers synced to the context and back, the hooked definition's tail call,
HOS_HLE_STATS's test and the host thread-local. nx2/guest_os.py rewrites each direct call, devirtualised call and tail
call of their hooked definitions into the field access itself through Ctx.thread (the HosThread running the context);
GetThreadPriority only about the calling thread's own ThreadType, the hooked definition as before about another. Calls
through a pointer still reach the HLE functions. HosThread now starts with guest_tt, tls_slots and priority, at offsets
_Static_assert'ed in host/hos_thread.c. Lift time, on by default (bit-exact against the HLE functions by test);
A64_OS_INLINE=0 lifts as before. HOS_HLE_STATS=1 no longer counts the inlined calls.

tests/test_guest_os.py: the rewrite (calls, the devirtualised arm, tail calls, other calls kept, =0 and a lift without
the hooks unchanged), the wiring in nx2/lift.py and nx2/chunks.py, and the four HLE functions cut from hos_thread.c
against the inlined C over two threads' records, 16,000 calls; --bench times GetTlsValue and GetThreadPriority called
and inlined.

From 4d98f819 (its lifter and HosThread half) and bb27910b on exp/x86-asm-sync. vc_kick stays where main has it (the
mutex change that reads it from the context is a branch of its own).

Assisted-by: Claude Code
Member

Possibly related, for context:

  • PR #1952 WIP: perf(host): four switches read on per-call paths are read once — Same author and nearby lifted/host performance work; no direct overlap in changed code.
  • PR #1209 perf(hos): splice the host's memcpy, memset and string hooks into the lifted SDK — Merged precedent for splicing host fast paths into lifted code.
Possibly related, for context: - PR #1952 WIP: perf(host): four switches read on per-call paths are read once — Same author and nearby lifted/host performance work; no direct overlap in changed code. - PR #1209 perf(hos): splice the host's memcpy, memset and string hooks into the lifted SDK — Merged precedent for splicing host fast paths into lifted code.
BeanBot left a comment

Recommendation: Request changes. The new lifted accesses carry unsynchronized TLS and priority reads into code the compiler can optimize across call sites, and the TLS access indexes beyond the first C member. The Windows microbenchmark shows cheaper individual calls, but the default-on change still needs gameplay and build measurements.

The standalone lifter replaces four common OS calls with access to the running thread’s record. That can save host calls and register traffic, at the cost of coupling generated C to the host thread layout, adding work to the lift, and reducing the coverage of HLE call counts.

Does it help: Unproven in play. The author’s Windows 11 i7-9700 fixture at 974297f5 reported GetTlsValue at 2.56 → 1.13 ns in both runs and own GetThreadPriority at 2.96 → 1.14 ns and 2.95 → 1.13 ns. Each result is the best of five 20-million-call loops. There are no matched base-versus-HEAD gameplay runs, frame-time distributions, or clean-build time and size results.
Other systems: The change runs on the CPU across GPU vendors and drivers. Windows uses spawned emit workers, while Linux ordinarily forks. Both rely on the same thread layout and generated code, and neither has a reported production build and play run at HEAD. Virtual-core scheduling shares the TLS array and priority field. Fiber switches update Ctx.thread before resumption, making a migrated fiber an important test case. Profiling reports can undercount the four calls. The diff has no direct changes to rendering, audio, input, saves, or the renderer API.
Switches: A64_OS_INLINE is a new lift-time developer switch. Unset enables it and 0 disables it. The lift key includes it through the A64_ prefix, and the build reference documents it. The runtime switch table applies to switches the game reads. The default needs the correctness fixes and matched gameplay and build results. HOS_HLE_STATS remains off by default, and enabling it on a default lift yields partial counts for the four rewritten functions.

Reviewed: b922eadf4a against 2e1f6c32d3: Read nx2/guest_os.py, nx2/lift.py, nx2/chunks.py, tests/test_guest_os.py and docs/dev/reference/build.md in full. Read the full diff and commit, the changed host/hos_thread.c sections and the relevant setup, lifecycle, fiber, HLE and scheduler code. Traced hook resolution, register sync, worker initialization, the lift key, giant-function splitting, the test runner and profiling consumers. Read AGENTS.md, CONTRIBUTING.md, the PR template and developer standards. The local review checklist and sibling Tooling checkout were absent. Skimmed the recorded-run index and read the supplied testing, measurements and bot comment. No generated files or translations changed. Unrelated host code and historical runs were outside the detailed read.; nothing run (a code review).
Checked and fine:

  • The TLS index is truncated to uint32_t before the range check, matching the existing HLE. Slots 32 and above take the out-of-range path after truncation.
  • The real emitter places register save and reload operations around the rewritten direct and devirtualised calls. The rewrite leaves the link register assignment and the unmatched BLR arm in place.
  • Thread setup assigns Ctx.thread before lifted execution. Fiber entry and resumption assign the destination context the running thread’s identity before switching.
  • The lift key follows the new Python import and hashes A64_OS_INLINE. The worker table carries the configured map, and stats deltas are already aggregated by the emitter.
  • The change is one topic and the commit carries an Assisted-by trailer. The diff contains no game-derived artifacts, generated output, vendor changes or renderer API changes. The whitespace check was clean.

Blocking (to change before merge)

  1. nx2/guest_os.py:35: The cast points at guest_tt, the first uint64_t member. Indexing it as [1 + slot] crosses that scalar subobject, even for valid slot 0. SetTlsValue does the same. Address tls_slots as its actual array member, ideally through a shared typed layout used by the host and generated C.
  2. nx2/guest_os.py:38: The lifted TLS load and store race with vc_sync_tls. A same-core SetThreadCoreMask can write the running target’s cached core slot while the target calls GetTlsValue or SetTlsValue. The scheduler’s lock excludes slot allocation and freeing, while these accesses take no lock. Use consistent atomic access in the lifted code, the HLE and the scheduler, including vc_pref_core.
  3. nx2/guest_os.py:41: Own-thread priority can change from another thread through ChangeThreadPriority or SVC SetThreadPriority. The inlined plain load carries the existing C data race into lifted code, where repeated reads can be reused. Use atomic reads throughout and an atomic exchange for the setter that returns the old priority, including the scheduler and diagnostic readers.
  4. nx2/guest_os.py:70: A tail call to GetThreadPriority about another thread changes from a guaranteed TAILCALL to f_...(c) followed by return. That retains the caller frame if the compiler does not tail-call it. Keep TAILCALL in the other-thread arm for a tail site, including when the giant-function splitter handles it.
  5. nx2/guest_os.py:89: The wrapper also rewrites emitted PLT bodies. The lifter emits those bodies for indirect entry, so a function pointer to a PLT stub can now bypass the HLE despite the documented indirect-call behavior. Leave the emitted PLT body intact and test indirect entry through a stub. Direct calls already resolve through callee().
  6. nx2/guest_os.py:90: The regex scans every emitted function on a default standalone lift, including functions with no eligible callee. This adds work across roughly 1.5 GB of lifted C on a full build. Use the emitter’s existing call references to skip bodies with no candidate targets.
  7. docs/dev/reference/build.md:73: The normal HOS_HLE_STATS profiling workflow now silently sees a partial sample. For example, scripts/perf.py cpu --calls can treat the remaining indirect GetTlsValue calls as the full calls-per-frame count. Preserve counts in profiling mode or mark the output as partial and make the profiling workflow require a lift with A64_OS_INLINE=0 for full counts.
  8. tests/test_guest_os.py:90: The fixture compiles READS directly and checks wiring by source text. If the installed emitter calls inline() but discards its result, all three PASS messages can still print. Extend the existing spawn-emitter fixture with synthetic hooks and compile C from the real emitter. Exercise localized registers, the direct and devirtualised arms, tail and indirect paths, and compare single-process with actual spawned output with the switch both ways.
  9. tests/test_guest_os.py:32: Both the suite and standalone entry point force gcc. A supported Linux setup with CC=clang and no gcc cannot run the check, and a setup with both ignores CC. Honor --cc first and CC next, with a platform-appropriate compiler fallback, and have the runner check the compiler the fixture will use.

Waiting on the author (a run, a measurement or a decision; no edit does it)

  • The PR opened without the required in-game run. At the reviewed head, compare the merge base and default build in interleaved runs with the same save, settings and native resolution. Report average FPS, 1% lows, median and p99 frame time in the target and a heavy scene. Cover Windows and Linux Tier 1 NVIDIA and AMD systems plus Steam Deck, and include a fiber-heavy scene and an off build to isolate the change.
  • A fresh install or rebuild pays for the changed lift and compiles larger call sites. The fixture cannot measure that cost, including on low-core machines. After reducing the scan, compare clean lift and end-to-end build times and generated and linked sizes against the merge base on Windows and Linux.
  • The reported Windows suite timed out in test_invalidate_vcore.py’s immediate case, which passed on the tested baseline and sibling branches. Rerun that case and the suite on the reviewed head, check the base under matched conditions if the timeout repeats, and provide passing head CI.

Suggestions (the author may take or leave)

  • If the microbenchmark stays, use a monotonic clock and alternate the two variants. Its stand-ins for cur and HLE stats and its fixed synthetic context limit how closely the timings represent a production build.

Testing: Read-only review. The author reports 16,000 fixture comparisons passing on Windows at 73b63e17. The changed suite there reported 51 passed, 13 failed and 3 skipped, with 12 failures also on the corresponding baseline. No game run, Linux standalone build, or passing CI at b922eadf4a was supplied.
Overlaps and merge order: The supplied bot comment identifies #1952 as related with no direct file overlap and no merge-order dependency. #1209 is a merged precedent. The description mentions sibling mutex branches carrying the lift change, with no prerequisite shown.


Automated code review by BeanBot (gpt-6.2-sol), on the pull request's opening. Nothing was run; a maintainer decides.

**Recommendation: Request changes.** The new lifted accesses carry unsynchronized TLS and priority reads into code the compiler can optimize across call sites, and the TLS access indexes beyond the first C member. The Windows microbenchmark shows cheaper individual calls, but the default-on change still needs gameplay and build measurements. The standalone lifter replaces four common OS calls with access to the running thread’s record. That can save host calls and register traffic, at the cost of coupling generated C to the host thread layout, adding work to the lift, and reducing the coverage of HLE call counts. **Does it help:** Unproven in play. The author’s Windows 11 i7-9700 fixture at 974297f5 reported GetTlsValue at 2.56 → 1.13 ns in both runs and own GetThreadPriority at 2.96 → 1.14 ns and 2.95 → 1.13 ns. Each result is the best of five 20-million-call loops. There are no matched base-versus-HEAD gameplay runs, frame-time distributions, or clean-build time and size results. **Other systems:** The change runs on the CPU across GPU vendors and drivers. Windows uses spawned emit workers, while Linux ordinarily forks. Both rely on the same thread layout and generated code, and neither has a reported production build and play run at HEAD. Virtual-core scheduling shares the TLS array and priority field. Fiber switches update Ctx.thread before resumption, making a migrated fiber an important test case. Profiling reports can undercount the four calls. The diff has no direct changes to rendering, audio, input, saves, or the renderer API. **Switches:** A64_OS_INLINE is a new lift-time developer switch. Unset enables it and 0 disables it. The lift key includes it through the A64_ prefix, and the build reference documents it. The runtime switch table applies to switches the game reads. The default needs the correctness fixes and matched gameplay and build results. HOS_HLE_STATS remains off by default, and enabling it on a default lift yields partial counts for the four rewritten functions. **Reviewed:** b922eadf4a99 against 2e1f6c32d30e: Read nx2/guest_os.py, nx2/lift.py, nx2/chunks.py, tests/test_guest_os.py and docs/dev/reference/build.md in full. Read the full diff and commit, the changed host/hos_thread.c sections and the relevant setup, lifecycle, fiber, HLE and scheduler code. Traced hook resolution, register sync, worker initialization, the lift key, giant-function splitting, the test runner and profiling consumers. Read AGENTS.md, CONTRIBUTING.md, the PR template and developer standards. The local review checklist and sibling Tooling checkout were absent. Skimmed the recorded-run index and read the supplied testing, measurements and bot comment. No generated files or translations changed. Unrelated host code and historical runs were outside the detailed read.; nothing run (a code review). **Checked and fine:** - The TLS index is truncated to uint32_t before the range check, matching the existing HLE. Slots 32 and above take the out-of-range path after truncation. - The real emitter places register save and reload operations around the rewritten direct and devirtualised calls. The rewrite leaves the link register assignment and the unmatched BLR arm in place. - Thread setup assigns Ctx.thread before lifted execution. Fiber entry and resumption assign the destination context the running thread’s identity before switching. - The lift key follows the new Python import and hashes A64_OS_INLINE. The worker table carries the configured map, and stats deltas are already aggregated by the emitter. - The change is one topic and the commit carries an Assisted-by trailer. The diff contains no game-derived artifacts, generated output, vendor changes or renderer API changes. The whitespace check was clean. **Blocking** (to change before merge) 1. nx2/guest_os.py:35: The cast points at guest_tt, the first uint64_t member. Indexing it as [1 + slot] crosses that scalar subobject, even for valid slot 0. SetTlsValue does the same. Address tls_slots as its actual array member, ideally through a shared typed layout used by the host and generated C. 2. nx2/guest_os.py:38: The lifted TLS load and store race with vc_sync_tls. A same-core SetThreadCoreMask can write the running target’s cached core slot while the target calls GetTlsValue or SetTlsValue. The scheduler’s lock excludes slot allocation and freeing, while these accesses take no lock. Use consistent atomic access in the lifted code, the HLE and the scheduler, including vc_pref_core. 3. nx2/guest_os.py:41: Own-thread priority can change from another thread through ChangeThreadPriority or SVC SetThreadPriority. The inlined plain load carries the existing C data race into lifted code, where repeated reads can be reused. Use atomic reads throughout and an atomic exchange for the setter that returns the old priority, including the scheduler and diagnostic readers. 4. nx2/guest_os.py:70: A tail call to GetThreadPriority about another thread changes from a guaranteed TAILCALL to f_...(c) followed by return. That retains the caller frame if the compiler does not tail-call it. Keep TAILCALL in the other-thread arm for a tail site, including when the giant-function splitter handles it. 5. nx2/guest_os.py:89: The wrapper also rewrites emitted PLT bodies. The lifter emits those bodies for indirect entry, so a function pointer to a PLT stub can now bypass the HLE despite the documented indirect-call behavior. Leave the emitted PLT body intact and test indirect entry through a stub. Direct calls already resolve through callee(). 6. nx2/guest_os.py:90: The regex scans every emitted function on a default standalone lift, including functions with no eligible callee. This adds work across roughly 1.5 GB of lifted C on a full build. Use the emitter’s existing call references to skip bodies with no candidate targets. 7. docs/dev/reference/build.md:73: The normal HOS_HLE_STATS profiling workflow now silently sees a partial sample. For example, scripts/perf.py cpu --calls can treat the remaining indirect GetTlsValue calls as the full calls-per-frame count. Preserve counts in profiling mode or mark the output as partial and make the profiling workflow require a lift with A64_OS_INLINE=0 for full counts. 8. tests/test_guest_os.py:90: The fixture compiles READS directly and checks wiring by source text. If the installed emitter calls inline() but discards its result, all three PASS messages can still print. Extend the existing spawn-emitter fixture with synthetic hooks and compile C from the real emitter. Exercise localized registers, the direct and devirtualised arms, tail and indirect paths, and compare single-process with actual spawned output with the switch both ways. 9. tests/test_guest_os.py:32: Both the suite and standalone entry point force gcc. A supported Linux setup with CC=clang and no gcc cannot run the check, and a setup with both ignores CC. Honor --cc first and CC next, with a platform-appropriate compiler fallback, and have the runner check the compiler the fixture will use. **Waiting on the author** (a run, a measurement or a decision; no edit does it) - The PR opened without the required in-game run. At the reviewed head, compare the merge base and default build in interleaved runs with the same save, settings and native resolution. Report average FPS, 1% lows, median and p99 frame time in the target and a heavy scene. Cover Windows and Linux Tier 1 NVIDIA and AMD systems plus Steam Deck, and include a fiber-heavy scene and an off build to isolate the change. - A fresh install or rebuild pays for the changed lift and compiles larger call sites. The fixture cannot measure that cost, including on low-core machines. After reducing the scan, compare clean lift and end-to-end build times and generated and linked sizes against the merge base on Windows and Linux. - The reported Windows suite timed out in test_invalidate_vcore.py’s immediate case, which passed on the tested baseline and sibling branches. Rerun that case and the suite on the reviewed head, check the base under matched conditions if the timeout repeats, and provide passing head CI. **Suggestions** (the author may take or leave) - If the microbenchmark stays, use a monotonic clock and alternate the two variants. Its stand-ins for cur and HLE stats and its fixed synthetic context limit how closely the timings represent a production build. **Testing:** Read-only review. The author reports 16,000 fixture comparisons passing on Windows at 73b63e17. The changed suite there reported 51 passed, 13 failed and 3 skipped, with 12 failures also on the corresponding baseline. No game run, Linux standalone build, or passing CI at b922eadf4a99 was supplied. **Overlaps and merge order:** The supplied bot comment identifies #1952 as related with no direct file overlap and no merge-order dependency. #1209 is a merged precedent. The description mentions sibling mutex branches carrying the lift change, with no prerequisite shown. --- <sub>Automated code review by BeanBot (gpt-6.2-sol), on the pull request's opening. Nothing was run; a maintainer decides.</sub>
@ -70,6 +70,7 @@ Read by the build, not the game, so they are not in `switches.def`.
| `A64_FASTFP`, `A64_LOCAL_REGS`, `A64_LOCAL_FLAGS`, `A64_LOCAL_FUNCS` | Lift-time switches (`nx2/fastfp.py`, `nx2/guest_locals.py`); they change the lift key |
| `A64_FASTFP_STORES=0`, `A64_FASTFP_VMOVES=0` | Fast float regions (`nx2/fastfp.py`) stop at a guest store, or at a vector move or shuffle (`nx2/fastvec.py`'s `a64_vld16`, `a64_vdup`, `A64_EXT16`, `A64_EXT8`, `a64_tbl1`), as before. Both on by default: a region may hold stores but no load after one, and its fast copy keeps their addresses and values and stores them, in order, once its MXCSR check passed (its exact copy is unchanged). Lift-time switches: they change the lift key |
| `A64_VECFP=0` or a list of `elem`, `vec`, `minmax`, `fcvt` | The SSE fast paths for the float forms the translator otherwise does lane by lane (`nx2/vecfp.py`); each keeps a result only where it is provably Arm's, else the instruction's own C runs. All on by default; `0` lifts exactly as without them. A lift-time switch: it changes the lift key |
| `A64_OS_INLINE=0` | nn::os::GetCurrentThread, GetTlsValue, SetTlsValue and GetThreadPriority called as HLE functions, as before. On by default: each direct call (or tail call) of their hooked definitions is the field read or write itself, through the context's thread pointer (`nx2/guest_os.py`; the thread record's head is fixed and asserted in `host/hos_thread.c`); GetThreadPriority so only about the calling thread's own ThreadType, a call as before about another. The same x0, other registers and records as the HLE functions (`tests/test_guest_os.py`), no checkpoint skipped (they had none); calls through a pointer still reach the HLE functions, and `HOS_HLE_STATS=1` does not count the inlined ones. A lift-time switch: it changes the lift key |
Member

Mark partial counts in the profiling output and document the required off lift for full counts, or preserve full counts when HOS_HLE_STATS is enabled.

Mark partial counts in the profiling output and document the required off lift for full counts, or preserve full counts when HOS_HLE_STATS is enabled.
nx2/guest_os.py Outdated
@ -0,0 +32,4 @@
READS = {
"_ZN2nn2os16GetCurrentThreadEv": "{ c->x[0] = ((const uint64_t *)c->thread)[0]; }",
"_ZN2nn2os11GetTlsValueENS0_7TlsSlotE":
"{ const uint32_t _slot = (uint32_t)c->x[0]; c->x[0] = _slot < %du ? ((const uint64_t *)c->thread)[1 + _slot] : 0; }"
Member

The uint64_t pointer starts at guest_tt, so [1 + slot] crosses a scalar member even for slot 0. Address the real tls_slots array in both accessors.

The uint64_t pointer starts at guest_tt, so [1 + slot] crosses a scalar member even for slot 0. Address the real tls_slots array in both accessors.
nx2/guest_os.py Outdated
@ -0,0 +35,4 @@
"{ const uint32_t _slot = (uint32_t)c->x[0]; c->x[0] = _slot < %du ? ((const uint64_t *)c->thread)[1 + _slot] : 0; }"
% SLOTS,
"_ZN2nn2os11SetTlsValueENS0_7TlsSlotEm":
"{ const uint32_t _slot = (uint32_t)c->x[0]; if (_slot < %du) ((uint64_t *)c->thread)[1 + _slot] = c->x[1]; }" % SLOTS,
Member

vc_sync_tls can write this slot from another thread during a same-core mask change. Use atomic accesses on the lifted, HLE and scheduler paths.

vc_sync_tls can write this slot from another thread during a same-core mask change. Use atomic accesses on the lifted, HLE and scheduler paths.
nx2/guest_os.py Outdated
@ -0,0 +38,4 @@
"{ const uint32_t _slot = (uint32_t)c->x[0]; if (_slot < %du) ((uint64_t *)c->thread)[1 + _slot] = c->x[1]; }" % SLOTS,
"_ZN2nn2os17GetThreadPriorityEPKNS0_10ThreadTypeE":
"{ const uint8_t *_th = (const uint8_t *)c->thread; if (c->x[0] == *(const uint64_t *)_th) "
"c->x[0] = (uint32_t)*(const int32_t *)(_th + %d); else @F@(c); }" % PRIORITY_AT,
Member

Another thread can set this priority through nn::os or SVC. Use an atomic load here and in host readers, with an atomic exchange for the setter.

Another thread can set this priority through nn::os or SVC. Use an atomic load here and in host readers, with an atomic exchange for the setter.
nx2/guest_os.py Outdated
@ -0,0 +67,4 @@
if body is None:
return m.group(0)
n += 1
return body + (" return;" if tail else "")
Member

Keep the guaranteed TAILCALL on the other-thread fallback at tail sites. An ordinary call followed by return can retain the caller frame.

Keep the guaranteed TAILCALL on the other-thread fallback at tail sites. An ordinary call followed by return can retain the caller frame.
nx2/guest_os.py Outdated
@ -0,0 +86,4 @@
def emit_function(self, e, texts):
text, refs = base(self, e, texts)
table = getattr(self, "os_inline", None)
if table:
Member

Emitted PLT bodies can be entered by an indirect call. Keep their HLE tail calls so function pointers through those stubs retain the documented behavior.

Emitted PLT bodies can be entered by an indirect call. Keep their HLE tail calls so function pointers through those stubs retain the documented behavior.
nx2/guest_os.py Outdated
@ -0,0 +87,4 @@
text, refs = base(self, e, texts)
table = getattr(self, "os_inline", None)
if table:
text, n = inline(text, table)
Member

Use refs to skip scanning emitted functions with no eligible callees. The full lift emits roughly 1.5 GB of C.

Use refs to skip scanning emitted functions with no eligible callees. The full lift emits roughly 1.5 GB of C.
@ -0,0 +29,4 @@
import kit # noqa: E402,F401 (a64-kit's folders on the path)
import guest_os # noqa: E402
RUN_ALL_ARGS = ["--cc", "gcc"]
Member

Honor CC in the suite and standalone entry point so a supported Clang-only Linux setup can run this fixture.

Honor CC in the suite and standalone entry point so a supported Clang-only Linux setup can run this fixture.
@ -0,0 +87,4 @@
"""nx2/lift.py installs the wrapper at its module level (the emit's new processes run it too) and configures it in the
standalone lift after the hooks are known; nx2/chunks.py sends the table to those processes"""
import chunks
check("os_inline" in chunks.EMIT_TABLES, "nx2/chunks.py's EMIT_TABLES sends os_inline to the emit's new processes")
Member

These checks pass if the wrapper discards its rewrite result. Compile real emitted C and compare single-process with actual spawned output using synthetic hooks.

These checks pass if the wrapper discards its rewrite result. Compile real emitted C and compare single-process with actual spawned output using synthetic hooks.
futondestroyer left a comment

Recommendation: Request changes. Use the actual TLS array and synchronize access to the slots and priority before enabling these shortcuts. Preserve tail calls and indirect entry behavior in the emitted code. Existing exact-head review

The TLS access indexes from guest_tt, a scalar member. The virtual-core scheduler can also write a running thread’s core slot while its lifted code accesses it, and another guest thread can change its priority. A shared typed layout and consistent atomic access across the lifted code, HLE and scheduler paths would address both issues. On the emitter side, keep the other-thread priority fallback as a tail call and skip PLT bodies that can be entered through a function pointer. Extend the fixture to compile real emitted C in both single-process and spawned modes. Existing TLS finding Scheduler finding Priority finding Tail call finding Indirect entry finding Fixture finding

Does it help? Unproven in play. The author’s Windows i7-9700 fixture reports 2.56 → 1.13 ns per GetTlsValue call and 2.96 → 1.14 ns for a thread’s own GetThreadPriority, best of five. A64_OS_INLINE defaults on for standalone lifts and 0 restores the HLE calls. The lift key includes it. A default lift counts only the remaining HLE calls when profiling with HOS_HLE_STATS. Existing profiling finding

Testing and merge order. Rebase through the renderer compile and Linux link fixes before collecting build or gameplay evidence. The Windows fixture passed before the last rebase. Rerun the Windows suite, including the test_invalidate_vcore.py timeout. That fixture uses its own thread layout and compiles helpers unchanged by this diff. Compare the rebased main and candidate in warmed, interleaved Market and heavy-scene runs. Include average FPS, 1% lows, median and p99 frame times, a build with the switch off, and clean lift/build time and size on Windows and Linux. Cover Tier 1 platforms, including Steam Deck. Head CI is pending and the farm has no jobs or reports. The register-pinning experiment also touches the lifter, so check generated code with both enabled if the changes are combined. Renderer compile fix Linux link fix Farm Register pinning PR

Reviewed b922eadf against 2e1f6c32. Read all six changed files at the diff, used revision-matched Graphify and checked the affected emitter, thread setup, fiber switch and scheduler source. The slot truncation and return convention match the HLE, and local register sync remains around direct and devirtualized calls. Nothing from the PR was run.

**Recommendation: Request changes.** Use the actual TLS array and synchronize access to the slots and priority before enabling these shortcuts. Preserve tail calls and indirect entry behavior in the emitted code. [Existing exact-head review](https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/1956#issuecomment-62005) The TLS access indexes from `guest_tt`, a scalar member. The virtual-core scheduler can also write a running thread’s core slot while its lifted code accesses it, and another guest thread can change its priority. A shared typed layout and consistent atomic access across the lifted code, HLE and scheduler paths would address both issues. On the emitter side, keep the other-thread priority fallback as a tail call and skip PLT bodies that can be entered through a function pointer. Extend the fixture to compile real emitted C in both single-process and spawned modes. [Existing TLS finding](https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/1956#issuecomment-61996) [Scheduler finding](https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/1956#issuecomment-61997) [Priority finding](https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/1956#issuecomment-61998) [Tail call finding](https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/1956#issuecomment-61999) [Indirect entry finding](https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/1956#issuecomment-62000) [Fixture finding](https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/1956#issuecomment-62003) **Does it help?** Unproven in play. The author’s Windows i7-9700 fixture reports 2.56 → 1.13 ns per GetTlsValue call and 2.96 → 1.14 ns for a thread’s own GetThreadPriority, best of five. `A64_OS_INLINE` defaults on for standalone lifts and `0` restores the HLE calls. The lift key includes it. A default lift counts only the remaining HLE calls when profiling with `HOS_HLE_STATS`. [Existing profiling finding](https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/1956#issuecomment-62002) **Testing and merge order.** Rebase through the renderer compile and Linux link fixes before collecting build or gameplay evidence. The Windows fixture passed before the last rebase. Rerun the Windows suite, including the `test_invalidate_vcore.py` timeout. That fixture uses its own thread layout and compiles helpers unchanged by this diff. Compare the rebased main and candidate in warmed, interleaved Market and heavy-scene runs. Include average FPS, 1% lows, median and p99 frame times, a build with the switch off, and clean lift/build time and size on Windows and Linux. Cover Tier 1 platforms, including Steam Deck. Head CI is pending and the farm has no jobs or reports. The register-pinning experiment also touches the lifter, so check generated code with both enabled if the changes are combined. [Renderer compile fix](https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/1949) [Linux link fix](https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/1961) [Farm](https://beanstalk.magicbean.dev/prs/1956) [Register pinning PR](https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/1981) **Reviewed** `b922eadf` against `2e1f6c32`. Read all six changed files at the diff, used revision-matched Graphify and checked the affected emitter, thread setup, fiber switch and scheduler source. The slot truncation and return convention match the HLE, and local register sync remains around direct and devirtualized calls. Nothing from the PR was run.
siantm force-pushed perf/os-inline from b922eadf4a
Some checks failed
PR description / Description follows the template (pull_request_target) Successful in 8s
Website / Website on Cloudflare (pull_request_target) Has been skipped
CI / Conventional Commit title (pull_request) Has been cancelled
CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Has been cancelled
CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Has been cancelled
CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Has been cancelled
CI / Workflow lint (actionlint) (pull_request) Has been cancelled
CI / Website build and checks (pull_request) Has been cancelled
Guide screenshots / Guide images and local UI source review (pull_request) Has been cancelled
Third-party parts / As their notes say (pull_request) Has been cancelled
Third-party parts / The same as upstream (pull_request) Has been cancelled
Website / Website on Cloudflare (pull_request) Has been cancelled
to 2a1ad8f217
Some checks are pending
CI / Conventional Commit title (pull_request) Blocked by required conditions
CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Blocked by required conditions
CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Blocked by required conditions
CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Blocked by required conditions
CI / Workflow lint (actionlint) (pull_request) Blocked by required conditions
CI / Website build and checks (pull_request) Blocked by required conditions
Guide screenshots / Guide images and local UI source review (pull_request) Blocked by required conditions
Third-party parts / As their notes say (pull_request) Blocked by required conditions
Third-party parts / The same as upstream (pull_request) Blocked by required conditions
Website / Website on Cloudflare (pull_request) Blocked by required conditions
Website / Website on Cloudflare (pull_request_target) Has been skipped
beanstalk/approval waiting for a BigBean's 👍 on the request
PR description / Description follows the template (pull_request_target) Successful in 6s
2026-11-21 04:40:25 +00:00
Compare
Author
Contributor

@Beanstalk This lifter change (A64_OS_INLINE, on by default, read at lift time) answers GetCurrentThread, GetTlsValue, SetTlsValue and a thread's own GetThreadPriority inline in the lifted code (nx2/guest_os.py, the HosThread head in host/hos_thread.c), so the PR's build lifts the game again and its lifted C differs from main's by design. It has not run in the game here with this head yet. Please time the lift against main's, then play Castle Town Market for two minutes, this PR against main, on a Windows PC with an AMD processor, a Linux PC with an AMD processor, a PC with 6 or fewer logical processors and the Steam Deck. No crash or hang, and the same game; frame time p99 equal to main's or better, within the run-to-run spread; lift time within main's spread or better.

@Beanstalk This lifter change (A64_OS_INLINE, on by default, read at lift time) answers GetCurrentThread, GetTlsValue, SetTlsValue and a thread's own GetThreadPriority inline in the lifted code (nx2/guest_os.py, the HosThread head in host/hos_thread.c), so the PR's build lifts the game again and its lifted C differs from main's by design. It has not run in the game here with this head yet. Please time the lift against main's, then play Castle Town Market for two minutes, this PR against main, on a Windows PC with an AMD processor, a Linux PC with an AMD processor, a PC with 6 or fewer logical processors and the Steam Deck. No crash or hang, and the same game; frame time p99 equal to main's or better, within the run-to-run spread; lift time within main's spread or better.
Member

@siantm only members of BigBeans can start the farm. One of them can approve this request by reacting 👍 to it or to this comment.

@siantm only members of `BigBeans` can start the farm. One of them can approve this request by reacting 👍 to it or to this comment.
Author
Contributor

@futondestroyer thanks for the review (62741). Fixed in 2a1ad8f2, with its tests in 7df10ab4 first. The branch is rebased onto main @ f1f2dc5b, which includes #1949. git range-diff shows the PR's own commit unchanged (aca07f9c = b922eadf).

  • The actual TLS array. The record's head is now a type, HosThreadHead (guest_tt, tls_slots[32], priority). It is declared once, in the generated a64.h (nx2/rtgen.py THREAD_HEAD), which lifted code and the host both include. host/hos_thread.c asserts that HosThread's fields have the same offsets and sizes as HosThreadHead. The inlined C now indexes ((HosThreadHead *)c->thread)->tls_slots[_slot], so it no longer reaches past guest_tt.
  • Synchronised slots and priority. Every access to a TLS slot and to the priority is now atomic (relaxed). That covers the lifted code, the GetTlsValue/SetTlsValue HLE functions, vc_pref_core's read and vc_sync_tls's write of the core cell, and hos_thread_priority, which now uses an exchange when it sets so it can return the old value. It also covers every scheduler and diagnostic read of the priority, which now go through an atomic hos_thread_prio. I removed the unused hos_tls_slots, which handed out the raw array. I kept this inside the PR: it is one file of host code plus the generated C. Two related things stay as they are on main, because they are not fields this PR inlines. The slot index core_tls_slot is still a volatile read outside tls_slot_lock in vc_pref_core. hos_thread_adopt_guest still writes guest_tt plainly. The description says so.
  • Tail calls. At a tail site, GetThreadPriority about another thread is a TAILCALL of the hooked definition again, not a call followed by a return. nx2/split_giants.py keeps it a tail call: the piece returns the tail id and the dispatcher's case is TAILCALL(f_…). tests/test_guest_os.py checks this through a cut giant, and its C fixture runs the tail form against the HLE function.
  • Indirect entry. A PLT stub's emitted body is left whole, so a function pointer to the stub still reaches the HLE function. Direct calls through a stub are resolved to their target by the lifter and still inlined.
  • Real emitted C, both modes. tests/test_spawn_emit.py has a new configuration, test_os_inline. Its made-up image hooks four functions, which are reached by BLs, a devirtualised arm, a B and a PLT stub. It runs the real emitter with A64_LOCAL_REGS=4 in 1 process and in 3 spawned ones (the wrapper is installed at the file's module level, as nx2/lift.py does) and gets the same files both ways. Each site holds the expected C, the stub stays whole, and the files holding sites compile against the generated a64.h. It also runs with A64_OS_INLINE=0, with no site.

Test results, from this PC's job queue (Windows 11, llvm-mingw clang 23.1.5):

  • At 2a1ad8f2, in job fb-1956-2:
    • tests/test_guest_os.py passes.
    • tests/test_spawn_emit.py passes. At the test commit 7df10ab4 it fails: the PLT stub's body was rewritten, so 9 sites against 8.
    • tests/test_invalidate_vcore.py passes, run alone twice and once on main @ f1f2dc5b.
    • run_all.py --strict --changed origin/main gives 81 passed and 15 failed. All 15 also fail on main with the same messages, test_vcore_escape/test_vcore_mask0 included (their harnesses do not compile on main).
  • In the run before the rebase (fb-1956-1), test_guest_os.py also passed with CC set, and failed at the test commit.

Still open on my side: the in-game runs (warmed and interleaved Market and a heavy scene, against the rebased main, with a switch-off build and a fiber-heavy scene), clean lift and build time and size on Windows and Linux, Linux, Tier 1 (including the Steam Deck), and CI at this head.

@futondestroyer thanks for the review ([62741](https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/1956#issuecomment-62741)). Fixed in 2a1ad8f2, with its tests in 7df10ab4 first. The branch is rebased onto `main` @ f1f2dc5b, which includes #1949. `git range-diff` shows the PR's own commit unchanged (aca07f9c = b922eadf). - **The actual TLS array.** The record's head is now a type, `HosThreadHead` (`guest_tt`, `tls_slots[32]`, `priority`). It is declared once, in the generated `a64.h` (`nx2/rtgen.py` `THREAD_HEAD`), which lifted code and the host both include. `host/hos_thread.c` asserts that `HosThread`'s fields have the same offsets and sizes as `HosThreadHead`. The inlined C now indexes `((HosThreadHead *)c->thread)->tls_slots[_slot]`, so it no longer reaches past `guest_tt`. - **Synchronised slots and priority.** Every access to a TLS slot and to the priority is now atomic (relaxed). That covers the lifted code, the `GetTlsValue`/`SetTlsValue` HLE functions, `vc_pref_core`'s read and `vc_sync_tls`'s write of the core cell, and `hos_thread_priority`, which now uses an exchange when it sets so it can return the old value. It also covers every scheduler and diagnostic read of the priority, which now go through an atomic `hos_thread_prio`. I removed the unused `hos_tls_slots`, which handed out the raw array. I kept this inside the PR: it is one file of host code plus the generated C. Two related things stay as they are on `main`, because they are not fields this PR inlines. The slot *index* `core_tls_slot` is still a `volatile` read outside `tls_slot_lock` in `vc_pref_core`. `hos_thread_adopt_guest` still writes `guest_tt` plainly. The description says so. - **Tail calls.** At a tail site, `GetThreadPriority` about another thread is a `TAILCALL` of the hooked definition again, not a call followed by a return. `nx2/split_giants.py` keeps it a tail call: the piece returns the tail id and the dispatcher's case is `TAILCALL(f_…)`. `tests/test_guest_os.py` checks this through a cut giant, and its C fixture runs the tail form against the HLE function. - **Indirect entry.** A PLT stub's emitted body is left whole, so a function pointer to the stub still reaches the HLE function. Direct calls through a stub are resolved to their target by the lifter and still inlined. - **Real emitted C, both modes.** `tests/test_spawn_emit.py` has a new configuration, `test_os_inline`. Its made-up image hooks four functions, which are reached by BLs, a devirtualised arm, a B and a PLT stub. It runs the real emitter with `A64_LOCAL_REGS=4` in 1 process and in 3 spawned ones (the wrapper is installed at the file's module level, as `nx2/lift.py` does) and gets the same files both ways. Each site holds the expected C, the stub stays whole, and the files holding sites compile against the generated `a64.h`. It also runs with `A64_OS_INLINE=0`, with no site. Test results, from this PC's job queue (Windows 11, llvm-mingw clang 23.1.5): - At 2a1ad8f2, in job fb-1956-2: - `tests/test_guest_os.py` passes. - `tests/test_spawn_emit.py` passes. At the test commit 7df10ab4 it fails: the PLT stub's body was rewritten, so 9 sites against 8. - `tests/test_invalidate_vcore.py` passes, run alone twice and once on `main` @ f1f2dc5b. - `run_all.py --strict --changed origin/main` gives 81 passed and 15 failed. All 15 also fail on `main` with the same messages, `test_vcore_escape`/`test_vcore_mask0` included (their harnesses do not compile on `main`). - In the run before the rebase (fb-1956-1), `test_guest_os.py` also passed with `CC` set, and failed at the test commit. Still open on my side: the in-game runs (warmed and interleaved Market and a heavy scene, against the rebased `main`, with a switch-off build and a fiber-heavy scene), clean lift and build time and size on Windows and Linux, Linux, Tier 1 (including the Steam Deck), and CI at this head.
Author
Contributor

Point by point for BeanBot's review (62005) at 2a1ad8f2. The tests are in 7df10ab4 and the fix is 2a1ad8f2. The branch is rebased onto main @ f1f2dc5b, and the PR's own commit is unchanged. These are the same changes as in my reply to futondestroyer's review.

  1. 61996 tls_slots through guest_tt: fixed. The record's head is now the shared type HosThreadHead, declared in the generated a64.h (nx2/rtgen.py) and included by the host too. The lifted code indexes its tls_slots array, and hos_thread.c asserts the offsets against it.
  2. 61997 TLS race with vc_sync_tls: fixed. TLS slot accesses are atomic (relaxed) in the lifted code, the HLE GetTlsValue/SetTlsValue, vc_pref_core and vc_sync_tls. The slot index core_tls_slot stays a volatile read as on main; the description says so.
  3. 61998 priority race: fixed. Priority reads are atomic in the lifted code, in hos_thread_prio and in every scheduler and diagnostic reader in hos_thread.c. The setter hos_thread_priority uses an atomic exchange that returns the old value.
  4. 61999 TAILCALL in the other-thread arm: fixed. tests/test_guest_os.py checks it directly and through nx2/split_giants.py's pieces. tests/test_spawn_emit.py checks it in the real emit.
  5. 62000 PLT bodies: fixed. The rewrite skips e in self.plt. test_os_inline checks that the stub still tail-calls the hooked definition.
  6. 62001 scan via refs: fixed. A function whose refs name none of the hooked definitions is not scanned. test_os_inline counts the scans: there is one per function with a hooked call, against all 4,500 functions before.
  7. 62002 partial HOS_HLE_STATS counts: marked and documented, not preserved. Counting at each inlined site would add the check the change removes. scripts/perf.py cpu --calls now says in its report that the four functions' counts are partial with A64_OS_INLINE on. docs/dev/reference/build.md and profiling.md say to profile a lift with A64_OS_INLINE=0 for full counts.
  8. 62003 real emitted C, spawned vs one process: done in tests/test_spawn_emit.py test_os_inline. It covers A64_LOCAL_REGS=4, BLs, the devirtualised arm, a tail call, a PLT stub and the unchanged indirect A64_ICALL. Output is the same in 1 and 3 spawned processes, with the switch on and with =0, and the files with sites compile against the generated a64.h. At the test commit 7df10ab4, before the fix, it fails.
  9. 62004 honour CC: fixed. The order is --cc, then CC, then LONGSHOT_MINGW's clang (Windows), then clang, gcc or cc. RUN_ALL_ARGS no longer forces gcc. This is the same helper as #1954's.

Waiting items: the test_invalidate_vcore.py timeout. Answered: it passed alone twice at b922eadf (HostLiftPRs' batch 8), and twice alone at 2a1ad8f2 plus once on main @ f1f2dc5b (fb-1956-2). Its failures happened only in the parallel changed-file runs, and both were timeouts: the immediate case at 10 s, and once its g++ compile. The in-game A/B (merge base vs head, switch off vs on, a fiber-heavy scene, Tier 1 incl. Steam Deck) and the clean lift/build time and size on Windows and Linux are pending on my side; they need runs I have not made. The suggestion about the micro-benchmark's clock is not taken this round. Its figures in Measurements are from 974297f5's form of the inlined C and were not run again at this head.

Point by point for BeanBot's review ([62005](https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/1956#issuecomment-62005)) at 2a1ad8f2. The tests are in 7df10ab4 and the fix is 2a1ad8f2. The branch is rebased onto `main` @ f1f2dc5b, and the PR's own commit is unchanged. These are the same changes as in my reply to futondestroyer's review. 1. [61996](https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/1956#issuecomment-61996) `tls_slots` through `guest_tt`: fixed. The record's head is now the shared type `HosThreadHead`, declared in the generated `a64.h` (`nx2/rtgen.py`) and included by the host too. The lifted code indexes its `tls_slots` array, and `hos_thread.c` asserts the offsets against it. 2. [61997](https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/1956#issuecomment-61997) TLS race with `vc_sync_tls`: fixed. TLS slot accesses are atomic (relaxed) in the lifted code, the HLE `GetTlsValue`/`SetTlsValue`, `vc_pref_core` and `vc_sync_tls`. The slot index `core_tls_slot` stays a `volatile` read as on `main`; the description says so. 3. [61998](https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/1956#issuecomment-61998) priority race: fixed. Priority reads are atomic in the lifted code, in `hos_thread_prio` and in every scheduler and diagnostic reader in `hos_thread.c`. The setter `hos_thread_priority` uses an atomic exchange that returns the old value. 4. [61999](https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/1956#issuecomment-61999) `TAILCALL` in the other-thread arm: fixed. `tests/test_guest_os.py` checks it directly and through `nx2/split_giants.py`'s pieces. `tests/test_spawn_emit.py` checks it in the real emit. 5. [62000](https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/1956#issuecomment-62000) PLT bodies: fixed. The rewrite skips `e in self.plt`. `test_os_inline` checks that the stub still tail-calls the hooked definition. 6. [62001](https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/1956#issuecomment-62001) scan via refs: fixed. A function whose refs name none of the hooked definitions is not scanned. `test_os_inline` counts the scans: there is one per function with a hooked call, against all 4,500 functions before. 7. [62002](https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/1956#issuecomment-62002) partial `HOS_HLE_STATS` counts: marked and documented, not preserved. Counting at each inlined site would add the check the change removes. `scripts/perf.py cpu --calls` now says in its report that the four functions' counts are partial with `A64_OS_INLINE` on. `docs/dev/reference/build.md` and `profiling.md` say to profile a lift with `A64_OS_INLINE=0` for full counts. 8. [62003](https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/1956#issuecomment-62003) real emitted C, spawned vs one process: done in `tests/test_spawn_emit.py` `test_os_inline`. It covers `A64_LOCAL_REGS=4`, BLs, the devirtualised arm, a tail call, a PLT stub and the unchanged indirect `A64_ICALL`. Output is the same in 1 and 3 spawned processes, with the switch on and with `=0`, and the files with sites compile against the generated `a64.h`. At the test commit 7df10ab4, before the fix, it fails. 9. [62004](https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/1956#issuecomment-62004) honour `CC`: fixed. The order is `--cc`, then `CC`, then `LONGSHOT_MINGW`'s clang (Windows), then clang, gcc or cc. `RUN_ALL_ARGS` no longer forces gcc. This is the same helper as #1954's. Waiting items: the `test_invalidate_vcore.py` timeout. Answered: it passed alone twice at b922eadf (HostLiftPRs' batch 8), and twice alone at 2a1ad8f2 plus once on `main` @ f1f2dc5b (fb-1956-2). Its failures happened only in the parallel changed-file runs, and both were timeouts: the `immediate` case at 10 s, and once its g++ compile. The in-game A/B (merge base vs head, switch off vs on, a fiber-heavy scene, Tier 1 incl. Steam Deck) and the clean lift/build time and size on Windows and Linux are pending on my side; they need runs I have not made. The suggestion about the micro-benchmark's clock is not taken this round. Its figures in Measurements are from 974297f5's form of the inlined C and were not run again at this head.
futondestroyer left a comment

Recommendation: Request changes. Make HosThreadHead the actual first member of HosThread so the emitted field accesses use the same object type as the host. The atomic slot and priority accesses, priority tail fallback, and PLT exclusion address the corresponding findings from the earlier review. Previous review

At nx2/guest_os.py:39, Ctx.thread still points to a HosThread, while the emitted code casts it to a separate HosThreadHead type. The static assertions check layout. Access through the cast relies on how the compiler handles aliasing between the two types. The game build disables strict aliasing, but the executable parity fixture compiles this same access at -O2 with the compiler default. Embedding the head at offset zero gives both paths the same typed storage without duplicating the fields. WG14 discussion of structure prefixes, Q39

Testing. The author’s Windows parity and spawned-emitter fixtures passed at 2a1ad8f2. The invalidate fixture passed twice on this head and once on its base. The changed-file suite reported 81 passes, 15 reported baseline failures and 3 skips. The two modified virtual-core fixtures still fail to compile in that run, so the results leave the scheduler changes untested by those fixtures. Linux build and gameplay are open, and required CI is pending.

Does it help? Unproven. The call timings are from the older plain-load implementation. Measure the revised default and switch-off builds against the rebased main with warmed, interleaved Market and heavy or fiber-heavy runs. Report average FPS, 1% lows, median and p99 frame times, and clean lift/build time and size on Windows and Linux. A64_OS_INLINE=0 keeps the calls and is covered by the emitter comparison. Profiling now explains that the default lift gives partial HLE counts.

Farm and merge order. Both earlier renderer fixes are in this base. Current main’s FPX change edits the same imports and setup in nx2/lift.py, so preserve both on rebase and rerun the emitter fixture. Keep the PR WIP while the layout and author gameplay are open. The existing request for AMD CPU Windows/Linux, low-core PCs and Steam Deck is awaiting a BigBean. Revisit it after the corrected head builds and passes the author’s gameplay checks. Merged FPX change Existing farm request Farm status

Reviewed 2a1ad8f2 against f1f2dc5b and compared the revision with b922eadf. Verified both revision maps and checked the affected host, scheduler and emitter paths in source. Nothing from the PR was run.

**Recommendation: Request changes.** Make `HosThreadHead` the actual first member of `HosThread` so the emitted field accesses use the same object type as the host. The atomic slot and priority accesses, priority tail fallback, and PLT exclusion address the corresponding findings from the earlier review. [Previous review](https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/1956#issuecomment-62741) At `nx2/guest_os.py:39`, `Ctx.thread` still points to a `HosThread`, while the emitted code casts it to a separate `HosThreadHead` type. The static assertions check layout. Access through the cast relies on how the compiler handles aliasing between the two types. The game build disables strict aliasing, but the executable parity fixture compiles this same access at `-O2` with the compiler default. Embedding the head at offset zero gives both paths the same typed storage without duplicating the fields. [WG14 discussion of structure prefixes, Q39](https://open-std.org/jtc1/sc22/wg14/www/docs/n2222.htm) **Testing.** The author’s Windows parity and spawned-emitter fixtures passed at `2a1ad8f2`. The invalidate fixture passed twice on this head and once on its base. The changed-file suite reported 81 passes, 15 reported baseline failures and 3 skips. The two modified virtual-core fixtures still fail to compile in that run, so the results leave the scheduler changes untested by those fixtures. Linux build and gameplay are open, and required CI is pending. **Does it help?** Unproven. The call timings are from the older plain-load implementation. Measure the revised default and switch-off builds against the rebased main with warmed, interleaved Market and heavy or fiber-heavy runs. Report average FPS, 1% lows, median and p99 frame times, and clean lift/build time and size on Windows and Linux. `A64_OS_INLINE=0` keeps the calls and is covered by the emitter comparison. Profiling now explains that the default lift gives partial HLE counts. **Farm and merge order.** Both earlier renderer fixes are in this base. Current main’s FPX change edits the same imports and setup in `nx2/lift.py`, so preserve both on rebase and rerun the emitter fixture. Keep the PR WIP while the layout and author gameplay are open. The existing request for AMD CPU Windows/Linux, low-core PCs and Steam Deck is awaiting a BigBean. Revisit it after the corrected head builds and passes the author’s gameplay checks. [Merged FPX change](https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/1905) [Existing farm request](https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/1956#issuecomment-63539) [Farm status](https://beanstalk.magicbean.dev/prs/1956) **Reviewed** `2a1ad8f2` against `f1f2dc5b` and compared the revision with `b922eadf`. Verified both revision maps and checked the affected host, scheduler and emitter paths in source. Nothing from the PR was run.
@ -0,0 +36,4 @@
# (the HLE functions' bodies as C on the context, through HosThreadHead; SLOTS: hos.h's HOS_TLS_SLOTS. @CALL@: the call of
# the hooked definition, for the case the inlined C leaves to it)
SLOTS = 32
_TH = "((%sHosThreadHead *)c->thread)"
Member

Make HosThreadHead the actual first member of HosThread. Ctx.thread currently points to the full HosThread and the offset assertions leave the two struct types distinct. The lift disables strict aliasing, but the executable parity fixture compiles the same cast at -O2 with its compiler's default aliasing rules.

Make HosThreadHead the actual first member of HosThread. Ctx.thread currently points to the full HosThread and the offset assertions leave the two struct types distinct. The lift disables strict aliasing, but the executable parity fixture compiles the same cast at -O2 with its compiler's default aliasing rules.
Some checks are pending
CI / Conventional Commit title (pull_request) Blocked by required conditions
CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Blocked by required conditions
CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Blocked by required conditions
CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Blocked by required conditions
CI / Workflow lint (actionlint) (pull_request) Blocked by required conditions
CI / Website build and checks (pull_request) Blocked by required conditions
Guide screenshots / Guide images and local UI source review (pull_request) Blocked by required conditions
Third-party parts / As their notes say (pull_request) Blocked by required conditions
Third-party parts / The same as upstream (pull_request) Blocked by required conditions
Website / Website on Cloudflare (pull_request) Blocked by required conditions
Website / Website on Cloudflare (pull_request_target) Has been skipped
beanstalk/approval waiting for a BigBean's 👍 on the request
PR description / Description follows the template (pull_request_target) Successful in 6s
This pull request has changes conflicting with the target branch.
  • docs/dev/reference/build.md
  • nx2/lift.py
Some workflows are waiting to be reviewed.
View command line instructions

Manual merge helper

Use this merge commit message when completing the merge manually.

Checkout

From your project repository, check out a new branch and test the changes.
git fetch -u 'perf/os-inline:siantm-perf/os-inline'
git switch 'siantm-perf/os-inline'
Sign in to join this conversation.
No reviewers
No milestone
No project
No assignees
4 participants
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
MagicBeanDev/Longshot!1956
No description provided.