perf(hos): splice the host's memcpy, memset and string hooks into the lifted SDK #1209

Open
dyst wants to merge 2 commits from dyst/Longshot:perf/host-libc-hooks into main
Member

Type: perf · Risk: medium · Tested on: Windows 11, RTX 3080, in the game (Kokiri Forest, Hyrule Field) and game-free; Linux game-free · Needs: a Linux run in the game

Why

The game copies, clears and compares memory with the SDK's own memcpy, memset, memcmp and string functions, run as
translated code: one byte-for-byte translation of ARM copy loops for every call, and for memcpy an indirect jump through
the SDK's own import table on top. The SDK's memcpy alone is 58th in the hot list, called from 1,112 places in the game.
host/nn_libc.c has had host versions of all six since the fork, but nothing in Longshot's lift ever called them, so
they never ran. This is for the CPU-bound frames in the worklist's 2.1 rows (Castle Town and Hyrule Field when the CPU is
the limit, Kokiri Forest while the camera turns).

What changed

The standalone lift now starts the SDK's memcpy, memmove, memset, memcmp, strlen and strcmp with a call of
the host's version. The host does the call when it stays inside the guest window and returns at once; otherwise, or with
HOS_GUEST_LIBC=1, the SDK's own translated code still runs, unchanged, right behind it.

  • lift: nx2/entry_hooks.py (new) reads the ENTRY_HOOK_SYM(sym) hooks host/*.c defines, finds each symbol where the
    sdk module defines it (the image's symbol table, as the HLE set is found), and wraps the emit so that function's C
    starts with { extern int entry_sym_<sym>(Ctx *A64_NOESC); if (entry_sym_<sym>(c)) return; }. Direct calls, tail calls
    and indirect calls all enter through that line. nx2/lift.py installs the wrap at module level (so the emit's new
    processes have it too) and calls entry_hooks.add in the standalone lift before discovery; it logs
    entry hooks: 6 of 6 (...) and entry hooks spliced: N. The table goes to the emit processes with the others
    (nx2/chunks.py EMIT_TABLES), and the hooked functions keep their registers in the context (Lifter.no_local).
    nx2/make_game.py lift_key names the hooks, so adding or removing one lifts again.
  • size gates: ENTRY_HOOK_SYM_FROM(sym, min) (host/hos.h) is a hook called only when x2, the size, is min or more.
    The spliced line tests the size itself (if (c->x[2] >= 8ULL && entry_sym_memset(c)) return;), so a smaller call stays
    in the lifted code at the cost of one compare. memset is gated at 8 bytes and memcmp at 16: below those the call cost
    1-4 ns more than it saved (micro-benchmark below).
  • large copies: Windows' memmove switches to non-temporal stores at about 1.75 MB. Its copy then took 1.4x the lifted
    copy's time and left the destination out of the cache, so the game's next read of it took 4-5x as long (micro-benchmark
    below). Copies and fills over 256 KB (LIBC_PIECE) now go in pieces below that size (copy_pieces, fill_pieces).
    Each piece is marked once written, and the lifted loop's interrupt poll (A64_POLL, the virtual cores' hand-off) runs
    between pieces. Overlapping copies go forward or from the end, as memmove would.
  • host: host/nn_libc.c. memcmp and strcmp now return their result zero-extended in x0, as the SDK's 32-bit result
    leaves it (they sign-extended before). The first hooked call logs which version is in use (libc: ...). With HOS_HLE_STATS=1 the hle: lines
    count the calls the host took (libc_memcpy, libc_memmove, libc_memset, libc_memcmp, libc_strlen, libc_strcmp)
    and those done in pieces (libc_pieced). The counting is out of line, so the small calls' path stays as short. How the switch is
    read is unchanged apart from atomic loads. host/hos.h: the comments say what splices ENTRY_HOOK_SYM, and that
    ENTRY_HOOK(addr) is not spliced in Longshot.
  • What the SDK's functions do, as I read them in the lift (behaviour, no code): its memcpy jumps through its import table
    to an optimized copy that handles overlapping ranges in either direction, as memmove does. Its memmove handles
    overlap and passes non-overlapping copies on to memcpy. Its memset stores the value's low byte. All three return the
    destination in x0 unchanged. memcmp and strcmp return the difference of the first unequal bytes, compared unsigned,
    as a 32-bit value. strlen counts up to the NUL. The host's memmove, memset, memcmp loop and strlen give the
    same result in every case.
  • Write marks: the host versions mark what they write with HOS_WRITTEN (hos_mark_written), which marks every 4 KB page
    of the written range after the data is written (for each piece, once that piece is written). The translated stores mark
    the same pages: each store marks the page of its first and its last byte, and a copy writes every byte of its range. The
    set of pages marked is identical; the only difference is timing: the host marks a piece's pages once it is copied, where
    the lifted loop marks them as it goes. One mark per page, so a copy inside one page is one store.
  • docs: docs/development.md has a row for HOS_GUEST_LIBC.

Measurements

Verdict: in the game, a small gain in Kokiri Forest with the camera turning, 38.6 fps against 37.7 (+2.3%, all three of this PR's runs above all three of main's), and level in CPU-bound Hyrule Field (50.6 against 50.6), where the first version of these hooks lost 3.8%. Micro-benchmark
(below): the host path is as fast or faster than the SDK's lifted functions at every size it now takes, 2-9x faster from
128 bytes, and large copies no longer leave their destination out of the cache.

Hardware: Windows 11 Pro, Ryzen 9 3900X, one thread pinned (core 4), the job queue idle
Scene: none (micro-benchmark); in the game: Kokiri Forest from the repository's kokiri-start save, the camera turning, and Hyrule Field (the debug teleport to HyruleField_Child), the camera still; uncapped, headless
Method: the SDK's lifted memcpy (f_8572f020, whose import-table jump goes to the optimized copy f_8575cdc0), memset,
memmove, memcmp, strlen and strcmp, cut out of the lift of main @ c167 and compiled as the hot pieces are (-O2,
x86-64-v3, write marks on, the call site's indirect-jump cache). Each was timed against the same function with this PR's
spliced line first, and against the hooks before this change (5839334c). The window was at the game's guest base, warm
caches, best of 7 runs of 4,000,000/(n+32) calls, medians over 3-4 runs of the whole set. For large copies: one copy of
cache-warm data, then a read of every line of the destination.
How to reproduce: the harness is not in the PR because it compiles the game's lifted code; any lifted
gen/code_*.c with the functions above and the hooks from host/nn_libc.c will do

ns a call (unless marked) Before: main @ 71b1edd8 (lifted SDK) After, the earlier hooks: this PR @ 5839334c After: this PR @ 00ad3e5b
memcpy 1 B / 8 B / 64 B / 256 B / 4 KB (aligned) 9.8 / 7.6 / 8.2 / 25.9 / 367 7.5 / 7.6 / 9.4 / 11.0 / 42 7.5 / 7.6 / 9.9 / 11.0 / 41
memcpy 8 B / 64 B / 256 B (dst +3, src +7) 7.6 / 8.8 / 26.5 7.5 / 9.2 / 10.2 7.6 / 9.3 / 10.1
memset 1 B / 4 B / 8 B / 64 B / 4 KB 5.6 / 8.4 / 11.7 / 24.4 / 332 7.8 / 7.9 / 7.8 / 9.2 / 75 5.8 / 8.3 (both lifted: gated) / 7.8 / 9.3 / 75
memmove overlapping 16 B / 1 KB 48.0 / 161 7.9 / 23 7.5 / 23
memcmp 4 B / 16 B / 256 B (equal) 5.4 / 8.6 / 74 7.3 / 7.2 / 14.0 5.6 (lifted: gated) / 7.6 / 14.9
strlen / strcmp, 64 chars 14.1 / 19.6 10.7 / 15.0 10.8 / 13.5
strcmp, 0-2 chars 3.9-5.7 5.9 6.3-6.4
2 MB copy, then reading it (us) 188-234 + 24-30 266-325 + 116-119 51-54 + 24-25
4 MB copy, then reading it (us) 367-619 + 48-61 509-647 + 232-328 103-104 + 48-53
UCRT memmove alone (cache-warm data, one copy, then read): 1536 KB: 60 us + 28.6 us read; 1792 KB: 372 us + 115.0 us read
(4.01 ns a line: past the cache); 4096 KB: 727 us + 257.7 us. The same 4 MB in 4 KB memcpy pieces: 154 us + 66.6 us.
memset stays cached up to 4 MB.
PASS: 1683333 calls (78208 under a size gate, 190 interrupt polls between pieces): the window, x0 and the page marks are the SDK's lifted functions' (marks after the data, 216724 marked writes); past the window the game's own; logged once
entry hooks: 6 of 6 (memcpy sdk+0x78f020, memmove sdk+0x78f040, memset sdk+0x78f300, strlen sdk+0x7a7d40, memcmp sdk+0x7bd280, strcmp sdk+0x7bd380)

The first in-game A/B (main @ 71b1edd8 against 5839334c, Ryzen 9 3900X, RTX 3080, warm) lost 3.8% in CPU-bound Hyrule
Field (1% lows 29.5 to 24.7 fps) and was level in Kokiri Forest. The micro-benchmark found no small-size cost large
enough to explain that. It did find the non-temporal large copies, which match both the lower average and the worse 1%
lows. This commit fixes those copies.

In the game, the re-run (main @ 5c284721 against this PR @ 00ad3e5b; Windows 11 Pro N, Ryzen 9 3900X, 64 GB, RTX 3080, a hard drive; one game folder updated in place between the two (make_game.py --update, warm stores), an unmeasured run of each scene first, then the order main, PR, PR, main, main, PR, one 45 s window per scene after each install with ab2.py; through the PC's job queue (--mode timing); medians of 3):

Before: main @ 5c284721 After: this PR @ 00ad3e5b
Kokiri, camera turning, fps (median of 3) 37.7 (37.97, 37.68, 37.61) 38.6 (39.14, 38.05, 38.55)
Kokiri, 1% low 23.3, 15.9, 18.8 20.8, 21.4, 21.2
Hyrule Field, fps (median of 3) 50.6 (50.41, 51.67, 50.64) 50.6 (51.90, 50.22, 50.59)
Hyrule Field, 1% low 29.0, 32.1, 26.6 30.1, 26.8, 28.8
Hyrule Field, p99 frame time 31.2, 28.3, 33.0 ms 30.3, 33.5, 29.7 ms

With HOS_HLE_STATS=1 (one untimed Hyrule Field run), the host takes in a typical 5 s of play about 229,000 memcpy, 202,000 memset, 100,000 strcmp, 56,000 strlen, 29,000 memcmp and 4,800 memmove calls, and copies in pieces about once (974 times in the 5 s that held the teleport's load). So the large copies happen in loads, rarely in play: they fit the first version's worse 1% lows, but the Field average it lost may also have been the tiny memsets now gated below 8 bytes (the busiest hook), or partly that set's own spread (its two runs were 5% apart).

[hos MainThread] libc: memcpy, memmove, memset, memcmp, strlen and strcmp are the host's (HOS_GUEST_LIBC=1: the game's own)
entry hooks: 6 of 6 (memcpy sdk+0x78f020, memmove sdk+0x78f040, memset sdk+0x78f300, strlen sdk+0x7a7d40, memcmp sdk+0x7bd280, strcmp sdk+0x7bd380)
entry hooks spliced: 6
[hos host] hle: 228712 libc_memcpy;
[hos host] hle: 1 libc_pieced;

Flags and settings

HOS_GUEST_LIBC existed already but did nothing, because the hooks never ran. Now it switches them off.

Flag What it is Before this PR After this PR Effect Helps / doesn't help / untested Recommendation
HOS_GUEST_LIBC (set) the SDK's own memcpy, memmove, memset, memcmp, strlen and strcmp for every call unset (no effect: the hooks were never called) unset: the host's versions set: every call runs the translated SDK code, on the same exe untested in the game: for A/B only leave unset; set only for A/B or to rule this change out

Testing

Tested: in the game on Windows 11 (Measurements: the re-run, the counters, the log lines). Game-free on Windows 11 (TDM gcc for the fixtures, llvm-mingw clang for the objects), as follows.

  • tests/test_libc_hooks.py (new):
    • The splice over a made-up image with a real a64_lift.Lifter: the hooked function starts with the hook's line and
      the rest of its text is unchanged. A symbol in another module, an HLE'd symbol, an address that is no entry and an
      undefined symbol are left out.
    • The hooks against the model of the SDK's functions. Sizes 0 to 300 and up to 12 KB across pages, every alignment
      mod 16, every overlap in both directions. memcmp and strcmp with bytes above 0x7f. The marks must be set after the
      data.
    • Pieces: sizes from 256 KB - 1 to 768 KB, with every overlap of a piece boundary in both directions, and the polls
      between pieces counted.
    • Gates: memset under 8 bytes and memcmp under 16 are declined (nothing written or marked), and the gated line runs the
      lifted body under its size and the host from it.
    • The spliced line in a lifted-shaped function, and HOS_GUEST_LIBC=1.
    • Mutations caught:
      • marking one byte short, or before the write (copy and fill);
      • the old sign-extended result;
      • a forward copy over an overlap;
      • an unmarked piece, or no poll between pieces;
      • the memset gate gone from the hook or from the splice;
      • pieces of 4 MB.
  • python -I -B tests/run_all.py --jobs 8 --changed longshot/main (the checks that read the changed files, on main @ 5c284721):
    96 passed, 8 skipped, 10 failed. The 10 failures also fail on a clean main worktree, with the same failing cases:
    test_make_game_cache, test_linux_preflight, test_walk_key, test_failed_compile_keeps_objects, test_job_queue,
    test_missing_nup, test_run_all_changed, test_crash_dialog, test_vk_occlusion_queries, and test_walk (no lz4
    module in this Python).
  • host/nn_libc.c compiled with llvm-mingw clang at the build's host flags. A lifted-shaped function with the spliced
    line compiled against the generated a64.h.
  • Linux (Docker python:3.12, gcc 14): nn_libc.c compiled; test_libc_hooks, test_guest_copies (its mmap fixture)
    and test_lift_entry_below pass.

Not tested: Linux and macOS in the game; Castle Town Market (its teleport's measured window was a blank screen, on main too).

Must be tested:

  • Linux, any GPU, by hand: the same two scenes, main against this PR, expecting the libc: ... are the host's line, no picture or behaviour change, and fps the same or higher.

Risks

What could break:

  • A host version that differs from the SDK's for some call would change game behaviour. The check compares all six with
    the SDK's behaviour, including overlapping copies and the exact x0.
  • A missed write mark would leave stale data on the GPU. The marks are the same pages as the lifted stores', set after the
    data.
  • A copy over 256 KB polls between its pieces, as the lifted loop does every 64 bytes. A virtual-core hand-off waits at
    most one piece (about 10 us).
  • The size gates and the piece size were chosen on one CPU (Zen 2) with Windows' CRT. On Linux, glibc's memmove has its
    own non-temporal threshold (a share of the L3). The pieces keep copies below it there too, but that was not measured.
  • The play profile (PGO) no longer matches the six functions' changed C; the rest of the profile is unaffected.

Turning it off: HOS_GUEST_LIBC=1 (same exe), or revert the commit.

Compatibility: The lift key changes, so the next build lifts the game again and recompiles the lifted files that hold
these six functions. Saves, longshot_options.ini, the shader and pipeline caches and the vk ABI are unchanged.

Questions for reviewers

  1. memcmp/strcmp now zero-extend x0, which is exactly what the SDK's code leaves there. Any caller reads only w0, so
    this changes nothing for compiled code, but it does differ from the hooks as written before. Is the change welcome?
  2. Only the sdk module's definitions are hooked; a game module with its own memcpy keeps it. Should that be wider?
  3. ENTRY_HOOK(addr) (by address) is still not spliced, and no hook uses it. Should this splice handle it too, or should
    the macro be removed?
  4. The gates (memset 8, memcmp 16) and the 256 KB piece size come from one Zen 2 PC. Should they be checked on an Intel
    CPU and on Linux before merge?
  5. The splice is an emit wrapper (as nx2/lift_fixes.py does for its entry goto) rather than a lifter table, because
    third_party/a64-kit stays untouched. Is a wrapper OK here?

Checklist

Everyone

  • One topic.
  • The title is a Conventional Commit (type(scope): description) of the status line's type.
  • Why gives the reason plainly and links the issues that fit (the tracker searched).
  • Measurements has a verdict, or one line saying why there is nothing to measure.
  • Testing names what was tested, what wasn't, and what must still be tested, where and how.
  • Rebased on current main.
  • python -I -B tests/run_all.py passes (any failure that also fails on main is named).
  • No game files, dumps, build output or generated files.
  • third_party/ untouched (a new folder with a PROVENANCE.md instead).
  • The license is OK: GPL-3.0-or-later, nothing incompatible and nothing decompiled.

When it applies

  • AI assistance recorded with Assisted-by: / Co-Authored-By: trailers.
  • Switches documented (docs/development.md; the README for a menu row) and shipped at their recommended value.
  • A check under tests/ that reads changed source updated in this PR, not weakened.
  • Host code builds on both Windows (llvm-mingw) and Linux, with UTF-8 paths.
  • Docs updated.

🤖 Generated with Claude Code

**Type:** perf · **Risk:** medium · **Tested on:** Windows 11, RTX 3080, in the game (Kokiri Forest, Hyrule Field) and game-free; Linux game-free · **Needs:** a Linux run in the game ## Why The game copies, clears and compares memory with the SDK's own `memcpy`, `memset`, `memcmp` and string functions, run as translated code: one byte-for-byte translation of ARM copy loops for every call, and for `memcpy` an indirect jump through the SDK's own import table on top. The SDK's `memcpy` alone is 58th in the hot list, called from 1,112 places in the game. `host/nn_libc.c` has had host versions of all six since the fork, but nothing in Longshot's lift ever called them, so they never ran. This is for the CPU-bound frames in the worklist's 2.1 rows (Castle Town and Hyrule Field when the CPU is the limit, Kokiri Forest while the camera turns). ## What changed The standalone lift now starts the SDK's `memcpy`, `memmove`, `memset`, `memcmp`, `strlen` and `strcmp` with a call of the host's version. The host does the call when it stays inside the guest window and returns at once; otherwise, or with `HOS_GUEST_LIBC=1`, the SDK's own translated code still runs, unchanged, right behind it. - lift: `nx2/entry_hooks.py` (new) reads the `ENTRY_HOOK_SYM(sym)` hooks `host/*.c` defines, finds each symbol where the sdk module defines it (the image's symbol table, as the HLE set is found), and wraps the emit so that function's C starts with `{ extern int entry_sym_<sym>(Ctx *A64_NOESC); if (entry_sym_<sym>(c)) return; }`. Direct calls, tail calls and indirect calls all enter through that line. `nx2/lift.py` installs the wrap at module level (so the emit's new processes have it too) and calls `entry_hooks.add` in the standalone lift before discovery; it logs `entry hooks: 6 of 6 (...)` and `entry hooks spliced: N`. The table goes to the emit processes with the others (`nx2/chunks.py` `EMIT_TABLES`), and the hooked functions keep their registers in the context (`Lifter.no_local`). `nx2/make_game.py` `lift_key` names the hooks, so adding or removing one lifts again. - size gates: `ENTRY_HOOK_SYM_FROM(sym, min)` (`host/hos.h`) is a hook called only when x2, the size, is `min` or more. The spliced line tests the size itself (`if (c->x[2] >= 8ULL && entry_sym_memset(c)) return;`), so a smaller call stays in the lifted code at the cost of one compare. `memset` is gated at 8 bytes and `memcmp` at 16: below those the call cost 1-4 ns more than it saved (micro-benchmark below). - large copies: Windows' `memmove` switches to non-temporal stores at about 1.75 MB. Its copy then took 1.4x the lifted copy's time and left the destination out of the cache, so the game's next read of it took 4-5x as long (micro-benchmark below). Copies and fills over 256 KB (`LIBC_PIECE`) now go in pieces below that size (`copy_pieces`, `fill_pieces`). Each piece is marked once written, and the lifted loop's interrupt poll (`A64_POLL`, the virtual cores' hand-off) runs between pieces. Overlapping copies go forward or from the end, as `memmove` would. - host: `host/nn_libc.c`. `memcmp` and `strcmp` now return their result zero-extended in x0, as the SDK's 32-bit result leaves it (they sign-extended before). The first hooked call logs which version is in use (`libc: ...`). With `HOS_HLE_STATS=1` the `hle:` lines count the calls the host took (`libc_memcpy`, `libc_memmove`, `libc_memset`, `libc_memcmp`, `libc_strlen`, `libc_strcmp`) and those done in pieces (`libc_pieced`). The counting is out of line, so the small calls' path stays as short. How the switch is read is unchanged apart from atomic loads. `host/hos.h`: the comments say what splices `ENTRY_HOOK_SYM`, and that `ENTRY_HOOK(addr)` is not spliced in Longshot. - What the SDK's functions do, as I read them in the lift (behaviour, no code): its `memcpy` jumps through its import table to an optimized copy that handles overlapping ranges in either direction, as `memmove` does. Its `memmove` handles overlap and passes non-overlapping copies on to `memcpy`. Its `memset` stores the value's low byte. All three return the destination in x0 unchanged. `memcmp` and `strcmp` return the difference of the first unequal bytes, compared unsigned, as a 32-bit value. `strlen` counts up to the NUL. The host's `memmove`, `memset`, `memcmp` loop and `strlen` give the same result in every case. - Write marks: the host versions mark what they write with `HOS_WRITTEN` (`hos_mark_written`), which marks every 4 KB page of the written range after the data is written (for each piece, once that piece is written). The translated stores mark the same pages: each store marks the page of its first and its last byte, and a copy writes every byte of its range. The set of pages marked is identical; the only difference is timing: the host marks a piece's pages once it is copied, where the lifted loop marks them as it goes. One mark per page, so a copy inside one page is one store. - docs: `docs/development.md` has a row for `HOS_GUEST_LIBC`. ## Measurements **Verdict:** in the game, a small gain in Kokiri Forest with the camera turning, **38.6 fps against 37.7** (+2.3%, all three of this PR's runs above all three of main's), and level in CPU-bound Hyrule Field (50.6 against 50.6), where the first version of these hooks lost 3.8%. Micro-benchmark (below): the host path is as fast or faster than the SDK's lifted functions at every size it now takes, 2-9x faster from 128 bytes, and large copies no longer leave their destination out of the cache. Hardware: Windows 11 Pro, Ryzen 9 3900X, one thread pinned (core 4), the job queue idle Scene: none (micro-benchmark); in the game: Kokiri Forest from the repository's `kokiri-start` save, the camera turning, and Hyrule Field (the debug teleport to `HyruleField_Child`), the camera still; uncapped, headless Method: the SDK's lifted `memcpy` (`f_8572f020`, whose import-table jump goes to the optimized copy `f_8575cdc0`), `memset`, `memmove`, `memcmp`, `strlen` and `strcmp`, cut out of the lift of main @ c167 and compiled as the hot pieces are (-O2, x86-64-v3, write marks on, the call site's indirect-jump cache). Each was timed against the same function with this PR's spliced line first, and against the hooks before this change (5839334c). The window was at the game's guest base, warm caches, best of 7 runs of 4,000,000/(n+32) calls, medians over 3-4 runs of the whole set. For large copies: one copy of cache-warm data, then a read of every line of the destination. How to reproduce: the harness is not in the PR because it compiles the game's lifted code; any lifted `gen/code_*.c` with the functions above and the hooks from `host/nn_libc.c` will do | ns a call (unless marked) | Before: main @ 71b1edd8 (lifted SDK) | After, the earlier hooks: this PR @ 5839334c | After: this PR @ 00ad3e5b | |---|---|---|---| | memcpy 1 B / 8 B / 64 B / 256 B / 4 KB (aligned) | 9.8 / 7.6 / 8.2 / 25.9 / 367 | 7.5 / 7.6 / 9.4 / 11.0 / 42 | 7.5 / 7.6 / 9.9 / 11.0 / 41 | | memcpy 8 B / 64 B / 256 B (dst +3, src +7) | 7.6 / 8.8 / 26.5 | 7.5 / 9.2 / 10.2 | 7.6 / 9.3 / 10.1 | | memset 1 B / 4 B / 8 B / 64 B / 4 KB | 5.6 / 8.4 / 11.7 / 24.4 / 332 | 7.8 / 7.9 / 7.8 / 9.2 / 75 | 5.8 / 8.3 (both lifted: gated) / 7.8 / 9.3 / 75 | | memmove overlapping 16 B / 1 KB | 48.0 / 161 | 7.9 / 23 | 7.5 / 23 | | memcmp 4 B / 16 B / 256 B (equal) | 5.4 / 8.6 / 74 | 7.3 / 7.2 / 14.0 | 5.6 (lifted: gated) / 7.6 / 14.9 | | strlen / strcmp, 64 chars | 14.1 / 19.6 | 10.7 / 15.0 | 10.8 / 13.5 | | strcmp, 0-2 chars | 3.9-5.7 | 5.9 | 6.3-6.4 | | 2 MB copy, then reading it (us) | 188-234 + 24-30 | 266-325 + 116-119 | 51-54 + 24-25 | | 4 MB copy, then reading it (us) | 367-619 + 48-61 | 509-647 + 232-328 | 103-104 + 48-53 | ``` UCRT memmove alone (cache-warm data, one copy, then read): 1536 KB: 60 us + 28.6 us read; 1792 KB: 372 us + 115.0 us read (4.01 ns a line: past the cache); 4096 KB: 727 us + 257.7 us. The same 4 MB in 4 KB memcpy pieces: 154 us + 66.6 us. memset stays cached up to 4 MB. PASS: 1683333 calls (78208 under a size gate, 190 interrupt polls between pieces): the window, x0 and the page marks are the SDK's lifted functions' (marks after the data, 216724 marked writes); past the window the game's own; logged once entry hooks: 6 of 6 (memcpy sdk+0x78f020, memmove sdk+0x78f040, memset sdk+0x78f300, strlen sdk+0x7a7d40, memcmp sdk+0x7bd280, strcmp sdk+0x7bd380) ``` The first in-game A/B (main @ 71b1edd8 against 5839334c, Ryzen 9 3900X, RTX 3080, warm) lost 3.8% in CPU-bound Hyrule Field (1% lows 29.5 to 24.7 fps) and was level in Kokiri Forest. The micro-benchmark found no small-size cost large enough to explain that. It did find the non-temporal large copies, which match both the lower average and the worse 1% lows. This commit fixes those copies. **In the game, the re-run** (main @ 5c284721 against this PR @ 00ad3e5b; Windows 11 Pro N, Ryzen 9 3900X, 64 GB, RTX 3080, a hard drive; one game folder updated in place between the two (`make_game.py --update`, warm stores), an unmeasured run of each scene first, then the order main, PR, PR, main, main, PR, one 45 s window per scene after each install with `ab2.py`; through the PC's job queue (`--mode timing`); medians of 3): | | Before: main @ 5c284721 | After: this PR @ 00ad3e5b | |---|---|---| | Kokiri, camera turning, fps (median of 3) | 37.7 (37.97, 37.68, 37.61) | **38.6** (39.14, 38.05, 38.55) | | Kokiri, 1% low | 23.3, 15.9, 18.8 | 20.8, 21.4, 21.2 | | Hyrule Field, fps (median of 3) | 50.6 (50.41, 51.67, 50.64) | 50.6 (51.90, 50.22, 50.59) | | Hyrule Field, 1% low | 29.0, 32.1, 26.6 | 30.1, 26.8, 28.8 | | Hyrule Field, p99 frame time | 31.2, 28.3, 33.0 ms | 30.3, 33.5, 29.7 ms | With `HOS_HLE_STATS=1` (one untimed Hyrule Field run), the host takes in a typical 5 s of play about 229,000 `memcpy`, 202,000 `memset`, 100,000 `strcmp`, 56,000 `strlen`, 29,000 `memcmp` and 4,800 `memmove` calls, and copies in pieces about once (974 times in the 5 s that held the teleport's load). So the large copies happen in loads, rarely in play: they fit the first version's worse 1% lows, but the Field average it lost may also have been the tiny `memset`s now gated below 8 bytes (the busiest hook), or partly that set's own spread (its two runs were 5% apart). ``` [hos MainThread] libc: memcpy, memmove, memset, memcmp, strlen and strcmp are the host's (HOS_GUEST_LIBC=1: the game's own) entry hooks: 6 of 6 (memcpy sdk+0x78f020, memmove sdk+0x78f040, memset sdk+0x78f300, strlen sdk+0x7a7d40, memcmp sdk+0x7bd280, strcmp sdk+0x7bd380) entry hooks spliced: 6 [hos host] hle: 228712 libc_memcpy; [hos host] hle: 1 libc_pieced; ``` ### Flags and settings `HOS_GUEST_LIBC` existed already but did nothing, because the hooks never ran. Now it switches them off. | Flag | What it is | Before this PR | After this PR | Effect | Helps / doesn't help / untested | Recommendation | |---|---|---|---|---|---|---| | `HOS_GUEST_LIBC` (set) | the SDK's own memcpy, memmove, memset, memcmp, strlen and strcmp for every call | unset (no effect: the hooks were never called) | unset: the host's versions | set: every call runs the translated SDK code, on the same exe | untested in the game: for A/B only | leave unset; set only for A/B or to rule this change out | ## Testing **Tested:** in the game on Windows 11 (Measurements: the re-run, the counters, the log lines). Game-free on Windows 11 (TDM gcc for the fixtures, llvm-mingw clang for the objects), as follows. - `tests/test_libc_hooks.py` (new): - The splice over a made-up image with a real `a64_lift.Lifter`: the hooked function starts with the hook's line and the rest of its text is unchanged. A symbol in another module, an HLE'd symbol, an address that is no entry and an undefined symbol are left out. - The hooks against the model of the SDK's functions. Sizes 0 to 300 and up to 12 KB across pages, every alignment mod 16, every overlap in both directions. memcmp and strcmp with bytes above 0x7f. The marks must be set after the data. - Pieces: sizes from 256 KB - 1 to 768 KB, with every overlap of a piece boundary in both directions, and the polls between pieces counted. - Gates: memset under 8 bytes and memcmp under 16 are declined (nothing written or marked), and the gated line runs the lifted body under its size and the host from it. - The spliced line in a lifted-shaped function, and `HOS_GUEST_LIBC=1`. - Mutations caught: - marking one byte short, or before the write (copy and fill); - the old sign-extended result; - a forward copy over an overlap; - an unmarked piece, or no poll between pieces; - the memset gate gone from the hook or from the splice; - pieces of 4 MB. - `python -I -B tests/run_all.py --jobs 8 --changed longshot/main` (the checks that read the changed files, on main @ 5c284721): 96 passed, 8 skipped, 10 failed. The 10 failures also fail on a clean `main` worktree, with the same failing cases: `test_make_game_cache`, `test_linux_preflight`, `test_walk_key`, `test_failed_compile_keeps_objects`, `test_job_queue`, `test_missing_nup`, `test_run_all_changed`, `test_crash_dialog`, `test_vk_occlusion_queries`, and `test_walk` (no `lz4` module in this Python). - `host/nn_libc.c` compiled with llvm-mingw clang at the build's host flags. A lifted-shaped function with the spliced line compiled against the generated `a64.h`. - Linux (Docker `python:3.12`, gcc 14): `nn_libc.c` compiled; `test_libc_hooks`, `test_guest_copies` (its mmap fixture) and `test_lift_entry_below` pass. **Not tested:** Linux and macOS in the game; Castle Town Market (its teleport's measured window was a blank screen, on main too). **Must be tested:** - Linux, any GPU, by hand: the same two scenes, main against this PR, expecting the `libc: ... are the host's` line, no picture or behaviour change, and fps the same or higher. ## Risks **What could break:** - A host version that differs from the SDK's for some call would change game behaviour. The check compares all six with the SDK's behaviour, including overlapping copies and the exact x0. - A missed write mark would leave stale data on the GPU. The marks are the same pages as the lifted stores', set after the data. - A copy over 256 KB polls between its pieces, as the lifted loop does every 64 bytes. A virtual-core hand-off waits at most one piece (about 10 us). - The size gates and the piece size were chosen on one CPU (Zen 2) with Windows' CRT. On Linux, glibc's memmove has its own non-temporal threshold (a share of the L3). The pieces keep copies below it there too, but that was not measured. - The play profile (PGO) no longer matches the six functions' changed C; the rest of the profile is unaffected. **Turning it off:** `HOS_GUEST_LIBC=1` (same exe), or revert the commit. **Compatibility:** The lift key changes, so the next build lifts the game again and recompiles the lifted files that hold these six functions. Saves, `longshot_options.ini`, the shader and pipeline caches and the vk ABI are unchanged. ## Questions for reviewers 1. `memcmp`/`strcmp` now zero-extend x0, which is exactly what the SDK's code leaves there. Any caller reads only w0, so this changes nothing for compiled code, but it does differ from the hooks as written before. Is the change welcome? 2. Only the sdk module's definitions are hooked; a game module with its own `memcpy` keeps it. Should that be wider? 3. `ENTRY_HOOK(addr)` (by address) is still not spliced, and no hook uses it. Should this splice handle it too, or should the macro be removed? 4. The gates (memset 8, memcmp 16) and the 256 KB piece size come from one Zen 2 PC. Should they be checked on an Intel CPU and on Linux before merge? 5. The splice is an emit wrapper (as `nx2/lift_fixes.py` does for its entry goto) rather than a lifter table, because `third_party/a64-kit` stays untouched. Is a wrapper OK here? ## Checklist ### Everyone - [x] One topic. - [x] The title is a Conventional Commit (`type(scope): description`) of the status line's type. - [x] Why gives the reason plainly and links the issues that fit (the tracker searched). - [x] Measurements has a verdict, or one line saying why there is nothing to measure. - [x] Testing names what was tested, what wasn't, and what must still be tested, where and how. - [x] Rebased on current `main`. - [x] `python -I -B tests/run_all.py` passes (any failure that also fails on `main` is named). - [x] No game files, dumps, build output or generated files. - [x] `third_party/` untouched (a new folder with a `PROVENANCE.md` instead). - [x] The license is OK: GPL-3.0-or-later, nothing incompatible and nothing decompiled. ### When it applies - [x] AI assistance recorded with `Assisted-by:` / `Co-Authored-By:` trailers. - [x] Switches documented (docs/development.md; the README for a menu row) and shipped at their recommended value. - [x] A check under `tests/` that reads changed source updated in this PR, not weakened. - [x] Host code builds on both Windows (llvm-mingw) and Linux, with UTF-8 paths. - [x] Docs updated. 🤖 Generated with [Claude Code](https://claude.com/claude-code)
host/nn_libc.c has defined memcpy, memmove, memset, memcmp, strlen and strcmp
as ENTRY_HOOK_SYMs, but nothing in Longshot's lift called them: the game ran
the SDK's lifted copies (memcpy through a PLT jump and an indirect-call lookup
every time). nx2/entry_hooks.py finds each hooked symbol where the sdk module
defines it and starts that lifted function's C with the hook's call; the lifted
function stays behind it for the calls the host declines (past the guest
window, or HOS_GUEST_LIBC=1).

memcmp and strcmp now leave x0 zero-extended, as the SDK's w0 does. The first
hooked call logs whether the host's or the game's are in use. The lift key
names the hooks, so a new one lifts again.

tests/test_libc_hooks.py: the splice over a made-up image, and the hooks
against a byte-at-a-time model of the SDK's functions (window, x0 and page
marks equal, marks after the data) over sizes, alignments and overlaps.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
perf(hos): large host copies in cached pieces, small memset and memcmp left to the SDK
Some checks failed
CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Has been skipped
CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Has been skipped
CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Has been skipped
Third-party parts / The same as upstream (pull_request) Has been skipped
Website / Website on Cloudflare (pull_request) Has been skipped
CI / Conventional Commit title (pull_request) Successful in 8s
Website / Website on Cloudflare (pull_request_target) Has been skipped
Guide screenshots / Guide images and local UI source review (pull_request) Failing after 13s
CI / Workflow lint (actionlint) (pull_request) Successful in 13s
PR description / Description follows the template (pull_request_target) Successful in 10s
Third-party parts / As their notes say (pull_request) Successful in 19s
00ad3e5b36
The in-game A/B of the spliced hooks lost 3.8% in CPU-bound Hyrule Field.
Measured against the lifted SDK functions (one thread, warm caches), the
host's copies are as fast or faster at every size, but Windows' memmove
switches to non-temporal stores at about 1.75 MB: a 2-4 MB copy took 1.4x
the lifted copy's time and left its destination out of the cache, so
reading it after took 4-5x as long.

- Copies and fills over 256 KB go in pieces (each marked once written),
  with the lifted loop's interrupt poll (A64_POLL) between them: 4 MB now
  copies in 104 us against the lifted 385 us, and reads back as fast.
- ENTRY_HOOK_SYM_FROM(sym, min): the spliced line tests x2 before it
  calls, so memset under 8 bytes and memcmp under 16 (where the call cost
  1-4 ns more than it saved) stay in the lifted code.
- HOS_HLE_STATS=1 counts the calls the host took (libc_memcpy, ...,
  libc_pieced), out of line so the small calls' path stays as short.

Co-Authored-By: Claude Opus 5.5 (1M context) <[email protected]>
Author
Member

Tested in game: Windows 11, RTX 3080: Kokiri Forest (turning) and Hyrule Field, main 5c284721 against this PR, 3 warm interleaved runs each: Kokiri +2.3%, Field level; all 6 hooks spliced and logged as the host's

Tested in game: Windows 11, RTX 3080: Kokiri Forest (turning) and Hyrule Field, main 5c284721 against this PR, 3 warm interleaved runs each: Kokiri +2.3%, Field level; all 6 hooks spliced and logged as the host's
dyst changed title from WIP: perf(hos): splice the host's memcpy, memset and string hooks into the lifted SDK to perf(hos): splice the host's memcpy, memset and string hooks into the lifted SDK 2026-11-16 18:48:13 +00:00
futondestroyer left a comment

Reviewed 00ad3e5b36328883bc4065ddc08d21f9350b1dc7, code and posted evidence only. Request changes; not ready for final merge approval. No builds, tests, generators or games were run for this review, and no source changes were made.

Code finding: the newly activated memcmp, strlen and strcmp hooks do not enforce the guest-window boundary promised by this PR and documented by hos_guest_range() in host/hos.h. See the inline comment. Copy/fill hooks have the check; compare/string hooks still form unchecked GP() pointers and enter the native CRT. The new boundary fixture iterates only through MEMSET, so it misses all three readers. Please close that path and extend the fixture before enabling these hooks for all callers.

Readiness evidence to settle before marking ready:

  • Testing explicitly requires a Linux in-game main/PR comparison, and it is still missing. The Linux object/fixture checks are useful but do not satisfy that gameplay requirement. Keep this WIP until the required run is recorded.
  • The full-suite checkbox is ticked, but the test results shown are tests/run_all.py --changed longshot/main (96 passed, 8 skipped, 10 baseline-matched failures). Current guidance requires the whole suite before a PR is ready. Please provide that full-run result and baseline explanations, or leave the box unticked with the reason and keep WIP. I am not treating the documented baseline failures or skips as passes.
  • The diff has no docs/tested/ files for the new measured runs. Add one self-contained run record per measured session, with build/settings/method/results, and retain the applicable tested-configurations checklist item. The documented six warm interleaved gameplay sessions should have their corresponding records.

The implementation otherwise has several good safeguards: SDK-only symbol selection, preserving the original lifted body on a declined hook, no_local for hooked entries, emit-worker transport of the hook table, and lift-key inclusion of hook names/size gates. Large overlapping copies select the correct direction, mark each completed piece after writing it, and poll between pieces. The new test exercises these semantics across alignments, overlap boundaries, size gates and signed byte differences. Its worker-table/splice integration assertions are source/wiring checks, not an actual spawned-worker integration run.

Evidence acknowledged, not independently rerun: the author's revised Windows A/B reports Kokiri 37.7 -> 38.6 fps (+2.3%, three PR runs above all three baseline runs), Hyrule Field 50.6 -> 50.6 fps, and Linux game-free compilation/fixtures. The earlier 3.8% Field loss was followed by a revised implementation and another interleaved A/B; I am reviewing that revision, not repeating the old regression as an unresolved finding. Market was not successfully measured, as disclosed. Linux gameplay and portability/performance of the platform CRT remain unproven here.

On the reviewer questions: zero-extending the documented 32-bit SDK compare result is appropriate; keep the initial scope at SDK definitions; there is no need to add address hooks or remove the unused macro in this PR; and the emit wrapper is a reasonable way to keep vendored a64-kit untouched. Linux validation is already mandatory in Testing; Intel CPU threshold testing would strengthen the recommendation but I am not inventing an additional merge gate beyond the stated coverage.

Related open work: #1202 rewrites lifted lane-permute idioms; it is not the same optimization as activating these six SDK entry hooks. No duplicate libc-hook PR was found in the open-title search. Main has moved beyond the measured merge base, so any subsequent rebase should retain the documented rationale for the evidence or remeasure affected paths.

Reviewed `00ad3e5b36328883bc4065ddc08d21f9350b1dc7`, code and posted evidence only. **Request changes; not ready for final merge approval.** No builds, tests, generators or games were run for this review, and no source changes were made. **Code finding:** the newly activated `memcmp`, `strlen` and `strcmp` hooks do not enforce the guest-window boundary promised by this PR and documented by `hos_guest_range()` in `host/hos.h`. See the inline comment. Copy/fill hooks have the check; compare/string hooks still form unchecked `GP()` pointers and enter the native CRT. The new boundary fixture iterates only through `MEMSET`, so it misses all three readers. Please close that path and extend the fixture before enabling these hooks for all callers. **Readiness evidence to settle before marking ready:** - Testing explicitly requires a Linux in-game main/PR comparison, and it is still missing. The Linux object/fixture checks are useful but do not satisfy that gameplay requirement. Keep this WIP until the required run is recorded. - The full-suite checkbox is ticked, but the test results shown are `tests/run_all.py --changed longshot/main` (96 passed, 8 skipped, 10 baseline-matched failures). Current guidance requires the whole suite before a PR is ready. Please provide that full-run result and baseline explanations, or leave the box unticked with the reason and keep WIP. I am not treating the documented baseline failures or skips as passes. - The diff has no `docs/tested/` files for the new measured runs. Add one self-contained run record per measured session, with build/settings/method/results, and retain the applicable tested-configurations checklist item. The documented six warm interleaved gameplay sessions should have their corresponding records. The implementation otherwise has several good safeguards: SDK-only symbol selection, preserving the original lifted body on a declined hook, `no_local` for hooked entries, emit-worker transport of the hook table, and lift-key inclusion of hook names/size gates. Large overlapping copies select the correct direction, mark each completed piece after writing it, and poll between pieces. The new test exercises these semantics across alignments, overlap boundaries, size gates and signed byte differences. Its worker-table/splice integration assertions are source/wiring checks, not an actual spawned-worker integration run. Evidence acknowledged, not independently rerun: the author's revised Windows A/B reports Kokiri 37.7 -> 38.6 fps (+2.3%, three PR runs above all three baseline runs), Hyrule Field 50.6 -> 50.6 fps, and Linux game-free compilation/fixtures. The earlier 3.8% Field loss was followed by a revised implementation and another interleaved A/B; I am reviewing that revision, not repeating the old regression as an unresolved finding. Market was not successfully measured, as disclosed. Linux gameplay and portability/performance of the platform CRT remain unproven here. On the reviewer questions: zero-extending the documented 32-bit SDK compare result is appropriate; keep the initial scope at SDK definitions; there is no need to add address hooks or remove the unused macro in this PR; and the emit wrapper is a reasonable way to keep vendored a64-kit untouched. Linux validation is already mandatory in Testing; Intel CPU threshold testing would strengthen the recommendation but I am not inventing an additional merge gate beyond the stated coverage. Related open work: https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/1202 rewrites lifted lane-permute idioms; it is not the same optimization as activating these six SDK entry hooks. No duplicate libc-hook PR was found in the open-title search. Main has moved beyond the measured merge base, so any subsequent rebase should retain the documented rationale for the evidence or remeasure affected paths.
@ -56,2 +131,2 @@
ENTRY_HOOK_SYM(memcmp) {
if (guest_libc()) return 0;
ENTRY_HOOK_SYM_FROM(memcmp, 16) {
if (guest_libc() || ARG(2) < 16) return 0;
Member

[P2] Check the guest window before taking the compare/string hooks. Unlike memcpy/memmove/memset above, memcmp does not validate either [argument, argument+n) before GP() and the native CRT call; strlen and strcmp below likewise accept unchecked starting addresses and do not bound their scans to HOS_SPACE_END. For example, memcmp with x0=HOS_SPACE_END-8 and x2=16 still returns handled rather than declining before a host read crosses the window. hos.h explicitly documents that GP alone can wrap to an arbitrary host address. These bodies existed before, but this PR newly splices them into the SDK entries, making the unchecked paths live by default. Validate both memcmp ranges before forming the pointers, and use bounded guest-window handling for the string readers that declines before any out-of-window access. Extend the boundary fixture to the three readers: its current loop stops at MEMSET, so the advertised outside-window fallback is not checked for them. No in-game crash is claimed from this code-only review.

[P2] Check the guest window before taking the compare/string hooks. Unlike memcpy/memmove/memset above, memcmp does not validate either [argument, argument+n) before GP() and the native CRT call; strlen and strcmp below likewise accept unchecked starting addresses and do not bound their scans to HOS_SPACE_END. For example, memcmp with x0=HOS_SPACE_END-8 and x2=16 still returns handled rather than declining before a host read crosses the window. hos.h explicitly documents that GP alone can wrap to an arbitrary host address. These bodies existed before, but this PR newly splices them into the SDK entries, making the unchecked paths live by default. Validate both memcmp ranges before forming the pointers, and use bounded guest-window handling for the string readers that declines before any out-of-window access. Extend the boundary fixture to the three readers: its current loop stops at MEMSET, so the advertised outside-window fallback is not checked for them. No in-game crash is claimed from this code-only review.
TrashPanda left a comment

The copies and fills check the guest window, and a large one goes in pieces that are marked after they are written. I read head 00ad3e5b. It applies onto current main. The three readers do not, which is the change already asked for on this head.

memcmp, strlen and strcmp form a host pointer with GP and call the C library before either range is known to lie in the guest window. hos_guest_range is what the copies use, and its comment says GP alone can wrap to any host address. A memcmp whose first address is 8 bytes before the end of the window and whose length is 16 still returns handled. strlen and strcmp will scan until a NUL, including past that end.

Decline those three before any host read. memcmp takes both ranges through hos_guest_range. strlen and strcmp stop at the end of the window, and a missing NUL there is the lifted body. The outside-window check in tests/test_libc_hooks.py stops at memset. It should include these three.

The Linux run in Must be tested is still open: the same two scenes, the libc: line, no change in the picture, and fps the same or higher. The measured sessions have no file under docs/tested/. The full-suite box is ticked, and Testing names run_all.py --changed.

The copies and fills check the guest window, and a large one goes in pieces that are marked after they are written. I read head `00ad3e5b`. It applies onto current main. The three readers do not, which is the change already asked for on this head. `memcmp`, `strlen` and `strcmp` form a host pointer with `GP` and call the C library before either range is known to lie in the guest window. `hos_guest_range` is what the copies use, and its comment says `GP` alone can wrap to any host address. A `memcmp` whose first address is 8 bytes before the end of the window and whose length is 16 still returns handled. `strlen` and `strcmp` will scan until a NUL, including past that end. Decline those three before any host read. `memcmp` takes both ranges through `hos_guest_range`. `strlen` and `strcmp` stop at the end of the window, and a missing NUL there is the lifted body. The outside-window check in `tests/test_libc_hooks.py` stops at `memset`. It should include these three. The Linux run in Must be tested is still open: the same two scenes, the `libc:` line, no change in the picture, and fps the same or higher. The measured sessions have no file under `docs/tested/`. The full-suite box is ticked, and Testing names `run_all.py --changed`.
Contributor

A Linux run in the game, on a Steam Deck OLED: the hooks run, the frame rate is the same, and the game reaches play 4 to 7 s sooner.

unset (the host's) HOS_GUEST_LIBC=1 (the game's own)
A save on Death Mountain Trail, standing (the game's code limits), fps, three runs in turn 27.15, 21.10, 23.47 24.24, 23.97, 22.40
the same runs, seconds from the start to play 41, 39, 44 48, 49, 45
A wide view of about 2,100 draws (the replay limits), fps 28.70 28.70
the same runs, seconds from the start to play 35 42
  • The log says libc: memcpy, memmove, memset, memcmp, strlen and strcmp are the host's unset, and ... are the game's own with the switch.
  • The trail save does not load twice in the same state (659 to 1,158 draws in these six runs), so its frame rate cannot show a few percent either way; one unset run had an uneven frame (sd 13.7 ms). The wide view is level to the hundredth.
  • The time to play is shorter in all four pairs. Main 544229a2 alone, the same day: 29.05 and 29.16 fps in the wide view.

Hardware and method: Steam Deck OLED (AMD Custom APU 0932, 4 cores / 8 threads, RADV VANGOGH), SteamOS 3.9.5, 16 GB, on the charger, the GPU at a manual 1600 MHz. One game build on the Deck by the Linux builder (--jobs 4), no hot list: main 544229a2 with this PR and three others merged locally (#1209 00ad3e5b, #1158 4e889b09, #1377 e4a1471d, #875 d51401bc), each measured by its own switch in that one binary. Headless runs from a save, 720 presents measured once the scene has settled, the player's settings (a low internal resolution, 768x432 passes; a texture budget of 2,800 MB) with the frame cap lifted (HOS_FPS_MAX=1000), numbers from the game's own log lines. No picture check (no display). 0 fatal lines in the 18 starts and 2 tours of the session.

**A Linux run in the game, on a Steam Deck OLED: the hooks run, the frame rate is the same, and the game reaches play 4 to 7 s sooner.** | | unset (the host's) | `HOS_GUEST_LIBC=1` (the game's own) | |---|---|---| | A save on Death Mountain Trail, standing (the game's code limits), fps, three runs in turn | 27.15, 21.10, 23.47 | 24.24, 23.97, 22.40 | | the same runs, seconds from the start to play | 41, 39, 44 | 48, 49, 45 | | A wide view of about 2,100 draws (the replay limits), fps | 28.70 | 28.70 | | the same runs, seconds from the start to play | 35 | 42 | - The log says `libc: memcpy, memmove, memset, memcmp, strlen and strcmp are the host's` unset, and `... are the game's own` with the switch. - The trail save does not load twice in the same state (659 to 1,158 draws in these six runs), so its frame rate cannot show a few percent either way; one unset run had an uneven frame (sd 13.7 ms). The wide view is level to the hundredth. - The time to play is shorter in all four pairs. Main 544229a2 alone, the same day: 29.05 and 29.16 fps in the wide view. **Hardware and method:** Steam Deck OLED (AMD Custom APU 0932, 4 cores / 8 threads, RADV VANGOGH), SteamOS 3.9.5, 16 GB, on the charger, the GPU at a manual 1600 MHz. One game build on the Deck by the Linux builder (`--jobs 4`), no hot list: main 544229a2 with this PR and three others merged locally (#1209 00ad3e5b, #1158 4e889b09, #1377 e4a1471d, #875 d51401bc), each measured by its own switch in that one binary. Headless runs from a save, 720 presents measured once the scene has settled, the player's settings (a low internal resolution, 768x432 passes; a texture budget of 2,800 MB) with the frame cap lifted (`HOS_FPS_MAX=1000`), numbers from the game's own log lines. No picture check (no display). 0 fatal lines in the 18 starts and 2 tours of the session.
Some checks failed
CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Has been skipped
CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Has been skipped
CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Has been skipped
Third-party parts / The same as upstream (pull_request) Has been skipped
Website / Website on Cloudflare (pull_request) Has been skipped
CI / Conventional Commit title (pull_request) Successful in 8s
Website / Website on Cloudflare (pull_request_target) Has been skipped
Guide screenshots / Guide images and local UI source review (pull_request) Failing after 13s
CI / Workflow lint (actionlint) (pull_request) Successful in 13s
PR description / Description follows the template (pull_request_target) Successful in 10s
Third-party parts / As their notes say (pull_request) Successful in 19s
This pull request doesn't have enough approvals yet. 0 of 1 approvals granted.
This branch is out-of-date with the base branch
You are not authorized to merge this pull request.
View command line instructions

Checkout

From your project repository, check out a new branch and test the changes.
git fetch -u 'perf/host-libc-hooks:dyst-perf/host-libc-hooks'
git switch 'dyst-perf/host-libc-hooks'
Sign in to join this conversation.
No reviewers
No milestone
No project
No assignees
4 participants
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
MagicBeanDev/Longshot!1209
No description provided.