WIP: perf(os): mutex_lock's wait for a held mutex is out of line and cold #1953

Closed
siantm wants to merge 2 commits from siantm/Longshot:perf/mutex-lock-wait-cold into main
Contributor

Summary

mutex_lock (host/nn_os_sync.c), behind the game's nn::os mutex locks, has the uncontended lock (the owner check,
one compare-exchange) and the whole wait after a failed immediate acquire (the spin, the sleep, HOS_MUTEX_TRACE's log)
in one function, so the short path sits in a function shaped for the long one.

The spin and the sleep move word for word into mutex_lock_wait, marked noinline and cold; mutex_lock keeps the
recursion check and the compare-exchange and calls it whenever that immediate acquire fails: another thread holds the
mutex, or it is reserved for a woken condition-variable waiter while its owner is already NULL. The take, the waiter
registration and retry, and the unlock's wake are unchanged. In the production host object mutex_lock is then inlined
into its two callers and its uncontended lock has no frame of its own; hos_self is still a call (see Measurements).

No gain is measured yet. The one earlier figure (an uncontended lock and unlock, 18.55 ns on main and 18.03 ns
here) is within the rounds' spread. The short-contention micro-benchmark (a failed first CAS that the spin then wins,
no sleep; main against this head) is written and held for a quiet PC (Measurements). If the measurements on this
rebased head show no gain, this change is folded into #1957 (where mutex_lock takes the thread from its context and
the leaf becomes possible) or dropped.

This PR is now the split alone, rebased onto current main (with #2012's spin_begin, which runs inside the new
function). The three diagnostic fixes it carried (d3e02c70, 18e90465) are their own PR, #2063 (fix(os), on main,
as dyst suggested in 65813), with his
two notes on those lines. scripts/profiling/cpu_functions.json describes both functions for the CPU report.

From cafb4100 on exp/x86-asm-sync (its other half, each virtual core's owner slot on a cache line of its own, needs
lock-free slots main does not have, and is not here).

Read first: mutex_lock and mutex_lock_wait in host/nn_os_sync.c; tests/test_cv_handoff_threads.py, which cuts
the new function too (from its name on, so its attributes may change).

Overlap: #2063 (fix/os-mutex-wait-diagnostics @ 28cca9e5) changes lines inside the wait this PR moves; the two merge
without a conflict in either order (git merge-tree), the fixed lines then inside mutex_lock_wait. #2019
(fix/os-chain-flag-atomic @ 75107f9e), stacked on #2020 and merging together with it, also merges with this PR
without a conflict now (git merge-tree): this PR moves the const int chain = hos_chain_on; line it changes without
changing it, so merged it reads the flag as #2019 does inside mutex_lock_wait; the one-line conflict there is now
#2063's with #2019. #1957 (perf/mutex-ctx-thread @ a912c5b8) is stacked on this PR's earlier four commits and is
rebased onto this head (its owner's).

Testing

Tested: Windows 11, i7-9700, RTX 4060 (NVIDIA 610.88), llvm-mingw clang 23.1.5; game-free checks only. This
change has not run in the game on this PC yet (Still to test). Rebased onto main @ 9dd41326: the split commit is now
6288abca (git range-diff: the code unchanged; the test cuts mutex_lock_wait from its name on, as d3e02c70 did, and
the message no longer calls the uncontended lock a leaf), the CPU report's entries e4ea22d7 (unchanged). Each check
ran in a hidden console of its own, in one job of the PC's job queue (fb3-mx-1-light):

  • python -I -B tests/test_cv_handoff_threads.py (the mutexes and condition variables on real threads, the source as
    it is): passes at e4ea22d7.
  • host/nn_os_sync.c built into an object with the host build's own flags at main 9dd41326 and at e4ea22d7, and
    disassembled with llvm-objdump (Measurements).
  • python -I -B tests/run_all.py --strict --changed origin/main at e4ea22d7 (the checks that read a changed file, 8
    at once; build job fb3-mxn-changed): 13 passed, none failed, 1 skipped (test_tracy_funcs.py: Linux only).

Still to test:

  • Windows 11, this PC (i7-9700): the short-contention micro-benchmark, this PR against main 9dd41326, expecting the
    waiter's lock call and hand-over within the runs' spread or better and no sleep (by hand: held job fb3-1953-bench,
    written and built, not run yet)
  • Windows 11, this PC (i7-9700, RTX 4060 (NVIDIA 610.88)): Castle Town's market, 60 s, this PR and main interleaved
    and warmed, no switches set, average fps, 1 % lows, median and p99 frame times, expecting them within the runs'
    spread or better (by hand, queued)
  • Linux: the generated mutex_lock in the Linux host build against main's, and python3 -I -B tests/test_cv_handoff_threads.py, expecting it to pass (by hand)
  • Linux and Windows with other CPUs (an AMD CPU, fewer cores), and the Steam Deck: Castle Town's market for two
    minutes, this PR against main, expecting the same frame times as main or better and no stall (the Test Farm: play
    Castle Town's market for two minutes on PCs with AMD CPUs and on the Steam Deck, this PR against main, comparing frame
    times)

Measurements

Verdict: no gain shown: folded into #1957. The production code is shorter on the uncontended lock (13
instructions against 40, no frame), but the micro-benchmark at this head puts the uncontended lock and unlock about
1 ns apart inside overlapping ranges and shows no difference under contention (below). As dyst suggested, the split
goes on in #1957, where mutex_lock takes the thread from its context.

Production code (Windows only; no Linux toolchain on this PC): host/nn_os_sync.c built into nn_os_sync.o with the
host build's own flags (llvm-mingw clang 23.1.5, --target=x86_64-w64-mingw32 -O2 -g0 -march=native -mcx16 -ffp-contract=off -fms-extensions ..., taken from a local build's compile_commands.json; no LTO), at main @
9dd41326 and at this head e4ea22d7, read with llvm-objdump -dr:

  • main: mutex_lock is a function of its own, called from LockMutex's entry and from mutex_lock_owed. An
    uncontended lock runs 37 instructions in it, plus 3 in the caller (two argument moves and the call): 8 register
    pushes and 8 pops, a 0x78-byte frame with xmm6 saved and restored, three argument moves, the call to hos_self,
    the owner check, the reserved load, lock cmpxchg, the count store and the jump to the epilogue.
  • This PR: mutex_lock is no longer a function: the compiler inlines it into both callers. An uncontended lock is 13
    instructions inside LockMutex's entry: the call to hos_self (still a call, to host/hos_thread.c), the owner
    check, the reserved load, lock cmpxchg, the count store and the jump on; no push, pop or frame of its own
    (LockMutex's entry keeps its own 6 pushes and 0x48-byte frame, the same on both).
  • A failed immediate acquire: on main the call to mutex_lock and its frame came first, then the compare-exchange
    and the spin; here the compare-exchange fails inline, then three argument moves, the call to mutex_lock_wait and,
    after its return, a jump back (one argument move and one jump more than main, the same one call and return).
    mutex_lock_wait sets up the frame mutex_lock did on main (8 pushes, 0x78 bytes, xmm6) and runs the same spin.
    cold does not move it to a section of its own on this COFF target: it is in .text where mutex_lock was.
  • Sizes: LockMutex's entry 560 to 592 bytes, mutex_lock_owed 224 to 304, mutex_lock (1056) gone,
    mutex_lock_wait 998.

Measured (fb3-1953-bench, run alone on this PC, i7-9700, Windows 11, clang 23.1.5; median of 21 interleaved runs
a side, the runs' range):

Before: main @ 9dd41326 After: this PR @ e4ea22d7
uncontended lock + unlock, ns 16.33 (16.00-17.71) 15.24 (14.90-22.42)
the waiter's lock call under short contention, ns 1157.0 (1153.0-1170.7) 1159.7 (1153.7-1169.3)
hand-over, unlock to the waiter's return, ns 91.7 (84.3-98.3) 92.3 (85.0-98.3)
iterations that slept (of 20000) 9 (2-25) 8 (1-21)

How it was run: fb3-1953-bench, main 9dd41326 against e4ea22d7, mutex_lock
cut from each checkout and built with the host build's flags, hos_self, hos_spin_ns and hos_now_ns in a
translation unit of their own as in the host; two threads pinned to two cores, the holder keeping the mutex 1 us after
the waiter starts its lock (the spin is 5 us), 20000 hand-overs a run, 21 runs a side interleaved: the waiter's lock
call, the hand-over from unlock to the waiter's return, the number of iterations that slept (0 expected: the spin
wins), and the uncontended lock and unlock.

Earlier, before the rebase (main @ 8859673e, this change as 1e6a4721): an uncontended mutex_lock + mutex_unlock
through tests/test_cv_handoff_threads.py's fixture (its hos_self a local function), 21 interleaved rounds, in one
job of the PC's job queue (microbench.py mutex):

Before: main @ 8859673e After: this PR @ 1e6a4721 (the same change on 8859673e)
uncontended lock + unlock, ns (median of 21 rounds, range) 18.55 (18.14 to 19.68) 18.03 (17.26 to 18.98)
uncontended mutex_lock + mutex_unlock    main           median    18.55 ns  (rounds 18.14 to 19.68)
uncontended mutex_lock + mutex_unlock    PR             median    18.03 ns  (rounds 17.26 to 18.98)

Risks

The wait is the same code in another function, so a mistake would show as a lock that never waits or never wakes: a
stall, which tests/test_cv_handoff_threads.py and the other mutex checks cover. A lock whose immediate acquire fails
makes its one call after the compare-exchange instead of before it, with one move and one jump more, even when the
spin then wins at once: the held benchmark measures that case. LockMutex's entry and mutex_lock_owed grow by 32 and
80 bytes with the inlined lock. No switch: a revert is enough. Players notice nothing (no saves,
longshot_options.ini or caches change).

Checklist

Ready for review

  • One topic, on current main; the title is a Conventional Commit.
  • Tested on my own PC, in the game for a change the game runs, as Testing says.
  • mise run test passes (any failure that also fails on main is named).
  • Measured against main, or Measurements says why there is nothing to measure.
  • No game files, build output or hand-edited generated files; third_party/ untouched; nothing GPL-incompatible.

When it applies

  • A check under tests/ that reads changed source is updated, not weakened.
  • Host code builds on Windows (llvm-mingw) and Linux, with UTF-8 paths.
  • Docs updated.
  • AI assistance recorded with Assisted-by: / Co-Authored-By: trailers.

Ready to merge

  • Everything in Still to test has run, its results in Tested.
  • Tier 1 hardware it can affect is tested (docs/dev/standards.md, "Hardware tiers").
  • Approved in review, every comment answered.
  • CI passes on the head; on current main, measured again or a comment says why the numbers still hold.
## Summary `mutex_lock` (`host/nn_os_sync.c`), behind the game's `nn::os` mutex locks, has the uncontended lock (the owner check, one compare-exchange) and the whole wait after a failed immediate acquire (the spin, the sleep, `HOS_MUTEX_TRACE`'s log) in one function, so the short path sits in a function shaped for the long one. The spin and the sleep move word for word into `mutex_lock_wait`, marked `noinline` and `cold`; `mutex_lock` keeps the recursion check and the compare-exchange and calls it whenever that immediate acquire fails: another thread holds the mutex, or it is reserved for a woken condition-variable waiter while its owner is already NULL. The take, the waiter registration and retry, and the unlock's wake are unchanged. In the production host object `mutex_lock` is then inlined into its two callers and its uncontended lock has no frame of its own; `hos_self` is still a call (see Measurements). **No gain is measured yet.** The one earlier figure (an uncontended lock and unlock, 18.55 ns on `main` and 18.03 ns here) is within the rounds' spread. The short-contention micro-benchmark (a failed first CAS that the spin then wins, no sleep; `main` against this head) is written and held for a quiet PC (Measurements). If the measurements on this rebased head show no gain, this change is folded into #1957 (where `mutex_lock` takes the thread from its context and the leaf becomes possible) or dropped. This PR is now the split alone, rebased onto current `main` (with #2012's `spin_begin`, which runs inside the new function). The three diagnostic fixes it carried (d3e02c70, 18e90465) are their own PR, #2063 (`fix(os)`, on `main`, as dyst suggested in [65813](https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/1953#issuecomment-65813)), with his two notes on those lines. `scripts/profiling/cpu_functions.json` describes both functions for the CPU report. From cafb4100 on `exp/x86-asm-sync` (its other half, each virtual core's owner slot on a cache line of its own, needs lock-free slots `main` does not have, and is not here). Read first: `mutex_lock` and `mutex_lock_wait` in `host/nn_os_sync.c`; `tests/test_cv_handoff_threads.py`, which cuts the new function too (from its name on, so its attributes may change). Overlap: #2063 (`fix/os-mutex-wait-diagnostics` @ 28cca9e5) changes lines inside the wait this PR moves; the two merge without a conflict in either order (`git merge-tree`), the fixed lines then inside `mutex_lock_wait`. #2019 (`fix/os-chain-flag-atomic` @ 75107f9e), stacked on #2020 and merging together with it, also merges with this PR without a conflict now (`git merge-tree`): this PR moves the `const int chain = hos_chain_on;` line it changes without changing it, so merged it reads the flag as #2019 does inside `mutex_lock_wait`; the one-line conflict there is now #2063's with #2019. #1957 (`perf/mutex-ctx-thread` @ a912c5b8) is stacked on this PR's earlier four commits and is rebased onto this head (its owner's). ## Testing **Tested:** Windows 11, i7-9700, RTX 4060 (NVIDIA 610.88), llvm-mingw clang 23.1.5; game-free checks only. This change has not run in the game on this PC yet (Still to test). Rebased onto `main` @ 9dd41326: the split commit is now 6288abca (`git range-diff`: the code unchanged; the test cuts `mutex_lock_wait` from its name on, as d3e02c70 did, and the message no longer calls the uncontended lock a leaf), the CPU report's entries e4ea22d7 (unchanged). Each check ran in a hidden console of its own, in one job of the PC's job queue (fb3-mx-1-light): - `python -I -B tests/test_cv_handoff_threads.py` (the mutexes and condition variables on real threads, the source as it is): passes at e4ea22d7. - `host/nn_os_sync.c` built into an object with the host build's own flags at `main` 9dd41326 and at e4ea22d7, and disassembled with llvm-objdump (Measurements). - `python -I -B tests/run_all.py --strict --changed origin/main` at e4ea22d7 (the checks that read a changed file, 8 at once; build job fb3-mxn-changed): 13 passed, none failed, 1 skipped (`test_tracy_funcs.py`: Linux only). **Still to test:** - Windows 11, this PC (i7-9700): the short-contention micro-benchmark, this PR against `main` 9dd41326, expecting the waiter's lock call and hand-over within the runs' spread or better and no sleep (by hand: held job fb3-1953-bench, written and built, not run yet) - Windows 11, this PC (i7-9700, RTX 4060 (NVIDIA 610.88)): Castle Town's market, 60 s, this PR and `main` interleaved and warmed, no switches set, average fps, 1 % lows, median and p99 frame times, expecting them within the runs' spread or better (by hand, queued) - Linux: the generated `mutex_lock` in the Linux host build against `main`'s, and `python3 -I -B tests/test_cv_handoff_threads.py`, expecting it to pass (by hand) - Linux and Windows with other CPUs (an AMD CPU, fewer cores), and the Steam Deck: Castle Town's market for two minutes, this PR against `main`, expecting the same frame times as `main` or better and no stall (the Test Farm: play Castle Town's market for two minutes on PCs with AMD CPUs and on the Steam Deck, this PR against main, comparing frame times) ## Measurements **Verdict:** no gain shown: folded into #1957. The production code is shorter on the uncontended lock (13 instructions against 40, no frame), but the micro-benchmark at this head puts the uncontended lock and unlock about 1 ns apart inside overlapping ranges and shows no difference under contention (below). As dyst suggested, the split goes on in #1957, where `mutex_lock` takes the thread from its context. Production code (Windows only; no Linux toolchain on this PC): `host/nn_os_sync.c` built into `nn_os_sync.o` with the host build's own flags (llvm-mingw clang 23.1.5, `--target=x86_64-w64-mingw32 -O2 -g0 -march=native -mcx16 -ffp-contract=off -fms-extensions ...`, taken from a local build's `compile_commands.json`; no LTO), at `main` @ 9dd41326 and at this head e4ea22d7, read with `llvm-objdump -dr`: - `main`: `mutex_lock` is a function of its own, called from LockMutex's entry and from `mutex_lock_owed`. An uncontended lock runs 37 instructions in it, plus 3 in the caller (two argument moves and the call): 8 register pushes and 8 pops, a 0x78-byte frame with `xmm6` saved and restored, three argument moves, the call to `hos_self`, the owner check, the `reserved` load, `lock cmpxchg`, the count store and the jump to the epilogue. - This PR: `mutex_lock` is no longer a function: the compiler inlines it into both callers. An uncontended lock is 13 instructions inside LockMutex's entry: the call to `hos_self` (still a call, to `host/hos_thread.c`), the owner check, the `reserved` load, `lock cmpxchg`, the count store and the jump on; no push, pop or frame of its own (LockMutex's entry keeps its own 6 pushes and 0x48-byte frame, the same on both). - A failed immediate acquire: on `main` the call to `mutex_lock` and its frame came first, then the compare-exchange and the spin; here the compare-exchange fails inline, then three argument moves, the call to `mutex_lock_wait` and, after its return, a jump back (one argument move and one jump more than `main`, the same one call and return). `mutex_lock_wait` sets up the frame `mutex_lock` did on `main` (8 pushes, 0x78 bytes, `xmm6`) and runs the same spin. `cold` does not move it to a section of its own on this COFF target: it is in `.text` where `mutex_lock` was. - Sizes: LockMutex's entry 560 to 592 bytes, `mutex_lock_owed` 224 to 304, `mutex_lock` (1056) gone, `mutex_lock_wait` 998. Measured (`fb3-1953-bench`, run alone on this PC, i7-9700, Windows 11, clang 23.1.5; median of 21 interleaved runs a side, the runs' range): | | Before: main @ 9dd41326 | After: this PR @ e4ea22d7 | |---|---|---| | uncontended lock + unlock, ns | 16.33 (16.00-17.71) | 15.24 (14.90-22.42) | | the waiter's lock call under short contention, ns | 1157.0 (1153.0-1170.7) | 1159.7 (1153.7-1169.3) | | hand-over, unlock to the waiter's return, ns | 91.7 (84.3-98.3) | 92.3 (85.0-98.3) | | iterations that slept (of 20000) | 9 (2-25) | 8 (1-21) | How it was run: `fb3-1953-bench`, `main` 9dd41326 against e4ea22d7, `mutex_lock` cut from each checkout and built with the host build's flags, `hos_self`, `hos_spin_ns` and `hos_now_ns` in a translation unit of their own as in the host; two threads pinned to two cores, the holder keeping the mutex 1 us after the waiter starts its lock (the spin is 5 us), 20000 hand-overs a run, 21 runs a side interleaved: the waiter's lock call, the hand-over from unlock to the waiter's return, the number of iterations that slept (0 expected: the spin wins), and the uncontended lock and unlock. Earlier, before the rebase (`main` @ 8859673e, this change as 1e6a4721): an uncontended `mutex_lock` + `mutex_unlock` through `tests/test_cv_handoff_threads.py`'s fixture (its `hos_self` a local function), 21 interleaved rounds, in one job of the PC's job queue (`microbench.py mutex`): | | Before: main @ 8859673e | After: this PR @ 1e6a4721 (the same change on 8859673e) | |---|---|---| | uncontended lock + unlock, ns (median of 21 rounds, range) | 18.55 (18.14 to 19.68) | 18.03 (17.26 to 18.98) | ``` uncontended mutex_lock + mutex_unlock main median 18.55 ns (rounds 18.14 to 19.68) uncontended mutex_lock + mutex_unlock PR median 18.03 ns (rounds 17.26 to 18.98) ``` ## Risks The wait is the same code in another function, so a mistake would show as a lock that never waits or never wakes: a stall, which `tests/test_cv_handoff_threads.py` and the other mutex checks cover. A lock whose immediate acquire fails makes its one call after the compare-exchange instead of before it, with one move and one jump more, even when the spin then wins at once: the held benchmark measures that case. LockMutex's entry and `mutex_lock_owed` grow by 32 and 80 bytes with the inlined lock. No switch: a revert is enough. Players notice nothing (no saves, `longshot_options.ini` or caches change). ## Checklist ### Ready for review - [x] One topic, on current `main`; the title is a Conventional Commit. - [ ] Tested on my own PC, in the game for a change the game runs, as Testing says. - [ ] `mise run test` passes (any failure that also fails on `main` is named). - [ ] Measured against `main`, or Measurements says why there is nothing to measure. - [x] No game files, build output or hand-edited generated files; `third_party/` untouched; nothing GPL-incompatible. ### When it applies - [x] A check under `tests/` that reads changed source is updated, not weakened. - [ ] Host code builds on Windows (llvm-mingw) and Linux, with UTF-8 paths. - [x] Docs updated. - [x] AI assistance recorded with `Assisted-by:` / `Co-Authored-By:` trailers. ### Ready to merge - [ ] Everything in Still to test has run, its results in Tested. - [ ] Tier 1 hardware it can affect is tested (docs/dev/standards.md, "Hardware tiers"). - [ ] Approved in review, every comment answered. - [ ] CI passes on the head; on current `main`, measured again or a comment says why the numbers still hold.
perf(os): mutex_lock's wait for a held mutex is out of line and cold
Some checks failed
PR description / Description follows the template (pull_request_target) Successful in 11s
Website / Website on Cloudflare (pull_request_target) Has been skipped
CI / Conventional Commit title (pull_request) Has been cancelled
CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Has been cancelled
CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Has been cancelled
CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Has been cancelled
CI / Workflow lint (actionlint) (pull_request) Has been cancelled
CI / Website build and checks (pull_request) Has been cancelled
Guide screenshots / Guide images and local UI source review (pull_request) Has been cancelled
Third-party parts / As their notes say (pull_request) Has been cancelled
Third-party parts / The same as upstream (pull_request) Has been cancelled
95c0fd4152
mutex_lock's spin and sleep, taken only when another thread holds the mutex, are moved word for word into
mutex_lock_wait (noinline, cold), so the uncontended lock (owner check, one compare-exchange) is a small leaf that does
not save registers or set up a frame for the wait. tests/test_cv_handoff_threads.py cuts the new function too.

From cafb4100 on exp/x86-asm-sync (its other half, the virtual cores' owner slots on their own cache lines, needs the
lock-free slots main does not have).

Assisted-by: Claude Code
Member

Possibly related, for context:

  • issue #1504 On Linux the Windows lock shim sleeps at once on a busy lock and makes a system call on every wake, even when nobody waits — Related Linux lock performance investigation; targets different shim/lock path
Possibly related, for context: - issue #1504 On Linux the Windows lock shim sleeps at once on a busy lock and makes a system call on every wake, even when nobody waits — Related Linux lock performance investigation; targets different shim/lock path
BeanBot left a comment

Recommendation: Request changes. The split preserves the mutex protocol in source, but the fixture shows no reliable speedup and leaves the cost of brief contention unmeasured. The extracted wait also carries forward races and an unsafe owner pointer in its diagnostics.

Moves spinning, sleeping, and wait diagnostics out of the ordinary guest mutex lock. This gives the compiler a smaller fast path to lay out, at the cost of an out-of-line transfer whenever the first acquire fails. The existing threaded fixture compiles the extracted helper.

Does it help: Unproven. In 21 interleaved Windows rounds on the i7-9700 with llvm-mingw Clang 23.1.5, median uncontended lock plus unlock went from 18.55 ns on 8859673e to 18.03 ns on 1e6a4721. The ranges overlap (18.14–19.68 ns and 17.26–18.98 ns). The fixture uses a local thread lookup and a no-op spin. No in-game measurements or contended timings are reported for 95c0fd41 against 2e1f6c32.
Other systems: Direct LockMutex and condition-variable reacquisition without a reservation enter the changed path. The reserved reacquire path, TryLockMutex, and UnlockMutex retain their source logic. Short contention can affect frame pacing, audio, and other guest workers on both OSes. The pthread fixture models wake ordering, while Windows uses native SRW waits and Linux and Steam Deck use the futex-backed compatibility layer. GPU vendors share this CPU-side path. Tracy retains the zone label but records the helper as its source function. The CPU profiler still categorizes the helper as os/sync by object file. Saves, settings, caches, and renderer interfaces have no changes in the diff.
Switches: None added, changed, or removed. HOS_MUTEX_TRACE still defaults to 0 (off), which keeps optional trace logging off. Its existing switch-table row remains applicable.

Reviewed: 95c0fd4152 against 2e1f6c32d3: Read AGENTS.md, CONTRIBUTING.md, docs/dev/standards.md, the Forgejo PR template, and both changed files in full. No docs/review-checklist.md was present. Read the relevant host thread lifetime and wait code, Linux wait implementation, host build flags, Tracy and chain paths, source cutter, and test runner selection. Skimmed sibling checks, profiling metadata, and related recorded runs. No matching recorded run or tester comment was available. The earlier PR revisions named in Testing and Measurements were unavailable in the checkout. No translations or generated files changed.; nothing run (a code review).
Checked and fine:

  • The recursive owner check, reservation check, CAS ordering, waiter registration and retry, and unlock wake sequence are preserved.
  • The extraction adds no allocation, buffer, guest offset, file access, or CPU feature requirement. The attribute syntax already has precedent in the host and is supported by the Windows and Linux toolchains.
  • The fixture extracts the helper before its caller, clears inherited HOS_ switches, and retains both handoff modes and the existing failure controls.
  • The diff is one topic. The commit has an Assisted-by trailer, and the diff contains no game-derived artifacts, vendor SDKs, third-party or generated changes, or versioned API changes.

Blocking (to change before merge)

  1. host/nn_os_sync.c:635: The extracted trace path retains an unsynchronized lazy threshold. Two threads entering their first sleeping wait on different mutexes can read and write the same static trace_ms even with the switch unset. An oversized HOS_MUTEX_TRACE value also exceeds atoi's representable range. Parse with range checks and publish the result once with synchronization, or use atomic accesses and keep a local threshold for the whole wait.
  2. host/nn_os_sync.c:670: The moved diagnostic code retains an owner pointer without protecting its lifetime or name. An owner can unlock, exit, and be destroyed while the waiter is still blocked or reacquiring a virtual core. The trace then dereferences freed storage. Concurrent SetThreadName can also race with the Tracy and long-wait name reads. Capture the name by value through a lifetime- and rename-safe lookup, or log a stable owner identifier, in all three diagnostic paths.
  3. host/nn_os_sync.c:656: The moved ten-second log reads m->count under m->lk, while the guest mutex owner updates the count without taking m->lk. A recursive acquire or final unlock during the log gives a C data race. Omit the count from this diagnostic or make every access to the mutex count atomic.
  4. host/nn_os_sync.c:640: The extracted wait still reads the volatile hos_chain_on flag while the recorder thread starts or stops capture and a producer can clear it at capacity. Volatile gives neither synchronization nor publication of recs. Use atomic flag accesses throughout the recorder and at its call sites, with release publication and acquire reads before using the buffer.
  5. host/nn_os_sync.c:633: The spin moved into this helper still reaches spin_begin's plain lazy adaptive flag. Two x86-64 workers can first spin on separate mutexes at the same time and race initializing it. Use an atomic cached value or initialize the setting before workers start. The fixture replaces the spin, so it cannot exercise this path.
  6. tests/test_cv_handoff_threads.py:92: The fixture still forces gcc in the runner and defaults to gcc when invoked directly, ignoring CC. A documented Windows llvm-mingw setup can have Clang and winpthreads without gcc on PATH, so the check fails or is skipped before it covers the new helper. Honor --cc and CC, use the configured platform compiler as the fallback, and give the runner the same compiler selection.

Waiting on the author (a run, a measurement or a decision; no edit does it)

  • host/nn_os_sync.c:674: The production fast path still calls hos_self from a separate translation unit in the ordinary non-LTO host build. The fixture's local definition cannot establish the claimed leaf or frame shape. Compare the generated production callers on Windows and Linux, then show a gain on current base versus head in warmed, interleaved game runs with the same route and settings. Report average FPS, 1% lows, median and p99 frame time, and the cost of the targeted lock path.
  • host/nn_os_sync.c:676: A briefly held job-queue mutex now sends a waiter into cold code even when the retry succeeds without a sleep. That adds a taken transfer into a different part of the instruction stream on short handoffs, which the single-thread benchmark never enters. Measure failed-first-CAS acquisitions that finish without parking and compare frame-time tails on AMD and lower-core systems, including Steam Deck, as well as the Windows Intel system.
  • This host change opened before the required author game run. The reported older Windows check covered a source-cut fixture, and the strict changed suite reported one skip. Record the author's game run, Windows and Linux host builds, the full checks with the skip explained, and CI on the reviewed head. Record the outstanding Tier 1 play results before merge.

Suggestions (the author may take or leave)

  • Update the existing mutex_lock description in scripts/profiling/cpu_functions.json and add one for mutex_lock_wait so profiler reports explain the new symbol.
  • host/nn_os_sync.c:656: Holding m->lk does not protect count from the owner’s recursive acquire or unlock. Drop this diagnostic field or make all mutex count accesses atomic.
  • host/nn_os_sync.c:640: The recorder changes this volatile flag concurrently. Use atomic accesses and release/acquire publication for the recording buffer.

Testing: Read-only review. The author reports a passing threaded fixture and 11 passes, 0 failures, and 1 skip in the changed suite at a105a307. The fixture exercises sleeping on contention and handoff ordering, including a case that stalls if the helper call is removed. Its no-op spin leaves spin success and spin cost untested. The author reports no rerun at HEAD.
Overlaps and merge order: None found in the available history or review discussion. The description identifies a separate experimental virtual-core owner-slot change and gives no merge dependency.


Automated code review by BeanBot (gpt-6.2-sol), on the pull request's opening. Nothing was run; a maintainer decides.

**Recommendation: Request changes.** The split preserves the mutex protocol in source, but the fixture shows no reliable speedup and leaves the cost of brief contention unmeasured. The extracted wait also carries forward races and an unsafe owner pointer in its diagnostics. Moves spinning, sleeping, and wait diagnostics out of the ordinary guest mutex lock. This gives the compiler a smaller fast path to lay out, at the cost of an out-of-line transfer whenever the first acquire fails. The existing threaded fixture compiles the extracted helper. **Does it help:** Unproven. In 21 interleaved Windows rounds on the i7-9700 with llvm-mingw Clang 23.1.5, median uncontended lock plus unlock went from 18.55 ns on 8859673e to 18.03 ns on 1e6a4721. The ranges overlap (18.14–19.68 ns and 17.26–18.98 ns). The fixture uses a local thread lookup and a no-op spin. No in-game measurements or contended timings are reported for 95c0fd41 against 2e1f6c32. **Other systems:** Direct LockMutex and condition-variable reacquisition without a reservation enter the changed path. The reserved reacquire path, TryLockMutex, and UnlockMutex retain their source logic. Short contention can affect frame pacing, audio, and other guest workers on both OSes. The pthread fixture models wake ordering, while Windows uses native SRW waits and Linux and Steam Deck use the futex-backed compatibility layer. GPU vendors share this CPU-side path. Tracy retains the zone label but records the helper as its source function. The CPU profiler still categorizes the helper as os/sync by object file. Saves, settings, caches, and renderer interfaces have no changes in the diff. **Switches:** None added, changed, or removed. HOS_MUTEX_TRACE still defaults to 0 (off), which keeps optional trace logging off. Its existing switch-table row remains applicable. **Reviewed:** 95c0fd41526b against 2e1f6c32d30e: Read AGENTS.md, CONTRIBUTING.md, docs/dev/standards.md, the Forgejo PR template, and both changed files in full. No docs/review-checklist.md was present. Read the relevant host thread lifetime and wait code, Linux wait implementation, host build flags, Tracy and chain paths, source cutter, and test runner selection. Skimmed sibling checks, profiling metadata, and related recorded runs. No matching recorded run or tester comment was available. The earlier PR revisions named in Testing and Measurements were unavailable in the checkout. No translations or generated files changed.; nothing run (a code review). **Checked and fine:** - The recursive owner check, reservation check, CAS ordering, waiter registration and retry, and unlock wake sequence are preserved. - The extraction adds no allocation, buffer, guest offset, file access, or CPU feature requirement. The attribute syntax already has precedent in the host and is supported by the Windows and Linux toolchains. - The fixture extracts the helper before its caller, clears inherited HOS_ switches, and retains both handoff modes and the existing failure controls. - The diff is one topic. The commit has an Assisted-by trailer, and the diff contains no game-derived artifacts, vendor SDKs, third-party or generated changes, or versioned API changes. **Blocking** (to change before merge) 1. host/nn_os_sync.c:635: The extracted trace path retains an unsynchronized lazy threshold. Two threads entering their first sleeping wait on different mutexes can read and write the same static trace_ms even with the switch unset. An oversized HOS_MUTEX_TRACE value also exceeds atoi's representable range. Parse with range checks and publish the result once with synchronization, or use atomic accesses and keep a local threshold for the whole wait. 2. host/nn_os_sync.c:670: The moved diagnostic code retains an owner pointer without protecting its lifetime or name. An owner can unlock, exit, and be destroyed while the waiter is still blocked or reacquiring a virtual core. The trace then dereferences freed storage. Concurrent SetThreadName can also race with the Tracy and long-wait name reads. Capture the name by value through a lifetime- and rename-safe lookup, or log a stable owner identifier, in all three diagnostic paths. 3. host/nn_os_sync.c:656: The moved ten-second log reads m->count under m->lk, while the guest mutex owner updates the count without taking m->lk. A recursive acquire or final unlock during the log gives a C data race. Omit the count from this diagnostic or make every access to the mutex count atomic. 4. host/nn_os_sync.c:640: The extracted wait still reads the volatile hos_chain_on flag while the recorder thread starts or stops capture and a producer can clear it at capacity. Volatile gives neither synchronization nor publication of recs. Use atomic flag accesses throughout the recorder and at its call sites, with release publication and acquire reads before using the buffer. 5. host/nn_os_sync.c:633: The spin moved into this helper still reaches spin_begin's plain lazy adaptive flag. Two x86-64 workers can first spin on separate mutexes at the same time and race initializing it. Use an atomic cached value or initialize the setting before workers start. The fixture replaces the spin, so it cannot exercise this path. 6. tests/test_cv_handoff_threads.py:92: The fixture still forces gcc in the runner and defaults to gcc when invoked directly, ignoring CC. A documented Windows llvm-mingw setup can have Clang and winpthreads without gcc on PATH, so the check fails or is skipped before it covers the new helper. Honor --cc and CC, use the configured platform compiler as the fallback, and give the runner the same compiler selection. **Waiting on the author** (a run, a measurement or a decision; no edit does it) - host/nn_os_sync.c:674: The production fast path still calls hos_self from a separate translation unit in the ordinary non-LTO host build. The fixture's local definition cannot establish the claimed leaf or frame shape. Compare the generated production callers on Windows and Linux, then show a gain on current base versus head in warmed, interleaved game runs with the same route and settings. Report average FPS, 1% lows, median and p99 frame time, and the cost of the targeted lock path. - host/nn_os_sync.c:676: A briefly held job-queue mutex now sends a waiter into cold code even when the retry succeeds without a sleep. That adds a taken transfer into a different part of the instruction stream on short handoffs, which the single-thread benchmark never enters. Measure failed-first-CAS acquisitions that finish without parking and compare frame-time tails on AMD and lower-core systems, including Steam Deck, as well as the Windows Intel system. - This host change opened before the required author game run. The reported older Windows check covered a source-cut fixture, and the strict changed suite reported one skip. Record the author's game run, Windows and Linux host builds, the full checks with the skip explained, and CI on the reviewed head. Record the outstanding Tier 1 play results before merge. **Suggestions** (the author may take or leave) - Update the existing mutex_lock description in scripts/profiling/cpu_functions.json and add one for mutex_lock_wait so profiler reports explain the new symbol. - `host/nn_os_sync.c:656`: Holding m->lk does not protect count from the owner’s recursive acquire or unlock. Drop this diagnostic field or make all mutex count accesses atomic. - `host/nn_os_sync.c:640`: The recorder changes this volatile flag concurrently. Use atomic accesses and release/acquire publication for the recording buffer. **Testing:** Read-only review. The author reports a passing threaded fixture and 11 passes, 0 failures, and 1 skip in the changed suite at a105a307. The fixture exercises sleeping on contention and handoff ordering, including a case that stalls if the helper call is removed. Its no-op spin leaves spin success and spin cost untested. The author reports no rerun at HEAD. **Overlaps and merge order:** None found in the available history or review discussion. The description identifies a separate experimental virtual-core owner-slot change and gives no merge dependency. --- <sub>Automated code review by BeanBot (gpt-6.2-sol), on the pull request's opening. Nothing was run; a maintainer decides.</sub>
@ -634,1 +630,4 @@
/* mutex_lock's way when another thread holds the mutex: its spin, then its sleep. Out of line and cold, so the uncontended
lock below does not save registers and set up a stack frame for this part */
static __attribute__((noinline, cold)) void mutex_lock_wait(Obj *m, HosThread *self, uint64_t guest, Ctx *c) {
spin_until_free(m);
Member

Two first-time spinners can race initializing spin_begin’s adaptive flag. Initialize before workers start or cache it atomically.

Two first-time spinners can race initializing spin_begin’s adaptive flag. Initialize before workers start or cache it atomically.
@ -634,3 +632,4 @@
static __attribute__((noinline, cold)) void mutex_lock_wait(Obj *m, HosThread *self, uint64_t guest, Ctx *c) {
spin_until_free(m);
if (mutex_take(m, self)) { m->count = 1; return; }
/* HOS_MUTEX_TRACE=<ms> (diagnostic): waits longer than that are logged with the thread that held the mutex */
Member

Concurrent first waits race initializing trace_ms. Use a race-free cached threshold and range-check HOS_MUTEX_TRACE before converting it.

Concurrent first waits race initializing trace_ms. Use a race-free cached threshold and range-check HOS_MUTEX_TRACE before converting it.
@ -671,6 +670,12 @@ static void mutex_lock(Obj *m, uint64_t guest, Ctx *c) {
(unsigned long long)(c ? c->x[30] : 0), hos_thread_name(holder));
Member

The original owner can unlock and be destroyed before this log dereferences it. Use a lifetime-safe name snapshot or stable identifier here and in the other owner diagnostics.

The original owner can unlock and be destroyed before this log dereferences it. Use a lifetime-safe name snapshot or stable identifier here and in the other owner diagnostics.
@ -631,3 +630,1 @@
HosThread *self = hos_self();
if (__atomic_load_n(&m->owner, __ATOMIC_RELAXED) == self) { m->count++; return; }
if (mutex_take(m, self)) { m->count = 1; return; }
/* mutex_lock's way when another thread holds the mutex: its spin, then its sleep. Out of line and cold, so the uncontended
Member

A reservation can send this path a mutex whose owner is already NULL. Describe the trigger as a failed immediate acquire.

A reservation can send this path a mutex whose owner is already NULL. Describe the trigger as a failed immediate acquire.
@ -672,2 +671,4 @@
}
}
static void mutex_lock(Obj *m, uint64_t guest, Ctx *c) {
HosThread *self = hos_self();
Member

Production still calls the separately compiled hos_self. Show the fast-path assembly from the actual Windows and Linux builds and measure the change in play.

Production still calls the separately compiled hos_self. Show the fast-path assembly from the actual Windows and Linux builds and measure the change in play.
@ -674,0 +673,4 @@
static void mutex_lock(Obj *m, uint64_t guest, Ctx *c) {
HosThread *self = hos_self();
if (__atomic_load_n(&m->owner, __ATOMIC_RELAXED) == self) { m->count++; return; }
if (mutex_take(m, self)) { m->count = 1; return; }
Member

The first failed acquire now enters cold code even when the spin succeeds immediately. Measure that handoff separately and check frame-time tails on the affected CPUs.

The first failed acquire now enters cold code even when the spin succeeds immediately. Measure that handoff separately and check frame-time tails on the affected CPUs.
@ -90,2 +90,3 @@
"static int mutex_take(Obj *m, HosThread *self) {", "static int mutex_take_owed(Obj *m, HosThread *self) {",
"static void mutex_give(Obj *m) {", "static void mutex_lock(Obj *m, uint64_t guest, Ctx *c) {",
"static void mutex_give(Obj *m) {",
"static __attribute__((noinline, cold)) void mutex_lock_wait(Obj *m, HosThread *self, uint64_t guest, Ctx *c) {",
Member

The runner forces gcc and the direct default ignores CC. Use the configured compiler so the fixture runs on a standard llvm-mingw setup.

The runner forces gcc and the direct default ignores CC. Use the configured compiler so the fixture runs on a standard llvm-mingw setup.
Member

The fixture marker can start at void mutex_lock_wait so an attribute spelling change does not break a behavior check.

The fixture marker can start at `void mutex_lock_wait` so an attribute spelling change does not break a behavior check.
futondestroyer left a comment

Recommendation: Wait for author-side gameplay and a measured benefit. The cold split preserves the ownership and waiter protocol. The numbers so far leave its benefit unresolved and omit the path that gets a new call.

mutex_lock still checks for recursion and attempts the same immediate acquisition. The new call runs after any immediate take fails, including an owner-free mutex reserved for a condition-variable waiter. The reserved reacquire still uses mutex_take_owed. The spin, registration, retry, core handoff and wake code are carried forward. The existing exact-head review calls out diagnostic lifetime and initialization defects. Those lines are inherited from the merge base and deserve their own fixes. This split adds no protection around them.

Does it help? Unproven. The author's Windows/i7-9700 fixture reports 18.55 ns before and 18.03 ns after for an uncontended lock/unlock over 21 rounds, with overlapping ranges. It substitutes hos_self and the spin helper, so it cannot establish the generated production fast path or the cost of the newly out-of-line failed-first-take path. Include brief contention that resolves without sleeping in the next comparison, with production assembly and the author's warmed, interleaved Market runs against exact main.

Reviewed. 95c0fd41526b05ade41ab265166a3c5c4ba55f06 against 2e1f6c32d30ec3582093743a968c8447bb435962. Reviewed both changed files, the mutex take/give and CV reacquire callers, and the threaded fixture. No new switch. The author reports a passing source-cut fixture and 11 passes with one skip on the earlier rebase. No confirmed exact-head gameplay or farm results. No PR code run in this review.

Overlap. The cold helper is already carried in the context mutex stack. If the cold split earns its own landing, rebase that stack to avoid landing the same step twice.

**Recommendation: Wait for author-side gameplay and a measured benefit.** The cold split preserves the ownership and waiter protocol. The numbers so far leave its benefit unresolved and omit the path that gets a new call. `mutex_lock` still checks for recursion and attempts the same immediate acquisition. The new call runs after any immediate take fails, including an owner-free mutex reserved for a condition-variable waiter. The reserved reacquire still uses `mutex_take_owed`. The spin, registration, retry, core handoff and wake code are carried forward. The [existing exact-head review](https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/1953#issuecomment-61957) calls out diagnostic lifetime and initialization defects. Those lines are inherited from the merge base and deserve their own fixes. This split adds no protection around them. **Does it help?** Unproven. The author's Windows/i7-9700 fixture reports 18.55 ns before and 18.03 ns after for an uncontended lock/unlock over 21 rounds, with overlapping ranges. It substitutes `hos_self` and the spin helper, so it cannot establish the generated production fast path or the cost of the newly out-of-line failed-first-take path. Include brief contention that resolves without sleeping in the next comparison, with production assembly and the author's warmed, interleaved Market runs against exact main. **Reviewed.** `95c0fd41526b05ade41ab265166a3c5c4ba55f06` against `2e1f6c32d30ec3582093743a968c8447bb435962`. Reviewed both changed files, the mutex take/give and CV reacquire callers, and the threaded fixture. No new switch. The author reports a passing source-cut fixture and 11 passes with one skip on the earlier rebase. No confirmed exact-head gameplay or farm results. No PR code run in this review. **Overlap.** The cold helper is already carried in the [context mutex stack](https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/1957). If the cold split earns its own landing, rebase that stack to avoid landing the same step twice.
siantm force-pushed perf/mutex-lock-wait-cold from 95c0fd4152
Some checks failed
PR description / Description follows the template (pull_request_target) Successful in 11s
Website / Website on Cloudflare (pull_request_target) Has been skipped
CI / Conventional Commit title (pull_request) Has been cancelled
CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Has been cancelled
CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Has been cancelled
CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Has been cancelled
CI / Workflow lint (actionlint) (pull_request) Has been cancelled
CI / Website build and checks (pull_request) Has been cancelled
Guide screenshots / Guide images and local UI source review (pull_request) Has been cancelled
Third-party parts / As their notes say (pull_request) Has been cancelled
Third-party parts / The same as upstream (pull_request) Has been cancelled
to 8443bba1f9
Some checks failed
Website / Website on Cloudflare (pull_request_target) Has been skipped
beanstalk/approval waiting for a BigBean's 👍 on the request
PR description / Description follows the template (pull_request_target) Successful in 17s
CI / Conventional Commit title (pull_request) Has been cancelled
CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Has been cancelled
CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Has been cancelled
CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Has been cancelled
CI / Workflow lint (actionlint) (pull_request) Has been cancelled
CI / Website build and checks (pull_request) Has been cancelled
Guide screenshots / Guide images and local UI source review (pull_request) Has been cancelled
Third-party parts / As their notes say (pull_request) Has been cancelled
Third-party parts / The same as upstream (pull_request) Has been cancelled
Website / Website on Cloudflare (pull_request) Has been cancelled
2026-11-21 03:25:58 +00:00
Compare
Author
Contributor

Pushed d3e02c70 (test), 18e90465 (fix) and 8443bba1 (profiling descriptions) for BeanBot's review, with the branch rebased onto main @ 6e1521a7 (from 2e1f6c32; git range-diff shows 95c0fd41 unchanged as 93f9b35a). The description is updated.

  • Blocking 1, trace_ms: fixed in 18e90465. HOS_MUTEX_TRACE is cached through an atomic (two first waits store the same value), kept in a local for the whole wait, and parsed with strtoll over the whole string: only 1 to 2147483647 ms turns it on, anything else is off (-1 used to log every wait). The switch's row in gpu/vk/switches.def says so.
  • Blocking 2, owner pointer: fixed for the lifetime in 18e90465. The holder's name is copied with the wait counted in mwait and m->lk held; the holder's unlock (mutex_give) then waits for m->lk, so it cannot end and be freed during the copy. The Tracy text and the long-wait log use that copy, and the 10 s log reads the name under m->lk too. The rename race (hos_thread_set_name writing a name while another thread reads it) is in host/hos_thread.c and affects every name reader. It is pre-existing on main, unchanged by this PR, and left for a separate change.
  • Blocking 3, m->count in the 10 s log: fixed in 18e90465, the count is dropped (and mwait is read as an atomic).
  • Blocking 4, volatile hos_chain_on: not changed here. The flag, its writer and HOS_CHAIN live in host/hos_chain.c, host/hos.h and six call sites in nn_os_sync.c, all as they are on main; this PR moves one read of it. Making the recorder atomic with release/acquire publication of recs is a separate change, listed for the owner and named in the description.
  • Blocking 5, spin_begin's flag: not changed here. spin_begin is unchanged by this PR (it moves the call to spin_until_free). The race is pre-existing on main and left for a separate change, named in the description.
  • Blocking 6, gcc forced: fixed in d3e02c70. RUN_ALL_ARGS is gone; the compiler is --cc, else CC, else LONGSHOT_MINGW's clang on Windows, else clang, gcc or cc, as #1954's test does. The queue run below gave it no --cc, so it used the configured llvm-mingw clang.
  • Inline, fixture marker: taken in d3e02c70: the cut starts at void mutex_lock_wait(, and the test cuts it as a plain static function.
  • Inline, the trigger: taken in 18e90465 and in the description: a failed immediate acquire, including a reserved mutex with no owner.
  • Suggestion, cpu_functions.json: taken in 8443bba1 (mutex_lock updated, mutex_lock_wait added).
  • Waiting, production assembly and play and the failed-first-take handoff: pending, for the owner. Nothing is measured for them yet; both are in Still to test, and the description no longer claims the leaf shape.
  • The author's game run, host builds on both OSes, the full checks and CI: pending. Run at 8443bba1 through this PC's job queue: tests/test_cv_handoff_threads.py passes, and it fails at d3e02c70 (the log read a holder that had ended). tests/run_all.py --strict --changed origin/main: 32 passed, 7 failed, 3 skipped; the 7 fail the same way on main @ 6e1521a7. host/nn_os_sync.c compiles with the host build's flags with and without Tracy.
Pushed d3e02c70 (test), 18e90465 (fix) and 8443bba1 (profiling descriptions) for [BeanBot's review](https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/1953#issuecomment-61957), with the branch rebased onto `main` @ 6e1521a7 (from 2e1f6c32; `git range-diff` shows 95c0fd41 unchanged as 93f9b35a). The description is updated. - [Blocking 1, trace_ms](https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/1953#issuecomment-61949): fixed in 18e90465. `HOS_MUTEX_TRACE` is cached through an atomic (two first waits store the same value), kept in a local for the whole wait, and parsed with `strtoll` over the whole string: only 1 to 2147483647 ms turns it on, anything else is off (`-1` used to log every wait). The switch's row in `gpu/vk/switches.def` says so. - [Blocking 2, owner pointer](https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/1953#issuecomment-61950): fixed for the lifetime in 18e90465. The holder's name is copied with the wait counted in `mwait` and `m->lk` held; the holder's unlock (`mutex_give`) then waits for `m->lk`, so it cannot end and be freed during the copy. The Tracy text and the long-wait log use that copy, and the 10 s log reads the name under `m->lk` too. The rename race (`hos_thread_set_name` writing a name while another thread reads it) is in `host/hos_thread.c` and affects every name reader. It is pre-existing on `main`, unchanged by this PR, and left for a separate change. - Blocking 3, `m->count` in the 10 s log: fixed in 18e90465, the count is dropped (and `mwait` is read as an atomic). - Blocking 4, `volatile hos_chain_on`: not changed here. The flag, its writer and `HOS_CHAIN` live in `host/hos_chain.c`, `host/hos.h` and six call sites in `nn_os_sync.c`, all as they are on `main`; this PR moves one read of it. Making the recorder atomic with release/acquire publication of `recs` is a separate change, listed for the owner and named in the description. - [Blocking 5, spin_begin's flag](https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/1953#issuecomment-61951): not changed here. `spin_begin` is unchanged by this PR (it moves the call to `spin_until_free`). The race is pre-existing on `main` and left for a separate change, named in the description. - [Blocking 6, gcc forced](https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/1953#issuecomment-61952): fixed in d3e02c70. `RUN_ALL_ARGS` is gone; the compiler is `--cc`, else `CC`, else `LONGSHOT_MINGW`'s clang on Windows, else clang, gcc or cc, as #1954's test does. The queue run below gave it no `--cc`, so it used the configured llvm-mingw clang. - [Inline, fixture marker](https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/1953#issuecomment-61955): taken in d3e02c70: the cut starts at `void mutex_lock_wait(`, and the test cuts it as a plain static function. - [Inline, the trigger](https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/1953#issuecomment-61956): taken in 18e90465 and in the description: a failed immediate acquire, including a reserved mutex with no owner. - Suggestion, `cpu_functions.json`: taken in 8443bba1 (`mutex_lock` updated, `mutex_lock_wait` added). - [Waiting, production assembly and play](https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/1953#issuecomment-61953) and [the failed-first-take handoff](https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/1953#issuecomment-61954): pending, for the owner. Nothing is measured for them yet; both are in Still to test, and the description no longer claims the leaf shape. - The author's game run, host builds on both OSes, the full checks and CI: pending. Run at 8443bba1 through this PC's job queue: `tests/test_cv_handoff_threads.py` passes, and it fails at d3e02c70 (the log read a holder that had ended). `tests/run_all.py --strict --changed origin/main`: 32 passed, 7 failed, 3 skipped; the 7 fail the same way on `main` @ 6e1521a7. `host/nn_os_sync.c` compiles with the host build's flags with and without Tracy.
Author
Contributor

@futondestroyer, on your review:

  • The benefit is still unshown. The description no longer claims the leaf shape. Still to test now lists the production mutex_lock against main's, brief contention that ends without a sleep (the new out-of-line path), and warmed, interleaved Market runs against exact main with fps, 1 % lows, median and p99 frame times. None of these has run yet; they are for the owner.
  • The inherited diagnostic defects: the three that sit inside the new function (the HOS_MUTEX_TRACE cache and parse, the holder's name read after it may have ended, m->count in the 10 s log) are fixed in their own commits, d3e02c70 (failing test) and 18e90465 (fix), on top of the split. If you would rather land them as a PR of their own, they can move as they are. hos_chain_on, spin_begin's cached flag and the thread-name rename race are left on main for separate changes, and the description says so.
  • #1957 carries this commit. It will be rebased onto this head once this PR is settled, so the step does not land twice.
@futondestroyer, on [your review](https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/1953#issuecomment-62896): - The benefit is still unshown. The description no longer claims the leaf shape. Still to test now lists the production `mutex_lock` against `main`'s, brief contention that ends without a sleep (the new out-of-line path), and warmed, interleaved Market runs against exact `main` with fps, 1 % lows, median and p99 frame times. None of these has run yet; they are for the owner. - The inherited diagnostic defects: the three that sit inside the new function (the `HOS_MUTEX_TRACE` cache and parse, the holder's name read after it may have ended, `m->count` in the 10 s log) are fixed in their own commits, d3e02c70 (failing test) and 18e90465 (fix), on top of the split. If you would rather land them as a PR of their own, they can move as they are. `hos_chain_on`, `spin_begin`'s cached flag and the thread-name rename race are left on `main` for separate changes, and the description says so. - #1957 carries this commit. It will be rebased onto this head once this PR is settled, so the step does not land twice.
Author
Contributor

@Beanstalk This moves the guest mutex lock's wait for a held mutex out of line and cold (host/nn_os_sync.c, mutex_lock): every guest mutex lock whose first attempt fails takes the new path; the uncontended path gets shorter. No switch: a revert gives back main's code. It has not run in the game on these PCs yet. Please play Castle Town Market for two minutes, the PR against main on the same PC, on a Windows PC with an AMD processor, a Linux PC with an AMD processor, a PC with 6 or fewer logical processors and the Steam Deck. No hang or stall and no audio or frame pacing trouble; frame time p99 equal to main's or better, within the run-to-run spread.

@Beanstalk This moves the guest mutex lock's wait for a held mutex out of line and cold (host/nn_os_sync.c, mutex_lock): every guest mutex lock whose first attempt fails takes the new path; the uncontended path gets shorter. No switch: a revert gives back main's code. It has not run in the game on these PCs yet. Please play Castle Town Market for two minutes, the PR against main on the same PC, on a Windows PC with an AMD processor, a Linux PC with an AMD processor, a PC with 6 or fewer logical processors and the Steam Deck. No hang or stall and no audio or frame pacing trouble; frame time p99 equal to main's or better, within the run-to-run spread.
Member

@siantm only members of BigBeans can start the farm.

One of them can approve this request by reacting 👍 to it or to this comment. This request was for 8443bba, and the PR has changed since (e4ea22d), so a 👍 no longer approves it. Comment @Beanstalk again to ask for the new commit.

@siantm only members of `BigBeans` can start the farm. ~~One of them can approve this request by reacting 👍 to it or to this comment.~~ This request was for `8443bba`, and the PR has changed since (`e4ea22d`), so a 👍 no longer approves it. Comment `@Beanstalk` again to ask for the new commit.
Author
Contributor

The three pre-existing defects this PR left on main (named in BeanBot's review) are now fixes of their own, each with a game-free test that fails on main (this PC's job queue, fb3-sep-1/2); none has run in the game yet:

  • Blocking 5, spin_begin's flag: #2012 (merged as 56da0e2e). tests/test_spin_always_once.py: 8 threads first spinning at once read HOS_SPIN_ALWAYS 8 times on main, once with the fix.
  • Blocking 4, volatile hos_chain_on: #2019 (fix/os-chain-flag-atomic @ a6fffab8). The flag is atomic (release once the buffer is there, acquire before using it, relaxed at the six waits); a note's record is published by its kind (release) and the recorder waits for it (acquire) instead of sleeping 50 ms. tests/test_chain_recorder.py: a note held across the stop left a record of zeros on main. It conflicts with this PR on the one moved line const int chain = hos_chain_on;; whichever lands second is rebased.
  • Blocking 2, the rename part: #2020 (fix/os-thread-names-kept @ 67baca96). A name is never written over: a rename publishes a kept name with one atomic store, so every hos_thread_name reader gets a whole name. tests/test_thread_rename.py: a reader during a rename read half of each name on main. It merges with this PR without a conflict.
The three pre-existing defects this PR left on `main` (named in [BeanBot's review](https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/1953#issuecomment-61957)) are now fixes of their own, each with a game-free test that fails on `main` (this PC's job queue, fb3-sep-1/2); none has run in the game yet: - [Blocking 5, spin_begin's flag](https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/1953#issuecomment-61951): #2012 (merged as 56da0e2e). `tests/test_spin_always_once.py`: 8 threads first spinning at once read `HOS_SPIN_ALWAYS` 8 times on `main`, once with the fix. - Blocking 4, `volatile hos_chain_on`: #2019 (`fix/os-chain-flag-atomic` @ a6fffab8). The flag is atomic (release once the buffer is there, acquire before using it, relaxed at the six waits); a note's record is published by its kind (release) and the recorder waits for it (acquire) instead of sleeping 50 ms. `tests/test_chain_recorder.py`: a note held across the stop left a record of zeros on `main`. It conflicts with this PR on the one moved line `const int chain = hos_chain_on;`; whichever lands second is rebased. - [Blocking 2, the rename part](https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/1953#issuecomment-61950): #2020 (`fix/os-thread-names-kept` @ 67baca96). A name is never written over: a rename publishes a kept name with one atomic store, so every `hos_thread_name` reader gets a whole name. `tests/test_thread_rename.py`: a reader during a rename read half of each name on `main`. It merges with this PR without a conflict.
Author
Contributor

Description: Summary says which PRs now carry the three defects left on main (#2012 merged, #2019, #2020) and the landing order with #2019 (one shared line).

Description: Summary says which PRs now carry the three defects left on `main` (#2012 merged, #2019, #2020) and the landing order with #2019 (one shared line).
Author
Contributor

The three renderer checks (test_present_queue_default, test_target_reuse_memory, test_vk_occlusion_queries) did not compile in the earlier run (no Vulkan headers on that job's path); rerun with them in job fb3-vk-2, at the head 8443bba1 and on main @ 6e1521a7: all three fail at both with the same first error, so they fail on main too; Tested's sentence and counts corrected.

The three renderer checks (`test_present_queue_default`, `test_target_reuse_memory`, `test_vk_occlusion_queries`) did not compile in the earlier run (no Vulkan headers on that job's path); rerun with them in job fb3-vk-2, at the head 8443bba1 and on `main` @ 6e1521a7: all three fail at both with the same first error, so they fail on `main` too; Tested's sentence and counts corrected.
futondestroyer left a comment

Recommendation: Wait for a rebased comparison and the owner's queued measurements. The new commits address the mutex-wait diagnostic defects that could be fixed within this patch. Last pass's performance gap remains open.

The trace threshold is now read atomically and is bounded to a positive 32-bit value. The wait copies the holder's name while its waiter count keeps a normal unlock at the mutex lock, and it drops the unsynchronized count from the ten-second message. The holder copy is used after the wait instead of dereferencing the former owner. The focused fixture checks the invalid values, first-read cache, and a holder that has already unlocked and ended before the waiter reports. The inherited chain flag and thread-name publication remain tracked separately. Author follow-up

Does it help? Unproven. The earlier uncontended fixture measured 18.55 ns on main and 18.03 ns with the extraction, with overlapping ranges. The failed-first-take path still pays the new call before trying the spin. For that case the current branch and its merge base both predate the now-merged HOS_SPIN_ALWAYS initialization and fast-read change, which runs inside the extracted spin path. A Farm approval here would compare 8443bba1 to 6e1521a7. Rebase the proposed landing on current main, then compare the updated production code and a short-contention workload that tries the spin without parking. The owner's interleaved Market run is still listed as queued. Merged spin initialization fix

Testing. The author reports the focused CV fixture passing at this head and failing before the diagnostic fix, Windows syntax checks with and without Tracy, and 32 broader passes, seven base-matched failures and three Linux-only skips. CI and Linux coverage remain open. The author’s cross-host Market request is pending. Its scene can check gameplay stability and frame times on the refreshed pair, alongside the owner’s brief-contention result. Existing Farm request

Dependencies. Keep the chain-flag fix coordinated at the moved read. The context-mutex experiment carries equivalents of these four commits and its own lift dependency. Once the proposed mutex split is settled, rebase that stack and measure its incremental change against the settled prerequisite, as well as the product against main. Chain flag · Context mutex stack

Reviewed. 8443bba1f9c56480bb3a26f8470bff52a87fe317 against actual merge base 6e1521a70a32603ed60a8263e4072625885669b7. Read the four-file diff, mutex release and CV handoff, the guest thread destroy path, and the updated diagnostic fixture. Both revision-matched Graphify maps verified and queried. Source only.

**Recommendation: Wait for a rebased comparison and the owner's queued measurements.** The new commits address the mutex-wait diagnostic defects that could be fixed within this patch. Last pass's performance gap remains open. The trace threshold is now read atomically and is bounded to a positive 32-bit value. The wait copies the holder's name while its waiter count keeps a normal unlock at the mutex lock, and it drops the unsynchronized count from the ten-second message. The holder copy is used after the wait instead of dereferencing the former owner. The focused fixture checks the invalid values, first-read cache, and a holder that has already unlocked and ended before the waiter reports. The inherited chain flag and thread-name publication remain tracked separately. [Author follow-up](https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/1953#issuecomment-64025) **Does it help? Unproven.** The earlier uncontended fixture measured 18.55 ns on main and 18.03 ns with the extraction, with overlapping ranges. The failed-first-take path still pays the new call before trying the spin. For that case the current branch and its merge base both predate the now-merged `HOS_SPIN_ALWAYS` initialization and fast-read change, which runs inside the extracted spin path. A Farm approval here would compare `8443bba1` to `6e1521a7`. Rebase the proposed landing on current main, then compare the updated production code and a short-contention workload that tries the spin without parking. The owner's interleaved Market run is still listed as queued. [Merged spin initialization fix](https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/2012) **Testing.** The author reports the focused CV fixture passing at this head and failing before the diagnostic fix, Windows syntax checks with and without Tracy, and 32 broader passes, seven base-matched failures and three Linux-only skips. CI and Linux coverage remain open. The author’s cross-host Market request is pending. Its scene can check gameplay stability and frame times on the refreshed pair, alongside the owner’s brief-contention result. [Existing Farm request](https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/1953#issuecomment-63064) **Dependencies.** Keep the chain-flag fix coordinated at the moved read. The context-mutex experiment carries equivalents of these four commits and its own lift dependency. Once the proposed mutex split is settled, rebase that stack and measure its incremental change against the settled prerequisite, as well as the product against main. [Chain flag](https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/2019) · [Context mutex stack](https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/1957) **Reviewed.** `8443bba1f9c56480bb3a26f8470bff52a87fe317` against actual merge base `6e1521a70a32603ed60a8263e4072625885669b7`. Read the four-file diff, mutex release and CV handoff, the guest thread destroy path, and the updated diagnostic fixture. Both revision-matched Graphify maps verified and queried. Source only.
dyst left a comment

Verdict: Request changes. The diagnostic fixes are right and well tested, but this is a perf change with no measured gain: the one number is within the spread. The fast path it is meant to shorten still calls hos_self() in another translation unit, so it cannot become the small leaf the split aims for. The cost on the contended path is still unmeasured, and the branch has not been rebased onto the main that changed the spin inside the new cold function, as futondestroyer asked.

For a player: a guest mutex lock that finds the mutex free should cost a little less, and one that has to wait goes through a separate cold function. Nothing visible, unless the gain or the cost shows in frame times. The three diagnostic fixes (the HOS_MUTEX_TRACE cache and parse, the holder's name read after it may have ended, m->count in the 10 s log) make the code better on their own. The split, as it stands, has no shown gain to set against its cost.

Reviewed: 8443bba1 against the merge base 6e1521a7 (forge/main is now 9dd41326, 15 commits on, #2012 among them): host/nn_os_sync.c (the diff in full, plus mutex_take, mutex_give, mutex_take_owed, mutex_unlock, mutex_release_all, spin_until_free), tests/test_cv_handoff_threads.py (the diff in full: compiler(), the cut, the fixture's HosThread, the hos_cv_sleep hook, trace_wait/trace_cases, the runner loop), gpu/vk/switches.def, scripts/profiling/cpu_functions.json; hos_self in host/hos_thread.c; the sibling PRs' diffs of nn_os_sync.c. A re-review of BeanBot's and futondestroyer's points at 95c0fd41 (no longer reachable here), judged from the author's replies and the current code. Nothing run (a code review).
Fixed in this review: nothing.
Checked and fine:

  • The split keeps the protocol: the recursion check, the take (reserved check, then the CAS), the spin, the waiter count, the retry loop and the wake in mutex_give are the same code. mutex_lock_wait is a tail call from mutex_lock.
  • BeanBot 1 (trace_ms): fixed. The cache is atomic and the value stays in a local for the whole wait. strtoll with a whole-string and 1..INT32_MAX check turns off -1, 1x and values too large for an int, and the switch row says so.
  • BeanBot 2 (owner lifetime): fixed for the lifetime. The name is copied after mwait's seq_cst increment, with m->lk held. A holder seen as owner then reaches mutex_give, which reads mwait after clearing the owner (both seq_cst), sees the waiter and blocks on m->lk, so it cannot end during the copy. The 10 s log reads under m->lk with mwait still counted. Non-Tracy builds copy the name only when the trace is on (name_holder), so the usual path pays nothing. The rename race moved to #2020.
  • BeanBot 3 (m->count): dropped, and mwait is read atomically.
  • BeanBot 4 (hos_chain_on) and 5 (spin_begin): code this PR only moves, now in #2019 and the merged #2012. That is fine for this PR's scope.
  • BeanBot 6 (gcc forced): fixed. RUN_ALL_ARGS is gone, and compiler() honours --cc, then CC, then LONGSHOT_MINGW's clang (which run_all.py sets on Windows), then clang, gcc or cc. The fixture defines _POSIX_C_SOURCE 200809L, so setenv is declared under -std=c11 -Werror on glibc, and _putenv on Windows.
  • The fixture marker starts at void mutex_lock_wait(, and the runner adds static to the cut. The trigger is described as a failed immediate acquire.
  • The new trace test would catch a revert: the fake hos_thread_name CHECKs a dead record, and the hos_cv_sleep hook holds the waiter until the holder has unlocked and ended. Reading the name after the wait, as on main, fails, as the author's run at d3e02c70 shows. The value table checks the cache too, because each value runs in a fresh process with the variable flipped between the two waits.
  • __attribute__((noinline, cold)) is accepted by gcc and clang (including llvm-mingw and clang-cl) on Windows and Linux. It is already used in host/nn_libc.c:27, and the file depends on GNU __atomic_* builtins anyway.
  • cpu_functions.json describes both functions. No game data in the diff. No switch added, so the default-at-first-push rule does not apply.
  • The branch still merges with current main without a textual conflict (git merge-tree).

Blocking (to change before merge)

  1. host/nn_os_sync.c:697: the gain the split is for is neither shown nor reachable as described. mutex_lock still begins with hos_self(), an out-of-line function in host/hos_thread.c:96. In the ordinary (non-LTO) host build, m, guest and c must live across that call, so the uncontended lock still saves registers and sets up a frame. The split can at most trim a few saves. The one measurement (18.55 vs 18.03 ns, ranges overlapping) is within the spread, and docs/dev/standards.md says "No measured gain, no merge". To fix, show mutex_lock's generated code before and after from the production host object (objdump of nn_os_sync.o, Windows and Linux) and a gain against current main. #1957 is where self comes from the context and the leaf becomes possible, so the split may belong there (see Suggestions).
  2. host/nn_os_sync.c:633, :700: the cost is unmeasured on the path the change makes slower. Every failed first take (a briefly held job or queue mutex) now pays a call into code the compiler treats as unlikely and may place away from its caller. gcc documents that cold also optimises the function for size, along with the spin it inlines. The fixture's spin is a no-op, so nothing so far enters that path. BeanBot's and futondestroyer's request for a short-contention measurement (a failed first CAS that the spin then wins, no sleep) is still open. standards.md wants a cost measured and stated in Risks.
  3. Not rebased onto current main, as futondestroyer asked. #2012 changed spin_begin, which now runs inside the new cold function, so any comparison made from this branch (the farm request included) measures a spin path main no longer has. Rebase onto current main and measure that pair.

Suggestions (the author may take or leave)

  • You offered to move the diagnostic fixes into their own PR, and I'd take that. d3e02c70 and 18e90465 are a fix(os) that is right on its own and needs no perf evidence, so it could land now. The split could then move into #1957, where mutex_lock takes self from the context and becomes a leaf, so its gain can be measured with the change that makes it possible. Or keep it here if the rebased measurement shows a gain alone.
  • host/nn_os_sync.c:664: the owner load that guards the copy pairs with mutex_give's seq_cst exchange and mwait load (a store-buffer pattern). It is right on x86-64 because the lock xadd before it is a full barrier, but C11 only guarantees it with this load at __ATOMIC_SEQ_CST. That is the same plain mov on x86-64, so it costs nothing.
  • gpu/vk/switches.def: "with the thread that held the mutex as the wait began". The name is taken after the spin and m->lk, so it is the holder when the waiter counts itself in. "when the wait started to sleep" would be exact. -1 used to log every wait and nothing does now; "1 logs nearly every wait" would tell a user what to use instead.

Testing: the author's game-free runs on Windows 11 / i7-9700 / llvm-mingw clang: the CV fixture passes at 8443bba1 and fails at the test commit, the changed-checks run shows 7 failures that fail on main too, and the file compiles with and without Tracy. Missing: any game run, the production codegen, the short-contention measurement, a Linux build, and CI (pending, blocked on approval). The farm request (63064) is for this head but compares against 6e1521a7, so it should wait for the rebase.
Overlaps and merge order: #1957 carries these four commits under other hashes and conflicts with this branch. Land one or the other, not both. #2019 conflicts on the moved hos_chain_on read (one line), and whichever lands second rebases. #1974, 1971, 1970, 1969, 1967 and 1947 merge with this branch cleanly. gpu/vk/switches.def will conflict textually with most open PRs, as usual.

**Verdict: Request changes.** The diagnostic fixes are right and well tested, but this is a `perf` change with no measured gain: the one number is within the spread. The fast path it is meant to shorten still calls `hos_self()` in another translation unit, so it cannot become the small leaf the split aims for. The cost on the contended path is still unmeasured, and the branch has not been rebased onto the `main` that changed the spin inside the new cold function, as futondestroyer asked. For a player: a guest mutex lock that finds the mutex free should cost a little less, and one that has to wait goes through a separate cold function. Nothing visible, unless the gain or the cost shows in frame times. The three diagnostic fixes (the `HOS_MUTEX_TRACE` cache and parse, the holder's name read after it may have ended, `m->count` in the 10 s log) make the code better on their own. The split, as it stands, has no shown gain to set against its cost. **Reviewed:** 8443bba1 against the merge base 6e1521a7 (forge/main is now 9dd41326, 15 commits on, #2012 among them): `host/nn_os_sync.c` (the diff in full, plus `mutex_take`, `mutex_give`, `mutex_take_owed`, `mutex_unlock`, `mutex_release_all`, `spin_until_free`), `tests/test_cv_handoff_threads.py` (the diff in full: `compiler()`, the cut, the fixture's `HosThread`, the `hos_cv_sleep` hook, `trace_wait`/`trace_cases`, the runner loop), `gpu/vk/switches.def`, `scripts/profiling/cpu_functions.json`; `hos_self` in `host/hos_thread.c`; the sibling PRs' diffs of `nn_os_sync.c`. A re-review of BeanBot's and futondestroyer's points at 95c0fd41 (no longer reachable here), judged from the author's replies and the current code. Nothing run (a code review). **Fixed in this review:** nothing. **Checked and fine:** - The split keeps the protocol: the recursion check, the take (`reserved` check, then the CAS), the spin, the waiter count, the retry loop and the wake in `mutex_give` are the same code. `mutex_lock_wait` is a tail call from `mutex_lock`. - BeanBot 1 (`trace_ms`): fixed. The cache is atomic and the value stays in a local for the whole wait. `strtoll` with a whole-string and 1..INT32_MAX check turns off `-1`, `1x` and values too large for an int, and the switch row says so. - BeanBot 2 (owner lifetime): fixed for the lifetime. The name is copied after `mwait`'s seq_cst increment, with `m->lk` held. A holder seen as owner then reaches `mutex_give`, which reads `mwait` after clearing the owner (both seq_cst), sees the waiter and blocks on `m->lk`, so it cannot end during the copy. The 10 s log reads under `m->lk` with `mwait` still counted. Non-Tracy builds copy the name only when the trace is on (`name_holder`), so the usual path pays nothing. The rename race moved to https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/2020. - BeanBot 3 (`m->count`): dropped, and `mwait` is read atomically. - BeanBot 4 (`hos_chain_on`) and 5 (`spin_begin`): code this PR only moves, now in https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/2019 and the merged https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/2012. That is fine for this PR's scope. - BeanBot 6 (gcc forced): fixed. `RUN_ALL_ARGS` is gone, and `compiler()` honours `--cc`, then `CC`, then `LONGSHOT_MINGW`'s clang (which `run_all.py` sets on Windows), then clang, gcc or cc. The fixture defines `_POSIX_C_SOURCE 200809L`, so `setenv` is declared under `-std=c11 -Werror` on glibc, and `_putenv` on Windows. - The fixture marker starts at `void mutex_lock_wait(`, and the runner adds `static` to the cut. The trigger is described as a failed immediate acquire. - The new trace test would catch a revert: the fake `hos_thread_name` CHECKs a dead record, and the `hos_cv_sleep` hook holds the waiter until the holder has unlocked and ended. Reading the name after the wait, as on `main`, fails, as the author's run at d3e02c70 shows. The value table checks the cache too, because each value runs in a fresh process with the variable flipped between the two waits. - `__attribute__((noinline, cold))` is accepted by gcc and clang (including llvm-mingw and clang-cl) on Windows and Linux. It is already used in `host/nn_libc.c:27`, and the file depends on GNU `__atomic_*` builtins anyway. - `cpu_functions.json` describes both functions. No game data in the diff. No switch added, so the default-at-first-push rule does not apply. - The branch still merges with current `main` without a textual conflict (`git merge-tree`). **Blocking** (to change before merge) 1. `host/nn_os_sync.c:697`: the gain the split is for is neither shown nor reachable as described. `mutex_lock` still begins with `hos_self()`, an out-of-line function in `host/hos_thread.c:96`. In the ordinary (non-LTO) host build, `m`, `guest` and `c` must live across that call, so the uncontended lock still saves registers and sets up a frame. The split can at most trim a few saves. The one measurement (18.55 vs 18.03 ns, ranges overlapping) is within the spread, and docs/dev/standards.md says "No measured gain, no merge". To fix, show `mutex_lock`'s generated code before and after from the production host object (objdump of `nn_os_sync.o`, Windows and Linux) and a gain against current `main`. https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/1957 is where `self` comes from the context and the leaf becomes possible, so the split may belong there (see Suggestions). 2. `host/nn_os_sync.c:633`, `:700`: the cost is unmeasured on the path the change makes slower. Every failed first take (a briefly held job or queue mutex) now pays a call into code the compiler treats as unlikely and may place away from its caller. gcc documents that `cold` also optimises the function for size, along with the spin it inlines. The fixture's spin is a no-op, so nothing so far enters that path. BeanBot's and futondestroyer's request for a short-contention measurement (a failed first CAS that the spin then wins, no sleep) is still open. standards.md wants a cost measured and stated in Risks. 3. Not rebased onto current `main`, as futondestroyer asked. https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/2012 changed `spin_begin`, which now runs inside the new cold function, so any comparison made from this branch (the farm request included) measures a spin path `main` no longer has. Rebase onto current `main` and measure that pair. **Suggestions** (the author may take or leave) - You offered to move the diagnostic fixes into their own PR, and I'd take that. d3e02c70 and 18e90465 are a `fix(os)` that is right on its own and needs no perf evidence, so it could land now. The split could then move into https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/1957, where `mutex_lock` takes `self` from the context and becomes a leaf, so its gain can be measured with the change that makes it possible. Or keep it here if the rebased measurement shows a gain alone. - `host/nn_os_sync.c:664`: the owner load that guards the copy pairs with `mutex_give`'s seq_cst exchange and `mwait` load (a store-buffer pattern). It is right on x86-64 because the `lock xadd` before it is a full barrier, but C11 only guarantees it with this load at `__ATOMIC_SEQ_CST`. That is the same plain `mov` on x86-64, so it costs nothing. - `gpu/vk/switches.def`: "with the thread that held the mutex as the wait began". The name is taken after the spin and `m->lk`, so it is the holder when the waiter counts itself in. "when the wait started to sleep" would be exact. `-1` used to log every wait and nothing does now; "1 logs nearly every wait" would tell a user what to use instead. **Testing:** the author's game-free runs on Windows 11 / i7-9700 / llvm-mingw clang: the CV fixture passes at 8443bba1 and fails at the test commit, the changed-checks run shows 7 failures that fail on `main` too, and the file compiles with and without Tracy. Missing: any game run, the production codegen, the short-contention measurement, a Linux build, and CI (pending, blocked on approval). The farm request (63064) is for this head but compares against 6e1521a7, so it should wait for the rebase. **Overlaps and merge order:** https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/1957 carries these four commits under other hashes and conflicts with this branch. Land one or the other, not both. https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/2019 conflicts on the moved `hos_chain_on` read (one line), and whichever lands second rebases. https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/1974, 1971, 1970, 1969, 1967 and 1947 merge with this branch cleanly. `gpu/vk/switches.def` will conflict textually with most open PRs, as usual.
@ -103,3 +103,3 @@
SW(HOS_MOUSE_SENS, player, float, "the menu's mouse sensitivity (or user/mouse_sens.txt, moved into the menu once)", "Sets the mouse sensitivity and stops the old mouse_sens.txt being moved into the menu.")
SW(HOS_MOVIE_PACE, kill, bool, "on: a 60 FPS cap is raised for a movie's pictures", "=0 leaves the pacing at the frame rate cap while a movie plays.")
SW(HOS_MUTEX_TRACE, diag, int, "0: off", "Logs mutex waits longer than <ms>, with the thread that held the mutex.")
SW(HOS_MUTEX_TRACE, diag, int, "0: off", "Logs mutex waits longer than <ms> (1 to 2147483647; any other value is off), with the thread that held the mutex as the wait began; read once, at the first wait that sleeps.")
Member

Nit: the name is taken when the waiter counts itself in (after the spin and m->lk), not as the wait began. -1 used to log every wait; say what to use instead (1).

Nit: the name is taken when the waiter counts itself in (after the spin and m->lk), not as the wait began. -1 used to log every wait; say what to use instead (1).
@ -634,0 +630,4 @@
/* mutex_lock's way when its immediate acquire failed (another thread holds the mutex, or it is reserved for a woken
condition variable waiter while its owner is already NULL): its spin, then its sleep. Out of line and cold, so the
uncontended lock below does not save registers and set up a stack frame for this part */
static __attribute__((noinline, cold)) void mutex_lock_wait(Obj *m, HosThread *self, uint64_t guest, Ctx *c) {
Member

Every failed first take now calls this cold function, which gcc also optimises for size along with the spin it inlines. The fixture's spin is a no-op, so the case where the spin wins without a sleep is unmeasured. Measure it, and rebase first so the spin is #2012's.

Every failed first take now calls this cold function, which gcc also optimises for size along with the spin it inlines. The fixture's spin is a no-op, so the case where the spin wins without a sleep is unmeasured. Measure it, and rebase first so the spin is #2012's.
@ -651,0 +661,4 @@
holds for the 10 s log below, which reads it with m->lk held again */
char held_by[48] = "nobody";
if (name_holder) {
HosThread *h = __atomic_load_n(&m->owner, __ATOMIC_RELAXED);
Member

Nit: this load pairs with mutex_give's seq_cst exchange and mwait load (store-buffer pattern). It is right on x86-64 after the lock xadd, but C11 guarantees it only at __ATOMIC_SEQ_CST, which is the same mov here.

Nit: this load pairs with mutex_give's seq_cst exchange and mwait load (store-buffer pattern). It is right on x86-64 after the lock xadd, but C11 guarantees it only at __ATOMIC_SEQ_CST, which is the same mov here.
@ -672,2 +694,4 @@
}
}
static void mutex_lock(Obj *m, uint64_t guest, Ctx *c) {
HosThread *self = hos_self();
Member

Still an out-of-line call to hos_self() (host/hos_thread.c:96) in the non-LTO host build: m, guest and c live across it, so this cannot be the frameless leaf the split is for. Show the production codegen before and after, and a gain against current main (no measured gain, no merge). #1957 is where self comes from the context.

Still an out-of-line call to hos_self() (host/hos_thread.c:96) in the non-LTO host build: m, guest and c live across it, so this cannot be the frameless leaf the split is for. Show the production codegen before and after, and a gain against current main (no measured gain, no merge). #1957 is where self comes from the context.
siantm force-pushed perf/mutex-lock-wait-cold from 8443bba1f9
Some checks failed
Website / Website on Cloudflare (pull_request_target) Has been skipped
beanstalk/approval waiting for a BigBean's 👍 on the request
PR description / Description follows the template (pull_request_target) Successful in 17s
CI / Conventional Commit title (pull_request) Has been cancelled
CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Has been cancelled
CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Has been cancelled
CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Has been cancelled
CI / Workflow lint (actionlint) (pull_request) Has been cancelled
CI / Website build and checks (pull_request) Has been cancelled
Guide screenshots / Guide images and local UI source review (pull_request) Has been cancelled
Third-party parts / As their notes say (pull_request) Has been cancelled
Third-party parts / The same as upstream (pull_request) Has been cancelled
Website / Website on Cloudflare (pull_request) Has been cancelled
to e4ea22d75b
Some checks failed
Website / Website on Cloudflare (pull_request_target) Has been skipped
PR description / Description follows the template (pull_request_target) Successful in 6s
Website / Website on Cloudflare (pull_request) Has been cancelled
CI / Conventional Commit title (pull_request) Has been cancelled
CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Has been cancelled
CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Has been cancelled
CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Has been cancelled
CI / Workflow lint (actionlint) (pull_request) Has been cancelled
CI / Website build and checks (pull_request) Has been cancelled
Guide screenshots / Guide images and local UI source review (pull_request) Has been cancelled
Third-party parts / As their notes say (pull_request) Has been cancelled
Third-party parts / The same as upstream (pull_request) Has been cancelled
beanstalk/approval changed since the request: ask again
2026-11-21 11:17:55 +00:00
Compare
Author
Contributor

Pushed e4ea22d7 on main @ 9dd41326 (from 8443bba1 on 6e1521a7), as dyst's review 65813 and futondestroyer's 65533 asked: this PR is now the split alone (6288abca, e4ea22d7); the diagnostic fixes are #2063. Description rewritten: no gain measured yet; the production mutex_lock before and after from llvm-objdump (Measurements); the short-contention benchmark held for a quiet PC; folded into #1957 or dropped if the rebased measurements show no gain; #2019 now merges with this head without a conflict (the one-line conflict moved to #2063); Tested on the new base (fb3-mx-1-light, fb3-mxn-changed).

Pushed e4ea22d7 on `main` @ 9dd41326 (from 8443bba1 on 6e1521a7), as dyst's review 65813 and futondestroyer's 65533 asked: this PR is now the split alone (6288abca, e4ea22d7); the diagnostic fixes are #2063. Description rewritten: no gain measured yet; the production `mutex_lock` before and after from llvm-objdump (Measurements); the short-contention benchmark held for a quiet PC; folded into #1957 or dropped if the rebased measurements show no gain; #2019 now merges with this head without a conflict (the one-line conflict moved to #2063); Tested on the new base (fb3-mx-1-light, fb3-mxn-changed).
Author
Contributor

@dyst, on your review, and @futondestroyer, on yours: pushed e4ea22d7, rebased onto main @ 9dd41326 (#2012's spin included).

  • Your suggestion to take the diagnostic fixes out: done. They are #2063 (fix(os), on main): 7492bb61 is the test, which fails on main at HOS_MUTEX_TRACE=1 (the ended holder's record is read), and 28cca9e5 is the fix, adapted to main's mutex_lock. They were d3e02c70 and 18e90465 here. This PR is now the split alone: 6288abca is the split (the code is unchanged; the test cuts mutex_lock_wait from its name on, as d3e02c70 did) and e4ea22d7 adds the CPU report's entries.
  • 65807 (hos_self out of line; production code before and after): I built nn_os_sync.o with the host build's own flags (llvm-mingw clang, Windows, no LTO) at main 9dd41326 and at e4ea22d7, and read it with llvm-objdump -dr.
    • On main, an uncontended lock is a call to mutex_lock and 37 instructions inside it: 8 pushes, 8 pops, a 0x78-byte frame with xmm6 saved, the call to hos_self and the compare-exchange.
    • Here mutex_lock is inlined into LockMutex's entry and into mutex_lock_owed. The uncontended lock is 13 instructions with no frame of its own.
    • hos_self is still a call, so you are right that it is not a leaf. The commit message no longer says it is.
    • No gain is measured. The description says so, and says the split goes into #1957 or is dropped if the rebased measurements show no gain.
    • The Linux code is not compared here (this PC has no Linux toolchain); it is in Still to test.
  • 65809 (a failed first take): the code is rebased first, so the spin is #2012's.
    • In the object, a failed take makes the same single call and return as on main, now after the compare-exchange instead of before it, with one more argument move and one more jump. cold did not move the function to a section of its own on this COFF target.
    • The benchmark you describe is written and built: a first CAS that fails and a spin that then wins without sleeping, main against this head, with sleeps counted to show there are none. A smoke run built and ran both sides.
    • It is held for a quiet PC under the queue's rule for timed runs and has not run yet. Its figures go into Measurements when they exist.
  • 65811 (seq_cst owner load) and 65812 (the switch row): both taken in #2063's 28cca9e5, since those lines moved there.
    • The load is __ATOMIC_SEQ_CST, and so is the 10 s log's.
    • The switch row now says the name is the holder's when the wait started to sleep, and that 1 logs nearly every wait.
  • #2019: it merges with this head without a conflict now (git merge-tree). The one-line conflict on the hos_chain_on read moved to #2063 with the diagnostics, and both descriptions say so. #1957 gets rebased onto this head.
  • futondestroyer's other asks: the owner's interleaved Market run is still pending. Farm request 63064 compared 8443bba1 with 6e1521a7; the pair is now e4ea22d7 against 9dd41326.

Tested on 9dd41326: tests/test_cv_handoff_threads.py passes at e4ea22d7. The changed-file checks gave 13 passed, 1 skipped (Linux only).

@dyst, on [your review](https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/1953#issuecomment-65813), and @futondestroyer, on [yours](https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/1953#issuecomment-65533): pushed e4ea22d7, rebased onto `main` @ 9dd41326 (#2012's spin included). - Your suggestion to take the diagnostic fixes out: done. They are #2063 (`fix(os)`, on `main`): 7492bb61 is the test, which fails on `main` at `HOS_MUTEX_TRACE=1` (the ended holder's record is read), and 28cca9e5 is the fix, adapted to `main`'s `mutex_lock`. They were d3e02c70 and 18e90465 here. This PR is now the split alone: 6288abca is the split (the code is unchanged; the test cuts `mutex_lock_wait` from its name on, as d3e02c70 did) and e4ea22d7 adds the CPU report's entries. - [65807](https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/1953#issuecomment-65807) (`hos_self` out of line; production code before and after): I built `nn_os_sync.o` with the host build's own flags (llvm-mingw clang, Windows, no LTO) at `main` 9dd41326 and at e4ea22d7, and read it with `llvm-objdump -dr`. - On `main`, an uncontended lock is a call to `mutex_lock` and 37 instructions inside it: 8 pushes, 8 pops, a 0x78-byte frame with `xmm6` saved, the call to `hos_self` and the compare-exchange. - Here `mutex_lock` is inlined into LockMutex's entry and into `mutex_lock_owed`. The uncontended lock is 13 instructions with no frame of its own. - `hos_self` is still a call, so you are right that it is not a leaf. The commit message no longer says it is. - No gain is measured. The description says so, and says the split goes into #1957 or is dropped if the rebased measurements show no gain. - The Linux code is not compared here (this PC has no Linux toolchain); it is in Still to test. - [65809](https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/1953#issuecomment-65809) (a failed first take): the code is rebased first, so the spin is #2012's. - In the object, a failed take makes the same single call and return as on `main`, now after the compare-exchange instead of before it, with one more argument move and one more jump. `cold` did not move the function to a section of its own on this COFF target. - The benchmark you describe is written and built: a first CAS that fails and a spin that then wins without sleeping, `main` against this head, with sleeps counted to show there are none. A smoke run built and ran both sides. - It is held for a quiet PC under the queue's rule for timed runs and has not run yet. Its figures go into Measurements when they exist. - [65811](https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/1953#issuecomment-65811) (seq_cst owner load) and [65812](https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/1953#issuecomment-65812) (the switch row): both taken in #2063's 28cca9e5, since those lines moved there. - The load is `__ATOMIC_SEQ_CST`, and so is the 10 s log's. - The switch row now says the name is the holder's when the wait started to sleep, and that `1` logs nearly every wait. - #2019: it merges with this head without a conflict now (`git merge-tree`). The one-line conflict on the `hos_chain_on` read moved to #2063 with the diagnostics, and both descriptions say so. #1957 gets rebased onto this head. - futondestroyer's other asks: the owner's interleaved Market run is still pending. Farm request 63064 compared 8443bba1 with 6e1521a7; the pair is now e4ea22d7 against 9dd41326. Tested on 9dd41326: `tests/test_cv_handoff_threads.py` passes at e4ea22d7. The changed-file checks gave 13 passed, 1 skipped (Linux only).
Author
Contributor

@Beanstalk This moves the guest mutex lock's wait for a held mutex out of line and cold (host/nn_os_sync.c, mutex_lock): every guest mutex lock whose first attempt fails takes the new path; the uncontended path gets shorter. No switch: a revert gives back main's code. Please start from the market-entrance save and play Castle Town Market for two minutes, for frame times, the PR against main on the same PC, at least two interleaved runs a side, on a Windows PC with an AMD processor, a Linux PC with an AMD processor, a PC with 6 or fewer logical processors and the Steam Deck. No hang or stall and no audio or frame pacing trouble; frame time p99 equal to main's or better, within the run-to-run spread.

@Beanstalk This moves the guest mutex lock's wait for a held mutex out of line and cold (host/nn_os_sync.c, mutex_lock): every guest mutex lock whose first attempt fails takes the new path; the uncontended path gets shorter. No switch: a revert gives back main's code. Please start from the market-entrance save and play Castle Town Market for two minutes, for frame times, the PR against main on the same PC, at least two interleaved runs a side, on a Windows PC with an AMD processor, a Linux PC with an AMD processor, a PC with 6 or fewer logical processors and the Steam Deck. No hang or stall and no audio or frame pacing trouble; frame time p99 equal to main's or better, within the run-to-run spread.
Member

@siantm only members of BigBeans can start the farm. One of them can approve this request by reacting 👍 to it or to this comment.

@siantm only members of `BigBeans` can start the farm. One of them can approve this request by reacting 👍 to it or to this comment.
Author
Contributor

Measurements: the held micro-benchmark ran (fb3-1953-bench, main 9dd41326 vs e4ea22d7, 21 runs a side): uncontended lock+unlock 16.33 vs 15.24 ns in overlapping ranges, no difference under contention (1157.0 vs 1159.7 ns, hand-over 91.7 vs 92.3 ns). No gain shown: the split is folded into #1957, as dyst suggested.

Measurements: the held micro-benchmark ran (fb3-1953-bench, main 9dd41326 vs e4ea22d7, 21 runs a side): uncontended lock+unlock 16.33 vs 15.24 ns in overlapping ranges, no difference under contention (1157.0 vs 1159.7 ns, hand-over 91.7 vs 92.3 ns). No gain shown: the split is folded into #1957, as dyst suggested.
Author
Contributor

Measurements: the tables put in the template's form (Before/After columns naming the commits; Hardware, Scene, Method and How to reproduce lines). Same figures.

Measurements: the tables put in the template's form (Before/After columns naming the commits; Hardware, Scene, Method and How to reproduce lines). Same figures.
Author
Contributor

Folded into #1957, as dyst suggested in 65813: measured alone (job fb3-1953-bench, 66418) the split showed no gain, and #1957 carries it with its figures (66421). Its diagnostic fixes are #2063.

Folded into #1957, as dyst suggested in [65813](https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/1953#issuecomment-65813): measured alone (job fb3-1953-bench, [66418](https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/1953#issuecomment-66418)) the split showed no gain, and #1957 carries it with its figures ([66421](https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/1957#issuecomment-66421)). Its diagnostic fixes are #2063.
siantm closed this pull request 2026-11-21 11:51:01 +00:00
Some checks failed
Website / Website on Cloudflare (pull_request_target) Has been skipped
PR description / Description follows the template (pull_request_target) Successful in 6s
Website / Website on Cloudflare (pull_request) Has been cancelled
CI / Conventional Commit title (pull_request) Has been cancelled
CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Has been cancelled
CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Has been cancelled
CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Has been cancelled
CI / Workflow lint (actionlint) (pull_request) Has been cancelled
CI / Website build and checks (pull_request) Has been cancelled
Guide screenshots / Guide images and local UI source review (pull_request) Has been cancelled
Third-party parts / As their notes say (pull_request) Has been cancelled
Third-party parts / The same as upstream (pull_request) Has been cancelled
beanstalk/approval changed since the request: ask again

Pull request closed

Sign in to join this conversation.
No reviewers
No milestone
No project
No assignees
5 participants
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
MagicBeanDev/Longshot!1953
No description provided.