WIP: perf(os): mutex_lock's wait for a held mutex is out of line and cold #1953
Labels
No labels
area/accessibility
area/audio
area/input
area/install
area/menu
area/renderer
area/saves
area/tooling
area/upscaling
beanstalk: approved
beanstalk: failed
beanstalk: needs approval
beanstalk: running
beanstalk: tested
ci: approved
good first issue
gpu/amd
gpu/igpu
gpu/intel
gpu/nvidia
help wanted
kind
bug
kind
chore
kind
docs
kind
feature
kind
performance
kind
question
needs measurements
platform/linux
platform/macos
platform/steam-deck
platform/windows
preview
priority
critical
priority
high
priority
low
regression
risk
high
risk
low
risk
medium
status
blocked
status
confirmed
status
duplicate
status
invalid
status
needs-info
status
wontfix
ci: approved
No milestone
No project
No assignees
5 participants
Notifications
Due date
No due date set.
Dependencies
No dependencies set
Reference
MagicBeanDev/Longshot!1953
Loading…
Reference in a new issue
No description provided.
Delete branch "siantm/Longshot:perf/mutex-lock-wait-cold"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Summary
mutex_lock(host/nn_os_sync.c), behind the game'snn::osmutex locks, has the uncontended lock (the owner check,one compare-exchange) and the whole wait after a failed immediate acquire (the spin, the sleep,
HOS_MUTEX_TRACE's log)in one function, so the short path sits in a function shaped for the long one.
The spin and the sleep move word for word into
mutex_lock_wait, markednoinlineandcold;mutex_lockkeeps therecursion check and the compare-exchange and calls it whenever that immediate acquire fails: another thread holds the
mutex, or it is reserved for a woken condition-variable waiter while its owner is already NULL. The take, the waiter
registration and retry, and the unlock's wake are unchanged. In the production host object
mutex_lockis then inlinedinto its two callers and its uncontended lock has no frame of its own;
hos_selfis still a call (see Measurements).No gain is measured yet. The one earlier figure (an uncontended lock and unlock, 18.55 ns on
mainand 18.03 nshere) is within the rounds' spread. The short-contention micro-benchmark (a failed first CAS that the spin then wins,
no sleep;
mainagainst this head) is written and held for a quiet PC (Measurements). If the measurements on thisrebased head show no gain, this change is folded into #1957 (where
mutex_locktakes the thread from its context andthe leaf becomes possible) or dropped.
This PR is now the split alone, rebased onto current
main(with #2012'sspin_begin, which runs inside the newfunction). The three diagnostic fixes it carried (
d3e02c70,18e90465) are their own PR, #2063 (fix(os), onmain,as dyst suggested in 65813), with his
two notes on those lines.
scripts/profiling/cpu_functions.jsondescribes both functions for the CPU report.From cafb4100 on
exp/x86-asm-sync(its other half, each virtual core's owner slot on a cache line of its own, needslock-free slots
maindoes not have, and is not here).Read first:
mutex_lockandmutex_lock_waitinhost/nn_os_sync.c;tests/test_cv_handoff_threads.py, which cutsthe new function too (from its name on, so its attributes may change).
Overlap: #2063 (
fix/os-mutex-wait-diagnostics@28cca9e5) changes lines inside the wait this PR moves; the two mergewithout a conflict in either order (
git merge-tree), the fixed lines then insidemutex_lock_wait. #2019(
fix/os-chain-flag-atomic@75107f9e), stacked on #2020 and merging together with it, also merges with this PRwithout a conflict now (
git merge-tree): this PR moves theconst int chain = hos_chain_on;line it changes withoutchanging it, so merged it reads the flag as #2019 does inside
mutex_lock_wait; the one-line conflict there is now#2063's with #2019. #1957 (
perf/mutex-ctx-thread@a912c5b8) is stacked on this PR's earlier four commits and isrebased onto this head (its owner's).
Testing
Tested: Windows 11, i7-9700, RTX 4060 (NVIDIA 610.88), llvm-mingw clang 23.1.5; game-free checks only. This
change has not run in the game on this PC yet (Still to test). Rebased onto
main@9dd41326: the split commit is now6288abca(git range-diff: the code unchanged; the test cutsmutex_lock_waitfrom its name on, asd3e02c70did, andthe message no longer calls the uncontended lock a leaf), the CPU report's entries
e4ea22d7(unchanged). Each checkran in a hidden console of its own, in one job of the PC's job queue (fb3-mx-1-light):
python -I -B tests/test_cv_handoff_threads.py(the mutexes and condition variables on real threads, the source asit is): passes at
e4ea22d7.host/nn_os_sync.cbuilt into an object with the host build's own flags atmain9dd41326and ate4ea22d7, anddisassembled with llvm-objdump (Measurements).
python -I -B tests/run_all.py --strict --changed origin/mainate4ea22d7(the checks that read a changed file, 8at once; build job fb3-mxn-changed): 13 passed, none failed, 1 skipped (
test_tracy_funcs.py: Linux only).Still to test:
main9dd41326, expecting thewaiter's lock call and hand-over within the runs' spread or better and no sleep (by hand: held job fb3-1953-bench,
written and built, not run yet)
maininterleavedand warmed, no switches set, average fps, 1 % lows, median and p99 frame times, expecting them within the runs'
spread or better (by hand, queued)
mutex_lockin the Linux host build againstmain's, andpython3 -I -B tests/test_cv_handoff_threads.py, expecting it to pass (by hand)minutes, this PR against
main, expecting the same frame times asmainor better and no stall (the Test Farm: playCastle Town's market for two minutes on PCs with AMD CPUs and on the Steam Deck, this PR against main, comparing frame
times)
Measurements
Verdict: no gain shown: folded into #1957. The production code is shorter on the uncontended lock (13
instructions against 40, no frame), but the micro-benchmark at this head puts the uncontended lock and unlock about
1 ns apart inside overlapping ranges and shows no difference under contention (below). As dyst suggested, the split
goes on in #1957, where
mutex_locktakes the thread from its context.Production code (Windows only; no Linux toolchain on this PC):
host/nn_os_sync.cbuilt intonn_os_sync.owith thehost build's own flags (llvm-mingw clang 23.1.5,
--target=x86_64-w64-mingw32 -O2 -g0 -march=native -mcx16 -ffp-contract=off -fms-extensions ..., taken from a local build'scompile_commands.json; no LTO), atmain@9dd41326and at this heade4ea22d7, read withllvm-objdump -dr:main:mutex_lockis a function of its own, called from LockMutex's entry and frommutex_lock_owed. Anuncontended lock runs 37 instructions in it, plus 3 in the caller (two argument moves and the call): 8 register
pushes and 8 pops, a 0x78-byte frame with
xmm6saved and restored, three argument moves, the call tohos_self,the owner check, the
reservedload,lock cmpxchg, the count store and the jump to the epilogue.mutex_lockis no longer a function: the compiler inlines it into both callers. An uncontended lock is 13instructions inside LockMutex's entry: the call to
hos_self(still a call, tohost/hos_thread.c), the ownercheck, the
reservedload,lock cmpxchg, the count store and the jump on; no push, pop or frame of its own(LockMutex's entry keeps its own 6 pushes and 0x48-byte frame, the same on both).
mainthe call tomutex_lockand its frame came first, then the compare-exchangeand the spin; here the compare-exchange fails inline, then three argument moves, the call to
mutex_lock_waitand,after its return, a jump back (one argument move and one jump more than
main, the same one call and return).mutex_lock_waitsets up the framemutex_lockdid onmain(8 pushes, 0x78 bytes,xmm6) and runs the same spin.colddoes not move it to a section of its own on this COFF target: it is in.textwheremutex_lockwas.mutex_lock_owed224 to 304,mutex_lock(1056) gone,mutex_lock_wait998.Measured (
fb3-1953-bench, run alone on this PC, i7-9700, Windows 11, clang 23.1.5; median of 21 interleaved runsa side, the runs' range):
9dd41326e4ea22d7How it was run:
fb3-1953-bench,main9dd41326againste4ea22d7,mutex_lockcut from each checkout and built with the host build's flags,
hos_self,hos_spin_nsandhos_now_nsin atranslation unit of their own as in the host; two threads pinned to two cores, the holder keeping the mutex 1 us after
the waiter starts its lock (the spin is 5 us), 20000 hand-overs a run, 21 runs a side interleaved: the waiter's lock
call, the hand-over from unlock to the waiter's return, the number of iterations that slept (0 expected: the spin
wins), and the uncontended lock and unlock.
Earlier, before the rebase (
main@8859673e, this change as 1e6a4721): an uncontendedmutex_lock+mutex_unlockthrough
tests/test_cv_handoff_threads.py's fixture (itshos_selfa local function), 21 interleaved rounds, in onejob of the PC's job queue (
microbench.py mutex):8859673e8859673e)Risks
The wait is the same code in another function, so a mistake would show as a lock that never waits or never wakes: a
stall, which
tests/test_cv_handoff_threads.pyand the other mutex checks cover. A lock whose immediate acquire failsmakes its one call after the compare-exchange instead of before it, with one move and one jump more, even when the
spin then wins at once: the held benchmark measures that case. LockMutex's entry and
mutex_lock_owedgrow by 32 and80 bytes with the inlined lock. No switch: a revert is enough. Players notice nothing (no saves,
longshot_options.inior caches change).Checklist
Ready for review
main; the title is a Conventional Commit.mise run testpasses (any failure that also fails onmainis named).main, or Measurements says why there is nothing to measure.third_party/untouched; nothing GPL-incompatible.When it applies
tests/that reads changed source is updated, not weakened.Assisted-by:/Co-Authored-By:trailers.Ready to merge
main, measured again or a comment says why the numbers still hold.Possibly related, for context:
Recommendation: Request changes. The split preserves the mutex protocol in source, but the fixture shows no reliable speedup and leaves the cost of brief contention unmeasured. The extracted wait also carries forward races and an unsafe owner pointer in its diagnostics.
Moves spinning, sleeping, and wait diagnostics out of the ordinary guest mutex lock. This gives the compiler a smaller fast path to lay out, at the cost of an out-of-line transfer whenever the first acquire fails. The existing threaded fixture compiles the extracted helper.
Does it help: Unproven. In 21 interleaved Windows rounds on the i7-9700 with llvm-mingw Clang 23.1.5, median uncontended lock plus unlock went from 18.55 ns on
8859673eto 18.03 ns on 1e6a4721. The ranges overlap (18.14–19.68 ns and 17.26–18.98 ns). The fixture uses a local thread lookup and a no-op spin. No in-game measurements or contended timings are reported for95c0fd41against2e1f6c32.Other systems: Direct LockMutex and condition-variable reacquisition without a reservation enter the changed path. The reserved reacquire path, TryLockMutex, and UnlockMutex retain their source logic. Short contention can affect frame pacing, audio, and other guest workers on both OSes. The pthread fixture models wake ordering, while Windows uses native SRW waits and Linux and Steam Deck use the futex-backed compatibility layer. GPU vendors share this CPU-side path. Tracy retains the zone label but records the helper as its source function. The CPU profiler still categorizes the helper as os/sync by object file. Saves, settings, caches, and renderer interfaces have no changes in the diff.
Switches: None added, changed, or removed. HOS_MUTEX_TRACE still defaults to 0 (off), which keeps optional trace logging off. Its existing switch-table row remains applicable.
Reviewed:
95c0fd4152against2e1f6c32d3: Read AGENTS.md, CONTRIBUTING.md, docs/dev/standards.md, the Forgejo PR template, and both changed files in full. No docs/review-checklist.md was present. Read the relevant host thread lifetime and wait code, Linux wait implementation, host build flags, Tracy and chain paths, source cutter, and test runner selection. Skimmed sibling checks, profiling metadata, and related recorded runs. No matching recorded run or tester comment was available. The earlier PR revisions named in Testing and Measurements were unavailable in the checkout. No translations or generated files changed.; nothing run (a code review).Checked and fine:
Blocking (to change before merge)
Waiting on the author (a run, a measurement or a decision; no edit does it)
Suggestions (the author may take or leave)
host/nn_os_sync.c:656: Holding m->lk does not protect count from the owner’s recursive acquire or unlock. Drop this diagnostic field or make all mutex count accesses atomic.host/nn_os_sync.c:640: The recorder changes this volatile flag concurrently. Use atomic accesses and release/acquire publication for the recording buffer.Testing: Read-only review. The author reports a passing threaded fixture and 11 passes, 0 failures, and 1 skip in the changed suite at a105a307. The fixture exercises sleeping on contention and handoff ordering, including a case that stalls if the helper call is removed. Its no-op spin leaves spin success and spin cost untested. The author reports no rerun at HEAD.
Overlaps and merge order: None found in the available history or review discussion. The description identifies a separate experimental virtual-core owner-slot change and gives no merge dependency.
Automated code review by BeanBot (gpt-6.2-sol), on the pull request's opening. Nothing was run; a maintainer decides.
@ -634,1 +630,4 @@/* mutex_lock's way when another thread holds the mutex: its spin, then its sleep. Out of line and cold, so the uncontendedlock below does not save registers and set up a stack frame for this part */static __attribute__((noinline, cold)) void mutex_lock_wait(Obj *m, HosThread *self, uint64_t guest, Ctx *c) {spin_until_free(m);Two first-time spinners can race initializing spin_begin’s adaptive flag. Initialize before workers start or cache it atomically.
@ -634,3 +632,4 @@static __attribute__((noinline, cold)) void mutex_lock_wait(Obj *m, HosThread *self, uint64_t guest, Ctx *c) {spin_until_free(m);if (mutex_take(m, self)) { m->count = 1; return; }/* HOS_MUTEX_TRACE=<ms> (diagnostic): waits longer than that are logged with the thread that held the mutex */Concurrent first waits race initializing trace_ms. Use a race-free cached threshold and range-check HOS_MUTEX_TRACE before converting it.
@ -671,6 +670,12 @@ static void mutex_lock(Obj *m, uint64_t guest, Ctx *c) {(unsigned long long)(c ? c->x[30] : 0), hos_thread_name(holder));The original owner can unlock and be destroyed before this log dereferences it. Use a lifetime-safe name snapshot or stable identifier here and in the other owner diagnostics.
@ -631,3 +630,1 @@HosThread *self = hos_self();if (__atomic_load_n(&m->owner, __ATOMIC_RELAXED) == self) { m->count++; return; }if (mutex_take(m, self)) { m->count = 1; return; }/* mutex_lock's way when another thread holds the mutex: its spin, then its sleep. Out of line and cold, so the uncontendedA reservation can send this path a mutex whose owner is already NULL. Describe the trigger as a failed immediate acquire.
@ -672,2 +671,4 @@}}static void mutex_lock(Obj *m, uint64_t guest, Ctx *c) {HosThread *self = hos_self();Production still calls the separately compiled hos_self. Show the fast-path assembly from the actual Windows and Linux builds and measure the change in play.
@ -674,0 +673,4 @@static void mutex_lock(Obj *m, uint64_t guest, Ctx *c) {HosThread *self = hos_self();if (__atomic_load_n(&m->owner, __ATOMIC_RELAXED) == self) { m->count++; return; }if (mutex_take(m, self)) { m->count = 1; return; }The first failed acquire now enters cold code even when the spin succeeds immediately. Measure that handoff separately and check frame-time tails on the affected CPUs.
@ -90,2 +90,3 @@"static int mutex_take(Obj *m, HosThread *self) {", "static int mutex_take_owed(Obj *m, HosThread *self) {","static void mutex_give(Obj *m) {", "static void mutex_lock(Obj *m, uint64_t guest, Ctx *c) {","static void mutex_give(Obj *m) {","static __attribute__((noinline, cold)) void mutex_lock_wait(Obj *m, HosThread *self, uint64_t guest, Ctx *c) {",The runner forces gcc and the direct default ignores CC. Use the configured compiler so the fixture runs on a standard llvm-mingw setup.
The fixture marker can start at
void mutex_lock_waitso an attribute spelling change does not break a behavior check.Recommendation: Wait for author-side gameplay and a measured benefit. The cold split preserves the ownership and waiter protocol. The numbers so far leave its benefit unresolved and omit the path that gets a new call.
mutex_lockstill checks for recursion and attempts the same immediate acquisition. The new call runs after any immediate take fails, including an owner-free mutex reserved for a condition-variable waiter. The reserved reacquire still usesmutex_take_owed. The spin, registration, retry, core handoff and wake code are carried forward. The existing exact-head review calls out diagnostic lifetime and initialization defects. Those lines are inherited from the merge base and deserve their own fixes. This split adds no protection around them.Does it help? Unproven. The author's Windows/i7-9700 fixture reports 18.55 ns before and 18.03 ns after for an uncontended lock/unlock over 21 rounds, with overlapping ranges. It substitutes
hos_selfand the spin helper, so it cannot establish the generated production fast path or the cost of the newly out-of-line failed-first-take path. Include brief contention that resolves without sleeping in the next comparison, with production assembly and the author's warmed, interleaved Market runs against exact main.Reviewed.
95c0fd41526b05ade41ab265166a3c5c4ba55f06against2e1f6c32d30ec3582093743a968c8447bb435962. Reviewed both changed files, the mutex take/give and CV reacquire callers, and the threaded fixture. No new switch. The author reports a passing source-cut fixture and 11 passes with one skip on the earlier rebase. No confirmed exact-head gameplay or farm results. No PR code run in this review.Overlap. The cold helper is already carried in the context mutex stack. If the cold split earns its own landing, rebase that stack to avoid landing the same step twice.
95c0fd41528443bba1f9Pushed
d3e02c70(test),18e90465(fix) and8443bba1(profiling descriptions) for BeanBot's review, with the branch rebased ontomain@6e1521a7(from2e1f6c32;git range-diffshows95c0fd41unchanged as93f9b35a). The description is updated.18e90465.HOS_MUTEX_TRACEis cached through an atomic (two first waits store the same value), kept in a local for the whole wait, and parsed withstrtollover the whole string: only 1 to 2147483647 ms turns it on, anything else is off (-1used to log every wait). The switch's row ingpu/vk/switches.defsays so.18e90465. The holder's name is copied with the wait counted inmwaitandm->lkheld; the holder's unlock (mutex_give) then waits form->lk, so it cannot end and be freed during the copy. The Tracy text and the long-wait log use that copy, and the 10 s log reads the name underm->lktoo. The rename race (hos_thread_set_namewriting a name while another thread reads it) is inhost/hos_thread.cand affects every name reader. It is pre-existing onmain, unchanged by this PR, and left for a separate change.m->countin the 10 s log: fixed in18e90465, the count is dropped (andmwaitis read as an atomic).volatile hos_chain_on: not changed here. The flag, its writer andHOS_CHAINlive inhost/hos_chain.c,host/hos.hand six call sites innn_os_sync.c, all as they are onmain; this PR moves one read of it. Making the recorder atomic with release/acquire publication ofrecsis a separate change, listed for the owner and named in the description.spin_beginis unchanged by this PR (it moves the call tospin_until_free). The race is pre-existing onmainand left for a separate change, named in the description.d3e02c70.RUN_ALL_ARGSis gone; the compiler is--cc, elseCC, elseLONGSHOT_MINGW's clang on Windows, else clang, gcc or cc, as #1954's test does. The queue run below gave it no--cc, so it used the configured llvm-mingw clang.d3e02c70: the cut starts atvoid mutex_lock_wait(, and the test cuts it as a plain static function.18e90465and in the description: a failed immediate acquire, including a reserved mutex with no owner.cpu_functions.json: taken in8443bba1(mutex_lockupdated,mutex_lock_waitadded).8443bba1through this PC's job queue:tests/test_cv_handoff_threads.pypasses, and it fails atd3e02c70(the log read a holder that had ended).tests/run_all.py --strict --changed origin/main: 32 passed, 7 failed, 3 skipped; the 7 fail the same way onmain@6e1521a7.host/nn_os_sync.ccompiles with the host build's flags with and without Tracy.@futondestroyer, on your review:
mutex_lockagainstmain's, brief contention that ends without a sleep (the new out-of-line path), and warmed, interleaved Market runs against exactmainwith fps, 1 % lows, median and p99 frame times. None of these has run yet; they are for the owner.HOS_MUTEX_TRACEcache and parse, the holder's name read after it may have ended,m->countin the 10 s log) are fixed in their own commits,d3e02c70(failing test) and18e90465(fix), on top of the split. If you would rather land them as a PR of their own, they can move as they are.hos_chain_on,spin_begin's cached flag and the thread-name rename race are left onmainfor separate changes, and the description says so.@Beanstalk This moves the guest mutex lock's wait for a held mutex out of line and cold (host/nn_os_sync.c, mutex_lock): every guest mutex lock whose first attempt fails takes the new path; the uncontended path gets shorter. No switch: a revert gives back main's code. It has not run in the game on these PCs yet. Please play Castle Town Market for two minutes, the PR against main on the same PC, on a Windows PC with an AMD processor, a Linux PC with an AMD processor, a PC with 6 or fewer logical processors and the Steam Deck. No hang or stall and no audio or frame pacing trouble; frame time p99 equal to main's or better, within the run-to-run spread.
@siantm only members of
BigBeanscan start the farm.One of them can approve this request by reacting 👍 to it or to this comment.This request was for8443bba, and the PR has changed since (e4ea22d), so a 👍 no longer approves it. Comment@Beanstalkagain to ask for the new commit.The three pre-existing defects this PR left on
main(named in BeanBot's review) are now fixes of their own, each with a game-free test that fails onmain(this PC's job queue, fb3-sep-1/2); none has run in the game yet:56da0e2e).tests/test_spin_always_once.py: 8 threads first spinning at once readHOS_SPIN_ALWAYS8 times onmain, once with the fix.volatile hos_chain_on: #2019 (fix/os-chain-flag-atomic@a6fffab8). The flag is atomic (release once the buffer is there, acquire before using it, relaxed at the six waits); a note's record is published by its kind (release) and the recorder waits for it (acquire) instead of sleeping 50 ms.tests/test_chain_recorder.py: a note held across the stop left a record of zeros onmain. It conflicts with this PR on the one moved lineconst int chain = hos_chain_on;; whichever lands second is rebased.fix/os-thread-names-kept@67baca96). A name is never written over: a rename publishes a kept name with one atomic store, so everyhos_thread_namereader gets a whole name.tests/test_thread_rename.py: a reader during a rename read half of each name onmain. It merges with this PR without a conflict.Description: Summary says which PRs now carry the three defects left on
main(#2012 merged, #2019, #2020) and the landing order with #2019 (one shared line).The three renderer checks (
test_present_queue_default,test_target_reuse_memory,test_vk_occlusion_queries) did not compile in the earlier run (no Vulkan headers on that job's path); rerun with them in job fb3-vk-2, at the head8443bba1and onmain@6e1521a7: all three fail at both with the same first error, so they fail onmaintoo; Tested's sentence and counts corrected.Recommendation: Wait for a rebased comparison and the owner's queued measurements. The new commits address the mutex-wait diagnostic defects that could be fixed within this patch. Last pass's performance gap remains open.
The trace threshold is now read atomically and is bounded to a positive 32-bit value. The wait copies the holder's name while its waiter count keeps a normal unlock at the mutex lock, and it drops the unsynchronized count from the ten-second message. The holder copy is used after the wait instead of dereferencing the former owner. The focused fixture checks the invalid values, first-read cache, and a holder that has already unlocked and ended before the waiter reports. The inherited chain flag and thread-name publication remain tracked separately. Author follow-up
Does it help? Unproven. The earlier uncontended fixture measured 18.55 ns on main and 18.03 ns with the extraction, with overlapping ranges. The failed-first-take path still pays the new call before trying the spin. For that case the current branch and its merge base both predate the now-merged
HOS_SPIN_ALWAYSinitialization and fast-read change, which runs inside the extracted spin path. A Farm approval here would compare8443bba1to6e1521a7. Rebase the proposed landing on current main, then compare the updated production code and a short-contention workload that tries the spin without parking. The owner's interleaved Market run is still listed as queued. Merged spin initialization fixTesting. The author reports the focused CV fixture passing at this head and failing before the diagnostic fix, Windows syntax checks with and without Tracy, and 32 broader passes, seven base-matched failures and three Linux-only skips. CI and Linux coverage remain open. The author’s cross-host Market request is pending. Its scene can check gameplay stability and frame times on the refreshed pair, alongside the owner’s brief-contention result. Existing Farm request
Dependencies. Keep the chain-flag fix coordinated at the moved read. The context-mutex experiment carries equivalents of these four commits and its own lift dependency. Once the proposed mutex split is settled, rebase that stack and measure its incremental change against the settled prerequisite, as well as the product against main. Chain flag · Context mutex stack
Reviewed.
8443bba1f9c56480bb3a26f8470bff52a87fe317against actual merge base6e1521a70a32603ed60a8263e4072625885669b7. Read the four-file diff, mutex release and CV handoff, the guest thread destroy path, and the updated diagnostic fixture. Both revision-matched Graphify maps verified and queried. Source only.Verdict: Request changes. The diagnostic fixes are right and well tested, but this is a
perfchange with no measured gain: the one number is within the spread. The fast path it is meant to shorten still callshos_self()in another translation unit, so it cannot become the small leaf the split aims for. The cost on the contended path is still unmeasured, and the branch has not been rebased onto themainthat changed the spin inside the new cold function, as futondestroyer asked.For a player: a guest mutex lock that finds the mutex free should cost a little less, and one that has to wait goes through a separate cold function. Nothing visible, unless the gain or the cost shows in frame times. The three diagnostic fixes (the
HOS_MUTEX_TRACEcache and parse, the holder's name read after it may have ended,m->countin the 10 s log) make the code better on their own. The split, as it stands, has no shown gain to set against its cost.Reviewed:
8443bba1against the merge base6e1521a7(forge/main is now9dd41326, 15 commits on, #2012 among them):host/nn_os_sync.c(the diff in full, plusmutex_take,mutex_give,mutex_take_owed,mutex_unlock,mutex_release_all,spin_until_free),tests/test_cv_handoff_threads.py(the diff in full:compiler(), the cut, the fixture'sHosThread, thehos_cv_sleephook,trace_wait/trace_cases, the runner loop),gpu/vk/switches.def,scripts/profiling/cpu_functions.json;hos_selfinhost/hos_thread.c; the sibling PRs' diffs ofnn_os_sync.c. A re-review of BeanBot's and futondestroyer's points at95c0fd41(no longer reachable here), judged from the author's replies and the current code. Nothing run (a code review).Fixed in this review: nothing.
Checked and fine:
reservedcheck, then the CAS), the spin, the waiter count, the retry loop and the wake inmutex_giveare the same code.mutex_lock_waitis a tail call frommutex_lock.trace_ms): fixed. The cache is atomic and the value stays in a local for the whole wait.strtollwith a whole-string and 1..INT32_MAX check turns off-1,1xand values too large for an int, and the switch row says so.mwait's seq_cst increment, withm->lkheld. A holder seen as owner then reachesmutex_give, which readsmwaitafter clearing the owner (both seq_cst), sees the waiter and blocks onm->lk, so it cannot end during the copy. The 10 s log reads underm->lkwithmwaitstill counted. Non-Tracy builds copy the name only when the trace is on (name_holder), so the usual path pays nothing. The rename race moved to #2020.m->count): dropped, andmwaitis read atomically.hos_chain_on) and 5 (spin_begin): code this PR only moves, now in #2019 and the merged #2012. That is fine for this PR's scope.RUN_ALL_ARGSis gone, andcompiler()honours--cc, thenCC, thenLONGSHOT_MINGW's clang (whichrun_all.pysets on Windows), then clang, gcc or cc. The fixture defines_POSIX_C_SOURCE 200809L, sosetenvis declared under-std=c11 -Werroron glibc, and_putenvon Windows.void mutex_lock_wait(, and the runner addsstaticto the cut. The trigger is described as a failed immediate acquire.hos_thread_nameCHECKs a dead record, and thehos_cv_sleephook holds the waiter until the holder has unlocked and ended. Reading the name after the wait, as onmain, fails, as the author's run atd3e02c70shows. The value table checks the cache too, because each value runs in a fresh process with the variable flipped between the two waits.__attribute__((noinline, cold))is accepted by gcc and clang (including llvm-mingw and clang-cl) on Windows and Linux. It is already used inhost/nn_libc.c:27, and the file depends on GNU__atomic_*builtins anyway.cpu_functions.jsondescribes both functions. No game data in the diff. No switch added, so the default-at-first-push rule does not apply.mainwithout a textual conflict (git merge-tree).Blocking (to change before merge)
host/nn_os_sync.c:697: the gain the split is for is neither shown nor reachable as described.mutex_lockstill begins withhos_self(), an out-of-line function inhost/hos_thread.c:96. In the ordinary (non-LTO) host build,m,guestandcmust live across that call, so the uncontended lock still saves registers and sets up a frame. The split can at most trim a few saves. The one measurement (18.55 vs 18.03 ns, ranges overlapping) is within the spread, and docs/dev/standards.md says "No measured gain, no merge". To fix, showmutex_lock's generated code before and after from the production host object (objdump ofnn_os_sync.o, Windows and Linux) and a gain against currentmain. #1957 is whereselfcomes from the context and the leaf becomes possible, so the split may belong there (see Suggestions).host/nn_os_sync.c:633,:700: the cost is unmeasured on the path the change makes slower. Every failed first take (a briefly held job or queue mutex) now pays a call into code the compiler treats as unlikely and may place away from its caller. gcc documents thatcoldalso optimises the function for size, along with the spin it inlines. The fixture's spin is a no-op, so nothing so far enters that path. BeanBot's and futondestroyer's request for a short-contention measurement (a failed first CAS that the spin then wins, no sleep) is still open. standards.md wants a cost measured and stated in Risks.main, as futondestroyer asked. #2012 changedspin_begin, which now runs inside the new cold function, so any comparison made from this branch (the farm request included) measures a spin pathmainno longer has. Rebase onto currentmainand measure that pair.Suggestions (the author may take or leave)
d3e02c70and18e90465are afix(os)that is right on its own and needs no perf evidence, so it could land now. The split could then move into #1957, wheremutex_locktakesselffrom the context and becomes a leaf, so its gain can be measured with the change that makes it possible. Or keep it here if the rebased measurement shows a gain alone.host/nn_os_sync.c:664: the owner load that guards the copy pairs withmutex_give's seq_cst exchange andmwaitload (a store-buffer pattern). It is right on x86-64 because thelock xaddbefore it is a full barrier, but C11 only guarantees it with this load at__ATOMIC_SEQ_CST. That is the same plainmovon x86-64, so it costs nothing.gpu/vk/switches.def: "with the thread that held the mutex as the wait began". The name is taken after the spin andm->lk, so it is the holder when the waiter counts itself in. "when the wait started to sleep" would be exact.-1used to log every wait and nothing does now; "1 logs nearly every wait" would tell a user what to use instead.Testing: the author's game-free runs on Windows 11 / i7-9700 / llvm-mingw clang: the CV fixture passes at
8443bba1and fails at the test commit, the changed-checks run shows 7 failures that fail onmaintoo, and the file compiles with and without Tracy. Missing: any game run, the production codegen, the short-contention measurement, a Linux build, and CI (pending, blocked on approval). The farm request (63064) is for this head but compares against6e1521a7, so it should wait for the rebase.Overlaps and merge order: #1957 carries these four commits under other hashes and conflicts with this branch. Land one or the other, not both. #2019 conflicts on the moved
hos_chain_onread (one line), and whichever lands second rebases. #1974, 1971, 1970, 1969, 1967 and 1947 merge with this branch cleanly.gpu/vk/switches.defwill conflict textually with most open PRs, as usual.@ -103,3 +103,3 @@SW(HOS_MOUSE_SENS, player, float, "the menu's mouse sensitivity (or user/mouse_sens.txt, moved into the menu once)", "Sets the mouse sensitivity and stops the old mouse_sens.txt being moved into the menu.")SW(HOS_MOVIE_PACE, kill, bool, "on: a 60 FPS cap is raised for a movie's pictures", "=0 leaves the pacing at the frame rate cap while a movie plays.")SW(HOS_MUTEX_TRACE, diag, int, "0: off", "Logs mutex waits longer than <ms>, with the thread that held the mutex.")SW(HOS_MUTEX_TRACE, diag, int, "0: off", "Logs mutex waits longer than <ms> (1 to 2147483647; any other value is off), with the thread that held the mutex as the wait began; read once, at the first wait that sleeps.")Nit: the name is taken when the waiter counts itself in (after the spin and m->lk), not as the wait began. -1 used to log every wait; say what to use instead (1).
@ -634,0 +630,4 @@/* mutex_lock's way when its immediate acquire failed (another thread holds the mutex, or it is reserved for a wokencondition variable waiter while its owner is already NULL): its spin, then its sleep. Out of line and cold, so theuncontended lock below does not save registers and set up a stack frame for this part */static __attribute__((noinline, cold)) void mutex_lock_wait(Obj *m, HosThread *self, uint64_t guest, Ctx *c) {Every failed first take now calls this cold function, which gcc also optimises for size along with the spin it inlines. The fixture's spin is a no-op, so the case where the spin wins without a sleep is unmeasured. Measure it, and rebase first so the spin is #2012's.
@ -651,0 +661,4 @@holds for the 10 s log below, which reads it with m->lk held again */char held_by[48] = "nobody";if (name_holder) {HosThread *h = __atomic_load_n(&m->owner, __ATOMIC_RELAXED);Nit: this load pairs with mutex_give's seq_cst exchange and mwait load (store-buffer pattern). It is right on x86-64 after the lock xadd, but C11 guarantees it only at __ATOMIC_SEQ_CST, which is the same mov here.
@ -672,2 +694,4 @@}}static void mutex_lock(Obj *m, uint64_t guest, Ctx *c) {HosThread *self = hos_self();Still an out-of-line call to hos_self() (host/hos_thread.c:96) in the non-LTO host build: m, guest and c live across it, so this cannot be the frameless leaf the split is for. Show the production codegen before and after, and a gain against current main (no measured gain, no merge). #1957 is where self comes from the context.
8443bba1f9e4ea22d75bPushed
e4ea22d7onmain@9dd41326(from8443bba1on6e1521a7), as dyst's review 65813 and futondestroyer's 65533 asked: this PR is now the split alone (6288abca,e4ea22d7); the diagnostic fixes are #2063. Description rewritten: no gain measured yet; the productionmutex_lockbefore and after from llvm-objdump (Measurements); the short-contention benchmark held for a quiet PC; folded into #1957 or dropped if the rebased measurements show no gain; #2019 now merges with this head without a conflict (the one-line conflict moved to #2063); Tested on the new base (fb3-mx-1-light, fb3-mxn-changed).@dyst, on your review, and @futondestroyer, on yours: pushed
e4ea22d7, rebased ontomain@9dd41326(#2012's spin included).fix(os), onmain):7492bb61is the test, which fails onmainatHOS_MUTEX_TRACE=1(the ended holder's record is read), and28cca9e5is the fix, adapted tomain'smutex_lock. They wered3e02c70and18e90465here. This PR is now the split alone:6288abcais the split (the code is unchanged; the test cutsmutex_lock_waitfrom its name on, asd3e02c70did) ande4ea22d7adds the CPU report's entries.hos_selfout of line; production code before and after): I builtnn_os_sync.owith the host build's own flags (llvm-mingw clang, Windows, no LTO) atmain9dd41326and ate4ea22d7, and read it withllvm-objdump -dr.main, an uncontended lock is a call tomutex_lockand 37 instructions inside it: 8 pushes, 8 pops, a 0x78-byte frame withxmm6saved, the call tohos_selfand the compare-exchange.mutex_lockis inlined into LockMutex's entry and intomutex_lock_owed. The uncontended lock is 13 instructions with no frame of its own.hos_selfis still a call, so you are right that it is not a leaf. The commit message no longer says it is.main, now after the compare-exchange instead of before it, with one more argument move and one more jump.colddid not move the function to a section of its own on this COFF target.mainagainst this head, with sleeps counted to show there are none. A smoke run built and ran both sides.28cca9e5, since those lines moved there.__ATOMIC_SEQ_CST, and so is the 10 s log's.1logs nearly every wait.git merge-tree). The one-line conflict on thehos_chain_onread moved to #2063 with the diagnostics, and both descriptions say so. #1957 gets rebased onto this head.8443bba1with6e1521a7; the pair is nowe4ea22d7against9dd41326.Tested on
9dd41326:tests/test_cv_handoff_threads.pypasses ate4ea22d7. The changed-file checks gave 13 passed, 1 skipped (Linux only).@Beanstalk This moves the guest mutex lock's wait for a held mutex out of line and cold (host/nn_os_sync.c, mutex_lock): every guest mutex lock whose first attempt fails takes the new path; the uncontended path gets shorter. No switch: a revert gives back main's code. Please start from the market-entrance save and play Castle Town Market for two minutes, for frame times, the PR against main on the same PC, at least two interleaved runs a side, on a Windows PC with an AMD processor, a Linux PC with an AMD processor, a PC with 6 or fewer logical processors and the Steam Deck. No hang or stall and no audio or frame pacing trouble; frame time p99 equal to main's or better, within the run-to-run spread.
@siantm only members of
BigBeanscan start the farm. One of them can approve this request by reacting 👍 to it or to this comment.Measurements: the held micro-benchmark ran (fb3-1953-bench, main
9dd41326vse4ea22d7, 21 runs a side): uncontended lock+unlock 16.33 vs 15.24 ns in overlapping ranges, no difference under contention (1157.0 vs 1159.7 ns, hand-over 91.7 vs 92.3 ns). No gain shown: the split is folded into #1957, as dyst suggested.Measurements: the tables put in the template's form (Before/After columns naming the commits; Hardware, Scene, Method and How to reproduce lines). Same figures.
Folded into #1957, as dyst suggested in 65813: measured alone (job fb3-1953-bench, 66418) the split showed no gain, and #1957 carries it with its figures (66421). Its diagnostic fixes are #2063.
Pull request closed