WIP: perf(os): mutex_lock's wait for a held mutex is out of line and cold #1953
No reviewers
Labels
No labels
area/accessibility
area/audio
area/input
area/install
area/menu
area/renderer
area/saves
area/tooling
area/upscaling
beanstalk: approved
beanstalk: failed
beanstalk: needs approval
beanstalk: running
beanstalk: tested
ci: approved
good first issue
gpu/amd
gpu/igpu
gpu/intel
gpu/nvidia
help wanted
kind
bug
kind
chore
kind
docs
kind
feature
kind
performance
kind
question
needs measurements
platform/linux
platform/macos
platform/steam-deck
platform/windows
preview
priority
critical
priority
high
priority
low
regression
risk
high
risk
low
risk
medium
status
blocked
status
confirmed
status
duplicate
status
invalid
status
needs-info
status
wontfix
ci: approved
No milestone
No project
No assignees
4 participants
Notifications
Due date
No due date set.
Dependencies
No dependencies set
Reference
MagicBeanDev/Longshot!1953
Loading…
Reference in a new issue
No description provided.
Delete branch "siantm/Longshot:perf/mutex-lock-wait-cold"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Summary
mutex_lock(host/nn_os_sync.c), behind the game'snn::osmutex locks, had the uncontended lock (the owner check,one compare-exchange) and the whole wait after a failed immediate acquire (the spin, the sleep,
HOS_MUTEX_TRACE's log)in one function, so the short path sat in a function shaped for the long one.
The spin and the sleep move into
mutex_lock_wait, markednoinlineandcold;mutex_lockkeeps the recursion checkand the compare-exchange and calls it whenever that immediate acquire fails: another thread holds the mutex, or it is
reserved for a woken condition-variable waiter while its owner is already NULL. The aim is that the uncontended lock can
compile to a small leaf that saves no registers and sets up no frame for the wait (not shown yet: Still to test). The
take, the waiter registration and retry, and the unlock's wake are unchanged.
The moved wait carried three races in its diagnostics, fixed here in their own commits (
d3e02c70, the failing testfirst, then
18e90465), insidemutex_lock_wait:HOS_MUTEX_TRACEwas cached in a plain static two first waits could write at once, and parsed withatoi. It isread through an atomic and kept in a local for the wait; only a number of milliseconds from 1 to 2147483647 turns it
on, anything else is off (
-1used to log every wait).HosThreadafter the wait, when it could haveended and been freed. The name is copied with the wait counted in
mwaitandm->lkheld, where the holder's unlockwaits for
m->lk; the 10 s log reads it there too.m->count, which the owner changes withoutm->lk: dropped.Left as they are on
mainby this PR, each fixed in a pull request of its own:spin_begin's lazily cachedHOS_SPIN_ALWAYS(#2012, merged intomain; this branch merges with it without a conflict);hos_chain_on, avolatileflag the recorder thread writes (#2019);hos_thread_set_namewriting a thread's name while others read it(#2020, no conflict with this branch). #2019 and this PR both change the line of
mutex_lock_waitthat reads the flag(
const int chain = hos_chain_on;, moved here): whichever lands second is rebased, the line then reading__atomic_load_n(&hos_chain_on, __ATOMIC_RELAXED).From cafb4100 on
exp/x86-asm-sync(its other half, each virtual core's owner slot on a cache line of its own, needslock-free slots
maindoes not have, and is not here).scripts/profiling/cpu_functions.jsondescribes bothfunctions for the CPU report.
Read first:
mutex_lockandmutex_lock_waitinhost/nn_os_sync.c;tests/test_cv_handoff_threads.py, which cutsthe new function too and runs
HOS_MUTEX_TRACE's cases.Testing
Tested: Windows 11, i7-9700, RTX 4060 (NVIDIA 610.88), llvm-mingw clang 23.1.5; game-free checks only. This
change has not run in the game on this PC yet (Still to test). Each check ran in a hidden console of its own, through
the PC's job queue, on this branch rebased onto
main@6e1521a7(head8443bba1):python -I -B tests/test_cv_handoff_threads.py(the mutexes and condition variables on real threads, the source asit is; now with
HOS_MUTEX_TRACEunset, 0, 1, -1, 1x, 2147483648 and 99999999999 in fresh processes, the variableflipped after the first wait, and a holder that has ended before the waiter logs; the compiler from
--cc,CCorLONGSHOT_MINGW, no--ccgiven): passes at8443bba1; at the test commitd3e02c70it fails (the log read the endedholder's record).
python -I -B tests/run_all.py --strict --changed origin/main@8443bba1(8 at once): 32 passed, 7 failed, 3 skipped.4 of the 7 (
test_crash_dialog,test_dev_ab,test_frame_overlap,test_linux_pressure; picked becausegpu/vk/switches.defchanged) fail the same way onmain@6e1521a7in the queue; the skips are Linux-only checks.The other three (
test_present_queue_default,test_target_reuse_memory,test_vk_occlusion_queries) compilednothing in that run nor in the run on
main(no Vulkan headers on the job's path: "not found: vulkan_headers(--strict)"). Rerun with the headers (job fb3-vk-2,
RX_CPMset) at the head8443bba1and onmain@6e1521a7, eachfails at both with the same first error:
test_present_queue_default"srccut: '{TAB_DEBUG, "Present queue",&opts.present_queue, ...' is not in gpu/vk/vk_menu.cpp";
test_target_reuse_memory2 of its 8 tests("'vmaGetAllocationInfo2' unexpectedly found" in the VMA source, and its fixture's "use of undeclared identifier
'destroy_image_later'");
test_vk_occlusion_queries"the ring fixture did not compile" ("no member named 'policy'").None fails on this branch alone.
host/nn_os_sync.ccompiled (syntax only) with the host build's own flags, with and withoutTRACY_ENABLE, at93f9b35a(this PR's first commit) and8443bba1: no errors.Earlier, the same change on
main@8859673e(1e6a4721):tests/test_cv_handoff_threads.pypassed, and themicro-benchmark in Measurements.
Still to test:
mutex_lockin the production host buildagainst
main's (by hand)against
main, a micro-benchmark through the fixture (by hand)maininterleaved and warmed, no switches set, averagefps, 1 % lows, median and p99 frame times, expecting them within the runs' spread or better (by hand, queued)
python3 -I -B tests/test_cv_handoff_threads.py, expecting it to pass (by hand)minutes, this PR against
main, expecting the same frame times asmainor better and no stall (@Beanstalk playCastle Town's market for two minutes on PCs with AMD CPUs and on the Steam Deck, this PR against main, comparing frame
times)
Measurements
Verdict: micro-benchmark only, not measured in play yet: an uncontended lock and unlock took 18.55 ns on
mainand18.03 ns with this change, within the rounds' spread (no gain shown).
Setup: Windows 11, i7-9700, llvm-mingw clang 23.1.5,
-O2 -march=nativeas the host is built; an uncontendedmutex_lock+mutex_unlockthroughtests/test_cv_handoff_threads.py's own fixture (each checkout'snn_os_sync.ccut as that test cuts it); 21 interleaved rounds, the median and the rounds' range, in one job of the PC's job queue
(
microbench.py mutex). Measured onmain@8859673ewith this change as it was there (1e6a4721); the rebase ontofc4a6769changed none of its lines.8859673e8859673e)The figures are from before the rebase and the diagnostics' fixes, which change only
mutex_lock_wait(the lock thatwaits), not the uncontended lock measured. The cost of the failed-first-take path is not measured (Still to test).
Risks
The wait is the same code in another function, so a mistake would show as a lock that never waits or never wakes: a
stall, which
tests/test_cv_handoff_threads.pyand the other mutex checks cover. A lock whose immediate acquire failspays one more call, even when the spin then succeeds at once.
HOS_MUTEX_TRACE(a diagnostic) is now off for a valuethat is not a number of milliseconds from 1 to 2147483647. No switch: a revert is enough. Players notice nothing (no
saves,
longshot_options.inior caches change).Checklist
Ready for review
main; the title is a Conventional Commit.mise run testpasses (any failure that also fails onmainis named).main, or Measurements says why there is nothing to measure.third_party/untouched; nothing GPL-incompatible.When it applies
tests/that reads changed source is updated, not weakened.Assisted-by:/Co-Authored-By:trailers.Ready to merge
main, measured again or a comment says why the numbers still hold.Possibly related, for context:
Recommendation: Request changes. The split preserves the mutex protocol in source, but the fixture shows no reliable speedup and leaves the cost of brief contention unmeasured. The extracted wait also carries forward races and an unsafe owner pointer in its diagnostics.
Moves spinning, sleeping, and wait diagnostics out of the ordinary guest mutex lock. This gives the compiler a smaller fast path to lay out, at the cost of an out-of-line transfer whenever the first acquire fails. The existing threaded fixture compiles the extracted helper.
Does it help: Unproven. In 21 interleaved Windows rounds on the i7-9700 with llvm-mingw Clang 23.1.5, median uncontended lock plus unlock went from 18.55 ns on
8859673eto 18.03 ns on 1e6a4721. The ranges overlap (18.14–19.68 ns and 17.26–18.98 ns). The fixture uses a local thread lookup and a no-op spin. No in-game measurements or contended timings are reported for95c0fd41against2e1f6c32.Other systems: Direct LockMutex and condition-variable reacquisition without a reservation enter the changed path. The reserved reacquire path, TryLockMutex, and UnlockMutex retain their source logic. Short contention can affect frame pacing, audio, and other guest workers on both OSes. The pthread fixture models wake ordering, while Windows uses native SRW waits and Linux and Steam Deck use the futex-backed compatibility layer. GPU vendors share this CPU-side path. Tracy retains the zone label but records the helper as its source function. The CPU profiler still categorizes the helper as os/sync by object file. Saves, settings, caches, and renderer interfaces have no changes in the diff.
Switches: None added, changed, or removed. HOS_MUTEX_TRACE still defaults to 0 (off), which keeps optional trace logging off. Its existing switch-table row remains applicable.
Reviewed:
95c0fd4152against2e1f6c32d3: Read AGENTS.md, CONTRIBUTING.md, docs/dev/standards.md, the Forgejo PR template, and both changed files in full. No docs/review-checklist.md was present. Read the relevant host thread lifetime and wait code, Linux wait implementation, host build flags, Tracy and chain paths, source cutter, and test runner selection. Skimmed sibling checks, profiling metadata, and related recorded runs. No matching recorded run or tester comment was available. The earlier PR revisions named in Testing and Measurements were unavailable in the checkout. No translations or generated files changed.; nothing run (a code review).Checked and fine:
Blocking (to change before merge)
Waiting on the author (a run, a measurement or a decision; no edit does it)
Suggestions (the author may take or leave)
host/nn_os_sync.c:656: Holding m->lk does not protect count from the owner’s recursive acquire or unlock. Drop this diagnostic field or make all mutex count accesses atomic.host/nn_os_sync.c:640: The recorder changes this volatile flag concurrently. Use atomic accesses and release/acquire publication for the recording buffer.Testing: Read-only review. The author reports a passing threaded fixture and 11 passes, 0 failures, and 1 skip in the changed suite at a105a307. The fixture exercises sleeping on contention and handoff ordering, including a case that stalls if the helper call is removed. Its no-op spin leaves spin success and spin cost untested. The author reports no rerun at HEAD.
Overlaps and merge order: None found in the available history or review discussion. The description identifies a separate experimental virtual-core owner-slot change and gives no merge dependency.
Automated code review by BeanBot (gpt-6.2-sol), on the pull request's opening. Nothing was run; a maintainer decides.
@ -634,1 +630,4 @@/* mutex_lock's way when another thread holds the mutex: its spin, then its sleep. Out of line and cold, so the uncontendedlock below does not save registers and set up a stack frame for this part */static __attribute__((noinline, cold)) void mutex_lock_wait(Obj *m, HosThread *self, uint64_t guest, Ctx *c) {spin_until_free(m);Two first-time spinners can race initializing spin_begin’s adaptive flag. Initialize before workers start or cache it atomically.
@ -634,3 +632,4 @@static __attribute__((noinline, cold)) void mutex_lock_wait(Obj *m, HosThread *self, uint64_t guest, Ctx *c) {spin_until_free(m);if (mutex_take(m, self)) { m->count = 1; return; }/* HOS_MUTEX_TRACE=<ms> (diagnostic): waits longer than that are logged with the thread that held the mutex */Concurrent first waits race initializing trace_ms. Use a race-free cached threshold and range-check HOS_MUTEX_TRACE before converting it.
@ -671,6 +670,12 @@ static void mutex_lock(Obj *m, uint64_t guest, Ctx *c) {(unsigned long long)(c ? c->x[30] : 0), hos_thread_name(holder));The original owner can unlock and be destroyed before this log dereferences it. Use a lifetime-safe name snapshot or stable identifier here and in the other owner diagnostics.
@ -631,3 +630,1 @@HosThread *self = hos_self();if (__atomic_load_n(&m->owner, __ATOMIC_RELAXED) == self) { m->count++; return; }if (mutex_take(m, self)) { m->count = 1; return; }/* mutex_lock's way when another thread holds the mutex: its spin, then its sleep. Out of line and cold, so the uncontendedA reservation can send this path a mutex whose owner is already NULL. Describe the trigger as a failed immediate acquire.
@ -672,2 +671,4 @@}}static void mutex_lock(Obj *m, uint64_t guest, Ctx *c) {HosThread *self = hos_self();Production still calls the separately compiled hos_self. Show the fast-path assembly from the actual Windows and Linux builds and measure the change in play.
@ -674,0 +673,4 @@static void mutex_lock(Obj *m, uint64_t guest, Ctx *c) {HosThread *self = hos_self();if (__atomic_load_n(&m->owner, __ATOMIC_RELAXED) == self) { m->count++; return; }if (mutex_take(m, self)) { m->count = 1; return; }The first failed acquire now enters cold code even when the spin succeeds immediately. Measure that handoff separately and check frame-time tails on the affected CPUs.
@ -90,2 +90,3 @@"static int mutex_take(Obj *m, HosThread *self) {", "static int mutex_take_owed(Obj *m, HosThread *self) {","static void mutex_give(Obj *m) {", "static void mutex_lock(Obj *m, uint64_t guest, Ctx *c) {","static void mutex_give(Obj *m) {","static __attribute__((noinline, cold)) void mutex_lock_wait(Obj *m, HosThread *self, uint64_t guest, Ctx *c) {",The runner forces gcc and the direct default ignores CC. Use the configured compiler so the fixture runs on a standard llvm-mingw setup.
The fixture marker can start at
void mutex_lock_waitso an attribute spelling change does not break a behavior check.Recommendation: Wait for author-side gameplay and a measured benefit. The cold split preserves the ownership and waiter protocol. The numbers so far leave its benefit unresolved and omit the path that gets a new call.
mutex_lockstill checks for recursion and attempts the same immediate acquisition. The new call runs after any immediate take fails, including an owner-free mutex reserved for a condition-variable waiter. The reserved reacquire still usesmutex_take_owed. The spin, registration, retry, core handoff and wake code are carried forward. The existing exact-head review calls out diagnostic lifetime and initialization defects. Those lines are inherited from the merge base and deserve their own fixes. This split adds no protection around them.Does it help? Unproven. The author's Windows/i7-9700 fixture reports 18.55 ns before and 18.03 ns after for an uncontended lock/unlock over 21 rounds, with overlapping ranges. It substitutes
hos_selfand the spin helper, so it cannot establish the generated production fast path or the cost of the newly out-of-line failed-first-take path. Include brief contention that resolves without sleeping in the next comparison, with production assembly and the author's warmed, interleaved Market runs against exact main.Reviewed.
95c0fd41526b05ade41ab265166a3c5c4ba55f06against2e1f6c32d30ec3582093743a968c8447bb435962. Reviewed both changed files, the mutex take/give and CV reacquire callers, and the threaded fixture. No new switch. The author reports a passing source-cut fixture and 11 passes with one skip on the earlier rebase. No confirmed exact-head gameplay or farm results. No PR code run in this review.Overlap. The cold helper is already carried in the context mutex stack. If the cold split earns its own landing, rebase that stack to avoid landing the same step twice.
95c0fd41528443bba1f9Pushed
d3e02c70(test),18e90465(fix) and8443bba1(profiling descriptions) for BeanBot's review, with the branch rebased ontomain@6e1521a7(from2e1f6c32;git range-diffshows95c0fd41unchanged as93f9b35a). The description is updated.18e90465.HOS_MUTEX_TRACEis cached through an atomic (two first waits store the same value), kept in a local for the whole wait, and parsed withstrtollover the whole string: only 1 to 2147483647 ms turns it on, anything else is off (-1used to log every wait). The switch's row ingpu/vk/switches.defsays so.18e90465. The holder's name is copied with the wait counted inmwaitandm->lkheld; the holder's unlock (mutex_give) then waits form->lk, so it cannot end and be freed during the copy. The Tracy text and the long-wait log use that copy, and the 10 s log reads the name underm->lktoo. The rename race (hos_thread_set_namewriting a name while another thread reads it) is inhost/hos_thread.cand affects every name reader. It is pre-existing onmain, unchanged by this PR, and left for a separate change.m->countin the 10 s log: fixed in18e90465, the count is dropped (andmwaitis read as an atomic).volatile hos_chain_on: not changed here. The flag, its writer andHOS_CHAINlive inhost/hos_chain.c,host/hos.hand six call sites innn_os_sync.c, all as they are onmain; this PR moves one read of it. Making the recorder atomic with release/acquire publication ofrecsis a separate change, listed for the owner and named in the description.spin_beginis unchanged by this PR (it moves the call tospin_until_free). The race is pre-existing onmainand left for a separate change, named in the description.d3e02c70.RUN_ALL_ARGSis gone; the compiler is--cc, elseCC, elseLONGSHOT_MINGW's clang on Windows, else clang, gcc or cc, as #1954's test does. The queue run below gave it no--cc, so it used the configured llvm-mingw clang.d3e02c70: the cut starts atvoid mutex_lock_wait(, and the test cuts it as a plain static function.18e90465and in the description: a failed immediate acquire, including a reserved mutex with no owner.cpu_functions.json: taken in8443bba1(mutex_lockupdated,mutex_lock_waitadded).8443bba1through this PC's job queue:tests/test_cv_handoff_threads.pypasses, and it fails atd3e02c70(the log read a holder that had ended).tests/run_all.py --strict --changed origin/main: 32 passed, 7 failed, 3 skipped; the 7 fail the same way onmain@6e1521a7.host/nn_os_sync.ccompiles with the host build's flags with and without Tracy.@futondestroyer, on your review:
mutex_lockagainstmain's, brief contention that ends without a sleep (the new out-of-line path), and warmed, interleaved Market runs against exactmainwith fps, 1 % lows, median and p99 frame times. None of these has run yet; they are for the owner.HOS_MUTEX_TRACEcache and parse, the holder's name read after it may have ended,m->countin the 10 s log) are fixed in their own commits,d3e02c70(failing test) and18e90465(fix), on top of the split. If you would rather land them as a PR of their own, they can move as they are.hos_chain_on,spin_begin's cached flag and the thread-name rename race are left onmainfor separate changes, and the description says so.@Beanstalk This moves the guest mutex lock's wait for a held mutex out of line and cold (host/nn_os_sync.c, mutex_lock): every guest mutex lock whose first attempt fails takes the new path; the uncontended path gets shorter. No switch: a revert gives back main's code. It has not run in the game on these PCs yet. Please play Castle Town Market for two minutes, the PR against main on the same PC, on a Windows PC with an AMD processor, a Linux PC with an AMD processor, a PC with 6 or fewer logical processors and the Steam Deck. No hang or stall and no audio or frame pacing trouble; frame time p99 equal to main's or better, within the run-to-run spread.
@siantm only members of
BigBeanscan start the farm. One of them can approve this request by reacting 👍 to it or to this comment.The three pre-existing defects this PR left on
main(named in BeanBot's review) are now fixes of their own, each with a game-free test that fails onmain(this PC's job queue, fb3-sep-1/2); none has run in the game yet:56da0e2e).tests/test_spin_always_once.py: 8 threads first spinning at once readHOS_SPIN_ALWAYS8 times onmain, once with the fix.volatile hos_chain_on: #2019 (fix/os-chain-flag-atomic@a6fffab8). The flag is atomic (release once the buffer is there, acquire before using it, relaxed at the six waits); a note's record is published by its kind (release) and the recorder waits for it (acquire) instead of sleeping 50 ms.tests/test_chain_recorder.py: a note held across the stop left a record of zeros onmain. It conflicts with this PR on the one moved lineconst int chain = hos_chain_on;; whichever lands second is rebased.fix/os-thread-names-kept@67baca96). A name is never written over: a rename publishes a kept name with one atomic store, so everyhos_thread_namereader gets a whole name.tests/test_thread_rename.py: a reader during a rename read half of each name onmain. It merges with this PR without a conflict.Description: Summary says which PRs now carry the three defects left on
main(#2012 merged, #2019, #2020) and the landing order with #2019 (one shared line).The three renderer checks (
test_present_queue_default,test_target_reuse_memory,test_vk_occlusion_queries) did not compile in the earlier run (no Vulkan headers on that job's path); rerun with them in job fb3-vk-2, at the head8443bba1and onmain@6e1521a7: all three fail at both with the same first error, so they fail onmaintoo; Tested's sentence and counts corrected.View command line instructions
Checkout
From your project repository, check out a new branch and test the changes.