WIP: perf(os): LockMutex and UnlockMutex take the calling thread and the core checkpoint's test from their context #1957
No reviewers
Labels
No labels
area/accessibility
area/audio
area/input
area/install
area/menu
area/renderer
area/saves
area/tooling
area/upscaling
beanstalk: approved
beanstalk: failed
beanstalk: needs approval
beanstalk: running
beanstalk: tested
ci: approved
good first issue
gpu/amd
gpu/igpu
gpu/intel
gpu/nvidia
help wanted
kind
bug
kind
chore
kind
docs
kind
feature
kind
performance
kind
question
needs measurements
platform/linux
platform/macos
platform/steam-deck
platform/windows
preview
priority
critical
priority
high
priority
low
regression
risk
high
risk
low
risk
medium
status
blocked
status
confirmed
status
duplicate
status
invalid
status
needs-info
status
wontfix
ci: approved
No milestone
No project
No assignees
4 participants
Notifications
Due date
No due date set.
Dependencies
No dependencies set
Reference
MagicBeanDev/Longshot!1957
Loading…
Reference in a new issue
No description provided.
Delete branch "siantm/Longshot:perf/mutex-ctx-thread"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Summary
This PR depends on #1956 (
perf/os-inline, theHosThreadhead) and #1953(
perf/mutex-lock-wait-cold,mutex_lock_wait's signature); their commits show in this diff until they merge.LockMutexandUnlockMutexare the game's busiestnn::oscalls. Each found the calling thread through the hostthread-local (
hos_self(), a call intohos_thread.c) and ran the virtual-core checkpoint as a call(
hos_vcore_checkpoint(), which reads the same thread-local) before its lookup. Their context already names the thread:Ctx.threadis theHosThreadrunning it (setup_guest; a fiber's context takes its thread's at each switch).This PR takes both from there:
mutex_lockandmutex_unlocktake the thread as an argument; the HLE functions passhos_ctx_thread(c), andmutex_lock_owed(the end of a condition variable's wait) still passeshos_self().obj_at(host/nn_os_sync.c) isobjfor an HLE function's own context: it tests the thread'svc_kickthroughthe context (
hos_vcore_checkpoint_atinhost/hos.h, a relaxed atomic load, ashos_vcore_checkpoint()reads it:a waiting thread sets it while this one runs) and calls
hos_vcore_checkpoint()only when a better waiter asked forthe core, which reads the kick again as it always has.
vc_kickmoves to theHosThreadhead, after #1956'sHosThreadHeadfields (offset 268,_Static_assert'ed);nothing else moves.
Not switched: the same thread and the same checkpoint, without the thread-local reads and the call.
Stacked: on #1956's head
2a1ad8f2(main@f1f2dc5bwithaca07f9c,7df10ab4,2a1ad8f2) and #1953's four commits at itshead
8443bba1, cherry-picked here asf02113e5,72ede7ee,6f64422bandca2e8c2f(the same lines; #1953 is onmain@6e1521a7, the parent off1f2dc5b). This PR's own commits are the last three:cd2d0384(the change),1af56b97(its test)and
a912c5b8(the kick read atomically). From 4d98f819 (itsnn_os_sync.chalf) onexp/x86-asm-sync.Read first:
obj_at,mutex_lock,mutex_unlockand the two HLE functions inhost/nn_os_sync.c;hos_ctx_threadandhos_vcore_checkpoint_atinhost/hos.h.Testing
Tested: Windows 11, i7-9700, RTX 4060 (NVIDIA 610.88), llvm-mingw clang 23.1.5; game-free checks only. This change
has not run in the game on this PC yet (Still to test). Each check ran in a hidden console of its own, through the PC's
job queue (jobs fb-1957-1 and fb-1957-2), at the head
a912c5b8(onmain@f1f2dc5b, as above), unless it says otherwise:python -I -B tests/test_cv_handoff_threads.py(no--cc:LONGSHOT_MINGW's clang; again withCCset to thatclang): exit 0 both times. It now runs
LockMutexandUnlockMutexthrough the realobj_at,hos_ctx_threadandhos_vcore_checkpoint_atwith a context whose thread is nothos_self()'s record, and its eight new broken variants(either entry through
obj()orobj_nockpt()or takinghos_self(), the kick read fromhos_self()'s record, thelookup before the checkpoint) each fail at their own check. At the test commit
1af56b97(before the atomic load) itpasses as well. With the two entries put back to
obj()andhos_self()it fails (FAIL:1204 ckpts() == calls),while this PR's earlier fixture (at
cd2d0384) passes that code.python -I -B tests/run_all.py --strict --changed origin/main: exit 1, "105 passed, 22 failed, 5 skipped" (skipped,Linux or the game's modules:
nx2/test_leaf.py,test_driver_env_choose,test_mesh_fallback_layout,test_posix_thread_priority,test_tracy_funcs). Passing among them:test_cv_handoff_threads,test_multiwait_wake,test_obj_cache_size,test_guest_os,test_spawn_emit,test_invalidate_vcoreand thetest_vcore_*checks that compile. All 22 failures fail onmain@f1f2dc5btoo (run there by name in the samejob):
test_crash_dialog,test_crash_record,test_dev_ab,test_dev_packs,test_frame_overlap,test_host_guest_writes,test_libc_hooks,test_lift_cache,test_linux_preflight,test_linux_pressure,test_make_game_ngx,test_mods_list,test_oot_shadow_fit,test_perf_report,test_present_queue_default,test_profiling_mode,test_sdiv_overflow,test_slow_frames,test_target_reuse_memory,test_vcore_escape,test_vcore_mask0,test_vk_occlusion_queries.host/nn_os_sync.candhost/hos_thread.ccompiled (syntax only) with the host build's flags, with and withoutTracy, against the
a64.hthis head'snx2/rtgen.pywrites (HosThreadHead; thevc_kickoffset assert): noerror.
Still to test:
and interleaved:
mainwith #1956 and #1953 (this PR's parent) against this PR, andmainagainst this PR; oncewith
HOS_VCORESat its default (on) and once withHOS_VCORES=0. Average fps, 1 % lows, median and p99 frametimes, no stall and the same
vcores:lines and escapes as the parent (by hand)against its parent,
HOS_VCORESat its default and off (by hand)python3 -I -B tests/test_cv_handoff_threads.py, expecting it to pass (by hand)minutes, this PR against
main, expecting the same frame times asmainor better and no stall (the test farm: arequest posted on this PR)
Measurements
Verdict: micro-benchmark only, not measured in play yet: an uncontended
LockMutex+UnlockMutextook 20.48 nson this PR's parent and 19.17 ns with this change, inside the rounds' spread (17.85 to 31.23 against 17.03 to 21.63): no
gain shown in this micro-benchmark. No play measurement exists yet. The figures are of earlier revisions (below) and were
not run again at this head: since then the kick is read with an atomic load (
a912c5b8) and the parents carry theirreview fixes.
Setup: Windows 11, i7-9700, llvm-mingw clang 23.1.5,
-O2 -march=native -ffp-contract=off -fno-strict-aliasingas thehost is built. An uncontended
LockMutex+UnlockMutexthrough the HLE functions themselves, each checkout'snn_os_sync.ccut as itstests/test_cv_handoff_threads.pythen cut it, with that fixture's stubs made as the game hasthem (
hos_self()andhos_vcore_checkpoint()out-of-line calls reading a thread-local, the lookup the checkpoint andthen the object); 21 interleaved rounds of 5,000,000 pairs in one
timing-cpujob of the PC's job queue, the median andthe rounds' range. Measured on
main@51f5f9da, with the parents as they were then.Baseline: this PR's parent 92647c07 (
main@51f5f9dawithperf/os-inlineandperf/mutex-lock-wait-cold), sothe table shows this PR's own commit alone.
Risks
If
Ctx.threadwere not the running thread's record, a lock would be owned by another thread's record and the kick testwould read another thread's request. Every guest context gets it in
setup_guestand a fiber's at each switch, ascurdoes.
tests/test_cv_handoff_threads.pyruns the two HLE functions through the realobj_at,hos_ctx_threadandhos_vcore_checkpoint_atwith a context whose thread is nothos_self()'s record, and fails if either entry takeshos_self()or skips a pending kick. The offset ofvc_kickis asserted. No switch: a revert is enough. Players noticenothing.
Checklist
Ready for review
main; the title is a Conventional Commit.mise run testpasses (any failure that also fails onmainis named).main, or Measurements says why there is nothing to measure.third_party/untouched; nothing GPL-incompatible.When it applies
tests/that reads changed source is updated, not weakened.Assisted-by:/Co-Authored-By:trailers.Ready to merge
main, measured again or a comment says why the numbers still hold.Recommendation: Request changes. The mutex fast path reads the virtual-core kick flag non-atomically, and the stacked lifter change adds unsynchronized reads and writes of thread state. The reported benchmark shows no clear gain, and there are no in-game frame-time results.
LockMutex and UnlockMutex use the calling context for mutex ownership and skip the checkpoint call when no kick is pending. The stack also moves contended waits out of line and inlines four nn::os calls in lifted code. This can save host calls, at the cost of coupling lifted code to the host thread layout, scanning generated C during the build, and losing coverage in runtime HLE counts.
Does it help: Unproven. The author's Windows i7-9700 fixture compared the mutex commit alone against its stacked parent over 21 interleaved rounds. The median fell from 20.48 ns to 19.17 ns per uncontended pair, but the ranges overlapped at 17.85–31.23 ns and 17.03–21.63 ns. The fixture used scheduler stubs. There are no gameplay, contended-wait, clean-build time, or executable-size results for this stack.
Other systems: The scheduler, mutex and CV handoff, and guest worker and fiber paths share the changed thread state. The reported Windows fixtures cover mutex/CV behavior and multiwait wakes, but stub out the new checkpoint. Linux and Steam Deck scheduling, Tier 1 frame pacing, and the movie decoder's contended load path still need coverage. The standalone lifter changes both forked and spawned emission and affects build time, the lift cache, and HLE profiling. No direct changes to the renderer API, GPU passes, saves, audio, or input were found.
Switches: The mutex commit adds no switch. The stack adds A64_OS_INLINE, on by default, with 0 restoring HLE calls. The build switch table documents it and the existing A64_ lift-key rule covers it. Gameplay and clean-build A/B results are missing for the default. HOS_HLE_STATS remains off by default, but enabling it on a default build omits inlined calls. HOS_VCORES remains on by default.
Reviewed:
0747cefa46against2e1f6c32d3: Read AGENTS.md, CONTRIBUTING.md, docs/dev/standards.md, the Forgejo template, all three commit messages, and all nine changed-file diffs. Read nx2/guest_os.py, nx2/lift.py, tests/test_guest_os.py, and docs/dev/reference/build.md in full. Read the changed sections and traced their dependencies in host/hos.h, host/hos_thread.c, host/nn_os_sync.c, nx2/chunks.py, and tests/test_cv_handoff_threads.py. Unrelated sections of those larger files, downstream build passes, and existing test records received targeted reads. No local docs/review-checklist.md, sibling Tooling checkout, or changed generated or translation files were present. Run evidence came from the PR description.; nothing run (a code review).Checked and fine:
Blocking (to change before merge)
Waiting on the author (a run, a measurement or a decision; no edit does it)
Suggestions (the author may take or leave)
host/nn_os_sync.c:645: Two first-time waiters can race on trace_ms. Set it before guest threads start or publish it atomically.host/nn_os_sync.c:664: The owner can change count while this log reads it, and it can be destroyed before name lookup here or in the final trace. Omit the racy count and use a lifetime-safe identifier or snapshot.Testing: Read-only review. The author reports Windows mutex/CV and multiwait fixture passes at af867cb6. The earlier changed-check batch reported 81 passed, 14 failed, and 3 skipped, with the same 14 failures on its then-base. Exact-head CI, lint and supported Windows/Linux build results are missing.
Overlaps and merge order: #1956 supplies the HosThread layout and lifter changes. #1953 supplies the cold mutex helper. Land the dependencies before #1957, then rebase the mutex commit onto current main with both present and verify the final diff and results. The checkout lists #1956 followed by the cherry-picked #1953.
Automated code review by BeanBot (gpt-6.2-sol), on the pull request's opening. Nothing was run; a maintainer decides.
@ -70,6 +70,7 @@ Read by the build, not the game, so they are not in `switches.def`.| `A64_FASTFP`, `A64_LOCAL_REGS`, `A64_LOCAL_FLAGS`, `A64_LOCAL_FUNCS` | Lift-time switches (`nx2/fastfp.py`, `nx2/guest_locals.py`); they change the lift key || `A64_FASTFP_STORES=0`, `A64_FASTFP_VMOVES=0` | Fast float regions (`nx2/fastfp.py`) stop at a guest store, or at a vector move or shuffle (`nx2/fastvec.py`'s `a64_vld16`, `a64_vdup`, `A64_EXT16`, `A64_EXT8`, `a64_tbl1`), as before. Both on by default: a region may hold stores but no load after one, and its fast copy keeps their addresses and values and stores them, in order, once its MXCSR check passed (its exact copy is unchanged). Lift-time switches: they change the lift key || `A64_VECFP=0` or a list of `elem`, `vec`, `minmax`, `fcvt` | The SSE fast paths for the float forms the translator otherwise does lane by lane (`nx2/vecfp.py`); each keeps a result only where it is provably Arm's, else the instruction's own C runs. All on by default; `0` lifts exactly as without them. A lift-time switch: it changes the lift key || `A64_OS_INLINE=0` | nn::os::GetCurrentThread, GetTlsValue, SetTlsValue and GetThreadPriority called as HLE functions, as before. On by default: each direct call (or tail call) of their hooked definitions is the field read or write itself, through the context's thread pointer (`nx2/guest_os.py`; the thread record's head is fixed and asserted in `host/hos_thread.c`); GetThreadPriority so only about the calling thread's own ThreadType, a call as before about another. The same x0, other registers and records as the HLE functions (`tests/test_guest_os.py`), no checkpoint skipped (they had none); calls through a pointer still reach the HLE functions, and `HOS_HLE_STATS=1` does not count the inlined ones. A lift-time switch: it changes the lift key |Preserve complete runtime HLE profiling, or document the required A64_OS_INLINE=0 rebuild in the profiling reference and HOS_HLE_STATS switch entry if maintainers accept that workflow.
@ -212,0 +217,4 @@static inline HosThread *hos_ctx_thread(const Ctx *c) { return (HosThread *)c->thread; }static inline void hos_vcore_checkpoint_at(const Ctx *c) {const char *t = (const char *)c->thread;if (__builtin_expect(t && *(const volatile int *)(t + HOS_THREAD_VC_KICK), 0)) hos_vcore_checkpoint();Use a relaxed atomic load here. A waiting worker can atomically set this flag while the holder is in LockMutex, and volatile does not make the accesses atomic.
@ -0,0 +33,4 @@"_ZN2nn2os16GetCurrentThreadEv": "{ c->x[0] = ((const uint64_t *)c->thread)[0]; }","_ZN2nn2os11GetTlsValueENS0_7TlsSlotE":"{ const uint32_t _slot = (uint32_t)c->x[0]; c->x[0] = _slot < %du ? ((const uint64_t *)c->thread)[1 + _slot] : 0; }"% SLOTS,A same-core affinity change can write this TLS slot while its owner runs. Make generated, HLE, and scheduler accesses consistently atomic.
@ -0,0 +35,4 @@"{ const uint32_t _slot = (uint32_t)c->x[0]; c->x[0] = _slot < %du ? ((const uint64_t *)c->thread)[1 + _slot] : 0; }"% SLOTS,"_ZN2nn2os11SetTlsValueENS0_7TlsSlotEm":"{ const uint32_t _slot = (uint32_t)c->x[0]; if (_slot < %du) ((uint64_t *)c->thread)[1 + _slot] = c->x[1]; }" % SLOTS,The cast points at the scalar guest_tt member. Index from the actual tls_slots array through a shared type or its asserted byte offset.
@ -0,0 +38,4 @@"{ const uint32_t _slot = (uint32_t)c->x[0]; if (_slot < %du) ((uint64_t *)c->thread)[1 + _slot] = c->x[1]; }" % SLOTS,"_ZN2nn2os17GetThreadPriorityEPKNS0_10ThreadTypeE":"{ const uint8_t *_th = (const uint8_t *)c->thread; if (c->x[0] == *(const uint64_t *)_th) ""c->x[0] = (uint32_t)*(const int32_t *)(_th + %d); else @F@(c); }" % PRIORITY_AT,Another thread can change this priority through HLE or SVC. Use atomic access in the generated load and all runtime readers and writers so the compiler cannot reuse a stale value across calls.
@ -0,0 +87,4 @@text, refs = base(self, e, texts)table = getattr(self, "os_inline", None)if table:text, n = inline(text, table)This scans every emitted function and calls Python for unrelated direct calls. Limit the pass to the configured targets or perform the substitution as the lifter emits those calls.
@ -224,6 +228,7 @@ static Obj *obj(uint64_t key, int kind) {CHECK(o->kind == kind);return o;}static Obj *obj_at(const Ctx *c, uint64_t key, int kind) { (void)c; return obj(key, kind); } /* (LockMutex, UnlockMutex) */This stub skips the code under review. Use the real checkpoint and lookup wrapper and make the test fail if either mutex entry skips a pending kick or uses hos_self() instead of the context's owner.
@ -0,0 +29,4 @@import kit # noqa: E402,F401 (a64-kit's folders on the path)import guest_os # noqa: E402RUN_ALL_ARGS = ["--cc", "gcc"]A Clang-only machine with CC=clang can fail strict mode before this test runs. Honor CC in both the test runner and direct invocation, including the CV fixture.
@ -0,0 +83,4 @@print("PASS: the lift's rewrite inlines the hooked calls and tail calls only, and A64_OS_INLINE=0 keeps them")def wiring():The source-string check and direct READS fixture still succeed if the real emitter drops the rewritten result. Emit and execute a synthetic caller through the real lifter, including spawned emission.
Recommendation: Request changes. The new mutex checkpoint reads
vc_kickthrough a volatile load while the scheduler stores to the same flag atomically. On a busy core, a waiting worker can request the holder's core while the holder enters either mutex function. Use the same relaxed atomic load ashos_vcore_checkpoint()before making the context-based path the default. The CV fixture stubs out the affected lookup and checkpoint, so it cannot catch that regression.I independently verified the scheduler path in the existing exact-head review. The waiter publishes a kick in
host/hos_thread.c:2089, and the old checkpoint reads it atomically at:2219.host/hos.h:220is the new unsynchronized reader. A focused fixture should use the real helper and both mutex HLE entries, checking that a set kick is processed before the lookup and that the caller's context supplies mutex ownership.Does it help? Unproven. The author's 21 interleaved fixture rounds on an i7-9700 gave 20.48 ns before and 19.17 ns after for an uncontended pair. The ranges overlap at 17.85–31.23 ns and 17.03–21.63 ns. That fixture compared earlier revisions of the final commit and its stacked parent. It does not give a gameplay, contended-wait or clean-build result. The Windows CV/mutex and multiwait passes were reported on the previous rebased head. The broader batch had 81 passes, 14 failures also reported on its main baseline, and three skips. Cached exact-head CI shows the description check passed, ten contexts pending and one skipped.
Checked and fine. Main and worker startup set
Ctx.threadbefore lifted code runs, and a fiber switch updates it before transfer. Condition-variable reacquisition still uses the current host thread. The mutex changes retain the recursive owner check, reservation-aware acquisition and release, and the object-cache flush after lookup. The lifted variants retain context register synchronization around calls and fall back to the existing HLE for another thread's priority. The standalone hook selection and spawn table are wired, and the lift key includes the build switch. No game artifacts, generated shader edits, renderer API changes or vendor modifications are in the diff.Dependencies and switches. The cold helper in
e8f21d1fmatches the patch in PR 1953, including its outstanding diagnostic initialization and owner-lifetime issues. PR 1956 is open WIP and is carried at its exact headb922eadf4a99dcb7d202770fecb6c2529d85035c. It enablesA64_OS_INLINEat lift time and still needs the atomic priority and TLS accesses, valid TLS array addressing, and real emitter coverage described in the exact-head prerequisite review. It also changes whatHOS_HLE_STATScounts on a default build. Resolve and measure both dependencies before rebasing the final mutex commit. That commit adds no switch.Testing needed. After source fixes and the prerequisites land, compare current main containing the prerequisites with the revised mutex head. Use warmed interleaved Market and heavy-scene gameplay, plus a route with contended workers. Check stalls, wait tails and virtual-core escapes with
HOS_VCORESat its default and off. Report average FPS, 1% lows, and median and p99 frame times. Game-free scheduler/CV checks and supported Windows and Linux builds should run on the revision being measured. Measure clean build time and game size separately for the lifter dependency and run its gameplay comparison with each build-time setting.Reviewed
0747cefa46b33c70b84f972ed1728d60c4fd3a63against merge base2e1f6c32d30ec3582093743a968c8447bb435962. Read all nine changed-file diffs, the surrounding scheduler, fiber, object-cache, lifter and check-runner code, and the cached Forgejo evidence. Used revision-matched local Graphify maps for navigation and checked the relevant source directly, including macro-generated HLE entries and the vendored emitter. No PR code was run. Guidance at Longshot6e1521a7, Tooling0b0059ac.0747cefa46a912c5b8e9Point by point for BeanBot's review (62205), at
a912c5b8. The branch is rebased onto #1956's head2a1ad8f2(onmain@f1f2dc5b) with #1953's four commits at its head8443bba1cherry-picked on top (f02113e5,72ede7ee,6f64422b,ca2e8c2f;git range-diffshows them unchanged), so it still depends on #1956 and #1953 and carries their review fixes. This PR's commit0747cefais nowcd2d0384. It had two conflicts, both kept from both sides:HosThread's head comment and asserts (#1956 now asserts againstHosThreadHead; thevc_kickassert stays), and the CV fixture's list of cut functions (#1953 cutsmutex_lock_waitfrom its name). The fixes for this commit's points are1af56b97(test) anda912c5b8(fix). The description is updated.This PR's own commit:
vc_kickread throughvolatile: fixed ina912c5b8.hos_vcore_checkpoint_atreads it with__atomic_load_n(..., __ATOMIC_RELAXED), ashos_vcore_checkpointdoes. It has no failing test of its own: the two loads behave the same in a run on this PC, where the difference is the C memory model's, and llvm-mingw has no thread sanitizer for Windows. The cases below run the changed function.1af56b97.tests/test_cv_handoff_threads.pynow cutsobjandobj_atfromhost/nn_os_sync.c, andhos_ctx_thread,hos_vcore_checkpoint_atandHOS_THREAD_VC_KICKfromhost/hos.h. Only the lookup itself andhos_thread.c'shos_vcore_checkpointare the fixture's; the latter counts its calls, numbered in one sequence with the lookups. A new group runs both entries with a context whose thread is a record other thanhos_self()'s. The mutex must be owned and let go by that record. No checkpoint may be called with no kick, or with a kick only onhos_self()'s record; with a kick on the context's record, each entry must call it once, before its lookup. Eight broken variants must each fail at their own check: either entry throughobj()orobj_nockpt(), either entry takinghos_self(), the kick read fromhos_self()'s record, and the lookup before the checkpoint. In job fb-1957-2 the fixture passes at the test commit and at the head. With the two entries put back toobj()andhos_self()it fails (ckpts() == calls), and the earlier fixture (atcd2d0384) passes that code.From #1956's commit (fixed on #1956; the rebase brings the fixes here; its reply):
2a1ad8f2. Priority reads are atomic in the lifted code,hos_thread_prioand every scheduler and diagnostic reader; the setter is an atomic exchange.2a1ad8f2. Slot accesses are atomic (relaxed) in the lifted code,GetTlsValue/SetTlsValue,vc_pref_coreandvc_sync_tls. The slot indexcore_tls_slotstays avolatileread as onmain(#1956's description says so).tls_slotscast: fixed in2a1ad8f2. The head is a shared type,HosThreadHead, in the generateda64.h; the lifted code indexes itstls_slotsarray andhos_thread.casserts the offsets.7df10ab4,tests/test_spawn_emit.pytest_os_inline: the real emit in one process and spawned, with the switch on and off.CC: fixed in7df10ab4fortests/test_guest_os.py, and for the CV fixture in #1953'sd3e02c70(72ede7eehere):--cc, thenCC, thenLONGSHOT_MINGW's clang, then clang, gcc or cc. The CV fixture passes at this head with no--cc(LONGSHOT_MINGW's clang) and withCCset (fb-1957-2).2a1ad8f2. A function whose refs name none of the hooked definitions is not scanned.HOS_HLE_STATS: not preserved, marked and documented in2a1ad8f2. #1956's reason: counting at each inlined site would add back the check the change removes.scripts/perf.py cpu --callssays the four counts are partial, andbuild.mdandprofiling.mdsay to lift withA64_OS_INLINE=0for full counts.From #1953's commit (fixed on #1953; its reply):
trace_ms: fixed in18e90465(6f64422bhere).HOS_MUTEX_TRACEis cached through an atomic and parsed over the whole string (1 to 2147483647 ms, anything else off).m->countand the holder's lifetime: fixed in18e90465. The count is gone from the 10 s log, and the holder's name is copied underm->lkwhile the holder cannot end; the final trace line uses that copy. Not fixed there: the rename race (hos_thread_set_nameagainst every name reader) is inhos_thread.c, pre-existing onmain, and left for a separate change.Waiting items: pending on my side, nothing measured yet. Interleaved play runs of this head against current
mainand against its parent (both dependencies), Market and a heavy scene; contended waits, stalls and escapes withHOS_VCORESat its default and off, on Windows, Linux and the Steam Deck; these are in Still to test. The lifter's clean build time and size andA64_OS_INLINEon/off are #1956's and pending there.Suggestions: sharing
obj's andobj_at's lookup and flush is not taken; the shared part is two lines, and the fixture now cuts both. The micro-benchmark clock suggestion is about #1956's benchmark and was not taken there.@futondestroyer, thanks for the review (62897). Pushed at
a912c5b8: the test is1af56b97and the fixa912c5b8, on top of this PR's commit (0747cefa, nowcd2d0384).vc_kickload. Fixed ina912c5b8:hos_vcore_checkpoint_atreads the kick with__atomic_load_n(..., __ATOMIC_RELAXED), the same load ashos_vcore_checkpoint().1af56b97. The fixture now runs the realobj_at,hos_ctx_threadandhos_vcore_checkpoint_at(cut from the sources) and both mutex entries with a context whose thread is a record other thanhos_self()'s. Each entry must call the checkpoint once for a kick set on the context's thread, before its lookup, and not at all with no kick or a kick only onhos_self()'s record. The mutex must be owned and released by the context's thread. Eight broken variants (either entry throughobj(),obj_nockpt()orhos_self(), the kick read fromhos_self(), the lookup first) each fail at their own check. With the entries put back toobj()andhos_self()the new fixture fails, and the earlier one passes that code.2a1ad8f2(onmain@f1f2dc5b) with #1953's four commits at its head8443bba1cherry-picked on top, unchanged bygit range-diff. So it carries the fixes your reviews of them asked for:HosThreadHead'stls_slotsarray, atomic priority and TLS accesses and the real-emitter checks (#1956's7df10ab4and2a1ad8f2; theHOS_HLE_STATScounts are marked partial and documented there, not preserved), and #1953'sHOS_MUTEX_TRACEinitialisation and holder lifetime (18e90465, here6f64422b). Two conflicts with them, inHosThread's head comment and asserts and in the fixture's list of cut functions, were resolved by keeping both sides. Neither dependency is measured yet. I rebased now rather than after they land so that this branch runs their fixed code. It will be rebased again when they merge.mainand against the parent with both dependencies, a contended-worker route withHOS_VCORESat its default and off, average fps, 1 % lows, median and p99. The game-free checks below ran at this head on Windows; Linux has not run yet.Checks at
a912c5b8(Windows 11, i7-9700, llvm-mingw clang 23.1.5, this PC's job queue, fb-1957-1 and fb-1957-2):tests/test_cv_handoff_threads.py: passes, with and withoutCC; at the test commit1af56b97too.tests/run_all.py --strict --changed origin/main: 105 passed, 22 failed, 5 skipped; all 22 also fail onmain@f1f2dc5b, run by name in the same job.host/nn_os_sync.candhost/hos_thread.ccompile (syntax only) with the host build's flags, with and without Tracy, against thea64.hthis head generates.@Beanstalk This makes the guest LockMutex and UnlockMutex take the calling thread and the virtual-core checkpoint's test from their context (host/nn_os_sync.c, host/hos.h). It is stacked on #1956 and #1953 (their commits are in its diff), so its build lifts the game again. No switch of its own: a revert gives back main's code. It has not run in the game on these PCs yet. Please play Castle Town Market for two minutes, the PR against main on the same PC, on a Windows PC with an AMD processor, a Linux PC with an AMD processor, a PC with 6 or fewer logical processors and the Steam Deck. No hang or stall; frame time p99 equal to main's or better, within the run-to-run spread.
@siantm only members of
BigBeanscan start the farm. One of them can approve this request by reacting 👍 to it or to this comment.View command line instructions
Checkout
From your project repository, check out a new branch and test the changes.