WIP: perf(os): virtual cores taken and released without vc_lock while no thread waits for one, each owner slot on its own cache line (HOS_VCORE_FAST, off) #1972
No reviewers
Labels
No labels
area/accessibility
area/audio
area/input
area/install
area/menu
area/renderer
area/saves
area/tooling
area/upscaling
beanstalk: approved
beanstalk: failed
beanstalk: needs approval
beanstalk: running
beanstalk: tested
ci: approved
good first issue
gpu/amd
gpu/igpu
gpu/intel
gpu/nvidia
help wanted
kind
bug
kind
chore
kind
docs
kind
feature
kind
performance
kind
question
needs measurements
platform/linux
platform/macos
platform/steam-deck
platform/windows
preview
priority
critical
priority
high
priority
low
regression
risk
high
risk
low
risk
medium
status
blocked
status
confirmed
status
duplicate
status
invalid
status
needs-info
status
wontfix
ci: approved
No milestone
No project
No assignees
4 participants
Notifications
Due date
No due date set.
Dependencies
No dependencies set
Reference
MagicBeanDev/Longshot!1972
Loading…
Reference in a new issue
No description provided.
Delete branch "siantm/Longshot:perf/vcore-fast"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Summary
Every take and every release of a virtual core goes through one lock,
vc_lock(host/hos_thread.c), also when nothread waits for a core: a guest thread takes its core after every block and gives it up at the next one, tens of
thousands of times a second, on all of the game's threads at once.
This adds an experiment, off by default, in two steps (one commit each; the second was #1973, folded in here as the
review asked):
HOS_VCORE_FAST=1: a take (vc_take_fast) and a release (vc_release,vc_free_slot) take no lock while nobodyis queued for a core. What holds without the lock (the comment above
vc_fastgives the whole argument):vc_nqand then looks at the cores again (vc_settle); a release frees the slot andthen reads
vc_nq: one of the two sees the other, so no waiter sleeps beside a free core;vc_nqagain after it, and gives the core to a waiter that came in between;(
vc_aff), asvc_free_fordoes. After its take it reads both again and writes the guest's TLS core cell in thesame
tls_slot_locksection (vc_fast_check); if either changed it gives the core up and chooses again. A maskchange stores the mask and then reads
vc_heldunder the lock (vc_note_mask_change); the fast take storesvc_heldand then reads the mask, a fence in each, so one of the two sees the other;DestroyThreadpassesvc_lockonce (vc_exited) before a thread'srecord is freed.
vc_owner[8]becomes eight 64-byte-aligned slots,.t): withstep 1 alone every take and release on one core takes the line of the eight slots from the holders of the others.
Round 1 of review (#1972 62911/62912, #1973 62913/62914, BeanBot 62391/62404) changed:
that core into the guest's cell over the cached one (the review's remask back to mask 0);
vc_last, the hold and kick times andvc_kickare read and written as atomics where anotherthread reads them;
hos_thread_corereadsvc_heldonce;vc_sync_tls_asreads the mask undertls_slot_lock;with the switch on the cached core is read under that lock, its slot validated there;
adds its last ones;
main'sequal-priority scan did;
core freed without the lock before the release that freed it takes the lock gets it as "found free", without the
releasing thread in its long-wait line (said in the comment above
vc_fast).With
HOS_VCORE_FASTunset, what changes againstmain: a test of the switch at each take, release and slot store(the stores themselves are
main's: relaxed atomics, the same instructions asmain's plain stores on x86-64, andvc_nq's release store; no fence invc_note_mask_change; the take count ismain'svc_st.takes++under thelock); the owner slots take 512 bytes instead of 64; the wait loop's scans read fewer slots;
vc_sync_tls_asreads themask inside its lock section; the 5 s line does one atomic exchange of the (then zero) fast-take counter. A mask-0
thread's cached core is read as on
main(withouttls_slot_lock) while the switch is unset.The switch is read once and said once in the log; its row is in
gpu/vk/switches.def(kindattempt). LeavingHOS_VCORE_FASTunset turns the fast path off.From d1e33769 (the host-sync integration) and cafb4100 (its
host/hos_thread.chalf) onexp/x86-asm-sync, where itwas on by default and never measured; rewritten over #327 (
a104182b).tests/test_vcore_escape.py's fixture declareshos_fps_timingtwice onmain(#1860, still open, fixes the sameline; whichever lands second rebases) and
tests/test_vcore_mask0.py's fixture has noenv_on_or: both are fixedhere.
tests/test_slow_frames.pylooked forvc_grant(c, t)invc_release, whichmainalready moved intovc_release_locked: it now checks both release paths.Read first: the comment above
vc_fast,vc_take_fast,vc_fast_check,vc_free_slot,vc_settleandvc_note_mask_changeinhost/hos_thread.c.Testing
Tested: Windows 11, i7-9700, RTX 4060 (NVIDIA 610.88), llvm-mingw clang 23.1.5; game-free checks and a
micro-benchmark only. This change has not run in the game on this PC yet (Still to test). Each check ran in a hidden
console of its own, through the PC's job queue (job fb3-1972-1), on this branch on
main@f1f2dc5b(headc9d646dc):python -I -B tests/test_vcore_mask0.py --cxx clang++: exit 0, "the defaults: ok", "HOS_VCORE_FAST=1: ok". At thetest commit
570f35da(before the fix) it fails: "the defaults: FAIL: a take's write of the cell after a same-coreremask's: the cell names the core (one-core mask)" (the unset run fails first, so its HOS_VCORE_FAST=1 run does not
start there). With
vc_fast_check's comparison taken out of this head's source (--source, a mutated copy), theHOS_VCORE_FAST=1 run fails on both remask cases, the review's remask back to mask 0 among them.
python -I -B tests/test_vcore_escape.py --cxx clang++: exit 0, "the defaults: ok", "the defaults,HOS_VCORE_STARVE_US=0: ok", "HOS_VCORE_FAST=1: ok", "HOS_VCORE_FAST=1, HOS_VCORE_STARVE_US=0: ok". On mutated copies
of this head's source its HOS_VCORE_FAST=1 run fails each time: the fast take's compare-and-swap made a store ("two
takers for core 0: the first to take it holds it"), its second look at the queue taken out ("a waiter that queued as
a take without vc_lock took its core is given that core at once"),
vc_exitedwithout its pass ofvc_lock, and adestroyed thread's last takes not added ("the 5 s line counts every take, a destroyed thread's last ones too").
python -I -B tests/test_slow_frames.py: exit 0, "slow frames: ok".python -I -B tests/test_switch_table.py --cxx clang++: exit 0, "ok: 735 switches (diag 295, tune 109, kill 129,attempt 81, hw 67, player 54), 144 removed, 743 names read".
python -I -B tests/run_all.py --strict --changed origin/main: exit 1, "44 passed, 11 failed, 4 skipped".test_vcore_escape,test_vcore_mask0,test_slow_framesandtest_switch_tablepass. 8 of the 11 failures fail onmain@f1f2dc5btoo (run by name in the same job, "6 passed, 11 failed"):test_crash_dialog,test_crash_record,test_dev_ab,test_dev_packs,test_frame_overlap,test_linux_pressure,test_profiling_mode,test_walk_key.The other three (
test_present_queue_default,test_target_reuse_memory,test_vk_occlusion_queries) compilednothing in that run nor in the run on
main(no Vulkan headers on the job's path: "not found: vulkan_headers(--strict)"). Rerun with the headers (job fb3-vk-1,
RX_CPMset) at the headc9d646dcand onmain@f1f2dc5b, eachfails at both with the same first error:
test_present_queue_default"srccut: '{TAB_DEBUG, "Present queue",&opts.present_queue, ...' is not in gpu/vk/vk_menu.cpp";
test_target_reuse_memory2 of its 8 tests("'vmaGetAllocationInfo2' unexpectedly found" in the VMA source, and its fixture's "use of undeclared identifier
'destroy_image_later'");
test_vk_occlusion_queries"the ring fixture did not compile" ("no member named 'policy'").None fails on this branch alone.
Still to test:
mainagainst this PR withHOS_VCORE_FASTunset and withHOS_VCORE_FAST=1, warmed and interleaved, on the same scripted route through Castle Town's market, a busy outdoorroute, and a route with queued waits for a core; average fps, 1 % lows, median and p99 frame times, and the "vcores:"
lines' waits, escapes and kicks, expecting the log line "vcores: takes and releases without vc_lock while nobody
waits (HOS_VCORE_FAST)" only with it set, no "ran without a core" line, no crash (by hand)
python3 -I -B tests/test_vcore_escape.pyandtests/test_vcore_mask0.py, expectingboth to pass with the switch off and on (by hand)
frame times within the runs' spread or better with the switch set and no "ran without a core" line (the Test Farm:
Castle Town's market for two minutes on PCs with 6 or fewer logical processors and on the Steam Deck,
mainagainstthis PR with HOS_VCORE_FAST unset and set, comparing frame times and looking for "ran without a core" in the log)
Measurements
Verdict: micro-benchmarks only; nothing has been measured in play.
HOS_VCORE_FASTships off (kindattempt).In the micro-benchmark, step 1 alone made a take and a release slower than the locked path (a loss); both steps
together made them faster. At this head, with the switch set, a take and a release cost 54 to 78 % less than on
main;with it unset, no difference from
mainwas measured (the ranges overlap; the medians are 1 to 10 % higher) (below).All micro-benchmarks: Windows 11, i7-9700 (8 cores, no SMT), llvm-mingw clang++ 23.1.5,
-O2;host/hos_thread.c'svcore block compiled with Windows' own SRW locks, 2, 4 and 6 threads each taking and giving up its own core
(
hos_vcore_back,hos_vcore_away) with 8 virtual cores, so nobody waits and the wait loop never runs; 15interleaved rounds of 400 ms, the median and the rounds' range, ns per take+release per thread; one timing job of the
PC's queue each.
Before this round's fixes (step 1 at 15e55f96 and both steps at 026a1abc, on
main@51f5f9da; locked: step 1 with theswitch unset):
Step 1 alone loses because the eight owner slots share one cache line, which every take and release on every core then
writes; it is not to be turned on without step 2, and here goes in only with it.
This head (
c9d646dc) againstmain@f1f2dc5b, the same micro-benchmark (one timing job of the PC's queue,fb3-1972-1-bench):main@f1f2dc5bc9d646dcc9d646dcWith the switch unset the ranges overlap
main's at all three thread counts: no difference measured, though everymedian is higher (1 to 10 %). With it set the ranges do not overlap
main's.The micro-benchmark measures a take and a release with nobody waiting, not a frame: what it saves or costs in play is
still to test (above), as is the wait loop's change, which it never reaches.
Risks
Off, a take and a release test a switch read once, their stores are
main's, and the owner slots are 64 bytes apart:Summary lists what else differs from
main. On, the scheduler's core hand-over runs without its lock while nobodywaits: a mistake would show as a lost grant (a thread waiting for a core beside a free one, until its 1 ms look finds
it, or an escape after 2 s), two threads on one core, or a stale guest core number. Turn it off by leaving
HOS_VCORE_FASTunset; a revert is enough. Players notice nothing: the switch is not set for them.Checklist
Ready for review
main; the title is a Conventional Commit.mise run testpasses (any failure that also fails onmainis named).main, or Measurements says why there is nothing to measure.third_party/untouched; nothing GPL-incompatible.When it applies
gpu/vk/switches.def.tests/that reads changed source is updated, not weakened.Assisted-by:/Co-Authored-By:trailers.Ready to merge
main, measured again or a comment says why the numbers still hold.Recommendation: Request changes. The fast path races with affinity changes, TLS updates and scheduler diagnostics. The experiment also changes the default hot path, whose cost was never measured against main.
Adds an opt-in virtual-core scheduler path that skips vc_lock on uncontended takes and releases. It preserves the common handoffs in the reported fixtures, but adds a second ownership protocol, more synchronization on the default path and new races when enabled.
@fixwill do (14)host/hos_thread.c:2117: The mask used for validation is separate from the mask used to choose a core. (more on the line)host/hos_thread.c:168: hos_thread_core reads vc_held twice with plain loads while the target can clear it atomically. A remote GetThreadCoreMask or pin_by_core can race a release. If the test sees a held core and the subtraction sees zero, it returns -1 and pin_by_core can shift by a negative count. Take one atomic snapshot here and audit the other cross-thread readers of vc_held.host/hos_thread.c:1932: The fast taker samples the mask before acquiring tls_slot_lock. (more on the line)host/hos_thread.c:1883: The fast mask-0 chooser reads the cached cell outside the lock used by vc_sync_tls_as. A same-core remask can pass its held check, then the target can release and start another take before the remask writes the cell. The preference read and pending write can overlap. vc_pref_core also reads vc_last while the fast taker can update it. Synchronize cache reads with writes and slot reuse, and use a synchronized snapshot for vc_last.host/hos_thread.c:2129: The owner CAS exposes t before the fast taker initializes vc_held_at and vc_kick_at. (more on the line)host/hos_thread.c:2039: vc_stats_locked reads takes normally and clears it with memset while fast takers can atomically add to it. A 64-take flush during the log call races the reset and can disappear. Keep takes outside the bulk reset and use an atomic exchange to capture the interval count. Flush each thread's final partial batch on exit so short-lived threads are counted.host/hos_thread.c:2299: A fast release can free a slot and then lose the lock to a timed-out waiter. (more on the line)tests/test_slow_frames.py:127: The source check still requires vc_grant(c, t) inside vc_release. That call exists at the merge base and is gone at HEAD. Even if the author's base run failed elsewhere, this assertion now fails on its own. Update it to cover attribution in both release paths, preferably by checking the resulting diagnostic behavior.tests/test_vcore_escape.py:312: The release hook runs after the waiter has queued and entered sleep. (more on the line)tests/test_vcore_mask0.py:270: The remask hook writes the field directly and calls vc_note_mask_change on the taker's host thread. (more on the line)tests/test_vcore_mask0.py:289: Both revised fixtures default to clang++ when called directly. A Linux machine with only g++ fails the documented standalone command even with CXX=g++ set. Keep --cxx as the explicit override and use CXX and the runner's compiler fallback for direct invocation in both files.host/hos_thread.c:2123: Put owner slots on separate cache lines in the same patch and measure that revision against main.host/hos_thread.c:2166: Keep the existing synchronization and counter increments on the locked path where possible. Measure the remaining differences against the merge base.For the author (2) (a run, a measurement or a decision)
host/hos_thread.c:2123: The enabled standalone path loses in its own target workload. (more on the line)host/hos_thread.c:2166: The benchmark calls the unset branch the main path, but both columns include the PR's sequentially consistent owner and held stores, atomic take counter and switch checks. (more on the line)Checks: Still running:
CI / Conventional Commit title (pull_request),CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request),CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request),CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request),CI / Workflow lint (actionlint) (pull_request),CI / Website build and checks (pull_request),Guide screenshots / Guide images and local UI source review (pull_request),Third-party parts / As their notes say (pull_request),Third-party parts / The same as upstream (pull_request),Website / Website on Cloudflare (pull_request).More: does it help, other systems, switches, what was read
Does it help: Unproven in play. The PR reports 15 interleaved 400 ms rounds on a Windows i7-9700 at 15e55f96. With 2, 4 and 6 threads, enabling the switch raised median take-plus-release time from 170.1, 366.5 and 587.0 ns to 218.5, 436.3 and 633.7 ns. The 4-thread ranges are separate, while the 2- and 6-thread ranges overlap. The separate owner-slot branch reports 56.0, 59.0 and 67.2 ns. Both sides of the standalone benchmark use the PR code, so it misses the default-path cost against main. No gameplay FPS, 1% low or frame-time results were supplied.
Other systems: The host scheduler runs on Windows and Linux across GPU vendors and Steam Deck. Guest synchronization, filesystem and video waits, and GPU waits all release and retake cores through this code. The switch can change their wake-up latency and frame pacing. Virtual-core counters and release attribution feed profiling and slow-frame diagnostics. No renderer API, save format, cache format, audio or input code changed.
Switches: HOS_VCORE_FAST is a developer-only attempt switch, read at startup and registered in switches.def and the developer reference. Unset or false keeps the locked scheduler. Off is the right default for the standalone experiment given the benchmark, though its default path still needs comparison with main.
Reviewed:
e6acd10c72against6e1521a70a: Read AGENTS.md, CONTRIBUTING.md, docs/dev/standards.md, the Forgejo PR template, both revised test fixtures in full, and all changed host functions. Traced scheduler, affinity, TLS, thread startup and teardown, Linux lock and wait implementations, and affected source checks. Reviewed the changed switch rows and nearby rules, and skimmed other callers, profiling consumers and prior mask-0 run summaries. No local review-checklist file or Tooling checkout was present. Unrelated host sections and the full switch catalog were outside the review.; nothing run (a code review).Checked and fine:
Testing: Read-only review. The author reports Windows Clang fixture passes from a line-identical rebase and a changed-file suite with 44 passed, 11 failed and 4 skipped. The affected slow-frame assertion differs in outcome at source level and is called out above. The full suite and lint results, Linux GCC and Clang runs, and Tier 1 gameplay runs on current code are still needed. The supplied head CI checks were running.
Overlaps and merge order: #1860 changes the same duplicate fixture declaration. Keep one fix and rebase whichever lands second. Group perf/vcore-owner-lines (026a1abc) into this PR before measuring or merging the experiment.
Automated code review by BeanBot (gpt-6.2-sol), on the pull request's opening. Nothing was run; a maintainer decides.
@ -1890,1 +1929,4 @@did under vc_lock: with HOS_VCORE_FAST, t may have let c go and taken another core without the lock since; that takestores vc_held before it writes the cell, and a release clears vc_held before it frees the slot) */static void vc_sync_tls_as(HosThread *t, int c, int held_only) {const uint32_t mask = vc_mask(t);L3: The fast taker samples the mask before acquiring tls_slot_lock. Pause it there, change a single-core mask to a multi-core mask that includes the held core, and let the setter clear the TLS cell. The taker can then acquire the lock and overwrite zero with core + 1 from its old snapshot. Sample the current mask under the TLS lock and use the same synchronization for the held-core validation and cell update.
@ -2026,0 +2114,4 @@static int vc_take_fast(HosThread *t) {if (__atomic_load_n(&vc_nq, __ATOMIC_SEQ_CST)) return 0;if (t->vc_kick) __atomic_store_n(&t->vc_kick, 0, __ATOMIC_SEQ_CST);const uint64_t mask = __atomic_load_n(&t->core_mask, __ATOMIC_RELAXED);L1: The mask used for validation is separate from the mask used to choose a core. A taker can snapshot core 0's mask, choose core 1 while the mask temporarily allows only core 1, and observe the original mask after the CAS. Both setters can see no held core, so neither side requests a move. The chooser also uses plain core_mask and ideal_core reads while affinity can change them. Choose from one synchronized affinity snapshot and validate the actual selected core against the current affinity after publishing the claim.
@ -2026,0 +2120,4 @@const int k = vc_free_for(t);HosThread *none = NULL;if (k < 0) return 0;if (__atomic_compare_exchange_n(&vc_owner[k], &none, t, 0, __ATOMIC_SEQ_CST, __ATOMIC_SEQ_CST)) c = k;L12: The enabled standalone path loses in its own target workload. At four threads, the complete reported ranges are 407.6–464.3 ns with the switch enabled and 345.9–391.0 ns with it unset. Evaluate the owner-slot change together with this code against the actual base, in play, before merging.
@ -2026,0 +2126,4 @@__atomic_store_n(&t->vc_held, c + 1, __ATOMIC_SEQ_CST);t->vc_last = c + 1;t->vc_starved = 0;t->vc_held_at = hos_now_ns(), t->vc_kick_at = 0; /* (vc_long_wait: how long a holder has held its core) */L5: The owner CAS exposes t before the fast taker initializes vc_held_at and vc_kick_at. A waiter can see that owner, set its kick time under vc_lock and have the taker race it or reset the timestamp to zero. The long-wait logger can also read vc_held_at during initialization. Publish initialized hold metadata and make accesses that can overlap a release or retake atomic or share a lock. Use atomic reads of vc_kick where the fast path can change it concurrently.
@ -2034,3 +2165,2 @@AcquireSRWLockExclusive(&vc_lock);vc_st.takes++;int c = vc_free_for(t);__atomic_fetch_add(&vc_st.takes, (int64_t)1, __ATOMIC_RELAXED);L13 (minor): The benchmark calls the unset branch the main path, but both columns include the PR's sequentially consistent owner and held stores, atomic take counter and switch checks. Compare the actual base with the corrected head in its shipping configuration and with the experiment enabled. Report FPS, 1% lows, median and p99 frame times and virtual-core waits on Windows, Linux and Steam Deck, including a busy scene on a low-core-count machine.
@ -2140,0 +2296,4 @@__atomic_store_n(&vc_owner[c], (HosThread *)NULL, __ATOMIC_SEQ_CST);if (!__atomic_load_n(&vc_nq, __ATOMIC_SEQ_CST)) return;AcquireSRWLockExclusive(&vc_lock);if (!vc_holder(c)) vc_grant(c, NULL, t);L7 (minor): A fast release can free a slot and then lose the lock to a timed-out waiter. vc_settle grants the slot with no releasing-thread data, so a long wait can be reported as 'found free' or 'given at a release' without the thread that held it or its hold and kick times. Preserve release attribution when settling an already queued waiter, or document that fast mode drops it and distinguish those grants in the trace.
@ -240,0 +309,4 @@/* a waiter and a release: the core goes to the waiter, asleep by then */thread(&t, "waiter", 16, 1);hook_thread = &h0, sleep_hook = gives_up;L9: The release hook runs after the waiter has queued and entered sleep. The fixture never forces two takers to claim the same free slot or a waiter to queue between a fast taker's two vc_nq reads. Replacing the fast CAS with a store or removing the post-CAS queue check would still pass these cases. Force both interleavings and assert one holder and timely handoff to the eligible waiter.
@ -231,0 +267,4 @@thread(&t, "remask", 16, 0);t.tls_slots[0] = 1; /* the guest cached core 0 */M = &t;wrapper_hook = [] { M->core_mask = 2; vc_note_mask_change(M); }; /* (in vc_pref_core: the choice) */L10: The remask hook writes the field directly and calls vc_note_mask_change on the taker's host thread. It catches removal of the final mask comparison, but bypasses hos_thread_affinity and cannot distinguish the production ordering from one with the fences removed. Drive the real setter and cover a concurrent remask, including a same-core update delayed across a TLS write.
Recommendation: Request changes. The fast take can lose the mask-0 core preference, the scheduler races remain, and the standalone experiment slows its target microbenchmark.
New finding. At
host/hos_thread.c:2147, the fast take overwrites the cached core before a detected change to mask 0 can act. A thread can leave core 0 with its cache still pointing there when an affinity change to core 1 arrives just as it enters a block. On return, it chooses core 1 and the setter restores mask 0 before the claim is published, so the setter sees no held core. The fast take detects the change and requests a checkpoint, butvc_sync_tlshas already replaced the cached preference with core 1 whenvc_yieldasksvc_pref_corewhere to move. The checkpoint keeps core 1 even if core 0 is still free. Use one synchronized affinity and preference snapshot through the claim and retry selection before changing the cache if the snapshot changes. Cover the transition to mask 0 in addition to the existing test for a transition away from it.The existing exact-head review also identifies the mixed reads in the affinity and held-core paths, stale same-core TLS writes, unsynchronized hold timestamps and take counters, and the stale slow-frame assertion. Exact main already delegates the grant from
vc_releasetovc_release_locked, so the assertion is already stale there. Update the fixture to follow the actual grant and the changed release signature. The transfer fixtures also leave the competing CAS and waiter-arrival windows untested. Existing exact-head reviewDoes it help? Unproven in gameplay. On the reported i7-9700 microbenchmark, median take-and-release time with the standalone switch enabled rose from 170.1 to 218.5 ns with two threads, 366.5 to 436.3 ns with four, and 587.0 to 633.7 ns with six. The four-thread ranges were separate. Both arms used the patched scheduler, so the benchmark leaves the shipping path's cost against main open. The reported owner-slot variant reached 56.0, 59.0 and 67.2 ns in the same setup. Fold the owner-slot change into this track before measuring the corrected revision against actual main with the switch unset and enabled. Owner-slot change
Other systems and switches.
HOS_VCORE_FASTis registered as a developer experiment and defaults off. Unset still changes synchronization and counter operations. The scheduler serves guest waits and wakeups across Windows, Linux and Steam Deck, including NVN waits, and supplies the slow-frame and profiling diagnostics. In the usual teardown, the post-join lock barrier drains scheduler readers before freeing the exiting thread record.Testing. The author reports Windows Clang fixture passes on an earlier rebase and 44 passes, 11 failures and four skips in the changed-file sweep. The saved Forgejo snapshot shows the description check passed and the relevant source checks pending. Gameplay and Linux host validation are outstanding. After the source fixes, compare main and the combined head with the switch unset and enabled, on the same scripted Market route and a busy outdoor route. Report average FPS, 1% lows, median and p99 frame times, and core-wait, escape and ownership diagnostics. Include a low-core-count system and Steam Deck in Tier 1 coverage.
Overlaps. The separate fixture cleanup touches the duplicate declaration. Keep one copy of that fix. Keep serial-core spinning out of this comparison so the scheduler effects remain attributable. Fixture cleanup · Serial-core spinning
Reviewed. Head
e6acd10c727c3fefbd3981129756c58bd3c005ebagainst merge base6e1521a70a32603ed60a8263e4072625885669b7. Read all five changed files at the touched areas, both fixtures in full, and relevant affinity, TLS, wait, lifecycle and statistics callers. Used revision-matched, local source-only Graphify maps and confirmed the findings in source. Longshot guidance6e1521a7, Tooling0b0059ac.@ -2026,0 +2144,4 @@__atomic_store_n(&t->vc_kick, 2, __ATOMIC_SEQ_CST);vc_wake_holder(t);}vc_sync_tls(t, c);A detected change to mask 0 can lose the original cached preference here. Let a core-0 thread give up its core after a remask to core 1 but before it services the kick. On its return from the block, it selects core 1. Have the setter restore mask 0 before the claim is published, while cached core 0 is free. The setter sees no core held. The taker requests a checkpoint but overwrites the cached preference with core 1, so vc_yield finds that it is already on its preferred core and drops the kick. Validate the selected core and the cached preference together before writing the cell, and retry with the current snapshot when they change. Please cover this transition to mask 0 in the fixture. Both cores are allowed under mask 0, so checking eligibility alone misses it.
e6acd10c72c9d646dcb9WIP: perf(os): HOS_VCORE_FAST=1, virtual cores taken and released without vc_lock while no thread waits for one (experiment, off)to WIP: perf(os): virtual cores taken and released without vc_lock while no thread waits for one, each owner slot on its own cache line (HOS_VCORE_FAST, off)Description updated for the push of
c9d646dc(onmain@f1f2dc5b): the title and Summary cover both steps (#1973's commit folded in as its own commit,1cde9675), this round's fixes, and what still differs frommainwithHOS_VCORE_FASTunset; Tested lists this head's checks (job fb3-1972-1); Measurements keeps the earlier micro-benchmark (step 1 alone a loss, both steps faster) and says its re-run on this head has not run yet.Pushed
c9d646dc(onmain@f1f2dc5b), with #1973's commit folded in as asked; #1973 is closed in favour of this PR.On 62911 / 62912:
a6ca3685. A take withoutvc_locknow chooses from one read of the mask and, under mask 0, of the cached core (vc_aff); after its compare-and-swap andvc_heldit reads both again and writes the TLS core cell in the sametls_slot_locksection (vc_fast_check), and if either changed it gives the core up and chooses again.tests/test_vcore_mask0.pydrives your interleaving with the real setter on another thread (core 0 cached, remask to core 1 while away, back to mask 0 during the take) and expects core 0 and the cell naming it. It passes on this head. The fixture fails at the test commit570f35da(its unset run already fails there on the cell-order case below, so its HOS_VCORE_FAST=1 run does not start), and withvc_fast_check's comparison taken out of this head's source the HOS_VCORE_FAST=1 run fails on this case (job fb3-1972-1).a6ca3685(list in its message); the slow-frame assertion follows both release paths inca0e9931, andtests/test_vcore_escape.pychecks the long-wait cause each gives. The competing compare-and-swap and the waiter arriving between a take's two queue reads are now forced intests/test_vcore_escape.py; with the compare-and-swap made a store, or the second queue read removed, it fails (run on mutated copies of the source in the same batch).hos_fps_timingstays here (61c51418); whichever lands second rebases.main's stores, no fence andmain's take count again; the description's Summary lists what still differs, and Measurements has a micro-benchmark ofmainagainst this head unset and set.main/ unset / set runs on the Market, a busy outdoor route and a route with queued core waits, low-core PCs and the Steam Deck are asked for with the person who does this PR's game runs, on this head.On #1973's 62913 / 62914:
1cde9675here, unchanged.c9d646dc. No check counts slot loads; the no-waiter micro-benchmark does not reach this loop, so the route with queued core waits is in the game runs above.a6ca3685,ca0e9931).BeanBot 62391 and #1973's 62404, in
c9d646dc:a6ca3685).hos_thread_corereadsvc_heldonce;vc_lastatomic (a6ca3685).tls_slot_lockwhere the cell is written; with the switch on the cached core read there with its slot validated (a6ca3685;tests/test_vcore_mask0.pycases fail at570f35da).vc_kickatomic where a waiter reads them (a6ca3685).a6ca3685; checked intests/test_vcore_escape.py).vc_fast; Tracy shows "given at a release" for a grant without a name (a6ca3685).ca0e9931.vc_exited's lock pass forced intests/test_vcore_escape.py; the mask cases drivehos_thread_affinityon another thread, including the same-core update against a take's cell write.--cxx, thenCXX, then clang++/g++/c++ in both fixtures.1cde9675); L13, L14: the unset path hasmain's stores and take count again, Summary says what still differs, Measurements comparesmain, unset and set. Play runs are pending (Still to test).@Beanstalk This adds a switch, HOS_VCORE_FAST, off by default: with HOS_VCORE_FAST=1, virtual cores are taken and released without their lock while no thread waits, and each virtual core's owner slot sits on its own cache line (host/hos_thread.c). It has not run in the game yet. Please play Castle Town Market for two minutes, the PR against main on the same PC, on the PR's side with HOS_VCORE_FAST=1 and with it unset, on PCs with 6 or fewer logical processors (Windows and Linux), on a Windows PC with an AMD processor, and on the Steam Deck. No crash or hang, and no "ran without a core" line in the log; set, the log must have the line "vcores: takes and releases without vc_lock while nobody waits (HOS_VCORE_FAST)" and frame time p99 equal to unset or better, within the run-to-run spread; unset, frame times as main's.
@siantm only members of
BigBeanscan start the farm. One of them can approve this request by reacting 👍 to it or to this comment.The three renderer checks (
test_present_queue_default,test_target_reuse_memory,test_vk_occlusion_queries) did not compile in the earlier run (no Vulkan headers on that job's path); rerun with them in job fb3-vk-1, at the headc9d646dcand onmain@f1f2dc5b: all three fail at both with the same first error, so they fail onmaintoo; Tested's sentence and counts corrected.Measurements: the take/release micro-benchmark on this head (
c9d646dc) againstmain@f1f2dc5b, job fb3-1972-1-bench: switch set 54-78 % less per take+release; unset, no difference measured (ranges overlap, medians 1-10 % higher). Micro-benchmark only, not play.View command line instructions
Checkout
From your project repository, check out a new branch and test the changes.