WIP: perf(os): HOS_VCORE_SERIAL_SPIN=1, the frame's serial threads spin for a virtual core and ask for one at once after host-only work (experiment, off) #1974
No reviewers
Labels
No labels
area/accessibility
area/audio
area/input
area/install
area/menu
area/renderer
area/saves
area/tooling
area/upscaling
beanstalk: approved
beanstalk: failed
beanstalk: needs approval
beanstalk: running
beanstalk: tested
ci: approved
good first issue
gpu/amd
gpu/igpu
gpu/intel
gpu/nvidia
help wanted
kind
bug
kind
chore
kind
docs
kind
feature
kind
performance
kind
question
needs measurements
platform/linux
platform/macos
platform/steam-deck
platform/windows
preview
priority
critical
priority
high
priority
low
regression
risk
high
risk
low
risk
medium
status
blocked
status
confirmed
status
duplicate
status
invalid
status
needs-info
status
wontfix
ci: approved
No milestone
No project
No assignees
4 participants
Notifications
Due date
No due date set.
Dependencies
No dependencies set
Reference
MagicBeanDev/Longshot!1974
Loading…
Reference in a new issue
No description provided.
Delete branch "siantm/Longshot:perf/vcore-serial-spin"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Summary
The game thread and the Presentation Thread (the frame's serial threads,
hos_thread_serial) wait for a virtual core asevery other thread does: when none is free they sleep on their condition variable at once, and when the holder has the
same priority they ask it for the core only after a time slice (
HOS_VCORE_SLICE_US, 2 ms). A frame waits on each ofthese threads in turn, so their wait for a core is on the frame's path.
This adds an experiment, off by default. With
HOS_VCORE_SERIAL_SPIN=1, a serial thread:hos_spin_ns,HOS_SPIN_US) for a core before it sleeps, so a holder that blocks withinthe spin hands the core over without a wake-up. The spin ends at the next of the thread's scheduler deadlines (the
slice, the starved time, the escape) if that comes first; after it the grant, a free core and those deadlines are
looked at under
vc_lockand the holders asked again before the thread sleeps;hos_vcore_back: a present, a replay, a file read) asks a holder as good as itself for thecore at once, not after the slice: on the console the thread would not have lost its core there. Back from one of the
guest's own waits (
hos_vcore_back_guest: its sleeps, waits, joins, a thread's start, and in the other files OoT's jobbarrier,
WaitTimerEvent,nn::oe's notification wait, the socket poll's timeout andnn::gs's timed wait for anative sync, through
hos_sleep_wait_ns,hos_sleep_ns's sleep with that way back) the slice stays. Each callersays which it is.
How long a waiter waits before it runs without a core follows its priority against the holders' of the cores it may
use (
HOS_VCORE_ESCAPE_USwhen one of them is worse, elseHOS_VCORE_ESCAPE_EQ_US), not whom it asked: an earlyrequest had chosen the better waiter's budget.
mainchose it the same way after the slice and for a starved waiter;with the defaults (2 s both) nothing changes there.
The virtual cores' switches (this one and the four wait times) are read once in
game_main, afterlongshot_env.txtand before the game's first thread (hos_vcore_env); a time that is not a whole number ofmicroseconds from 0 to an hour is the default. The switch's row is in
gpu/vk/switches.def(kindattempt). Off, thepaths are as on
mainapart from that read and the escape budget above.vc_acquirekeeps its name and takesowed;hos_vcore_backandhos_vcore_back_guestsharevc_back; #327'svc_wake_holderis kept for every ask.vc_takestores
vc_heldlast, with a release, as the spin reads it withoutvc_lock, and the release andhos_thread_core(GetThreadCoreMask's look at another thread) use atomics too. With the switch on, the 5 s
vcores:line counts theserial threads' waits, those a grant seen in the spin ended, and the early requests.
Left as on
main(they are not new here):SetThreadPriorityandSetThreadCoreMaskwrite fields the scheduler readsunder
vc_lockwithout it,vc_wake_holdercan race a fiber switch'srun_ctx, and the frame trace'sblocked_totaland
vc_wait_totalandhos_fps_timingare plain cross-thread fields.From d1e33769 (the host-sync integration) on
exp/x86-asm-sync, where it was on by default and not measured; rewrittenover #327's
vc_acquire. It does not touch #1860's starvation default.The two vcore checks fail on
mainbefore this PR:tests/test_vcore_escape.py's fixture declareshos_fps_timingtwice (#1860 and #1966 fix the same line) and
tests/test_vcore_mask0.py's fixture has noenv_on_or. Both are fixedhere, as the new cases need the fixtures to build;
tests/test_slow_frames.pynow cutsvc_release_lockedfor thegrant's attribution, which failed on
main.Read first:
vc_acquire,vc_backandhos_sleep_wait_nsinhost/hos_thread.c; theserial(),handoff(),spin_deadline()andbudgets()cases intests/test_vcore_escape.py.Testing
Tested: Windows 11, i7-9700, RTX 4060 (NVIDIA 610.88), llvm-mingw clang 23.1.5; game-free checks only. This change
has not run in the game on this PC yet (Still to test). Each check ran in a hidden console of its own, through the PC's
job queue (jobs fb3-1974-1 and fb3-1974-2), on this branch rebased onto
main@f1f2dc5b(head789c7663; the first job ranat 95c9d9f3, the same tree but for a one-line fix to
tests/invalidate_vcore_test.cpp's harness, then squashed):python -I -B tests/test_vcore_escape.py(and again withCXXset to llvm-mingw's clang++): exit 0, "ok" for thedefaults,
HOS_VCORE_SERIAL_SPIN=0and=1, each also withHOS_VCORE_STARVE_US=0, and the "budget" runsHOS_VCORE_ESCAPE_US=0andHOS_VCORE_ESCAPE_EQ_US=0, each with the switch unset and=1. At the test commit(
e4fbd3db) it fails: withHOS_VCORE_SERIAL_SPIN=1"its spin ends at the slice, where the holder as good is asked andanswers (not after the whole spin)"; with
HOS_VCORE_ESCAPE_US=0the equal serial thread back from host-only work ranwithout a core after 1.0 ms (switch on) and 4.0 ms (off), the equal thread back from a guest wait after 4.0 ms and the
starved worse one after 1002.0 ms, each with the holder still on the core and a budget of 2000 ms; with
HOS_VCORE_ESCAPE_EQ_US=0and the switch on the equal serial thread waited 2001.0 ms of a 0 ms budget.python -I -B tests/test_oot_barrier.py: exit 0, "PASS: the job barriers wait until their masks move, ...". At thetest commit: "FAIL:149 aways == a0 + 1 && backs == b0 + 1 ..." (the barrier went back as host-only work).
python -I -B tests/test_short_sleep_timer.py(and withCCset to llvm-mingw's clang): exit 0 (it prints nothingwhen it passes). At the test commit: "FAIL: host/hos_thread.c has no 'void hos_sleep_wait_ns(int64_t ns) {'".
python -I -B tests/test_vcore_mask0.py: exit 0, "ok".tests/test_slow_frames.py: exit 0, "slow frames: ok".tests/test_switch_table.py: exit 0, "ok: 735 switches (diag 295, tune 109, kill 129, attempt 81, hw 67, player 54),144 removed, 743 names read".
tests/test_spin_by_role.py,tests/test_reserved_core.py,tests/test_reserved_core_mask.py,tests/test_hle_guest_ranges.py: exit 0.python -I -B tests/test_invalidate_vcore.py(job fb3-1974-2, on789c7663): exit 0, eleven "PASS" lines, "PASS: vulkanactual invalidate wrapper" to "PASS: quick-declined actual invalidate wrapper";
test_vcore_escapeandtest_slow_framesran again there, exit 0.python -I -B tests/run_all.py --strict --changed origin/main(126 checks read the changed files; at 95c9d9f3): exit1, "108 passed, 18 failed, 12 skipped".
test_invalidate_vcorefailed there (the harness's own call tovc_acquirestill had one argument; fixed, above). 14 of the other 17 fail on
main@f1f2dc5btoo, run by name in the same job:test_crash_dialog,test_crash_record,test_dev_ab,test_dev_packs,test_display_startup_failure,test_frame_overlap,test_host_guest_writes,test_libc_hooks,test_lift_cache,test_linux_pressure,test_mods_list,test_perf_report,test_profiling_mode,test_walk_key. The other three(
test_present_queue_default,test_target_reuse_memory,test_vk_occlusion_queries) compiled nothing in that runnor in the run on
main(no Vulkan headers on the job's path: "not found: vulkan_headers (--strict)"). Rerun with theheaders (job fb3-vk-1,
RX_CPMset) at the head789c7663and onmain@f1f2dc5b, each fails at both with the samefirst error:
test_present_queue_default"srccut: '{TAB_DEBUG, "Present queue", &opts.present_queue, ...' is not ingpu/vk/vk_menu.cpp";
test_target_reuse_memory2 of its 8 tests ("'vmaGetAllocationInfo2' unexpectedly found" in theVMA source, and its fixture's "use of undeclared identifier 'destroy_image_later'");
test_vk_occlusion_queries"thering fixture did not compile" ("no member named 'policy'"). None fails on this branch alone.
Still to test:
main@f1f2dc5bagainst this PR withHOS_VCORE_SERIAL_SPINunset against this PR withHOS_VCORE_SERIAL_SPIN=1, the same build type, save and settings,NVN, yield and fast-take switches at their defaults (#1968, #1966 and #1972 not in the build), at least two warmed runs
of each, interleaved; average fps, 1% lows, median and p99 frame times and CPU time per frame for each, and with it set
the log line "vcores: the serial threads spin for a core, and ask for one at once after host-only work
(HOS_VCORE_SERIAL_SPIN)" and the 5 s
vcores:line's serial waits, grants seen in the spin and early requests,expecting unset the same as
main, no stall or crash, and with it on fps and frame times within the runs' spread orbetter (by hand)
python3 -I -B tests/test_vcore_escape.py,tests/test_vcore_mask0.pyandtests/test_short_sleep_timer.py, expecting them to pass (by hand)expecting fps and frame times within the runs' spread or better (@Beanstalk play Castle Town's market for two minutes
on main, on this PR without HOS_VCORE_SERIAL_SPIN and with HOS_VCORE_SERIAL_SPIN=1, on PCs with 6 or fewer logical
processors, with AMD hardware and on the Steam Deck, comparing frame times and CPU time and looking for a stall or a
crash)
Measurements
Verdict: not measured: no micro-benchmark was run for this PR, and no play measurement exists yet.
HOS_VCORE_SERIAL_SPINships off (kindattempt): it is an experiment, and nothing measured chooses it on. Thein-game A/B (
main, this PR unset, this PR set) is in Still to test; until then the table holds the checks only.Setup: Windows 11, i7-9700, RTX 4060 (NVIDIA 610.88), llvm-mingw clang 23.1.5; the vcore checks, each in a hidden console
of its own through the PC's job queue (job fb3-1974-1).
main's side not run again: its fixtures have the same twobuild errors at
f1f2dc5bas at2e1f6c32(read in the source).f1f2dc5b789c7663tests/test_vcore_escape.pyhos_fps_timingtwice)=0and=1, each also withHOS_VCORE_STARVE_US=0;HOS_VCORE_ESCAPE_US=0andHOS_VCORE_ESCAPE_EQ_US=0with the switch unset and settests/test_vcore_mask0.pyenv_on_orin its fixture)Risks
Off, a wait for a core tests a switch read once at startup, and the escape budget after the slice or for a starved
waiter is its priority's (the same with the default times). On, the game thread and the Presentation Thread spin up to
HOS_SPIN_US(20 us by default, never past their next scheduler deadline) on a processor before they sleep for a core,CPU another thread could have used; and after host-only work they ask a holder of equal priority for its core at its
next checkpoint, which moves that thread off its core sooner. A mistake would show as a stall or a lower frame rate with
the switch on. Turn it off by leaving
HOS_VCORE_SERIAL_SPINunset; a revert is enough. Players notice nothing: theswitch is not set for them.
Checklist
Ready for review
main; the title is a Conventional Commit.mise run testpasses (any failure that also fails onmainis named).main, or Measurements says why there is nothing to measure.third_party/untouched; nothing GPL-incompatible.When it applies
gpu/vk/switches.def.tests/that reads changed source is updated, not weakened.Assisted-by:/Co-Authored-By:trailers.Ready to merge
main, measured again or a comment says why the numbers still hold.Possibly related, for context:
Recommendation: Request changes. The experiment can preempt equal-priority holders after guest waits and can make an equal-priority waiter escape before its configured deadline. Fix those paths and validate the handoff before choosing a default from in-game measurements.
The game and Presentation Thread can try to take a virtual core without sleeping and can request an equal-priority holder's core sooner after host work. That could shorten frame-critical waits, at the cost of extra CPU time under contention and more preemption. The game-free fixtures now build in the reported Windows runs.
@fixwill do (17)host/oot_barrier.c:107: The default-enabled OoT job barrier replaces a guest SleepThread but returns through the newly privileged hos_vcore_back. A serial thread waking at the barrier can therefore kick an equal-priority holder immediately. WaitTimerEvent also sleeps through hos_sleep_ns, and the applet socket poll and notification wait have the same classification problem. Give guest waits an explicit reacquire path across files and preserve their existing sleep implementation and slice.host/hos_thread.c:2139: The sameworstpointer selects the kick target and the escape timeout. With HOS_VCORE_ESCAPE_US=0 and the default equal-priority timeout, an early kick at an equal-priority holder makes the serial waiter escape on the first timed look even if the holder is still running. That lets both threads use the same core's guest state. Select the escape budget independently of the early kick and cover unequal timeout settings in the fixture.host/hos_thread.c:2113: The spin can defer scheduler deadlines and then enter a timed sleep before checking them. (more on the line)host/hos_thread.c:1946: The acquire/release pair covers the new spinner, but GetThreadCoreMask can still call hos_thread_core on a different thread while its vc_held is being published or cleared. (more on the line)host/hos_thread.c:2095: The early-kick path inherits unsynchronized scheduler inputs. (more on the line)host/hos_thread.c:2107: The added kick path inherits a race with fiber switching. (more on the line)host/hos_thread.c:2044: Normal boot initializes the scheduler before guest threads start. (more on the line)host/hos_thread.c:2256: The new guest return duplicates the frame-trace update. (more on the line)host/hos_thread.c:2122: The spin count can include a grant that arrived after the spin expired while the waiter was reacquiring vc_lock. (more on the line)scripts/profiling/symbols.py:20: The profiling tools still know the old scheduler symbol. An external leaf attributed directly to vc_acquire_as can be classified as running because waiting() only uses the nearest Longshot frame. The CPU descriptions also only name vc_acquire. Update both for the new name, keeping old captures readable and distinguishing a spin from a sleep.tests/test_slow_frames.py:123: The acquisition check now cuts the one-line vc_acquire wrapper, which cannot contain the long-wait logging or timing accounting it asserts. The PR reports a base failure for this fixture, but the changed cut adds a separate failure. Point it at the implementation and cover the shared return accounting if it moves.tests/test_vcore_escape.py:290: The grant-in-spin case still passes if the atomic poll is removed and the waiter spins to its deadline. (more on the line)tests/test_vcore_escape.py:323: Both HOS_VCORE_STARVE_US=0 invocations return before serial(), so the four reported configurations exercise the serial policy in only two of them. (more on the line)tests/test_short_sleep_timer.py:120: Both return stubs increment the same counter. (more on the line)tests/test_short_sleep_timer.py:27: The suite forces gcc for this changed fixture even on an llvm-mingw Windows installation with clang and CC set. Strict mode can fail before exercising the test. Honor CC and choose an available supported C compiler when it is unset. The standalone vcore fixtures should likewise use CXX when there is no explicit --cxx.tests/test_vcore_mask0.py:267: The enabled run never makes a thread serial, and its spin budget is always zero. (more on the line)For the author (1) (a run, a measurement or a decision)
Checks: Still running:
CI / Conventional Commit title (pull_request),CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request),CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request),CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request),CI / Workflow lint (actionlint) (pull_request),CI / Website build and checks (pull_request),Guide screenshots / Guide images and local UI source review (pull_request),Third-party parts / As their notes say (pull_request),Third-party parts / The same as upstream (pull_request),Website / Website on Cloudflare (pull_request).More: does it help, other systems, switches, what was read
Does it help: Unproven. The author reports passing game-free checks on Windows at af519170, whose patch matches HEAD. There are zero in-game runs or timings for this change. Compare base
6e1521a7with HEAD off and HEAD on in interleaved runs, with the same hardware, build type, scene and settings. Report average FPS, 1% lows, median and p99 frame times, virtual-core waits and CPU time.Other systems: The shared scheduler runs on Windows and Linux across GPU vendors. Contention can change when workers, audio, file I/O, decode, replay and presentation resume. The job-barrier interaction also reaches the game thread directly. Steam Deck and other CPU-constrained hosts have the least spare CPU for the spin. Frame and CPU profiling depend on the changed accounting and symbols. The diff changes no save format, input mapping, renderer interface, GPU allocation or build pipeline.
Switches: Adds HOS_VCORE_SERIAL_SPIN, an off-by-default developer experiment listed in switches.def and the developer reference. Off limits exposure during testing. The repository requires measurements to choose a shipping default, and the PR supplies none. When enabled, the experiment also uses HOS_SPIN_US (20 µs by default) for virtual-core waits and bypasses HOS_VCORE_SLICE_US (2 ms by default) for equal-priority requests after host work. HOS_SPIN_US=0 disables the spin while leaving early requests enabled.
Reviewed:
e594135319against6e1521a70a: Read AGENTS.md, CONTRIBUTING.md, the Forgejo template, the applicable developer standards, the commit and all six diffs. Read all three changed fixtures and the changed host code with its scheduler, thread, wait, timing and profiling dependencies. Read the edited rows and surrounding switch documentation. Skimmed older vcore run records and the wider caller and source-check lists. No review-checklist file or local Tooling checkout was present. There were no changed translations, generated files or run records.; nothing run (a code review).Checked and fine:
Testing: Read-only review. The author reports targeted Windows fixture passes and a changed-check run at af519170 with 44 passes, 11 failures also reported on that base and four skips. No in-game results, Linux host build, GCC fixture results or full-suite pass are recorded. The supplied HEAD CI checks are pending.
Overlaps and merge order: #1860 reportedly edits the same duplicate declaration in test_vcore_escape.py. Either order can work if the second merge keeps one declaration and preserves both sets of cases. #327 is already in the base.
Automated code review by BeanBot (gpt-6.2-sol), on the pull request's opening. Nothing was run; a maintainer decides.
@ -2027,2 +2041,4 @@guest's own (vc_back_guest) or a yield: HOS_VCORE_SERIAL_SPIN, above */static void vc_acquire_as(HosThread *t, int owed) {static int64_t esc_better = -1, esc_eq, slice, starve;if (esc_better < 0) {L7 (minor): Normal boot initializes the scheduler before guest threads start. A negative HOS_VCORE_ESCAPE_US leaves the sentinel negative, so later acquisitions on multiple threads can re-enter initialization and race on the timeout values. Large values can overflow during conversion to nanoseconds. Validate the inputs and initialize the scheduler configuration once under vc_lock or during boot.
@ -2068,3 +2093,3 @@if (!worst && hos_now_ns() - t0 > slice)if (!worst && ((owed && serial) || hos_now_ns() - t0 > slice))for (int k = 0; k < vc_cores() && !worst; k++)if ((m >> k & 1) && vc_owner[k] && vc_owner[k]->priority == t->priority) worst = vc_owner[k];L5 (minor): The early-kick path inherits unsynchronized scheduler inputs. SetThreadPriority and SetThreadCoreMask write fields that the scheduler reads under vc_lock without taking the same lock for those writes. A concurrent priority or mask change can race the choice of which holder to kick or which waiter gets a grant. Synchronize those updates with scheduler selection and take consistent snapshots for external readers.
@ -1932,3 +1943,4 @@t->vc_starved = 0;t->vc_held_at = hos_now_ns(), t->vc_kick_at = 0;vc_sync_tls(t, c);__atomic_store_n(&t->vc_held, c + 1, __ATOMIC_RELEASE);L4: The acquire/release pair covers the new spinner, but GetThreadCoreMask can still call hos_thread_core on a different thread while its vc_held is being published or cleared. Its two plain reads can straddle a release and return -1 as the core number. Use a single atomic snapshot for this unlocked reader and atomic stores for both grant and release. Audit the other unlocked readers against the same ownership rule.
@ -2081,6 +2106,24 @@ static void vc_acquire(HosThread *t) {}vc_wake_holder(worst); /* (also repairs a request whose interrupt was already taken) */L6 (minor): The added kick path inherits a race with fiber switching. vc_wake_holder reads the volatile run_ctx while the holder can change it. A kick after fiber_switch checks the old poll flag and before it publishes the new context can land on the outgoing context only. Synchronize the context handoff with the kick or recheck the thread's pending kick as the new context is published.
@ -2084,0 +2110,4 @@/* (HOS_VCORE_SERIAL_SPIN) once, before the first sleep, with the lock let go: t is queued, so a release givesit the core under the lock (vc_grant: vc_held set last; the wake-up finds nobody asleep and is not needed).The look at vc_held under the lock after the spin is the one the sleep's loop makes: no grant is missed */const int64_t end = hos_now_ns() + spin_ns;L3: The spin can defer scheduler deadlines and then enter a timed sleep before checking them. HOS_SPIN_US accepts up to one second, so a guest waiter with the default 2 ms slice can delay its first equal-priority request by roughly a second. Cap the spin at the next scheduler deadline and recheck availability, priority and elapsed wait under vc_lock before sleeping.
@ -2084,0 +2119,4 @@}AcquireSRWLockExclusive(&vc_lock);if (t->vc_held) {vc_st.spun++;L9 (minor): The spin count can include a grant that arrived after the spin expired while the waiter was reacquiring vc_lock. It can also exceed the logged wait count when another thread resets the statistics after this wait starts and before it finishes. Define the metric as either a grant observed in the spin or a wait completed without sleeping, then count the numerator and denominator in the same interval.
@ -2202,0 +2253,4 @@HosThread *t = cur;if (!hos_vcores_on() || !t) return;t->vc_escaped = 0; /* (as hos_vcore_back) */if (t->away_at) t->blocked_total += hos_now_ns() - t->away_at, t->away_at = 0;L8 (minor): The new guest return duplicates the frame-trace update. The renderer reads blocked_total and vc_wait_total under threads_lock while guest threads write them without that lock, and the FPS timing flag is written atomically but read through volatile accesses. Share the return bookkeeping and use atomic loads and updates for the cross-thread timing fields. This matters when the Breakdown view is used to measure the experiment.
@ -117,6 +117,7 @@ static void hos_log(const char *fmt, ...) { (void)fmt; logged++; }static void hos_vcore_yield(void) { core_yields++; }static int hos_vcore_away(void) { aways++; return 1; }static void hos_vcore_back(void) { backs++; }static void vc_back_guest(void) { backs++; } /* (a guest sleep's way back: HOS_VCORE_SERIAL_SPIN's vc_back_guest) */L14 (minor): Both return stubs increment the same counter. Sending a guest SleepThread through the host return path, or sending a host pacing sleep through the guest return path, still passes. Count and assert the two returns separately.
@ -170,2 +320,4 @@int main(int argc, char **argv) {(void)argv;vc_lock_at = &vc_lock;if (argc > 1) { /* the old rule: HOS_VCORE_STARVE_US=0 */L13 (minor): Both HOS_VCORE_STARVE_US=0 invocations return before serial(), so the four reported configurations exercise the serial policy in only two of them. Run the serial cases in the old-rule invocations too, or label those results as starvation-only. The guest case also releases at the first sleep, so add a holder that stays through the slice and checks that a kick eventually arrives.
@ -169,0 +287,4 @@check(t.vc_held == 1 && vc_owner[0] == &t && vc_nq == 0, "the serial thread has the core the holder gave up");check(sleeps == s0 + (spin_on ? 0 : 1),"HOS_VCORE_SERIAL_SPIN: a core freed within a serial thread's spin is taken without a sleep (off: it sleeps at once)");check(vc_st.spun == (spin_on ? 1 : 0), "the wait counts as one a serial thread's spin ended");L12: The grant-in-spin case still passes if the atomic poll is removed and the waiter spins to its deadline. The hook gives it the core and the locked recheck still increments spun. The early-kick case also forces the holder to release without checking that its interrupt was raised. Assert that the observed grant ends the spin before its deadline, and have the holder take its checkpoint only after the kick reaches its poll flag.
@ -262,0 +264,4 @@subprocess.run([compiler, "-std=c++17", "-O1", "-w", "-include", ENV_ON_H, *flags, str(folder / "fixture.cpp"),"-o", str(exe)], check=True, timeout=120)env = {k: v for k, v in os.environ.items() if not k.startswith("HOS_")}for config in ({}, {"HOS_VCORE_SERIAL_SPIN": "1"}): # (the vcore block's switches: none changes these rules)L16 (minor): The enabled run never makes a thread serial, and its spin budget is always zero. It therefore cannot catch a mask-0 regression in the new handoff. Exercise a contended serial mask-0 waiter and verify its granted core and TLS cell, or keep this check single-mode and cover that interaction in the serial fixture.
Recommendation: Request changes. Preserve the guest wait policy and the scheduler’s escape and starvation deadlines before measuring the serial-core experiment in gameplay. Existing exact-head review
Confirmed blockers. The existing review covers these paths.
SleepThread. Its return athost/oot_barrier.c:107now calls the host-only reacquisition path and can immediately kick an equal-priority holder when the game thread wakes. Keep the regular guest slice at the barrier and at other guest waits such asWaitTimerEvent. Classify the reason at the caller and preserve the existing sleep implementation.host/hos_thread.c:2139. WithHOS_VCORE_ESCAPE_US=0and the default equal-priority timeout, an early kick can let a serial thread escape on its first timed check while the equal-priority holder still owns the core. That can put both threads on the same guest per-core state. Derive the timeout from the waiter’s priority relationship and cover unequal timeout settings with a holder that keeps running.host/hos_thread.c:2113can postpone scheduler decisions for up to one second and then enter a timed wait. Bound the spin by the next scheduler deadline and recheck the grant, eligible cores and deadlines under the lock before sleeping. Spin deadline findingDoes it help? Unproven. A successful spin can keep a serial thread from sleeping while a core becomes free. An immediate request can shorten the wait after host work and can also preempt a worker earlier. There are no gameplay measurements. Compare pinned main with a corrected head in both modes. The head with the switch unset still changes core publication and adds a switch check. Measure frame pacing and CPU time, and count grants that avoid a sleep separately from early kicks.
Switch and platform coverage.
HOS_VCORE_SERIAL_SPINis registered as a developer attempt, off by default. Enabled, it usesHOS_SPIN_USfor the game and Presentation Thread’s core waits and bypasses the default 2 ms equal-priority slice after host work.HOS_SPIN_US=0disables the new spin and keeps the early request behavior. The native Windows and Linux scheduler, including Steam Deck, shares the affected paths with game jobs, NVN, I/O and audio. Give CPU-constrained systems particular attention and choose the shipping policy from the measured results.Testing. The author reports the focused Windows clang fixtures passing at
af519170, with 44 passes, 11 failures and four skips in the changed-check sweep. The author reports patch parity across the rebase. The commits between the two bases leave all six touched paths unchanged. The recorded base failure intest_slow_frames.pyconcerns grant attribution. At this head the old source marker now selects a one-line wrapper, so it also loses the wait-time and long-wait assertions. Update it to exercise the real implementation. The mock grant case still passes if the atomic poll is removed and the spin runs to its deadline. Add a test that distinguishes a grant observed before the deadline and verifies delivery of the holder’s interrupt. The captured Forgejo snapshot had one successful description check, one skip and ten pending contexts. Author gameplay, Linux host build and affected Tier 1 gameplay coverage are open. PR and author evidence · Spin fixture findingOverlap. The NVN draft removes some virtual-core release and return events the serial experiment targets, including zero-time fence polls and unpaced texture acquisition. Hold its setting fixed for independent runs and repeat the relevant NVN and pacing checks if both advance. The yield-pause experiment and fast take/release sequence touch the same scheduler. Keep those variants out of this comparison, then recheck kick, grant and publication behavior on an integrated head if accepted. NVN no-idle-away · Yield pause · Fast take/release · Owner-slot layout
Reviewed. Head
e59413531971e49c9a168737e422ae9219ab4568against merge base6e1521a70a32603ed60a8263e4072625885669b7. Read all six changed files at the touched areas, the scheduler and guest/host wait callers, and the changed fixtures. The spin drops the scheduler lock and rechecks for a grant under that lock before sleeping, preserving a grant in that window. Both local source maps verified against their revisions. Confirmed affected edges and findings in Git source. Longshot guidance6e1521a7, Tooling0b0059ac. Source review only.e594135319789c766347Pushed
789c7663, rebased ontomain@f1f2dc5b(the PR's own commit unchanged, range-diff "="), with7bf55119(the switches read once at startup, shared way back, atomichos_thread_core, fixtures),e4fbd3db(tests that fail before the fix) and789c7663(the fix) for the review: guest waits in other files keep the slice, the escape budget follows the waiter's priority, and a serial thread's spin ends at its next deadline. Summary, Testing (new base and what ran), Still to test (main / head unset / head set), Measurements and Risks updated.On your review, pushed as
789c7663onmain@f1f2dc5b(7bf55119refactor,e4fbd3dbthe failing tests,789c7663the fix; the PR's own commit is unchanged by the rebase):host/oot_barrier.c:107,WaitTimerEventand the others). Each caller now says which way back it is: OoT's job barrier,WaitTimerEvent,nn::oe's notification wait, the socket poll's timeout andnn::gs's timed wait for a native sync go back throughhos_vcore_back_guest(the regular slice), the sleeping ones throughhos_sleep_wait_ns, which ishos_sleep_ns's sleep unchanged with the guest way back; host-only work keepshos_vcore_back(789c7663). Tests:tests/test_oot_barrier.pycounts the two ways back apart and fails ate4fbd3db(the barrier went back as host-only work);tests/test_short_sleep_timer.pycovershos_sleep_wait_nsand counts the host's and the game's sleeps' ways back apart.hos_thread.c:2139). The budget is now the waiter's priority against the holders' of the cores it may use (HOS_VCORE_ESCAPE_USwhen one is worse, elseHOS_VCORE_ESCAPE_EQ_US), not the kick target's (789c7663).tests/test_vcore_escape.py's "budget" runs setHOS_VCORE_ESCAPE_US=0andHOS_VCORE_ESCAPE_EQ_US=0, each with the switch off and on, with a holder that keeps running and never answers: equal, worse and better waiters, serial back from host-only work and not. Ate4fbd3dbwithHOS_VCORE_ESCAPE_US=0the equal serial thread back from host-only work ran without a core after 1.0 ms with the switch on (4.0 ms off), the equal one back from a guest wait after 4.0 ms and the starved worse one after 1002.0 ms, the holder still on the core, against a 2000 ms budget.mainpicked the budget the same way after the slice and for a starved waiter; that changes too (no change with the default 2 s both), as the description says.hos_thread.c:2113). The spin ends at the next of the thread's deadlines (slice, starved time, escape); after it the grant, a free core and the deadlines are looked at undervc_lockand the holders asked again before any sleep (789c7663). Test: a 1 s spin with a holder as good that answers only at its checkpoint; ate4fbd3dbit failed ("its spin ends at the slice ... not after the whole spin").tests/test_slow_frames.py. It now cutsvc_acquire(HosThread *t, int owed)(the name kept, so the profiler's wait frames and old captures still match),vc_back(both ways back's block time) andvc_release_locked(the grant's attribution, which failed onmain) (7bf55119). It passes on this head ("slow frames: ok").e4fbd3db,789c7663).Measurements: pending with the PC's game runs, asked for: pinned
mainagainst this head with the switch unset and set, frame pacing (fps, 1% lows, median and p99) and CPU per frame, and the 5 svcores:line, which now counts the serial waits, those a grant seen in the spin ended, and the early requests apart. #1968, #1966 and #1972 stay out of that comparison. Linux and the other Tier 1 setups are in Still to test.On BeanBot's review, in
789c7663(7bf55119,e4fbd3dbtests,789c7663fix):test_slow_frames, L12 grant in spin / interrupt: done, as in the reply to the maintainer's review.hos_thread_core: one atomic load; the release storesvc_heldatomically too. The other readers of another thread'svc_heldare undervc_lock.hos_vcore_envreads the switch and the four times once ingame_main, before the first guest thread; a time outside 0..3600 s or not a number is the default (tested, and the switch flipped after the read changes nothing).vc_back. The plainblocked_total/vc_wait_totaland the volatilehos_fps_timingare as onmain; the description names them as left for a separate change. Same for L5 priority/mask writes and L6run_ctx.vc_acquirekeeps its name, soscripts/profiling/symbols.pyand old captures need nothing; a spin sample is CPU time ofvc_acquire(running), the sleep after it waits inSleepConditionVariableSRW;cpu_functions.jsonsays so.test_short_sleep_timertakes--cc,$CC, else clang, gcc or cc; the vcore fixtures default to$CXX.test_vcore_mask0runs once (none of its cases waits); a contended serial mask-0 waiter, its granted core and TLS cell are intest_vcore_escape.@Beanstalk This adds a switch, HOS_VCORE_SERIAL_SPIN, off by default: with HOS_VCORE_SERIAL_SPIN=1, the frame's serial threads (the game thread and the Presentation Thread) spin for a virtual core, and ask for one at once after a host-only block (host/hos_thread.c). It has not run in the game yet. Please play Castle Town Market for two minutes, the PR against main on the same PC, on the PR's side with HOS_VCORE_SERIAL_SPIN=1 and with it unset, on PCs with 6 or fewer logical processors (Windows and Linux), on a Windows PC with an AMD processor, and on the Steam Deck. No stall, hang or crash; set, the log must have the line starting "vcores: the serial threads spin for a core" and frame time p99 equal to unset or better, within the run-to-run spread; unset, frame times as main's.
@siantm only members of
BigBeanscan start the farm. One of them can approve this request by reacting 👍 to it or to this comment.On point 5 of my reply: the grant-in-spin case from
e4fbd3dbfails without the atomic poll. In a scratch copy of 789c7663'shost/hos_thread.c(not the branch), the spin had no load ofvc_held: it ran to its end, and the grant was counted at the locked look afterwards, as before the fix.python -I -B tests/test_vcore_escape.py --source <that copy>(PC job queue, job fb3-1974-3) exited 1, with "FAIL: the grant seen in the spin ends it at once, not at its end (the poll of vc_held)" forHOS_VCORE_SERIAL_SPIN=1and forHOS_VCORE_SERIAL_SPIN=1, HOS_VCORE_STARVE_US=0. Every run with the switch unset or 0, and every budget run, was "ok".The three renderer checks (
test_present_queue_default,test_target_reuse_memory,test_vk_occlusion_queries) did not compile in the earlier run (no Vulkan headers on that job's path); rerun with them in job fb3-vk-1, at the head789c7663and onmain@f1f2dc5b: all three fail at both with the same first error, so they fail onmaintoo; Tested's sentence and counts corrected.Recommendation — Request changes. The revision fixes the named OS waits and the static-holder escape cases. The game's NVN frame wait still takes the host-only shortcut, and the first spin can pass an already due deadline or use a stale escape budget.
Blocking
host/hos_thread.c:2287now makes everyhos_vcore_backcaller eligible for an immediate equal-priority kick. With threaded rendering, the main thread'snvnSyncWaiton the previous present fence is a game-requested wait and returns through that path athost/nvn_native.c:1190. The blockingnvnWindowAcquireTexturepath athost/nvn_native.c:2643also returns that way even though the console blocks there for a display texture. When one of these calls wakes a serial thread into contention, it can cut an equal-priority worker's slice short. Keep the guest return policy for actual NVN waits and reserve early requests for host-added work such as replay. Cover an unsignaled fence and a paced acquisition on the serial path. Frame pipelinehost/hos_thread.c:2155drops deadlines that are already due on the first pass. WithHOS_VCORE_ESCAPE_EQ_US=0, an equal holder that stays on its core can send the waiter spinning to the slice. WithHOS_VCORE_SLICE_US=0andHOS_SPIN_US=1000000as well, the same check can spin for a second before handling the escape. Check deadlines before dropping the lock and only spin while the next pending decision is in the future. The fixture exercises a long spin with the regular slice and tests unequal budgets with a zero spin. Exercise the two together, including a starvation deadline already crossed at the first scan.host/hos_thread.c:2177checks a budget derived from the owners seen before releasingvc_lock. A serial waiter can start a spin against a lower-priority owner, then see that core granted to a higher-priority waiter during the spin. If the shorterHOS_VCORE_ESCAPE_UShas elapsed, the serial waiter can escape alongside the new owner even thoughHOS_VCORE_ESCAPE_EQ_USapplies. The condition-variable path also carries its snapshot across the unlocked wait. After a grant and free-core check, derive the escape decision from the current eligible owners under the reacquired lock. Add a case that changes ownership during the spin with unequal budgets.Previous review. The barrier, timer, notification and socket waits, and the native sync sleep now use the guest return. The revised scheduler selects the budget independently of an early kick when the holder stays put. The default slice bounds the long-spin fixture. The slow-frame fixture reads the actual acquisition, release and shared return bookkeeping. The author reports that removing the atomic poll makes both enabled fixture runs fail and that the new handoff fixture waits for the holder's real poll flag. The mask-zero case exercises core publication and the guest's TLS cell during a contended serial take. Prior review · Author's follow-up · Poll mutation result
Does it help? Unproven. There are no gameplay measurements. A grant observed during the spin can avoid a sleep. Earlier requests and added polling can also cost worker time. Compare a corrected, rebased head unset and enabled against its pinned main on the same hardware. Measure average FPS, 1% lows, median and p99 frame times, CPU time per frame, completed serial waits, grants seen during the spin, early requests and escapes. Verify that the route actually exercises the affected waits.
Switch and reach.
HOS_VCORE_SERIAL_SPINis an off-by-default developer experiment for the game and Presentation Thread.HOS_SPIN_US=0leaves early requests enabled when the experiment is on. With the switch unset the revised scheduler still changes core publication, startup parsing and how unequal escape budgets are selected. The virtual cores serve engine work, NVN, audio and I/O on Windows, Linux and Steam Deck. Low-CPU hosts are important for measuring the tradeoff.Testing. The author reports focused Windows 11 llvm-mingw Clang passes, including the escape and slow-frame fixtures and the invalidate harness rerun at
789c7663. The strict changed sweep at preceding95c9d9f3had 108 passes, 18 failures and 12 skips. The later invalidate change fixed its reported fixture failure. The author reports that 14 other failures also occur on the pinned base and that the three renderer fixtures fail at the same first error on both revisions after rerunning with Vulkan headers. The captured exact-head Forgejo statuses showed the description passing, ten pending contexts blocked by required conditions, one website skip and the Farm approval pending. The full suite, Linux host and fixture validation, author gameplay and affected Tier 1 coverage are still due. Author's testing · Renderer fixture rerunOverlap. #1968 changes which NVN operations release a core. It leaves the classification of real guest waits relevant. #1966 changes yielding over the same scheduler and shares a fixture repair. #1972, which now includes #1973, changes ownership publication used by the spinner and overlaps the scheduler fixture. Keep those variants out of the three-arm comparison. If the changes advance together, recheck the caller classifications, affinity and TLS state, and spin and kick behavior on their combined revision. NVN waits · Yield pause · Fast take and release
Reviewed. Head
789c766347f6aa0ce17784bb6ce09de6b9d52d4aagainst Git merge basef1f2dc5bd2821cef061798ba40a043d281ac1707. The original experiment commit is patch-identical across the rebase frome59413531971e49c9a168737e422ae9219ab4568. Reviewed the three follow-up commits, all 17 changed files at the changed areas, the guest and host wait callers, and the scheduler's grant, affinity and TLS paths. Verified and queried local source-only Graphify maps at base and head. Each extraction warned of 18 partially parsed files and 54 code files without extracted symbols. Confirmed the NVN return paths and the ownership and deadline checks in Git source. Longshot guidance9dd41326, Tooling0b0059ac. Source review only.@ -2085,0 +2152,4 @@int64_t end = now + spin_ns;const int64_t due[3] = {t0 + vc_slice, vc_starve > 0 && !t->vc_starved ? t0 + vc_starve : end, t0 + escape};for (int i = 0; i < 3; i++)if (due[i] > now && due[i] < end) end = due[i];A deadline already due is skipped. With HOS_VCORE_ESCAPE_EQ_US=0, HOS_VCORE_SLICE_US=0 and HOS_SPIN_US=1000000, a serial waiter against an equal holder can spin for a second before checking the already-due escape. Service elapsed deadlines before unlocking. Add a fixture that combines a positive spin with zero or already-due deadlines.
@ -2095,2 +2176,2 @@if (starve > 0 && waited > starve) t->vc_starved = 1;if (waited > (worst ? esc_better : esc_eq)) {if (vc_starve > 0 && waited > vc_starve) t->vc_starved = 1;if (waited > escape) {This budget comes from the owners seen before the spin or condition wait releases vc_lock. If a worse holder gives its core to a better queued waiter during the spin, the serial waiter can escape on the old shorter HOS_VCORE_ESCAPE_US while the new owner is still running. Select the budget from current eligible owners after reacquiring the lock. Cover a grant to another waiter during a spin with unequal budgets.
@ -2201,1 +2284,4 @@}/* what the other files call around host-only work (a submission's replay, a present, a file read): a serial threadback from it asks a holder as good as itself at once (HOS_VCORE_SERIAL_SPIN) */void hos_vcore_back(void) { vc_back(1); }This host-only return also covers the game's own NVN waits. host/nvn_native.c:1190 takes it after nvnSyncWait on the prior present fence, and :2643 takes it after texture acquisition can block as on the console. A serial thread can cut an equal holder's slice short at either wait. Keep the guest return for the actual guest waits and exercise a contended fence and paced acquire.
View command line instructions
Manual merge helper
Use this merge commit message when completing the merge manually.
Checkout
From your project repository, check out a new branch and test the changes.