WIP: perf(os): HOS_CV_PARK=1, events, semaphores and message queues sleep on a park that blocks at once after their own spin (experiment, off) #1970

Draft
siantm wants to merge 6 commits from siantm/Longshot:perf/event-park into main
Contributor

Summary

A guest wait on an event, a semaphore or a message queue spins for its flag first (spin_until_set) and then sleeps
with SleepConditionVariableSRW, which spins again before it blocks: 10240 cycles of pause in ntdll
(RtlSleepConditionVariableSRW: 10240 / KUSER_SHARED_DATA.CyclesPerYield pauses), after the wait's own spin has
already missed.

This adds an experiment, off by default: with HOS_CV_PARK=1 (Windows) those sleeps go to a park in
host/hos_thread.c (hos_park_sleep, hos_park_wake) that uses the condition variable's own block and wake
(NtWaitForAlertByThreadId, NtAlertThreadByThreadId) without its second spin when the caller has just spun (a wait's
first sleep after its own spin, or after the spin policy left that spin out). The waiters are an oldest-first list from
the cv's word under one of 64 striped SRW locks; a wake that finds a waiter still spinning makes no system call. A
woken waiter takes its virtual core back, then the object's lock once (hos_cv_sleep takes it, lets it go for the core
and takes it again). Mutexes, guest condition variables, read-write locks, timers and WaitAny stay on Windows'
condition variable. Unset, =0, an ntdll without the two calls, or a wait that does not answer at setup: everything is
as on main (hos_park_sleep is hos_cv_sleep, hos_park_wake the Win32 wake). Linux keeps its condition variable
(the same shims).

How the park keeps a wake its own (after the review): the switch and the two calls are set up once
(InitOnceExecuteOnce, the mode published after the pointers). An alert is the thread's, so a sleep the park has woken
leaves only when its wake is done with it (PARK_DONE, set after the alert is sent or has failed) and then takes any
alert still pending for its thread with a zero-time wait, so none reaches the thread's next wait: Windows'
WaitOnAddress returns at the first alert its thread gets, its wait block still listed, and the game's job barriers
(host/oot_barrier.c) wait that way on the same threads (on the local branch this came from, a start with the first
version of the park stopped with an access violation in WakeByAddressAll under hos_oot_barrier_wake, one start of
five). A failed alert or a failing wait is logged once; a blocked sleep waits at most 100 ms at a time and looks at its
state, and a failing wait goes on in 1 ms steps until its wake or deadline, on the list all along.
spin_until_set reads the thread's spin budget once (spin_begin hands it on): with the park off it makes as many
hos_spin_ns() calls as main when the spin is left out and one fewer when it spins.

Six commits: the park; a test that no alert is left for a thread after a park sleep; its fix; tests for the review's
points (first callers, alert ownership, failing wait and alert, timed semaphore and queue peek, first-sleep policy, one
budget lookup); their fix; the budget.

From the local branch exp/x86-asm-sync: c565d05a (the park), 78c83308 (its first part: one lock take after a park
wake), 939b5324 (the test), 76de03b5 (the fix). There the park was on by default; here it is off.

Read first: the park block after hos_cv_sleep in host/hos_thread.c (park_init, park_sleep, hos_park_wake,
hos_park_sleep); in host/nn_os_sync.c spin_begin, wait_until's park argument, spin_until_set's return,
event_wait, sema_acquire, queue_get; tests/test_event_park.py (new).

Testing

Tested: Windows 11, i7-9700, RTX 4060 (NVIDIA 610.88), llvm-mingw clang 23.1.5; game-free checks only. This change
has not run in the game on this PC yet (Still to test). Each check ran in a hidden console of its own, through the PC's
job queue (job fb3-1970-1), on this branch rebased onto main @ f1f2dc5b (head ff7e2362):

  • python -I -B tests/test_event_park.py at the test commit 1e2adc46 (job fb3-1970-1): exit 1, each new part failing as it
    should without its fix: "budget" ("spin_until_set read the spin budget 2 and 2 times, not once", with and without a
    budget), "first callers" with HOS_CV_PARK=1 ("8 first callers looked ntdll's two calls up 16 times (want 2)"),
    "own" ("a untimed park sleep left on another alert for its thread before its wake's alert was sent"), "failing wait"
    ("stalled": the 30 ms sleep never returned), "failing alert" ("an untimed sleep whose wake's alert failed ends (0 of
    1)"), "a wait that does not answer" ("a park whose wait answers STATUS_NOT_IMPLEMENTED is used"). The races, "timed"
    and "policy" pass there.
  • The same at e4b273b1 (the park fix, before the budget commit; job fb3-1970-2): exit 1, only "budget" failing (2 and 2).
  • The same at the head ff7e2362 (job fb3-1970-2): exit 0, every part passing, unset, HOS_CV_PARK=1, both with no spin
    first and HOS_CV_PARK=0 ("ok", 20,000 auto-clear signals, 2,000 manual-clear rounds of 4, 5,000 timed-wait signals,
    60,000 through a semaphore and a queue, each message once); "timed" 68 of 100 racing releases taken and 32 after the acquire timed out, 76 of 100 racing sends seen and 24 after the peek timed out (on the park); "policy": first sleeps told spun=1 and
    0 pauses before blocking with a budget, spun=0 and 66 pauses with none; "budget" 1 and 1; "first callers" 2 lookups;
    "own", "failing wait" (2 waits for a 30 ms sleep), "failing alert" pass; with the wait answering
    STATUS_NOT_IMPLEMENTED the sleeps stay on Windows' condition variable; both controls fail. With CC set to
    llvm-mingw's clang (job fb3-1970-1, before the own part's check was moved ahead of the held wake, where its control
    crashed instead of failing): every part passed the same way.
  • test_cv_handoff_threads, test_cv_signal_race, test_cv_signal_early, test_event_handoff,
    test_multiwait_wake, test_os_lifted_siblings, test_oot_barrier, test_switch_table (job fb3-1970-1) and
    test_spin_by_role (job fb3-1970-2, its wiring check now reading the budget where spin_begin reads it) at the head:
    exit 0.
  • python -I -B tests/run_all.py --strict --changed origin/main (job fb3-1970-1, at the head before the two test-only
    changes of job fb3-1970-2): exit 1, "69 passed, 19 failed, 4 skipped". Two were this PR's and are fixed since
    (test_event_park's own-part control, test_spin_by_role's wiring check, both passing in job fb3-1970-2); 14 of the
    other 17 fail on main @ f1f2dc5b too (run by name in the same job): test_crash_dialog, test_crash_record,
    test_dev_ab, test_dev_packs, test_frame_overlap, test_host_guest_writes, test_libc_hooks,
    test_linux_pressure, test_mods_list, test_profiling_mode, test_slow_frames, test_vcore_escape,
    test_vcore_mask0, test_walk_key. The other three (test_present_queue_default, test_target_reuse_memory,
    test_vk_occlusion_queries) compiled nothing in that run nor in the run on main (no Vulkan headers on the job's
    path: "not found: vulkan_headers (--strict)"). Rerun with the headers (job fb3-vk-1, RX_CPM set) at the head
    ff7e2362 and on main @ f1f2dc5b, each fails at both with the same first error: test_present_queue_default "srccut:
    '{TAB_DEBUG, "Present queue", &opts.present_queue, ...' is not in gpu/vk/vk_menu.cpp"; test_target_reuse_memory 2 of
    its 8 tests ("'vmaGetAllocationInfo2' unexpectedly found" in the VMA source, and its fixture's "use of undeclared
    identifier 'destroy_image_later'"); test_vk_occlusion_queries "the ring fixture did not compile" ("no member named
    'policy'"). None fails on this branch alone.

Still to test:

  • Windows 11, this PC (i7-9700, RTX 4060 (NVIDIA 610.88)): repeated starts through the title screen with
    HOS_CV_PARK=1 and unset (the earlier local branch failed one start in five), then Castle Town's market, main @
    f1f2dc5b against this PR with HOS_CV_PARK unset and with it set, at least two warmed, interleaved runs each, CPU per
    frame, average fps, 1% lows, median and p99 frame times, expecting the log line "os: HOS_CV_PARK: events, semaphores
    and message queues sleep on a park that does not spin again after their own spin" with it set, no crash (none under
    hos_oot_barrier_wake in particular), and the unset run within main's spread (by hand)
  • Linux: build the host (the park's shims) and run python3 -I -B tests/run_all.py --strict --changed origin/main,
    expecting tests/test_event_park.py skipped (Windows only) and the cut checks passing (by hand)
  • Windows 10, and Windows PCs with other CPUs (an AMD CPU, fewer cores): Castle Town's market, main against this PR
    unset and with HOS_CV_PARK=1, expecting the os: HOS_CV_PARK log line, no crash and frame times as unset or better

Measurements

Verdict: not measured: no micro-benchmark was run for this PR, and no play measurement exists yet. HOS_CV_PARK
ships off (kind attempt), an experiment for A/B runs; tests/test_event_park.py has no timing mode. The disabled
head against main and the in-game A/B (main, unset, HOS_CV_PARK=1) are in Still to test: a game-free benchmark of
the wait's entry would time a stand-in for hos_spin_ns() (it reads the host's thread table), not the call itself.
The table holds the fixes' tests, before and after.

Baseline: this PR's test commits, on main @ f1f2dc5b: main has no park for the tests to run on.
Setup: Windows 11, i7-9700, RTX 4060 (NVIDIA 610.88), llvm-mingw clang 23.1.5; tests/test_event_park.py, one run
each side, each in a hidden console of its own through the PC's job queue.

Before, the test commits: this PR @ 1e2adc46 (and e4b273b1 for the budget) After: this PR @ ff7e2362
tests/test_event_park.py "first callers", HOS_CV_PARK=1 fails: 16 lookups passes: 2
"own" (another alert, a deadline, the wake's alert held) fails: the sleep left on the other alert passes
"failing wait" / "a wait that does not answer" fails: stalled / the park used passes
"failing alert" fails: the untimed sleep never ended passes
"budget" (hos_spin_ns() calls a wait) fails: 2 and 2 (at e4b273b1) passes: 1 and 1

Risks

Off (the default), the events', semaphores' and queues' sleeps and wakes go through hos_park_sleep and
hos_park_wake, which call Windows' condition variable as before after one test of a switch read once (its first
caller runs the setup once, the others wait for it); spin_until_set returns whether it spun. On, the park relies on
two undocumented ntdll calls and two words of KUSER_SHARED_DATA (the unparked processor count and CyclesPerYield,
read as ntdll's own condition variable reads them); a missed wake would stall a guest thread, and a stale alert would
wake another wait on the thread. Each sleep the park blocks and then wakes makes one more zero-time
NtWaitForAlertByThreadId (to take a pending alert), and a long sleep wakes every 100 ms to look at its state.
Turn it off by leaving HOS_CV_PARK unset; a revert is enough. Players notice nothing: the switch is not set for them.

Checklist

Ready for review

  • One topic, on current main; the title is a Conventional Commit.
  • Tested on my own PC, in the game for a change the game runs, as Testing says. (the game run is still to come)
  • mise run test passes (any failure that also fails on main is named). (the checks reading the change ran, each failure named and failing on main too; not the whole suite)
  • Measured against main, or Measurements says why there is nothing to measure. (not measured yet: an experiment, off)
  • No game files, build output or hand-edited generated files; third_party/ untouched; nothing GPL-incompatible.

When it applies

  • A switch added or changed: it ships at the default the measurements chose, with its row in gpu/vk/switches.def. (the row is there; off as an experiment, not measured yet)
  • A check under tests/ that reads changed source is updated, not weakened.
  • Host code builds on Windows (llvm-mingw) and Linux, with UTF-8 paths. (only its functions compiled, in the checks, with llvm-mingw clang; no host or Linux build here)
  • Docs updated.
  • AI assistance recorded with Assisted-by: / Co-Authored-By: trailers.

Ready to merge

  • Everything in Still to test has run, its results in Tested.
  • Tier 1 hardware it can affect is tested (docs/dev/standards.md, "Hardware tiers").
  • Approved in review, every comment answered.
  • CI passes on the head; on current main, measured again or a comment says why the numbers still hold.
## Summary A guest wait on an event, a semaphore or a message queue spins for its flag first (`spin_until_set`) and then sleeps with `SleepConditionVariableSRW`, which spins again before it blocks: 10240 cycles of pause in ntdll (`RtlSleepConditionVariableSRW`: 10240 / `KUSER_SHARED_DATA.CyclesPerYield` pauses), after the wait's own spin has already missed. This adds an experiment, off by default: with `HOS_CV_PARK=1` (Windows) those sleeps go to a park in `host/hos_thread.c` (`hos_park_sleep`, `hos_park_wake`) that uses the condition variable's own block and wake (`NtWaitForAlertByThreadId`, `NtAlertThreadByThreadId`) without its second spin when the caller has just spun (a wait's first sleep after its own spin, or after the spin policy left that spin out). The waiters are an oldest-first list from the cv's word under one of 64 striped SRW locks; a wake that finds a waiter still spinning makes no system call. A woken waiter takes its virtual core back, then the object's lock once (`hos_cv_sleep` takes it, lets it go for the core and takes it again). Mutexes, guest condition variables, read-write locks, timers and WaitAny stay on Windows' condition variable. Unset, `=0`, an ntdll without the two calls, or a wait that does not answer at setup: everything is as on `main` (`hos_park_sleep` is `hos_cv_sleep`, `hos_park_wake` the Win32 wake). Linux keeps its condition variable (the same shims). How the park keeps a wake its own (after the review): the switch and the two calls are set up once (`InitOnceExecuteOnce`, the mode published after the pointers). An alert is the thread's, so a sleep the park has woken leaves only when its wake is done with it (`PARK_DONE`, set after the alert is sent or has failed) and then takes any alert still pending for its thread with a zero-time wait, so none reaches the thread's next wait: Windows' WaitOnAddress returns at the first alert its thread gets, its wait block still listed, and the game's job barriers (`host/oot_barrier.c`) wait that way on the same threads (on the local branch this came from, a start with the first version of the park stopped with an access violation in `WakeByAddressAll` under `hos_oot_barrier_wake`, one start of five). A failed alert or a failing wait is logged once; a blocked sleep waits at most 100 ms at a time and looks at its state, and a failing wait goes on in 1 ms steps until its wake or deadline, on the list all along. `spin_until_set` reads the thread's spin budget once (`spin_begin` hands it on): with the park off it makes as many `hos_spin_ns()` calls as `main` when the spin is left out and one fewer when it spins. Six commits: the park; a test that no alert is left for a thread after a park sleep; its fix; tests for the review's points (first callers, alert ownership, failing wait and alert, timed semaphore and queue peek, first-sleep policy, one budget lookup); their fix; the budget. From the local branch `exp/x86-asm-sync`: c565d05a (the park), 78c83308 (its first part: one lock take after a park wake), 939b5324 (the test), 76de03b5 (the fix). There the park was on by default; here it is off. Read first: the park block after `hos_cv_sleep` in `host/hos_thread.c` (`park_init`, `park_sleep`, `hos_park_wake`, `hos_park_sleep`); in `host/nn_os_sync.c` `spin_begin`, `wait_until`'s `park` argument, `spin_until_set`'s return, `event_wait`, `sema_acquire`, `queue_get`; `tests/test_event_park.py` (new). ## Testing **Tested:** Windows 11, i7-9700, RTX 4060 (NVIDIA 610.88), llvm-mingw clang 23.1.5; game-free checks only. This change has not run in the game on this PC yet (Still to test). Each check ran in a hidden console of its own, through the PC's job queue (job fb3-1970-1), on this branch rebased onto `main` @ f1f2dc5b (head ff7e2362): - `python -I -B tests/test_event_park.py` at the test commit 1e2adc46 (job fb3-1970-1): exit 1, each new part failing as it should without its fix: "budget" ("spin_until_set read the spin budget 2 and 2 times, not once", with and without a budget), "first callers" with `HOS_CV_PARK=1` ("8 first callers looked ntdll's two calls up 16 times (want 2)"), "own" ("a untimed park sleep left on another alert for its thread before its wake's alert was sent"), "failing wait" ("stalled": the 30 ms sleep never returned), "failing alert" ("an untimed sleep whose wake's alert failed ends (0 of 1)"), "a wait that does not answer" ("a park whose wait answers STATUS_NOT_IMPLEMENTED is used"). The races, "timed" and "policy" pass there. - The same at e4b273b1 (the park fix, before the budget commit; job fb3-1970-2): exit 1, only "budget" failing (2 and 2). - The same at the head ff7e2362 (job fb3-1970-2): exit 0, every part passing, unset, `HOS_CV_PARK=1`, both with no spin first and `HOS_CV_PARK=0` ("ok", 20,000 auto-clear signals, 2,000 manual-clear rounds of 4, 5,000 timed-wait signals, 60,000 through a semaphore and a queue, each message once); "timed" 68 of 100 racing releases taken and 32 after the acquire timed out, 76 of 100 racing sends seen and 24 after the peek timed out (on the park); "policy": first sleeps told spun=1 and 0 pauses before blocking with a budget, spun=0 and 66 pauses with none; "budget" 1 and 1; "first callers" 2 lookups; "own", "failing wait" (2 waits for a 30 ms sleep), "failing alert" pass; with the wait answering `STATUS_NOT_IMPLEMENTED` the sleeps stay on Windows' condition variable; both controls fail. With `CC` set to llvm-mingw's clang (job fb3-1970-1, before the own part's check was moved ahead of the held wake, where its control crashed instead of failing): every part passed the same way. - `test_cv_handoff_threads`, `test_cv_signal_race`, `test_cv_signal_early`, `test_event_handoff`, `test_multiwait_wake`, `test_os_lifted_siblings`, `test_oot_barrier`, `test_switch_table` (job fb3-1970-1) and `test_spin_by_role` (job fb3-1970-2, its wiring check now reading the budget where `spin_begin` reads it) at the head: exit 0. - `python -I -B tests/run_all.py --strict --changed origin/main` (job fb3-1970-1, at the head before the two test-only changes of job fb3-1970-2): exit 1, "69 passed, 19 failed, 4 skipped". Two were this PR's and are fixed since (`test_event_park`'s own-part control, `test_spin_by_role`'s wiring check, both passing in job fb3-1970-2); 14 of the other 17 fail on `main` @ f1f2dc5b too (run by name in the same job): `test_crash_dialog`, `test_crash_record`, `test_dev_ab`, `test_dev_packs`, `test_frame_overlap`, `test_host_guest_writes`, `test_libc_hooks`, `test_linux_pressure`, `test_mods_list`, `test_profiling_mode`, `test_slow_frames`, `test_vcore_escape`, `test_vcore_mask0`, `test_walk_key`. The other three (`test_present_queue_default`, `test_target_reuse_memory`, `test_vk_occlusion_queries`) compiled nothing in that run nor in the run on `main` (no Vulkan headers on the job's path: "not found: vulkan_headers (--strict)"). Rerun with the headers (job fb3-vk-1, `RX_CPM` set) at the head ff7e2362 and on `main` @ f1f2dc5b, each fails at both with the same first error: `test_present_queue_default` "srccut: '{TAB_DEBUG, "Present queue", &opts.present_queue, ...' is not in gpu/vk/vk_menu.cpp"; `test_target_reuse_memory` 2 of its 8 tests ("'vmaGetAllocationInfo2' unexpectedly found" in the VMA source, and its fixture's "use of undeclared identifier 'destroy_image_later'"); `test_vk_occlusion_queries` "the ring fixture did not compile" ("no member named 'policy'"). None fails on this branch alone. **Still to test:** - Windows 11, this PC (i7-9700, RTX 4060 (NVIDIA 610.88)): repeated starts through the title screen with `HOS_CV_PARK=1` and unset (the earlier local branch failed one start in five), then Castle Town's market, `main` @ f1f2dc5b against this PR with `HOS_CV_PARK` unset and with it set, at least two warmed, interleaved runs each, CPU per frame, average fps, 1% lows, median and p99 frame times, expecting the log line "os: HOS_CV_PARK: events, semaphores and message queues sleep on a park that does not spin again after their own spin" with it set, no crash (none under `hos_oot_barrier_wake` in particular), and the unset run within `main`'s spread (by hand) - Linux: build the host (the park's shims) and run `python3 -I -B tests/run_all.py --strict --changed origin/main`, expecting `tests/test_event_park.py` skipped (Windows only) and the cut checks passing (by hand) - Windows 10, and Windows PCs with other CPUs (an AMD CPU, fewer cores): Castle Town's market, `main` against this PR unset and with `HOS_CV_PARK=1`, expecting the `os: HOS_CV_PARK` log line, no crash and frame times as unset or better ## Measurements **Verdict:** not measured: no micro-benchmark was run for this PR, and no play measurement exists yet. `HOS_CV_PARK` ships off (kind `attempt`), an experiment for A/B runs; `tests/test_event_park.py` has no timing mode. The disabled head against `main` and the in-game A/B (`main`, unset, `HOS_CV_PARK=1`) are in Still to test: a game-free benchmark of the wait's entry would time a stand-in for `hos_spin_ns()` (it reads the host's thread table), not the call itself. The table holds the fixes' tests, before and after. Baseline: this PR's test commits, on `main` @ f1f2dc5b: `main` has no park for the tests to run on. Setup: Windows 11, i7-9700, RTX 4060 (NVIDIA 610.88), llvm-mingw clang 23.1.5; `tests/test_event_park.py`, one run each side, each in a hidden console of its own through the PC's job queue. | | Before, the test commits: this PR @ 1e2adc46 (and e4b273b1 for the budget) | After: this PR @ ff7e2362 | |---|---|---| | `tests/test_event_park.py` "first callers", `HOS_CV_PARK=1` | fails: 16 lookups | passes: 2 | | "own" (another alert, a deadline, the wake's alert held) | fails: the sleep left on the other alert | passes | | "failing wait" / "a wait that does not answer" | fails: stalled / the park used | passes | | "failing alert" | fails: the untimed sleep never ended | passes | | "budget" (`hos_spin_ns()` calls a wait) | fails: 2 and 2 (at e4b273b1) | passes: 1 and 1 | ## Risks Off (the default), the events', semaphores' and queues' sleeps and wakes go through `hos_park_sleep` and `hos_park_wake`, which call Windows' condition variable as before after one test of a switch read once (its first caller runs the setup once, the others wait for it); `spin_until_set` returns whether it spun. On, the park relies on two undocumented ntdll calls and two words of `KUSER_SHARED_DATA` (the unparked processor count and `CyclesPerYield`, read as ntdll's own condition variable reads them); a missed wake would stall a guest thread, and a stale alert would wake another wait on the thread. Each sleep the park blocks and then wakes makes one more zero-time `NtWaitForAlertByThreadId` (to take a pending alert), and a long sleep wakes every 100 ms to look at its state. Turn it off by leaving `HOS_CV_PARK` unset; a revert is enough. Players notice nothing: the switch is not set for them. ## Checklist ### Ready for review - [x] One topic, on current `main`; the title is a Conventional Commit. - [ ] Tested on my own PC, in the game for a change the game runs, as Testing says. (the game run is still to come) - [ ] `mise run test` passes (any failure that also fails on `main` is named). (the checks reading the change ran, each failure named and failing on `main` too; not the whole suite) - [ ] Measured against `main`, or Measurements says why there is nothing to measure. (not measured yet: an experiment, off) - [x] No game files, build output or hand-edited generated files; `third_party/` untouched; nothing GPL-incompatible. ### When it applies - [ ] A switch added or changed: it ships at the default the measurements chose, with its row in `gpu/vk/switches.def`. (the row is there; off as an experiment, not measured yet) - [x] A check under `tests/` that reads changed source is updated, not weakened. - [ ] Host code builds on Windows (llvm-mingw) and Linux, with UTF-8 paths. (only its functions compiled, in the checks, with llvm-mingw clang; no host or Linux build here) - [x] Docs updated. - [x] AI assistance recorded with `Assisted-by:` / `Co-Authored-By:` trailers. ### Ready to merge - [ ] Everything in Still to test has run, its results in Tested. - [ ] Tier 1 hardware it can affect is tested (docs/dev/standards.md, "Hardware tiers"). - [ ] Approved in review, every comment answered. - [ ] CI passes on the head; on current `main`, measured again or a comment says why the numbers still hold.
A wait on an event, a semaphore or a message queue spins for its flag first (spin_until_set) and then sleeps with
SleepConditionVariableSRW, which spins again for 10240 cycles of pause before it blocks (ntdll's
RtlSleepConditionVariableSRW: 10240 / KUSER_SHARED_DATA.CyclesPerYield pauses). With HOS_CV_PARK=1 those sleeps go to a
park in host/hos_thread.c (hos_park_sleep, hos_park_wake) that blocks at once with NtWaitForAlertByThreadId when the
caller has just spun (spin_until_set now says so) and is woken with NtAlertThreadByThreadId: the condition variable's
own block and wake, without its second spin. The waiters are an oldest-first list from the cv's word under one of 64
striped SRW locks; a wake that finds a waiter still spinning makes no call. A woken waiter takes its virtual core back
and then the object's lock once (hos_cv_sleep takes it, lets it go for the core and takes it again).

Mutexes, guest condition variables (wait_until's park < 0), read-write locks, timers and WaitAny stay on Windows'
condition variable. Unset (the default), HOS_CV_PARK=0, or an ntdll without the two calls: every sleep and wake is
Windows' condition variable's, as before (hos_park_sleep is hos_cv_sleep, hos_park_wake WakeConditionVariable or
WakeAllConditionVariable). Linux: the same shims, its condition variable.

tests/test_event_park.py (Windows) runs the real park and the real event, semaphore and queue code on threads, unset
and with HOS_CV_PARK=1, with and without the spin first: timeouts, oldest-first wakes, no lost wake over 20,000
auto-clear signals, manual-clear rounds, timed waits, a semaphore and a queue; controls (a wake of one of several
sleepers, a park wake without its alert) fail. The checks that cut wait_until, spin_until_set or the semaphore's wake
(test_cv_signal_race, test_cv_handoff_threads, test_event_handoff, test_multiwait_wake, test_os_lifted_siblings) take
the new signatures and the park's stubs.

From the local branch exp/x86-asm-sync: c565d05a (the park) and 78c83308's first bullet (one lock take after a park
wake). There it was on by default; here it is off: its in-play A/B on that branch measured no difference.

Assisted-by: Claude Code
tests/test_event_park.py asks, before and after every park sleep of its races (HOS_CV_PARK=1), whether an alert is
still pending for the thread (a zero-time NtWaitForAlertByThreadId), and fails when one was.

It fails at the commit before this one: a wake that finds its sleeper blocked sets the sleeper's state and then
alerts the thread; the sleeper leaves as soon as it sees the state, so an alert sent after that stays pending for the
thread. Windows' WaitOnAddress returns at the first alert its thread gets, its wait block still on the address's list,
and the next wake of that address reads the block where the stack has moved on. The game's job barriers
(host/oot_barrier.c) wait that way on the same threads: on the local branch this came from, a start of the game with
the park on stopped at the title screen with an access violation in WakeByAddressAll's walk of the waiters, under
hos_oot_barrier_wake, one start of five.

From the local branch exp/x86-asm-sync: 939b5324.

Assisted-by: Claude Code
fix(os): a park sleep leaves no alert for its thread, so the job barriers' WaitOnAddress is not woken by one
Some checks failed
PR description / Description follows the template (pull_request_target) Successful in 16s
Website / Website on Cloudflare (pull_request_target) Has been skipped
CI / Conventional Commit title (pull_request) Has been cancelled
CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Has been cancelled
CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Has been cancelled
CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Has been cancelled
CI / Workflow lint (actionlint) (pull_request) Has been cancelled
CI / Website build and checks (pull_request) Has been cancelled
Guide screenshots / Guide images and local UI source review (pull_request) Has been cancelled
Third-party parts / As their notes say (pull_request) Has been cancelled
Third-party parts / The same as upstream (pull_request) Has been cancelled
Website / Website on Cloudflare (pull_request) Has been cancelled
82d36ed100
A park wake (HOS_CV_PARK=1) that finds its sleeper blocked sets the sleeper's state and then alerts the thread
(NtAlertThreadByThreadId). park_sleep looked at the state before each wait, and again after a timeout that found it
off the list, and left at once when it said woken: an alert sent after that stayed pending for the thread. Windows'
WaitOnAddress returns at the first alert its thread gets with its wait block still on the address's list, and the next
wake of that address reads the block where the stack has moved on. The game's job barriers wait that way on the same
threads (host/oot_barrier.c): on the local branch this came from, a start with the park on stopped at the title screen
with an access violation under hos_oot_barrier_wake -> WakeByAddressAll, one start of five.

The sleep now waits first and leaves only at an alerted wait whose state says woken: a wake that finds PARK_BLOCK
always alerts, so there is always one to take, and it is taken once. A timeout that finds the sleeper on the list
takes it off and leaves as before (no wake has it: no alert).

tests/test_event_park.py's check from the commit before passes with it. HOS_CV_PARK unset: nothing changes.

From the local branch exp/x86-asm-sync: 76de03b5.

Assisted-by: Claude Code
Member

Possibly related, for context:

  • PR #1967 WIP: feat(os): HOS_WAKE_LAT also counts the event sleepers that slept more than once — Adjacent event-wait diagnostics in the same sync code
  • issue #1505 Every event, queue and semaphore signal locks and wakes all waiters, and takes a global lock while any thread waits in a multi-wait — Open performance issue on event, semaphore, and queue wake paths
  • PR #1662 perf(host): the game's job barriers wait for their mask words instead of sleeping 20 us (HOS_OOT_BARRIER_WAIT) — Merged job-barrier wait change shares thread wait/wake paths
Possibly related, for context: - PR #1967 WIP: feat(os): HOS_WAKE_LAT also counts the event sleepers that slept more than once — Adjacent event-wait diagnostics in the same sync code - issue #1505 Every event, queue and semaphore signal locks and wakes all waiters, and takes a global lock while any thread waits in a multi-wait — Open performance issue on event, semaphore, and queue wake paths - PR #1662 perf(host): the game's job barriers wait for their mask words instead of sleeping 20 us (HOS_OOT_BARRIER_WAIT) — Merged job-barrier wait change shares thread wait/wake paths
BeanBot left a comment

Recommendation: Request changes. The opt-in Windows park can race during initialization and can lose the correspondence between a wake and its thread alert. It adds a second synchronization implementation to the host, and there is no measured in-game gain against main.

Adds an off-by-default Windows experiment for guest events, semaphores and message queues. The park can skip a second spin and lets a woken thread reacquire its virtual core before retaking the object lock. The cost is a custom waiter list, striped locks and an undocumented alert protocol on paths that also serve audio, video and frame delivery.

@fix will do (13)

  1. L1 host/hos_thread.c:2272: The atomic flag publishes the pointers but does not protect their initialization. (more on the line)
  2. L2 host/hos_thread.c:2323: STATUS_ALERTED and PARK_WOKEN do not establish that the alert came from this wake. (more on the line)
  3. L3 host/hos_thread.c:2326: An unexpected status from the native wait retries immediately with the same inputs. (more on the line)
  4. L4 host/hos_thread.c:2359: The wake detaches the waiter, marks it woken and ignores the alert's return status. (more on the line)
  5. L5 (minor) host/hos_thread.c:2299: The spin divisor is not the advertised CyclesPerYield calculation. (more on the line)
  6. L6 tests/test_event_park.py:35: Both the suite declaration and the direct default hard-code gcc. (more on the line)
  7. L7 tests/test_event_park.py:226: The teardown writes signaled with a plain store while event_peek can still read or claim it without taking the lock. (more on the line)
  8. L8 (minor) tests/test_event_park.py:224: release_all treats WAIT_FAILED like a successful join. (more on the line)
  9. L9 tests/test_event_park.py:189: The check for blocking without a second spin waits for PARK_BLOCK before waking the thread. (more on the line)
  10. L10 (minor) tests/test_event_park.py:92: The stale-alert regression check depends on stress scheduling. (more on the line)
  11. L11 (minor) docs/dev/reference/switches.md:282: The switch reference and table describe the removed spin as about 2.5 µs without identifying the CPU or Windows build. (more on the line)
  12. L12 (its part an edit can do) host/hos_thread.c:2293: Record enqueue depth and lock time for the experiment. If contention shows up, keep a tail for constant-time insertion instead of traversing the list under the stripe lock.
  13. L14 (minor): Testing says 87 checks read the changed files, but 72 passed, 16 failed and 4 skipped totals 92. Clarify whether the figures count different sets and record the full suite and lint results against the revised head.

For the author (2) (a run, a measurement or a decision)

  • L12 (minor) host/hos_thread.c:2293: The test result measures the stale-alert fix. (more on the line)
  • L13 (minor): The repository requires an author-PC game run before opening a runtime PR. The author reports fixture runs only. Exercise repeated launches through the title screen with the park on and off, since the earlier local branch failed once in five launches, then cover the Market and audio, video and input on the affected Tier 1 setups. Record full Windows and Linux host build results and the suite results, with a Steam Deck smoke run for the Linux path.

Checks: Still running: CI / Conventional Commit title (pull_request), CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request), CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request), CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request), CI / Workflow lint (actionlint) (pull_request), CI / Website build and checks (pull_request), Guide screenshots / Guide images and local UI source review (pull_request), Third-party parts / As their notes say (pull_request), Third-party parts / The same as upstream (pull_request), Website / Website on Cloudflare (pull_request).

More: does it help, other systems, switches, what was read

Does it help: Unproven. On Windows 11 with an i7-9700 and RTX 4060, the author reports 27 stale alerts in one game-free run of the park before its fix and zero after it on an earlier rebase. The earlier local branch reportedly saw no in-play difference. There are no matched runs against 6e1521a7 for CPU time, wake latency, average FPS, 1% lows or median and p99 frame times.

Other systems: Normal, light and system events share the change, including the audio renderer, vsync, video and GPU completion signals. The job barriers use WaitOnAddress on some of the same threads, so a pending alert can affect a later wait. Queue producers and consumers share one park. WaitAny still waits on its own condition variable and receives the same notifications. Mutexes, guest condition variables, reader-writer locks and timers still use their existing waits. Linux and Steam Deck use the original futex-backed condition variable through the new wrappers. There is no direct renderer, save, input, build or VRAM change. Windows 10, AMD hardware, lower-core machines, Linux and Steam Deck have no reported integration coverage for this revision.

Switches: HOS_CV_PARK is a read-once Windows developer switch, classified as attempt, and listed in switches.def and the developer reference. Unset and =0 select the existing condition variable, as does a missing ntdll export. Off is the safe interim value while correctness is open. The repository requires measurements to choose a shipping default. HOS_SPIN_US and HOS_SPIN_OTHER_US retain their defaults and determine whether the park can omit its spin.

Reviewed: 82d36ed100 against 6e1521a70a: Read AGENTS.md, CONTRIBUTING.md, docs/dev/standards.md, the Forgejo template and tests/test_event_park.py in full. Read all three commit messages and all 11 file diffs, including every changed region of host/hos.h, host/hos_thread.c, host/nn_os_sync.c and the five existing fixtures. Traced object creation and reuse, wait and wake callers, virtual-core handoff, Tracy lock wrappers, the Linux shim, the job barrier, switch parsing and test-runner compiler discovery. Unrelated portions of existing files were outside the review. The supplied run summaries were the available evidence. There was no docs/review-checklist.md or adjacent Tooling checkout, and there were no changed translations, generated files or prior reviews.; nothing run (a code review).

Checked and fine:

  • One topic. All three commits carry Assisted-by trailers. No game-derived files, build products, shader outputs, third-party files or renderer API changes appear in the diff.
  • The switch is registered and documented. The fixture strips inherited HOS_ and T_ settings from its child runs and explicitly covers unset, zero and enabled.
  • Waiters register before dropping the object lock. The park drops the stripe before reacquiring a virtual core, and callers recheck readiness under the object lock.
  • Finite waits still clamp a millisecond step below INFINITE. WaitAny retains its readiness checks and notifications, and the guest condition-variable handoff keeps its existing wait path.

Testing: Read-only review. git diff --check passed. The author reports that the Windows fixtures passed on an earlier rebase and that the 16 selected-suite failures also failed on its base. The reported range-diff was unchanged through 82d36ed1. The supplied CI checks are still running, with no head-only failures listed.

Overlaps and merge order: None found in the checkout history or supplied PR material.


Automated code review by BeanBot (gpt-6.2-sol), on the pull request's opening. Nothing was run; a maintainer decides.

**Recommendation: Request changes.** The opt-in Windows park can race during initialization and can lose the correspondence between a wake and its thread alert. It adds a second synchronization implementation to the host, and there is no measured in-game gain against main. Adds an off-by-default Windows experiment for guest events, semaphores and message queues. The park can skip a second spin and lets a woken thread reacquire its virtual core before retaking the object lock. The cost is a custom waiter list, striped locks and an undocumented alert protocol on paths that also serve audio, video and frame delivery. **`@fix` will do (13)** 1. **L1** `host/hos_thread.c:2272`: The atomic flag publishes the pointers but does not protect their initialization. (more on the line) 2. **L2** `host/hos_thread.c:2323`: STATUS_ALERTED and PARK_WOKEN do not establish that the alert came from this wake. (more on the line) 3. **L3** `host/hos_thread.c:2326`: An unexpected status from the native wait retries immediately with the same inputs. (more on the line) 4. **L4** `host/hos_thread.c:2359`: The wake detaches the waiter, marks it woken and ignores the alert's return status. (more on the line) 5. **L5** (minor) `host/hos_thread.c:2299`: The spin divisor is not the advertised CyclesPerYield calculation. (more on the line) 6. **L6** `tests/test_event_park.py:35`: Both the suite declaration and the direct default hard-code gcc. (more on the line) 7. **L7** `tests/test_event_park.py:226`: The teardown writes signaled with a plain store while event_peek can still read or claim it without taking the lock. (more on the line) 8. **L8** (minor) `tests/test_event_park.py:224`: release_all treats WAIT_FAILED like a successful join. (more on the line) 9. **L9** `tests/test_event_park.py:189`: The check for blocking without a second spin waits for PARK_BLOCK before waking the thread. (more on the line) 10. **L10** (minor) `tests/test_event_park.py:92`: The stale-alert regression check depends on stress scheduling. (more on the line) 11. **L11** (minor) `docs/dev/reference/switches.md:282`: The switch reference and table describe the removed spin as about 2.5 µs without identifying the CPU or Windows build. (more on the line) 12. **L12** (its part an edit can do) `host/hos_thread.c:2293`: Record enqueue depth and lock time for the experiment. If contention shows up, keep a tail for constant-time insertion instead of traversing the list under the stripe lock. 13. **L14** (minor): Testing says 87 checks read the changed files, but 72 passed, 16 failed and 4 skipped totals 92. Clarify whether the figures count different sets and record the full suite and lint results against the revised head. **For the author (2)** (a run, a measurement or a decision) - **L12** (minor) `host/hos_thread.c:2293`: The test result measures the stale-alert fix. (more on the line) - **L13** (minor): The repository requires an author-PC game run before opening a runtime PR. The author reports fixture runs only. Exercise repeated launches through the title screen with the park on and off, since the earlier local branch failed once in five launches, then cover the Market and audio, video and input on the affected Tier 1 setups. Record full Windows and Linux host build results and the suite results, with a Steam Deck smoke run for the Linux path. **Checks:** Still running: `CI / Conventional Commit title (pull_request)`, `CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request)`, `CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request)`, `CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request)`, `CI / Workflow lint (actionlint) (pull_request)`, `CI / Website build and checks (pull_request)`, `Guide screenshots / Guide images and local UI source review (pull_request)`, `Third-party parts / As their notes say (pull_request)`, `Third-party parts / The same as upstream (pull_request)`, `Website / Website on Cloudflare (pull_request)`. <details> <summary>More: does it help, other systems, switches, what was read</summary> **Does it help:** Unproven. On Windows 11 with an i7-9700 and RTX 4060, the author reports 27 stale alerts in one game-free run of the park before its fix and zero after it on an earlier rebase. The earlier local branch reportedly saw no in-play difference. There are no matched runs against 6e1521a7 for CPU time, wake latency, average FPS, 1% lows or median and p99 frame times. **Other systems:** Normal, light and system events share the change, including the audio renderer, vsync, video and GPU completion signals. The job barriers use WaitOnAddress on some of the same threads, so a pending alert can affect a later wait. Queue producers and consumers share one park. WaitAny still waits on its own condition variable and receives the same notifications. Mutexes, guest condition variables, reader-writer locks and timers still use their existing waits. Linux and Steam Deck use the original futex-backed condition variable through the new wrappers. There is no direct renderer, save, input, build or VRAM change. Windows 10, AMD hardware, lower-core machines, Linux and Steam Deck have no reported integration coverage for this revision. **Switches:** HOS_CV_PARK is a read-once Windows developer switch, classified as attempt, and listed in switches.def and the developer reference. Unset and =0 select the existing condition variable, as does a missing ntdll export. Off is the safe interim value while correctness is open. The repository requires measurements to choose a shipping default. HOS_SPIN_US and HOS_SPIN_OTHER_US retain their defaults and determine whether the park can omit its spin. **Reviewed:** 82d36ed1000c against 6e1521a70a32: Read AGENTS.md, CONTRIBUTING.md, docs/dev/standards.md, the Forgejo template and tests/test_event_park.py in full. Read all three commit messages and all 11 file diffs, including every changed region of host/hos.h, host/hos_thread.c, host/nn_os_sync.c and the five existing fixtures. Traced object creation and reuse, wait and wake callers, virtual-core handoff, Tracy lock wrappers, the Linux shim, the job barrier, switch parsing and test-runner compiler discovery. Unrelated portions of existing files were outside the review. The supplied run summaries were the available evidence. There was no docs/review-checklist.md or adjacent Tooling checkout, and there were no changed translations, generated files or prior reviews.; nothing run (a code review). **Checked and fine:** - One topic. All three commits carry Assisted-by trailers. No game-derived files, build products, shader outputs, third-party files or renderer API changes appear in the diff. - The switch is registered and documented. The fixture strips inherited HOS_ and T_ settings from its child runs and explicitly covers unset, zero and enabled. - Waiters register before dropping the object lock. The park drops the stripe before reacquiring a virtual core, and callers recheck readiness under the object lock. - Finite waits still clamp a millisecond step below INFINITE. WaitAny retains its readiness checks and notifications, and the guest condition-variable handoff keeps its existing wait path. **Testing:** Read-only review. git diff --check passed. The author reports that the Windows fixtures passed on an earlier rebase and that the 16 selected-suite failures also failed on its base. The reported range-diff was unchanged through 82d36ed1. The supplied CI checks are still running, with no head-only failures listed. **Overlaps and merge order:** None found in the checkout history or supplied PR material. </details> --- <sub>Automated code review by BeanBot (gpt-6.2-sol), on the pull request's opening. Nothing was run; a maintainer decides.</sub>
@ -279,6 +279,7 @@ The memory budget and its `vk: budget:` lines are in [architecture](architecture
| `HOS_CORES=<1..8>` | The cores the game sees (6) |
| `HOS_RESERVE_CORE=0/1` | Keep a core for the replay (one is kept with 8 or more logical processors, but none on Windows CPUs whose L3 caches differ, unless `=1`) |
| `HOS_AB=<switch>=<value>[,<s>[,<warm-up s>]]` | Alternate a switch within one run for an A/B ([developing](../developing.md#an-ab-within-one-run)) |
| `HOS_CV_PARK=1` | Experiment, off, Windows: the game's events, semaphores and message queues sleep on a park that blocks at once after their own spin (`NtWaitForAlertByThreadId`, `NtAlertThreadByThreadId`: `hos_park_sleep` in `host/hos_thread.c`) instead of on Windows' condition variable, which spins 10240 cycles more (about 2.5 us) before it blocks. Other sleeps (a full queue's put, every wait with `HOS_SPIN_US=0`) keep that spin; mutexes, condition variables, read-write locks and WaitAny stay on Windows' condition variable. Read once; set, the log says which way they sleep (`os: HOS_CV_PARK: ...`) |
Member

L11 (minor): The switch reference and table describe the removed spin as about 2.5 µs without identifying the CPU or Windows build. The implementation also skips the spin when the adaptive policy skips the guest spin, and retains a park spin on timed semaphore waits and on sleeps after the first. Describe those cases and label the duration as an estimate for the reported setup.

**L11** (minor): The switch reference and table describe the removed spin as about 2.5 µs without identifying the CPU or Windows build. The implementation also skips the spin when the adaptive policy skips the guest spin, and retains a park spin on timed semaphore waits and on sleeps after the first. Describe those cases and label the duration as an estimate for the reported setup.
@ -2244,0 +2269,4 @@
v = 0;
if (env_on("HOS_CV_PARK")) {
const HMODULE nt = GetModuleHandleA("ntdll.dll");
park_wait = (ParkWaitFn)(void *)GetProcAddress(nt, "NtWaitForAlertByThreadId");
Member

L1: The atomic flag publishes the pointers but does not protect their initialization. Two first callers on different objects can both read -1. One can publish and start using park_wait while the other is still assigning the same plain function pointer. Initialize the switch and both exports once before worker threads start, or use a real once primitive. Exercise simultaneous first calls in the fixture, which currently initializes the park before creating workers.

**L1**: The atomic flag publishes the pointers but does not protect their initialization. Two first callers on different objects can both read -1. One can publish and start using park_wait while the other is still assigning the same plain function pointer. Initialize the switch and both exports once before worker threads start, or use a real once primitive. Exercise simultaneous first calls in the fixture, which currently initializes the park before creating workers.
@ -2244,0 +2290,4 @@
void **at = &((PCONDITION_VARIABLE)cv)->Ptr;
SRWLOCK *s = park_stripe(cv);
AcquireSRWLockExclusive(s);
while (*at) at = &((Parked *)*at)->next;
Member

L12 (minor): The test result measures the stale-alert fix. It provides no evidence of a player gain or of the cost when the experiment is off. Compare 6e1521a7 with the revised head both unset and enabled. Use interleaved runs in the Market and a heavy scene with matched settings, and report CPU per frame, wake latency, average FPS, 1% lows and median and p99 frame times on the affected Windows setups.

**L12** (minor): The test result measures the stale-alert fix. It provides no evidence of a player gain or of the cost when the experiment is off. Compare 6e1521a7 with the revised head both unset and enabled. Use interleaved runs in the Market and a heavy scene with matched settings, and report CPU per frame, wake latency, average FPS, 1% lows and median and p99 frame times on the affected Windows setups.
@ -2244,0 +2296,4 @@
ReleaseSRWLockExclusive(lk);
/* (as RtlSleepConditionVariableSRW: no spin with one processor unparked; KUSER_SHARED_DATA's UnparkedProcessorCount
and CyclesPerYield) */
const int spins = spun || *(volatile USHORT *)0x7FFE036A <= 1 ? 0 : 10240 / (*(volatile USHORT *)0x7FFE02D6 | 1);
Member

L5 (minor): The spin divisor is not the advertised CyclesPerYield calculation. For example, an even value of 2 is treated as 3, and zero produces 10,240 pauses. Export lookup also does not establish that the two shared-data fields have the expected layout on the running Windows build. Avoid the private offsets or validate the supported layout and use the exact nonzero divisor with a conservative fallback.

**L5** (minor): The spin divisor is not the advertised CyclesPerYield calculation. For example, an even value of 2 is treated as 3, and zero produces 10,240 pauses. Export lookup also does not establish that the two shared-data fields have the expected layout on the running Windows build. Avoid the private offsets or validate the supported layout and use the exact nonzero divisor with a conservative fallback.
@ -2244,0 +2320,4 @@
}
const LONG status = park_wait(cv, to);
if (status == 0x101 /* STATUS_ALERTED */) {
if (__atomic_load_n(&me.state, __ATOMIC_ACQUIRE) == PARK_WOKEN) break; /* (its wake's alert, taken) */
Member

L2: STATUS_ALERTED and PARK_WOKEN do not establish that the alert came from this wake. If a waiter receives an unrelated thread alert and the park waker publishes PARK_WOKEN before the waiter checks it, the waiter returns while its own alert is still to come. That later alert can reach the next job-barrier WaitOnAddress, the same failure mode reported in the PR. Use a waiter-specific notification whose delivery cannot be confused with another thread alert, and test an unrelated alert with the real wake held back.

**L2**: STATUS_ALERTED and PARK_WOKEN do not establish that the alert came from this wake. If a waiter receives an unrelated thread alert and the park waker publishes PARK_WOKEN before the waiter checks it, the waiter returns while its own alert is still to come. That later alert can reach the next job-barrier WaitOnAddress, the same failure mode reported in the PR. Use a waiter-specific notification whose delivery cannot be confused with another thread alert, and test an unrelated alert with the real wake held back.
@ -2244,0 +2323,4 @@
if (__atomic_load_n(&me.state, __ATOMIC_ACQUIRE) == PARK_WOKEN) break; /* (its wake's alert, taken) */
continue; /* (not this sleep's: none is left by the park, above; waited again as before) */
}
if (status != 0x102 /* STATUS_TIMEOUT */) continue;
Member

L3: An unexpected status from the native wait retries immediately with the same inputs. An implementation that exports the symbol but returns STATUS_NOT_IMPLEMENTED, or any persistent error, leaves the guest thread in a hot loop even after a timed wait's deadline. Establish which back end is usable before publishing it and define how an active park handles an unexpected wait result without dropping its registered waiter. Add a test that injects a persistent error.

**L3**: An unexpected status from the native wait retries immediately with the same inputs. An implementation that exports the symbol but returns STATUS_NOT_IMPLEMENTED, or any persistent error, leaves the guest thread in a hot loop even after a timed wait's deadline. Establish which back end is usable before publishing it and define how an active park handles an unexpected wait result without dropping its registered waiter. Add a test that injects a persistent error.
@ -2244,0 +2356,4 @@
while (w) { /* (off the list: only this wake reads them; each is the waiter's stack until its state is PARK_WOKEN) */
Parked *next = all ? (Parked *)w->next : NULL;
const DWORD tid = w->tid;
if (__atomic_exchange_n(&w->state, PARK_WOKEN, __ATOMIC_SEQ_CST) == PARK_BLOCK) park_alert((HANDLE)(uintptr_t)tid);
Member

L4: The wake detaches the waiter, marks it woken and ignores the alert's return status. If the alert fails, an infinite waiter has no notification and future wakes cannot find it. A timed waiter also switches to an infinite wait once it sees that it was detached. Use a wake protocol that handles delivery failure without abandoning the detached waiter, and exercise a failed alert in the fixture.

**L4**: The wake detaches the waiter, marks it woken and ignores the alert's return status. If the alert fails, an infinite waiter has no notification and future wakes cannot find it. A timed waiter also switches to an infinite wait once it sees that it was detached. Use a wake protocol that handles delivery failure without abandoning the detached waiter, and exercise a failed alert in the fixture.
@ -0,0 +32,4 @@
import tempfile
from pathlib import Path
RUN_ALL_ARGS = ["--cc", "gcc"]
Member

L6: Both the suite declaration and the direct default hard-code gcc. A supported bundled llvm-mingw installation with clang and CC set can still skip the new Windows check or fail to launch its compiler. Honor CC and use the project's Windows toolchain discovery for the default so the check runs with the supported compiler.

**L6**: Both the suite declaration and the direct default hard-code gcc. A supported bundled llvm-mingw installation with clang and CC set can still skip the new Windows check or fail to launch its compiler. Honor CC and use the project's Windows toolchain discovery for the default so the check runs with the supported compiler.
@ -0,0 +89,4 @@
now.QuadPart = 0;
while (park_on() && park_wait((void *)&alerts_left, &now) != 0x102 /* STATUS_TIMEOUT */) InterlockedIncrement(&alerts_left);
}
static int probed_park_sleep(void *cv, void *lk, unsigned ms, int spun) {
Member

L10 (minor): The stale-alert regression check depends on stress scheduling. Its two controls remove an alert or change wake-all, so neither forces the window fixed by the last commit. Pause the waker after it changes the blocked waiter's state and before it sends the alert, then prove the waiter stays in the park until delivery. A control restoring the old early return should fail on that exact assertion.

**L10** (minor): The stale-alert regression check depends on stress scheduling. Its two controls remove an alert or change wake-all, so neither forces the window fixed by the last commit. Pause the waker after it changes the blocked waiter's state and before it sends the alert, then prove the waiter stays in the park until delivery. A control restoring the old early return should fail on that exact assertion.
@ -0,0 +186,4 @@
WaitForMultipleObjects(4, h, TRUE, INFINITE);
if (parked()) for (int i = 0; i < 4; i++) CHECK(order[i] == i);
printf("wake-one order %ld %ld %ld %ld\n", (long)order[0], (long)order[1], (long)order[2], (long)order[3]);
if (parked()) { /* (a sleep that spun blocks at once: its wake is one alert) */
Member

L9: The check for blocking without a second spin waits for PARK_BLOCK before waking the thread. It passes if park_sleep always spins, and no assertion sees whether event, semaphore or queue callers pass the intended spun value. Instrument the spin and sleep paths and drive each caller to a known blocked state. Assert the first-sleep policy, including the zero-budget case, and cover each object's actual back end. The printed switch state alone would also pass if a semaphore or queue were switched back to Win32 for both sleep and wake.

**L9**: The check for blocking without a second spin waits for PARK_BLOCK before waking the thread. It passes if park_sleep always spins, and no assertion sees whether event, semaphore or queue callers pass the intended spun value. Instrument the spin and sleep paths and drive each caller to a known blocked state. Assert the first-sleep policy, including the zero-budget case, and cover each object's actual back end. The printed switch state alone would also pass if a semaphore or queue were switched back to Win32 for both sleep and wake.
@ -0,0 +221,4 @@
}
static void release_all(HANDLE *h, int n) { /* (stop set: the waiters still asleep are let out) */
__atomic_store_n(&stop, 1, __ATOMIC_SEQ_CST);
while (WaitForMultipleObjects(n, h, TRUE, 1) == WAIT_TIMEOUT) {
Member

L8 (minor): release_all treats WAIT_FAILED like a successful join. If one CreateThread call fails during an auto-clear case, the remaining workers can still take every signal, then the invalid handle makes teardown return before those workers exit. Check allocation, thread creation and wait results, join the workers before reusing the globals, and close their handles.

**L8** (minor): release_all treats WAIT_FAILED like a successful join. If one CreateThread call fails during an auto-clear case, the remaining workers can still take every signal, then the invalid handle makes teardown return before those workers exit. Check allocation, thread creation and wait results, join the workers before reusing the globals, and close their handles.
@ -0,0 +223,4 @@
__atomic_store_n(&stop, 1, __ATOMIC_SEQ_CST);
while (WaitForMultipleObjects(n, h, TRUE, 1) == WAIT_TIMEOUT) {
AcquireSRWLockExclusive(&ev->lk);
ev->signaled = 1;
Member

L7: The teardown writes signaled with a plain store while event_peek can still read or claim it without taking the lock. The auto-clear stress cases can therefore race in the fixture itself. Use the production atomic publication for the teardown flag. Also use atomic reads for the sleeps snapshots taken while workers increment that counter.

**L7**: The teardown writes signaled with a plain store while event_peek can still read or claim it without taking the lock. The auto-clear stress cases can therefore race in the fixture itself. Use the production atomic publication for the teardown flag. Also use atomic reads for the sleeps snapshots taken while workers increment that counter.
futondestroyer left a comment

Recommendation: Request changes. Fix how the Windows park initializes and delivers wakeups before testing it in the game. The existing exact-revision review documents those blockers. The disabled path also adds a spin-budget lookup that applies on Windows and x86-64 Linux. Existing review

The experiment lets events, semaphores and message queues block after their guest-side spin, and it takes the virtual core back before locking the object again. This could reduce CPU time spent waiting on a frame, but wakeup errors can stall the game or interfere with the next job-barrier wait on the same host thread.

Does it help? Unproven. The author reports 27 leftover alerts before the fix and none after it in one Windows fixture run per revision. That is evidence for the specific stale-alert fix. A matched game run against main is still needed to measure CPU time per frame, wake latency, average FPS, 1% lows, and median and p99 frame times.

Other systems. Normal, light and system events share the path. That includes notifications from host workers. The separate job barrier calls WaitOnAddress on some of the same threads. WaitAny retains its own condition variable and is still notified through poke_multiwait.

Switches. HOS_CV_PARK is a Windows developer experiment, off by default and documented in the switch table and reference. Linux and a Windows installation missing either export use the existing condition-variable implementation. Measurements are needed to choose a shipping default.

Reviewed. Head 82d36ed1000c93c10526953b3c316782ff2c2409 against merge base 6e1521a70a32603ed60a8263e4072625885669b7. Read all 11 changed files where they changed and traced the wait and wake paths, virtual-core handoff, Linux shim and job barrier with revision-matched local Graphify maps and direct source checks. Source review only. No changes made.

Checked and fine. The waiter registers before releasing the object lock. The park releases its stripe before retaking the virtual core and the object lock. The wake-one path selects the oldest waiter, and the guest predicate is rechecked under the object lock. Finite wait steps stay below INFINITE. The disabled path still calls the same platform condition-variable operations.

Blocking

  1. Keep the existing initialization and wake findings open. Concurrent first callers can race on the native function pointers. A thread-level alert can be mistaken for the wake of the current wait, and a failed alert after detaching a waiter leaves even a finite wait without a way to finish. Initialization · Alert ownership · Wait error · Wake error
  2. host/nn_os_sync.c:483 repeats hos_spin_ns() when adaptive spinning skips a wait or the thread has a zero spin budget. Events, untimed semaphores and blocking queue reads pay for this extra lookup on x86-64 Linux and on Windows with the experiment off. Both paths discard the result. Carry the budget or skip reason out of the first lookup, then measure the disabled head against main.

Testing. The reported Windows fixture passed at 06c08f8b, which the author says was rebased unchanged to this head. The strict changed-file run had 16 failures reported on both sides and skipped checks. CI was pending on this head, with the cached checks reporting blocked conditions. Full Windows and Linux host builds, in-game runs, and the full checks remain open. Add native Windows fixture coverage for timed semaphore acquisition and timed queue peek. Once the source issues are fixed, first validate repeated Windows starts through the title screen and Market play locally. Then compare exact main and the revised candidate with the switch on and off on supported Windows 10 and 11 CPUs, including AMD and a low-core system. The PR description proposes a farm run. The cached PR has no farm request comment.

Overlaps and merge order. The auto-clear event handoff branch already contains this exact park head. Fix and measure the park on its own before rebasing that branch, and test the four combinations of the two switches on the combined head. The wake-latency diagnostic changes the same event loop and conflicts in a read-only merge check. Preserve its sleep count and the park's first-sleep behavior when rebasing. Event handoff · Wake-latency diagnostic · Related wake-path issue

**Recommendation: Request changes.** Fix how the Windows park initializes and delivers wakeups before testing it in the game. The existing exact-revision review documents those blockers. The disabled path also adds a spin-budget lookup that applies on Windows and x86-64 Linux. [Existing review](https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/1970#issuecomment-62356) The experiment lets events, semaphores and message queues block after their guest-side spin, and it takes the virtual core back before locking the object again. This could reduce CPU time spent waiting on a frame, but wakeup errors can stall the game or interfere with the next job-barrier wait on the same host thread. **Does it help?** Unproven. The author reports 27 leftover alerts before the fix and none after it in one Windows fixture run per revision. That is evidence for the specific stale-alert fix. A matched game run against main is still needed to measure CPU time per frame, wake latency, average FPS, 1% lows, and median and p99 frame times. **Other systems.** Normal, light and system events share the path. That includes notifications from host workers. The separate job barrier calls `WaitOnAddress` on some of the same threads. WaitAny retains its own condition variable and is still notified through `poke_multiwait`. **Switches.** `HOS_CV_PARK` is a Windows developer experiment, off by default and documented in the switch table and reference. Linux and a Windows installation missing either export use the existing condition-variable implementation. Measurements are needed to choose a shipping default. **Reviewed.** Head `82d36ed1000c93c10526953b3c316782ff2c2409` against merge base `6e1521a70a32603ed60a8263e4072625885669b7`. Read all 11 changed files where they changed and traced the wait and wake paths, virtual-core handoff, Linux shim and job barrier with revision-matched local Graphify maps and direct source checks. Source review only. No changes made. **Checked and fine.** The waiter registers before releasing the object lock. The park releases its stripe before retaking the virtual core and the object lock. The wake-one path selects the oldest waiter, and the guest predicate is rechecked under the object lock. Finite wait steps stay below `INFINITE`. The disabled path still calls the same platform condition-variable operations. **Blocking** 1. Keep the existing initialization and wake findings open. Concurrent first callers can race on the native function pointers. A thread-level alert can be mistaken for the wake of the current wait, and a failed alert after detaching a waiter leaves even a finite wait without a way to finish. [Initialization](https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/1970#issuecomment-62344) · [Alert ownership](https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/1970#issuecomment-62345) · [Wait error](https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/1970#issuecomment-62346) · [Wake error](https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/1970#issuecomment-62347) 2. `host/nn_os_sync.c:483` repeats `hos_spin_ns()` when adaptive spinning skips a wait or the thread has a zero spin budget. Events, untimed semaphores and blocking queue reads pay for this extra lookup on x86-64 Linux and on Windows with the experiment off. Both paths discard the result. Carry the budget or skip reason out of the first lookup, then measure the disabled head against main. **Testing.** The reported Windows fixture passed at `06c08f8b`, which the author says was rebased unchanged to this head. The strict changed-file run had 16 failures reported on both sides and skipped checks. CI was pending on this head, with the cached checks reporting blocked conditions. Full Windows and Linux host builds, in-game runs, and the full checks remain open. Add native Windows fixture coverage for timed semaphore acquisition and timed queue peek. Once the source issues are fixed, first validate repeated Windows starts through the title screen and Market play locally. Then compare exact main and the revised candidate with the switch on and off on supported Windows 10 and 11 CPUs, including AMD and a low-core system. The PR description proposes a farm run. The cached PR has no farm request comment. **Overlaps and merge order.** The auto-clear event handoff branch already contains this exact park head. Fix and measure the park on its own before rebasing that branch, and test the four combinations of the two switches on the combined head. The wake-latency diagnostic changes the same event loop and conflicts in a read-only merge check. Preserve its sleep count and the park's first-sleep behavior when rebasing. [Event handoff](https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/1971) · [Wake-latency diagnostic](https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/1967) · [Related wake-path issue](https://git.magicbean.dev/MagicBeanDev/Longshot/issues/1505)
@ -470,2 +481,3 @@
int misses;
if (__atomic_load_n(flag, __ATOMIC_RELAXED) > 0 || !spin_begin(o, &misses)) return;
if (__atomic_load_n(flag, __ATOMIC_RELAXED) > 0) return 0;
if (!spin_begin(o, &misses)) return hos_spin_ns() > 0;
Member

spin_begin() already reads hos_spin_ns(). This repeats the role/budget lookup on adaptive skips and for a thread with a zero spin budget, even with the park disabled or x86-64 Linux discarding the result. Carry the budget or skip reason from the first lookup, and include a disabled-head versus main comparison.

spin_begin() already reads hos_spin_ns(). This repeats the role/budget lookup on adaptive skips and for a thread with a zero spin budget, even with the park disabled or x86-64 Linux discarding the result. Carry the budget or skip reason from the first lookup, and include a disabled-head versus main comparison.
@ -0,0 +305,4 @@
}
return 0;
}
static void test_sema_queue(void) {
Member

Please cover timed semaphore acquisition and timed queue peek with the real Windows backend. This fixture only invokes the untimed semaphore and queue paths. Run each timed path through deadline expiry and a racing release or send, then check the waiter is removed and readiness is preserved.

Please cover timed semaphore acquisition and timed queue peek with the real Windows backend. This fixture only invokes the untimed semaphore and queue paths. Run each timed path through deadline expiry and a racing release or send, then check the waiter is removed and readiness is preserved.
siantm force-pushed perf/event-park from 82d36ed100
Some checks failed
PR description / Description follows the template (pull_request_target) Successful in 16s
Website / Website on Cloudflare (pull_request_target) Has been skipped
CI / Conventional Commit title (pull_request) Has been cancelled
CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Has been cancelled
CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Has been cancelled
CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Has been cancelled
CI / Workflow lint (actionlint) (pull_request) Has been cancelled
CI / Website build and checks (pull_request) Has been cancelled
Guide screenshots / Guide images and local UI source review (pull_request) Has been cancelled
Third-party parts / As their notes say (pull_request) Has been cancelled
Third-party parts / The same as upstream (pull_request) Has been cancelled
Website / Website on Cloudflare (pull_request) Has been cancelled
to ff7e23622b
Some checks are pending
CI / Conventional Commit title (pull_request) Blocked by required conditions
CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Blocked by required conditions
CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Blocked by required conditions
CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Blocked by required conditions
CI / Workflow lint (actionlint) (pull_request) Blocked by required conditions
CI / Website build and checks (pull_request) Blocked by required conditions
Guide screenshots / Guide images and local UI source review (pull_request) Blocked by required conditions
Third-party parts / As their notes say (pull_request) Blocked by required conditions
Third-party parts / The same as upstream (pull_request) Blocked by required conditions
Website / Website on Cloudflare (pull_request) Blocked by required conditions
Website / Website on Cloudflare (pull_request_target) Has been skipped
beanstalk/approval waiting for a BigBean's 👍 on the request
PR description / Description follows the template (pull_request_target) Successful in 20s
2026-11-21 04:47:25 +00:00
Compare
Author
Contributor

Description updated for the push of ff7e2362 (on main @ f1f2dc5b): the Summary describes the review's fixes (set up once, a sleep ends only after its own wake is done, failed waits and alerts, one spin-budget lookup), Testing lists what ran in jobs fb3-1970-1 and fb3-1970-2, Still to test adds repeated starts and the main / unset / set A/B, Measurements holds the new checks before and after, and Risks names the park's added costs.

Description updated for the push of ff7e2362 (on `main` @ f1f2dc5b): the Summary describes the review's fixes (set up once, a sleep ends only after its own wake is done, failed waits and alerts, one spin-budget lookup), Testing lists what ran in jobs fb3-1970-1 and fb3-1970-2, Still to test adds repeated starts and the `main` / unset / set A/B, Measurements holds the new checks before and after, and Risks names the park's added costs.
Author
Contributor

@futondestroyer thanks. Pushed ff7e2362, rebased onto main @ f1f2dc5b (the first three commits unchanged, git range-diff "="). Each fix has a check in tests/test_event_park.py that fails at the test commit 1e2adc46 and passes at the head, run through the PC's job queue (jobs fb3-1970-1 and fb3-1970-2).

  • Review, blocking 1, the four findings (e4b273b1):
    • Initialization: park_init runs once (InitOnceExecuteOnce) and publishes the mode after the two pointers (release, acquire). Check "first callers": 8 threads call the park at once with ntdll's lookups slowed; 16 lookups at 1e2adc46, 2 at the head.
    • Alert ownership: the wake sets PARK_DONE after its alert (sent or failed); the sleep leaves only at PARK_DONE, then takes any alert still pending for its thread with a zero-time wait. Check "own": the wake's alert is held after it took the sleeper off the list, another alert is sent to the thread and, for a timed sleep, its deadline passes. At 1e2adc46 the sleep left on the other alert; at the head it stays asleep until the held alert. A control that leaves at PARK_WOKEN fails it.
    • Wait error: park_init tries a zero-time wait and keeps the park off unless it answers STATUS_TIMEOUT or STATUS_ALERTED. A later unexpected status is logged once, and the sleep goes on in Sleep(1) steps, still on the list, until its wake or its deadline. Checks "probe" and "fault_wait" (STATUS_NOT_IMPLEMENTED): at 1e2adc46 the park was used and a 30 ms sleep never returned. At the head the park stays off, the sleep times out after 2 waits, and a wake ends an untimed one.
    • Wake error: a failed alert is logged and PARK_DONE is still set. A blocked sleep waits at most 100 ms per step and then checks its state. Check "fault_alert": with the alert failing, an untimed sleep and a 5 s timed sleep end at the wake. At 1e2adc46 the untimed one never ended.
    • What this costs with the park on (in Risks): one more zero-time wait for each blocked sleep the park wakes, and a check every 100 ms during a long sleep.
  • :483: spin_begin hands the budget it read to spin_until_set and spin_until_free (ff7e2362). Check "budget" counts hos_spin_ns() calls: 2 before this commit, 1 at the head, whether the spin is left out, runs, or has no budget. With the park off, a wait now makes as many lookups as main when the spin is left out, and one fewer when it spins. The disabled head against main is pending, asked of HostLiftPRs (CPU per frame and frame times in the Market: main, head unset, head with HOS_CV_PARK=1). I did not run a game-free benchmark: it would time a stand-in for hos_spin_ns(), which reads the host's thread table, not the call itself.
  • tests/test_event_park.py:289: check "timed" runs sema_acquire_for and queue_get's timed peek on the real Windows backend, unset and on the park. It covers the deadline (it ends in the time asked, nothing taken, the list empty) and 100 rounds each against a release or send arriving 0-30 ms into a 5 ms wait. At the head on the park: 68 of 100 releases taken and 32 after the acquire timed out; 76 of 100 sends seen by the peek and 24 after it timed out. When it timed out, the count or message was still there; the peek left the message; the list was empty after each round. These also pass at 1e2adc46: coverage, no fault found.
  • In-game (repeated starts through the title screen, the Market, the A/B on Windows 10 and 11, AMD and a low-core PC): pending, asked of HostLiftPRs. Still to test lists them.
  • Overlaps: #1971 is rebased onto this head. A read-only git merge-tree of #1967's new head 4d2ebc2a with this head conflicts in event_wait at the spin's line (this PR's missed, #1967's naps and its comment): whichever lands second keeps #1967's sleep state (naps) and the park's first-sleep argument (missed and the first sleep only).
@futondestroyer thanks. Pushed ff7e2362, rebased onto `main` @ f1f2dc5b (the first three commits unchanged, `git range-diff` "="). Each fix has a check in `tests/test_event_park.py` that fails at the test commit 1e2adc46 and passes at the head, run through the PC's job queue (jobs fb3-1970-1 and fb3-1970-2). - [Review, blocking 1](https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/1970#issuecomment-62907), the four findings (e4b273b1): - [Initialization](https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/1970#issuecomment-62344): `park_init` runs once (`InitOnceExecuteOnce`) and publishes the mode after the two pointers (release, acquire). Check "first callers": 8 threads call the park at once with ntdll's lookups slowed; 16 lookups at 1e2adc46, 2 at the head. - [Alert ownership](https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/1970#issuecomment-62345): the wake sets `PARK_DONE` after its alert (sent or failed); the sleep leaves only at `PARK_DONE`, then takes any alert still pending for its thread with a zero-time wait. Check "own": the wake's alert is held after it took the sleeper off the list, another alert is sent to the thread and, for a timed sleep, its deadline passes. At 1e2adc46 the sleep left on the other alert; at the head it stays asleep until the held alert. A control that leaves at `PARK_WOKEN` fails it. - [Wait error](https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/1970#issuecomment-62346): `park_init` tries a zero-time wait and keeps the park off unless it answers `STATUS_TIMEOUT` or `STATUS_ALERTED`. A later unexpected status is logged once, and the sleep goes on in `Sleep(1)` steps, still on the list, until its wake or its deadline. Checks "probe" and "fault_wait" (`STATUS_NOT_IMPLEMENTED`): at 1e2adc46 the park was used and a 30 ms sleep never returned. At the head the park stays off, the sleep times out after 2 waits, and a wake ends an untimed one. - [Wake error](https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/1970#issuecomment-62347): a failed alert is logged and `PARK_DONE` is still set. A blocked sleep waits at most 100 ms per step and then checks its state. Check "fault_alert": with the alert failing, an untimed sleep and a 5 s timed sleep end at the wake. At 1e2adc46 the untimed one never ended. - What this costs with the park on (in Risks): one more zero-time wait for each blocked sleep the park wakes, and a check every 100 ms during a long sleep. - [`:483`](https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/1970#issuecomment-62905): `spin_begin` hands the budget it read to `spin_until_set` and `spin_until_free` (ff7e2362). Check "budget" counts `hos_spin_ns()` calls: 2 before this commit, 1 at the head, whether the spin is left out, runs, or has no budget. With the park off, a wait now makes as many lookups as `main` when the spin is left out, and one fewer when it spins. The disabled head against `main` is pending, asked of HostLiftPRs (CPU per frame and frame times in the Market: `main`, head unset, head with `HOS_CV_PARK=1`). I did not run a game-free benchmark: it would time a stand-in for `hos_spin_ns()`, which reads the host's thread table, not the call itself. - [`tests/test_event_park.py:289`](https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/1970#issuecomment-62906): check "timed" runs `sema_acquire_for` and `queue_get`'s timed peek on the real Windows backend, unset and on the park. It covers the deadline (it ends in the time asked, nothing taken, the list empty) and 100 rounds each against a release or send arriving 0-30 ms into a 5 ms wait. At the head on the park: 68 of 100 releases taken and 32 after the acquire timed out; 76 of 100 sends seen by the peek and 24 after it timed out. When it timed out, the count or message was still there; the peek left the message; the list was empty after each round. These also pass at 1e2adc46: coverage, no fault found. - In-game (repeated starts through the title screen, the Market, the A/B on Windows 10 and 11, AMD and a low-core PC): pending, asked of HostLiftPRs. Still to test lists them. - Overlaps: #1971 is rebased onto this head. A read-only `git merge-tree` of #1967's new head 4d2ebc2a with this head conflicts in `event_wait` at the spin's line (this PR's `missed`, #1967's `naps` and its comment): whichever lands second keeps #1967's sleep state (`naps`) and the park's first-sleep argument (`missed` and the first sleep only).
Author
Contributor

BeanBot's review, at ff7e2362 (test commit 1e2adc46, fixes e4b273b1 and ff7e2362; tests/test_event_park.py through the PC's job queue):

  • L1 (first-use race), L2 (alert ownership), L3 (wait error), L4 (wake error): fixed. See the reply to futondestroyer's review for each fix and its check.
  • L5 (minor, divisor): the exact CyclesPerYield divisor; 0 spins none. KUSER_SHARED_DATA has a fixed layout that Windows only appends to; its offsets are the ones ntdll's own condition variable reads.
  • L6 (gcc): the compiler is --cc, then $CC, then LONGSHOT_MINGW's clang, then clang, gcc or cc on PATH. RUN_ALL_ARGS is gone. It ran with no argument and with CC set.
  • L7 (teardown stores, sleep counts): atomic stores of ev->signaled and atomic reads of the counters.
  • L8 (minor, joins): thread creation, allocations and joins are checked; handles are closed and objects freed.
  • L9 (first-sleep policy): check "policy" drives an event, a timed event, a semaphore, a queue, a timed queue receive and an event whose spin is left out to a blocked first sleep. With a spin budget each is told it spun and the park makes no pause before it blocks; with none, it is told it did not and the park spins first (66 pauses here).
  • L10 (minor, stale-alert check by stress): check "own" holds the wake's alert and checks the sleep stays asleep through another alert and its deadline. The control leaving at PARK_WOKEN fails it.
  • L11 (minor, docs): the rows say which sleeps keep the spin and label 2.5 us as an estimate for the i7-9700 under Windows 11.
  • L12 (minor, enqueue depth, lock time): not instrumented. The list holds the sleepers of one object (a few guest threads); the in-game A/B measures the cost.
  • L13 (author: game run, builds): pending, asked of HostLiftPRs; in Still to test.
  • L14 (counts): Testing gives run_all's own totals now.
BeanBot's [review](https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/1970#issuecomment-62356), at ff7e2362 (test commit 1e2adc46, fixes e4b273b1 and ff7e2362; `tests/test_event_park.py` through the PC's job queue): - L1 (first-use race), L2 (alert ownership), L3 (wait error), L4 (wake error): fixed. See the reply to futondestroyer's review for each fix and its check. - L5 (minor, divisor): the exact `CyclesPerYield` divisor; 0 spins none. `KUSER_SHARED_DATA` has a fixed layout that Windows only appends to; its offsets are the ones ntdll's own condition variable reads. - L6 (gcc): the compiler is `--cc`, then `$CC`, then `LONGSHOT_MINGW`'s clang, then clang, gcc or cc on PATH. `RUN_ALL_ARGS` is gone. It ran with no argument and with `CC` set. - L7 (teardown stores, sleep counts): atomic stores of `ev->signaled` and atomic reads of the counters. - L8 (minor, joins): thread creation, allocations and joins are checked; handles are closed and objects freed. - L9 (first-sleep policy): check "policy" drives an event, a timed event, a semaphore, a queue, a timed queue receive and an event whose spin is left out to a blocked first sleep. With a spin budget each is told it spun and the park makes no pause before it blocks; with none, it is told it did not and the park spins first (66 pauses here). - L10 (minor, stale-alert check by stress): check "own" holds the wake's alert and checks the sleep stays asleep through another alert and its deadline. The control leaving at `PARK_WOKEN` fails it. - L11 (minor, docs): the rows say which sleeps keep the spin and label 2.5 us as an estimate for the i7-9700 under Windows 11. - L12 (minor, enqueue depth, lock time): not instrumented. The list holds the sleepers of one object (a few guest threads); the in-game A/B measures the cost. - L13 (author: game run, builds): pending, asked of HostLiftPRs; in Still to test. - L14 (counts): Testing gives run_all's own totals now.
Author
Contributor

@Beanstalk This adds a switch, HOS_CV_PARK, off by default: with HOS_CV_PARK=1, guest events, semaphores and message queues sleep on a park that does not spin again after their own spin (host/hos_thread.c, host/nn_os_sync.c). It has not run in the game yet. Please play Castle Town Market for two minutes, the PR against main on the same PC, on the PR's side with HOS_CV_PARK=1 and with it unset, on Windows 10, on a Windows 11 PC with an AMD processor and on a Windows PC with 6 or fewer logical processors. No crash or hang; set, the log must have the line starting "os: HOS_CV_PARK:" and frame time p99 equal to unset or better, within the run-to-run spread; unset, frame times as main's.

@Beanstalk This adds a switch, HOS_CV_PARK, off by default: with HOS_CV_PARK=1, guest events, semaphores and message queues sleep on a park that does not spin again after their own spin (host/hos_thread.c, host/nn_os_sync.c). It has not run in the game yet. Please play Castle Town Market for two minutes, the PR against main on the same PC, on the PR's side with HOS_CV_PARK=1 and with it unset, on Windows 10, on a Windows 11 PC with an AMD processor and on a Windows PC with 6 or fewer logical processors. No crash or hang; set, the log must have the line starting "os: HOS_CV_PARK:" and frame time p99 equal to unset or better, within the run-to-run spread; unset, frame times as main's.
Member

@siantm only members of BigBeans can start the farm. One of them can approve this request by reacting 👍 to it or to this comment.

@siantm only members of `BigBeans` can start the farm. One of them can approve this request by reacting 👍 to it or to this comment.
Author
Contributor

The three renderer checks (test_present_queue_default, test_target_reuse_memory, test_vk_occlusion_queries) did not compile in the earlier run (no Vulkan headers on that job's path); rerun with them in job fb3-vk-1, at the head ff7e2362 and on main @ f1f2dc5b: all three fail at both with the same first error, so they fail on main too; Tested's sentence and counts corrected.

The three renderer checks (`test_present_queue_default`, `test_target_reuse_memory`, `test_vk_occlusion_queries`) did not compile in the earlier run (no Vulkan headers on that job's path); rerun with them in job fb3-vk-1, at the head ff7e2362 and on `main` @ f1f2dc5b: all three fail at both with the same first error, so they fail on `main` too; Tested's sentence and counts corrected.
futondestroyer left a comment

Recommendation: Request changes. The new recovery path can return from a park with a native alert still pending. The next job-barrier wait on that Windows thread can receive it.

The experiment replaces the Windows condition variable used by guest events, semaphores and message queues. It skips the park's spin after the caller has already tried its own spin or the adaptive policy skipped it, and reacquires the virtual core before the object's lock. Normal, light and system events and notifications from host workers share this path. The OoT job barriers use WaitOnAddress on some of those same threads. Guest condition variables, mutexes, read-write locks and WaitAny keep their existing backend. WaitAny is still notified through poke_multiwait.

Does it help? Unproven. The author's Windows fixtures establish progress for the injected wait and alert errors, two native lookups for eight simultaneous first callers versus 16 before the fix, and one spin-budget lookup versus two. No matched game or microbenchmark comparison shows a gain. Measure CPU time per frame, wake latency, average FPS, 1% low FPS, and median and p99 frame times against main and the switched-off candidate. A blocked wake adds a zero-time system call and long sleeps check their state every 100 ms. The disabled Windows path still adds a mode check and the shared spin-budget change also affects x86-64 Linux.

Switches. HOS_CV_PARK is documented, Windows-only and off by default as an experiment pending measurement. Windows falls back to its condition variable when the native exports or the initialization probe are unavailable. Linux uses the existing condition-variable shims.

Reviewed. Head ff7e23622b749cff8ac14e28323bb8818addd44d against merge base f1f2dc5bd2821cef061798ba40a043d281ac1707 of the head and official target 9dd41326e71094d8b774459dac5154eb5a3306d9. Read all 12 changed files where they changed and the three new commits relative to the first review. Verified both source-only Graphify maps and their extraction logs. Traced the wait, wake, virtual-core, compatibility and job-barrier callers, and checked the native dispatch and macros directly in source. Source review only. Longshot guidance 9dd41326 and Tooling 0b0059ac.

Checked and fine. The first three patches are unchanged after the rebase. InitOnceExecuteOnce serializes setup and publishes the pointers with the mode. The PARK_DONE handshake now keeps a stack waiter alive until the waker finishes even when another alert arrives or the deadline passes. The 100 ms steps make a failed alert observable without waiting for the original deadline. A native wait error can reach a timed deadline, and the Windows fixture now covers the timed semaphore and queue-peek paths. The earlier duplicate spin-budget lookup is gone. Author's response

Blocking

  1. host/hos_thread.c:2369 discards the status of the final zero-time wait. With the runtime wait fault exercised by test_fault_wait, the wait can fail while the waker's NtAlertThreadByThreadId succeeds. The waiter sees PARK_DONE while polling, the drain also fails without taking the alert, and park_sleep returns. The next WaitOnAddress can consume the alert and leave the wait block behind for WakeByAddressAll, the same crash path described in the PR. The fixture disables its leftover-alert probe in this case, so it proves the waiter exits while the alert can still be pending. Make cleanup reliable after a wait error, or use a wake mechanism that cannot leave a thread alert for the next wait. Add a deterministic check that sends a real alert, makes the park's wait fail after successful initialization, then checks for a pending alert with the real native wait before the waiter thread exits.

Testing. The author reports the focused Windows checks passed at this head, and 17 other strict changed-run failures also reproduce on the base, including the three rerun with Vulkan headers. The complete suite, full Windows and Linux host builds, repeated startup through the title screen, and gameplay measurements remain open. The cached description check passed while CI was pending and the farm awaited BigBean approval. Once the alert path is fixed and the local Windows start check passes, compare main and both switch arms in the Market and a heavier scene on Windows 10 and 11 across AMD CPU and low-core targets. For enabled runs, verify the positive park-mode diagnostic and check for native-call failures. The prefix in the current request also matches fallback. Strict fixture follow-up · Farm request

Overlaps and merge order. The event-handoff branch contains this exact park head. Establish the standalone park result first, then rebase the event-handoff branch and test all four combinations of its switch and the park switch on Windows. The wake-latency work changes the same event_wait spin and sleep bookkeeping. Preserve the diagnostic's sleep count and the park's first-sleep argument in whichever lands second. Event handoff · Wake-latency diagnostic

**Recommendation: Request changes.** The new recovery path can return from a park with a native alert still pending. The next job-barrier wait on that Windows thread can receive it. The experiment replaces the Windows condition variable used by guest events, semaphores and message queues. It skips the park's spin after the caller has already tried its own spin or the adaptive policy skipped it, and reacquires the virtual core before the object's lock. Normal, light and system events and notifications from host workers share this path. The OoT job barriers use `WaitOnAddress` on some of those same threads. Guest condition variables, mutexes, read-write locks and WaitAny keep their existing backend. WaitAny is still notified through `poke_multiwait`. **Does it help?** Unproven. The author's Windows fixtures establish progress for the injected wait and alert errors, two native lookups for eight simultaneous first callers versus 16 before the fix, and one spin-budget lookup versus two. No matched game or microbenchmark comparison shows a gain. Measure CPU time per frame, wake latency, average FPS, 1% low FPS, and median and p99 frame times against main and the switched-off candidate. A blocked wake adds a zero-time system call and long sleeps check their state every 100 ms. The disabled Windows path still adds a mode check and the shared spin-budget change also affects x86-64 Linux. **Switches.** `HOS_CV_PARK` is documented, Windows-only and off by default as an experiment pending measurement. Windows falls back to its condition variable when the native exports or the initialization probe are unavailable. Linux uses the existing condition-variable shims. **Reviewed.** Head `ff7e23622b749cff8ac14e28323bb8818addd44d` against merge base `f1f2dc5bd2821cef061798ba40a043d281ac1707` of the head and official target `9dd41326e71094d8b774459dac5154eb5a3306d9`. Read all 12 changed files where they changed and the three new commits relative to the first review. Verified both source-only Graphify maps and their extraction logs. Traced the wait, wake, virtual-core, compatibility and job-barrier callers, and checked the native dispatch and macros directly in source. Source review only. Longshot guidance `9dd41326` and Tooling `0b0059ac`. **Checked and fine.** The first three patches are unchanged after the rebase. `InitOnceExecuteOnce` serializes setup and publishes the pointers with the mode. The `PARK_DONE` handshake now keeps a stack waiter alive until the waker finishes even when another alert arrives or the deadline passes. The 100 ms steps make a failed alert observable without waiting for the original deadline. A native wait error can reach a timed deadline, and the Windows fixture now covers the timed semaphore and queue-peek paths. The earlier duplicate spin-budget lookup is gone. [Author's response](https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/1970#issuecomment-63600) **Blocking** 1. `host/hos_thread.c:2369` discards the status of the final zero-time wait. With the runtime wait fault exercised by `test_fault_wait`, the wait can fail while the waker's `NtAlertThreadByThreadId` succeeds. The waiter sees `PARK_DONE` while polling, the drain also fails without taking the alert, and `park_sleep` returns. The next `WaitOnAddress` can consume the alert and leave the wait block behind for `WakeByAddressAll`, the same crash path described in the PR. The fixture disables its leftover-alert probe in this case, so it proves the waiter exits while the alert can still be pending. Make cleanup reliable after a wait error, or use a wake mechanism that cannot leave a thread alert for the next wait. Add a deterministic check that sends a real alert, makes the park's wait fail after successful initialization, then checks for a pending alert with the real native wait before the waiter thread exits. **Testing.** The author reports the focused Windows checks passed at this head, and 17 other strict changed-run failures also reproduce on the base, including the three rerun with Vulkan headers. The complete suite, full Windows and Linux host builds, repeated startup through the title screen, and gameplay measurements remain open. The cached description check passed while CI was pending and the farm awaited BigBean approval. Once the alert path is fixed and the local Windows start check passes, compare main and both switch arms in the Market and a heavier scene on Windows 10 and 11 across AMD CPU and low-core targets. For enabled runs, verify the positive park-mode diagnostic and check for native-call failures. The prefix in the current request also matches fallback. [Strict fixture follow-up](https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/1970#issuecomment-64867) · [Farm request](https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/1970#issuecomment-63613) **Overlaps and merge order.** The event-handoff branch contains this exact park head. Establish the standalone park result first, then rebase the event-handoff branch and test all four combinations of its switch and the park switch on Windows. The wake-latency work changes the same `event_wait` spin and sleep bookkeeping. Preserve the diagnostic's sleep count and the park's first-sleep argument in whichever lands second. [Event handoff](https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/1971) · [Wake-latency diagnostic](https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/1967)
@ -2244,0 +2366,4 @@
}
LARGE_INTEGER now;
now.QuadPart = 0;
park_wait(cv, &now); /* (the alert its wake sent, if another alert or a step's end woke the sleep first) */
Member

If the native wait fails at runtime while the wake successfully sends an alert, the polling waiter can reach PARK_DONE and return even when this drain fails. The pending alert can then reach the same thread's WaitOnAddress in the job barrier. Please make the cleanup reliable or use a wake primitive without a pending thread alert. Extend the wait-failure fixture to send a real alert and probe for it with the real native wait before the worker exits.

If the native wait fails at runtime while the wake successfully sends an alert, the polling waiter can reach PARK_DONE and return even when this drain fails. The pending alert can then reach the same thread's WaitOnAddress in the job barrier. Please make the cleanup reliable or use a wake primitive without a pending thread alert. Extend the wait-failure fixture to send a real alert and probe for it with the real native wait before the worker exits.
Some checks are pending
CI / Conventional Commit title (pull_request) Blocked by required conditions
CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Blocked by required conditions
CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Blocked by required conditions
CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Blocked by required conditions
CI / Workflow lint (actionlint) (pull_request) Blocked by required conditions
CI / Website build and checks (pull_request) Blocked by required conditions
Guide screenshots / Guide images and local UI source review (pull_request) Blocked by required conditions
Third-party parts / As their notes say (pull_request) Blocked by required conditions
Third-party parts / The same as upstream (pull_request) Blocked by required conditions
Website / Website on Cloudflare (pull_request) Blocked by required conditions
Website / Website on Cloudflare (pull_request_target) Has been skipped
beanstalk/approval waiting for a BigBean's 👍 on the request
PR description / Description follows the template (pull_request_target) Successful in 20s
This pull request has changes conflicting with the target branch.
  • host/nn_os_sync.c
Some workflows are waiting to be reviewed.
View command line instructions

Manual merge helper

Use this merge commit message when completing the merge manually.

Checkout

From your project repository, check out a new branch and test the changes.
git fetch -u 'perf/event-park:siantm-perf/event-park'
git switch 'siantm-perf/event-park'
Sign in to join this conversation.
No reviewers
No milestone
No project
No assignees
4 participants
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
MagicBeanDev/Longshot!1970
No description provided.