WIP: perf(os): HOS_EVENT_HANDOFF=1, an auto-clear event's signal that finds a sleeper hands the event to it (experiment, off) #1971

Draft
siantm wants to merge 9 commits from siantm/Longshot:perf/event-handoff into main
Contributor

Summary

Depends on #1970 (perf/event-park, the HOS_CV_PARK pull request): this branch is on it and uses its
hos_park_wake. Its six commits show in this diff until it merges; only the last three commits are this PR's.

An auto-clear event's signal sets the event and wakes a sleeper; whoever looks first takes it. A woken sleeper can find
it already taken by a spinner or a newcomer and sleep again (a wake system call and a sleep for nothing), and two
signals that come before the woken one looks let one waiter go for both.

This adds an experiment, off by default: with HOS_EVENT_HANDOFF=1 a signal of an auto-clear event that finds more
sleepers than signals already handed to them (e->mwait against e->grants, under e->lk) hands the event to the
sleepers that were in before it and leaves it clear, as Horizon's kernel releases one waiter and clears the event.
Sleepers are numbered as they come in under e->lk (e->wseq); a hand-off records the last number and its time
(e->grant[], oldest first, at most 8 waiting: with 8, a signal sets the event as before), and a sleeper takes the
oldest hand-off made while it was in, before anything else each time it is back under e->lk and before its deadline
is resolved there. A wait that began after the signal cannot take it; a spinner, a TryWait or WaitAny cannot take a
handed event. The wake is the one event_signal makes otherwise (one sleeper, or all of them when several sleep, the
first back taking it), so a sleeper on a free virtual core is not left asleep behind one whose core is busy; it works
with HOS_CV_PARK=1 and on Windows' condition variable. HOS_WAKE_LAT takes a handed sleeper's latency from its own
signal. Off: nothing is handed, the event's mode is not read in event_signal, and event_wait only numbers its
sleepers. Manual-clear events are not touched.

Also, from the review: the switch is read and published atomically (as cv_signal_always), the event's mode
(autoclear) is written and read atomically, and the audio renderer's event key is published to the renderer thread
only after its event is initialised.

Three commits: the hand-off; its tests (eligibility, busy virtual core, reservation, per-signal latency; they fail on
the hand-off as first written); the fix.

From the local branch exp/x86-asm-sync: 78c83308 (its third part).

Read first: event_handoff, event_signal and event_wait in host/nn_os_sync.c; tests/test_event_handoff.py; the
hand-off cases in tests/test_event_park.py.

Testing

Tested: Windows 11, i7-9700, RTX 4060 (NVIDIA 610.88), llvm-mingw clang 23.1.5; game-free checks only. This change
has not run in the game on this PC yet (Still to test). Each check ran in a hidden console of its own, through the PC's
job queue (job fb3-1971-1), on this branch on perf/event-park ff7e2362 (on main @ f1f2dc5b), head c438b914:

  • python -I -B tests/test_event_handoff.py --cxx <llvm-mingw clang++> (portable; each case a process of its own, the
    hand-off off and on) at the test commit 140850be: exit 1. On: "busy" fails ((runnable & 2) || parked < 2: the wake
    picked the sleeper on the busy core and left the free one asleep), "late" fails ("a timed wait started after the
    signal took the event handed to the sleeper before it"), "lat" fails (both latencies 14 and 21 us: from the last
    signal); off, all pass. At the head: exit 0, every case passes; "lat": 27 us and 31,973 us.
  • python -I -B tests/test_event_park.py at the head: exit 0. The four combinations of the two switches each pass the
    races (20,000 auto-clear signals, 2,000 manual-clear rounds of 4, 5,000 timed-wait signals, 60,000 through a
    semaphore and a queue): unset, HOS_CV_PARK=1, HOS_EVENT_HANDOFF=1 (with its "two signals let two sleepers go"),
    and both; #1970's own parts pass too, and the controls fail (including waiters that never take a hand-off, both
    ways).
  • test_multiwait_wake, test_cv_handoff_threads, test_cv_signal_race, test_cv_signal_early,
    test_os_lifted_siblings, test_oot_barrier, test_spin_by_role, test_switch_table at the head: exit 0.
  • python -I -B tests/run_all.py --strict --changed origin/main at the head: exit 1, "83 passed, 17 failed, 5 skipped".
    14 of the 17 fail on main @ f1f2dc5b too (run by name in the same job): test_crash_dialog, test_crash_record,
    test_dev_ab, test_dev_packs, test_frame_overlap, test_host_guest_writes, test_libc_hooks,
    test_linux_pressure, test_mods_list, test_profiling_mode, test_slow_frames, test_vcore_escape,
    test_vcore_mask0, test_walk_key. The other three (test_present_queue_default, test_target_reuse_memory,
    test_vk_occlusion_queries) compiled nothing in that run nor in the run on main (no Vulkan headers on the job's
    path: "not found: vulkan_headers (--strict)"). Rerun with the headers (job fb3-vk-1, RX_CPM set) at the head
    c438b914 and on main @ f1f2dc5b, each fails at both with the same first error: test_present_queue_default "srccut:
    '{TAB_DEBUG, "Present queue", &opts.present_queue, ...' is not in gpu/vk/vk_menu.cpp"; test_target_reuse_memory 2 of
    its 8 tests ("'vmaGetAllocationInfo2' unexpectedly found" in the VMA source, and its fixture's "use of undeclared
    identifier 'destroy_image_later'"); test_vk_occlusion_queries "the ring fixture did not compile" ("no member named
    'policy'"). None fails on this branch alone.
  • test_crash_dialog alone, on main @ f1f2dc5b and at the head: both fail the same way (a UnicodeDecodeError in its
    output reader, then TypeError: 'NoneType' object is not subscriptable at tests/test_crash_dialog.py:596).

Still to test:

  • Windows 11, this PC (i7-9700, RTX 4060 (NVIDIA 610.88)): Castle Town's market, same route and settings, separate
    processes, warmed and interleaved: main @ f1f2dc5b, #1970 alone, and this PR with HOS_EVENT_HANDOFF unset and
    =1 (each with HOS_CV_PARK unset and =1), at least two runs each, average fps, 1% lows, median and p99 frame times
    and their spread, and HOS_WAKE_LAT's signal-to-running latency (each hand-off timed from its own signal), expecting
    no stall or crash (by hand)
  • Linux: build the host and run python3 -I -B tests/run_all.py --strict --changed origin/main, expecting the cut
    checks (test_event_handoff, test_multiwait_wake) passing and test_event_park skipped (Windows only) (by hand)
  • Windows and Linux PCs with other CPUs (an AMD CPU, fewer cores), and the Steam Deck: Castle Town's market with
    HOS_EVENT_HANDOFF=1 against unset on this PR, expecting no stall or crash and frame times as unset or better

Measurements

Verdict: not measured: no micro-benchmark was run for this PR, and no play measurement exists yet.
HOS_EVENT_HANDOFF ships off (kind attempt), an experiment for A/B runs. The table holds the fixture's own count of
sleeps in its races (a count, not a timing). With the wake of all kept for several sleepers (the review's busy-core
point), the hand-off now makes more sleeps in these races than unset, where before it made fewer: the woken sleepers
that do not get it sleep again. The in-game A/B is in Still to test.

Baseline: this PR with HOS_EVENT_HANDOFF unset, which runs perf/event-park's code (the pull request this one depends
on), so the counts show the switch alone.
Setup: Windows 11, i7-9700, RTX 4060 (NVIDIA 610.88), llvm-mingw clang 23.1.5; tests/test_event_park.py's races
(4 waiters, the spin on), one run in a hidden console of its own through the PC's job queue (fb3-1971-1).

Before, HOS_EVENT_HANDOFF unset: this PR @ c438b914 After, HOS_EVENT_HANDOFF=1: this PR @ c438b914
sleeps for 20,000 auto-clear signals, park unset 29,794 44,441
the same, HOS_CV_PARK=1 31,457 44,980
sleeps for 5,000 timed-wait signals, park unset 6,859 10,036
the same, HOS_CV_PARK=1 7,440 9,327

Risks

Off (the default), a signal of an auto-clear event reads one more switch (read once, atomically) before anything else
and nothing else changes; event_wait numbers its sleepers under e->lk and finds no hand-off. On, a signal that finds
sleepers no longer leaves the event set for a TryWait, a spinner or WaitAny, as on Horizon; code that relied on the
host's looser order (two signals before a waiter looks releasing one waiter) sees one waiter released per signal. With
several sleepers every one is woken and all but one sleep again (more sleeps than unset in the fixture). A lost
hand-off would stall a waiter: one is made only while there are more sleepers in than hand-offs, each sleeper in then
may take it, and every sleeper looks before it leaves. Turn it off by leaving HOS_EVENT_HANDOFF unset; a revert is
enough. Players notice nothing: the switch is not set for them.

Checklist

Ready for review

  • One topic, on current main; the title is a Conventional Commit.
  • Tested on my own PC, in the game for a change the game runs, as Testing says. (the game run is still to come)
  • mise run test passes (any failure that also fails on main is named). (the checks reading the change ran, each failure named and failing on main too; not the whole suite)
  • Measured against main, or Measurements says why there is nothing to measure. (not measured yet: an experiment, off)
  • No game files, build output or hand-edited generated files; third_party/ untouched; nothing GPL-incompatible.

When it applies

  • A switch added or changed: it ships at the default the measurements chose, with its row in gpu/vk/switches.def. (the row is there; off as an experiment, not measured yet)
  • A check under tests/ that reads changed source is updated, not weakened.
  • Host code builds on Windows (llvm-mingw) and Linux, with UTF-8 paths. (only its functions compiled, in the checks, with llvm-mingw clang; no host or Linux build here)
  • Docs updated.
  • AI assistance recorded with Assisted-by: / Co-Authored-By: trailers.

Ready to merge

  • Everything in Still to test has run, its results in Tested.
  • Tier 1 hardware it can affect is tested (docs/dev/standards.md, "Hardware tiers").
  • Approved in review, every comment answered.
  • CI passes on the head; on current main, measured again or a comment says why the numbers still hold.
## Summary Depends on #1970 (`perf/event-park`, the `HOS_CV_PARK` pull request): this branch is on it and uses its `hos_park_wake`. Its six commits show in this diff until it merges; only the last three commits are this PR's. An auto-clear event's signal sets the event and wakes a sleeper; whoever looks first takes it. A woken sleeper can find it already taken by a spinner or a newcomer and sleep again (a wake system call and a sleep for nothing), and two signals that come before the woken one looks let one waiter go for both. This adds an experiment, off by default: with `HOS_EVENT_HANDOFF=1` a signal of an auto-clear event that finds more sleepers than signals already handed to them (`e->mwait` against `e->grants`, under `e->lk`) hands the event to the sleepers that were in before it and leaves it clear, as Horizon's kernel releases one waiter and clears the event. Sleepers are numbered as they come in under `e->lk` (`e->wseq`); a hand-off records the last number and its time (`e->grant[]`, oldest first, at most 8 waiting: with 8, a signal sets the event as before), and a sleeper takes the oldest hand-off made while it was in, before anything else each time it is back under `e->lk` and before its deadline is resolved there. A wait that began after the signal cannot take it; a spinner, a TryWait or WaitAny cannot take a handed event. The wake is the one `event_signal` makes otherwise (one sleeper, or all of them when several sleep, the first back taking it), so a sleeper on a free virtual core is not left asleep behind one whose core is busy; it works with `HOS_CV_PARK=1` and on Windows' condition variable. `HOS_WAKE_LAT` takes a handed sleeper's latency from its own signal. Off: nothing is handed, the event's mode is not read in `event_signal`, and `event_wait` only numbers its sleepers. Manual-clear events are not touched. Also, from the review: the switch is read and published atomically (as `cv_signal_always`), the event's mode (`autoclear`) is written and read atomically, and the audio renderer's event key is published to the renderer thread only after its event is initialised. Three commits: the hand-off; its tests (eligibility, busy virtual core, reservation, per-signal latency; they fail on the hand-off as first written); the fix. From the local branch `exp/x86-asm-sync`: 78c83308 (its third part). Read first: `event_handoff`, `event_signal` and `event_wait` in `host/nn_os_sync.c`; `tests/test_event_handoff.py`; the hand-off cases in `tests/test_event_park.py`. ## Testing **Tested:** Windows 11, i7-9700, RTX 4060 (NVIDIA 610.88), llvm-mingw clang 23.1.5; game-free checks only. This change has not run in the game on this PC yet (Still to test). Each check ran in a hidden console of its own, through the PC's job queue (job fb3-1971-1), on this branch on `perf/event-park` ff7e2362 (on `main` @ f1f2dc5b), head c438b914: - `python -I -B tests/test_event_handoff.py --cxx <llvm-mingw clang++>` (portable; each case a process of its own, the hand-off off and on) at the test commit 140850be: exit 1. On: "busy" fails (`(runnable & 2) || parked < 2`: the wake picked the sleeper on the busy core and left the free one asleep), "late" fails ("a timed wait started after the signal took the event handed to the sleeper before it"), "lat" fails (both latencies 14 and 21 us: from the last signal); off, all pass. At the head: exit 0, every case passes; "lat": 27 us and 31,973 us. - `python -I -B tests/test_event_park.py` at the head: exit 0. The four combinations of the two switches each pass the races (20,000 auto-clear signals, 2,000 manual-clear rounds of 4, 5,000 timed-wait signals, 60,000 through a semaphore and a queue): unset, `HOS_CV_PARK=1`, `HOS_EVENT_HANDOFF=1` (with its "two signals let two sleepers go"), and both; #1970's own parts pass too, and the controls fail (including waiters that never take a hand-off, both ways). - `test_multiwait_wake`, `test_cv_handoff_threads`, `test_cv_signal_race`, `test_cv_signal_early`, `test_os_lifted_siblings`, `test_oot_barrier`, `test_spin_by_role`, `test_switch_table` at the head: exit 0. - `python -I -B tests/run_all.py --strict --changed origin/main` at the head: exit 1, "83 passed, 17 failed, 5 skipped". 14 of the 17 fail on `main` @ f1f2dc5b too (run by name in the same job): `test_crash_dialog`, `test_crash_record`, `test_dev_ab`, `test_dev_packs`, `test_frame_overlap`, `test_host_guest_writes`, `test_libc_hooks`, `test_linux_pressure`, `test_mods_list`, `test_profiling_mode`, `test_slow_frames`, `test_vcore_escape`, `test_vcore_mask0`, `test_walk_key`. The other three (`test_present_queue_default`, `test_target_reuse_memory`, `test_vk_occlusion_queries`) compiled nothing in that run nor in the run on `main` (no Vulkan headers on the job's path: "not found: vulkan_headers (--strict)"). Rerun with the headers (job fb3-vk-1, `RX_CPM` set) at the head c438b914 and on `main` @ f1f2dc5b, each fails at both with the same first error: `test_present_queue_default` "srccut: '{TAB_DEBUG, "Present queue", &opts.present_queue, ...' is not in gpu/vk/vk_menu.cpp"; `test_target_reuse_memory` 2 of its 8 tests ("'vmaGetAllocationInfo2' unexpectedly found" in the VMA source, and its fixture's "use of undeclared identifier 'destroy_image_later'"); `test_vk_occlusion_queries` "the ring fixture did not compile" ("no member named 'policy'"). None fails on this branch alone. - `test_crash_dialog` alone, on `main` @ f1f2dc5b and at the head: both fail the same way (a `UnicodeDecodeError` in its output reader, then `TypeError: 'NoneType' object is not subscriptable` at `tests/test_crash_dialog.py:596`). **Still to test:** - Windows 11, this PC (i7-9700, RTX 4060 (NVIDIA 610.88)): Castle Town's market, same route and settings, separate processes, warmed and interleaved: `main` @ f1f2dc5b, #1970 alone, and this PR with `HOS_EVENT_HANDOFF` unset and `=1` (each with `HOS_CV_PARK` unset and `=1`), at least two runs each, average fps, 1% lows, median and p99 frame times and their spread, and `HOS_WAKE_LAT`'s signal-to-running latency (each hand-off timed from its own signal), expecting no stall or crash (by hand) - Linux: build the host and run `python3 -I -B tests/run_all.py --strict --changed origin/main`, expecting the cut checks (`test_event_handoff`, `test_multiwait_wake`) passing and `test_event_park` skipped (Windows only) (by hand) - Windows and Linux PCs with other CPUs (an AMD CPU, fewer cores), and the Steam Deck: Castle Town's market with `HOS_EVENT_HANDOFF=1` against unset on this PR, expecting no stall or crash and frame times as unset or better ## Measurements **Verdict:** not measured: no micro-benchmark was run for this PR, and no play measurement exists yet. `HOS_EVENT_HANDOFF` ships off (kind `attempt`), an experiment for A/B runs. The table holds the fixture's own count of sleeps in its races (a count, not a timing). With the wake of all kept for several sleepers (the review's busy-core point), the hand-off now makes more sleeps in these races than unset, where before it made fewer: the woken sleepers that do not get it sleep again. The in-game A/B is in Still to test. Baseline: this PR with `HOS_EVENT_HANDOFF` unset, which runs `perf/event-park`'s code (the pull request this one depends on), so the counts show the switch alone. Setup: Windows 11, i7-9700, RTX 4060 (NVIDIA 610.88), llvm-mingw clang 23.1.5; `tests/test_event_park.py`'s races (4 waiters, the spin on), one run in a hidden console of its own through the PC's job queue (fb3-1971-1). | | Before, HOS_EVENT_HANDOFF unset: this PR @ c438b914 | After, HOS_EVENT_HANDOFF=1: this PR @ c438b914 | |---|---|---| | sleeps for 20,000 auto-clear signals, park unset | 29,794 | 44,441 | | the same, `HOS_CV_PARK=1` | 31,457 | 44,980 | | sleeps for 5,000 timed-wait signals, park unset | 6,859 | 10,036 | | the same, `HOS_CV_PARK=1` | 7,440 | 9,327 | ## Risks Off (the default), a signal of an auto-clear event reads one more switch (read once, atomically) before anything else and nothing else changes; `event_wait` numbers its sleepers under `e->lk` and finds no hand-off. On, a signal that finds sleepers no longer leaves the event set for a TryWait, a spinner or WaitAny, as on Horizon; code that relied on the host's looser order (two signals before a waiter looks releasing one waiter) sees one waiter released per signal. With several sleepers every one is woken and all but one sleep again (more sleeps than unset in the fixture). A lost hand-off would stall a waiter: one is made only while there are more sleepers in than hand-offs, each sleeper in then may take it, and every sleeper looks before it leaves. Turn it off by leaving `HOS_EVENT_HANDOFF` unset; a revert is enough. Players notice nothing: the switch is not set for them. ## Checklist ### Ready for review - [x] One topic, on current `main`; the title is a Conventional Commit. - [ ] Tested on my own PC, in the game for a change the game runs, as Testing says. (the game run is still to come) - [ ] `mise run test` passes (any failure that also fails on `main` is named). (the checks reading the change ran, each failure named and failing on `main` too; not the whole suite) - [ ] Measured against `main`, or Measurements says why there is nothing to measure. (not measured yet: an experiment, off) - [x] No game files, build output or hand-edited generated files; `third_party/` untouched; nothing GPL-incompatible. ### When it applies - [ ] A switch added or changed: it ships at the default the measurements chose, with its row in `gpu/vk/switches.def`. (the row is there; off as an experiment, not measured yet) - [x] A check under `tests/` that reads changed source is updated, not weakened. - [ ] Host code builds on Windows (llvm-mingw) and Linux, with UTF-8 paths. (only its functions compiled, in the checks, with llvm-mingw clang; no host or Linux build here) - [x] Docs updated. - [x] AI assistance recorded with `Assisted-by:` / `Co-Authored-By:` trailers. ### Ready to merge - [ ] Everything in Still to test has run, its results in Tested. - [ ] Tier 1 hardware it can affect is tested (docs/dev/standards.md, "Hardware tiers"). - [ ] Approved in review, every comment answered. - [ ] CI passes on the head; on current `main`, measured again or a comment says why the numbers still hold.
A wait on an event, a semaphore or a message queue spins for its flag first (spin_until_set) and then sleeps with
SleepConditionVariableSRW, which spins again for 10240 cycles of pause before it blocks (ntdll's
RtlSleepConditionVariableSRW: 10240 / KUSER_SHARED_DATA.CyclesPerYield pauses). With HOS_CV_PARK=1 those sleeps go to a
park in host/hos_thread.c (hos_park_sleep, hos_park_wake) that blocks at once with NtWaitForAlertByThreadId when the
caller has just spun (spin_until_set now says so) and is woken with NtAlertThreadByThreadId: the condition variable's
own block and wake, without its second spin. The waiters are an oldest-first list from the cv's word under one of 64
striped SRW locks; a wake that finds a waiter still spinning makes no call. A woken waiter takes its virtual core back
and then the object's lock once (hos_cv_sleep takes it, lets it go for the core and takes it again).

Mutexes, guest condition variables (wait_until's park < 0), read-write locks, timers and WaitAny stay on Windows'
condition variable. Unset (the default), HOS_CV_PARK=0, or an ntdll without the two calls: every sleep and wake is
Windows' condition variable's, as before (hos_park_sleep is hos_cv_sleep, hos_park_wake WakeConditionVariable or
WakeAllConditionVariable). Linux: the same shims, its condition variable.

tests/test_event_park.py (Windows) runs the real park and the real event, semaphore and queue code on threads, unset
and with HOS_CV_PARK=1, with and without the spin first: timeouts, oldest-first wakes, no lost wake over 20,000
auto-clear signals, manual-clear rounds, timed waits, a semaphore and a queue; controls (a wake of one of several
sleepers, a park wake without its alert) fail. The checks that cut wait_until, spin_until_set or the semaphore's wake
(test_cv_signal_race, test_cv_handoff_threads, test_event_handoff, test_multiwait_wake, test_os_lifted_siblings) take
the new signatures and the park's stubs.

From the local branch exp/x86-asm-sync: c565d05a (the park) and 78c83308's first bullet (one lock take after a park
wake). There it was on by default; here it is off: its in-play A/B on that branch measured no difference.

Assisted-by: Claude Code
tests/test_event_park.py asks, before and after every park sleep of its races (HOS_CV_PARK=1), whether an alert is
still pending for the thread (a zero-time NtWaitForAlertByThreadId), and fails when one was.

It fails at the commit before this one: a wake that finds its sleeper blocked sets the sleeper's state and then
alerts the thread; the sleeper leaves as soon as it sees the state, so an alert sent after that stays pending for the
thread. Windows' WaitOnAddress returns at the first alert its thread gets, its wait block still on the address's list,
and the next wake of that address reads the block where the stack has moved on. The game's job barriers
(host/oot_barrier.c) wait that way on the same threads: on the local branch this came from, a start of the game with
the park on stopped at the title screen with an access violation in WakeByAddressAll's walk of the waiters, under
hos_oot_barrier_wake, one start of five.

From the local branch exp/x86-asm-sync: 939b5324.

Assisted-by: Claude Code
fix(os): a park sleep leaves no alert for its thread, so the job barriers' WaitOnAddress is not woken by one
Some checks failed
PR description / Description follows the template (pull_request_target) Successful in 16s
Website / Website on Cloudflare (pull_request_target) Has been skipped
CI / Conventional Commit title (pull_request) Has been cancelled
CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Has been cancelled
CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Has been cancelled
CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Has been cancelled
CI / Workflow lint (actionlint) (pull_request) Has been cancelled
CI / Website build and checks (pull_request) Has been cancelled
Guide screenshots / Guide images and local UI source review (pull_request) Has been cancelled
Third-party parts / As their notes say (pull_request) Has been cancelled
Third-party parts / The same as upstream (pull_request) Has been cancelled
Website / Website on Cloudflare (pull_request) Has been cancelled
82d36ed100
A park wake (HOS_CV_PARK=1) that finds its sleeper blocked sets the sleeper's state and then alerts the thread
(NtAlertThreadByThreadId). park_sleep looked at the state before each wait, and again after a timeout that found it
off the list, and left at once when it said woken: an alert sent after that stayed pending for the thread. Windows'
WaitOnAddress returns at the first alert its thread gets with its wait block still on the address's list, and the next
wake of that address reads the block where the stack has moved on. The game's job barriers wait that way on the same
threads (host/oot_barrier.c): on the local branch this came from, a start with the park on stopped at the title screen
with an access violation under hos_oot_barrier_wake -> WakeByAddressAll, one start of five.

The sleep now waits first and leaves only at an alerted wait whose state says woken: a wake that finds PARK_BLOCK
always alerts, so there is always one to take, and it is taken once. A timeout that finds the sleeper on the list
takes it off and leaves as before (no wake has it: no alert).

tests/test_event_park.py's check from the commit before passes with it. HOS_CV_PARK unset: nothing changes.

From the local branch exp/x86-asm-sync: 76de03b5.

Assisted-by: Claude Code
perf(os): HOS_EVENT_HANDOFF=1, an auto-clear event's signal that finds a sleeper hands the event to it (experiment, off)
Some checks failed
PR description / Description follows the template (pull_request_target) Successful in 15s
Website / Website on Cloudflare (pull_request_target) Has been skipped
CI / Conventional Commit title (pull_request) Has been cancelled
CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Has been cancelled
CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Has been cancelled
CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Has been cancelled
CI / Workflow lint (actionlint) (pull_request) Has been cancelled
CI / Website build and checks (pull_request) Has been cancelled
Guide screenshots / Guide images and local UI source review (pull_request) Has been cancelled
Third-party parts / As their notes say (pull_request) Has been cancelled
Third-party parts / The same as upstream (pull_request) Has been cancelled
Website / Website on Cloudflare (pull_request) Has been cancelled
203ed5f289
An auto-clear event's signal sets the event and wakes a sleeper; whoever looks first takes it. A woken sleeper often
finds it already taken by a spinner or a newcomer and sleeps again (a wake system call and a sleep for nothing), and
two signals that come before the woken one looks let one waiter go for both. With HOS_EVENT_HANDOFF=1 a signal that
finds more waiters than grants (e->mwait against e->grants, under e->lk) hands the event to a sleeper and leaves it
clear, as Horizon's kernel releases one waiter and clears the event: the woken sleeper takes its grant before
anything else when it is back under e->lk (event_wait). A spinner, a TryWait or WaitAny cannot take a granted event.
The wake is hos_park_wake's wake of one (HOS_CV_PARK, the commit before this branch's), so it works on the park and on
Windows' condition variable. Off (the default): grants are never given and event_wait is as before.

tests/test_event_park.py: with HOS_EVENT_HANDOFF=1, unset park and HOS_CV_PARK=1, two signals to two sleepers let both
go and leave the event clear (a TryWait between them fails), a signal with nobody asleep sets it, and the races (20,000
auto-clear signals, timed waits, manual-clear rounds) lose nothing; the control (waiters that do not look for a grant)
fails both ways. test_multiwait_wake's locked-region check has a control for a checkpoint in the hand-off under e->lk;
test_event_handoff's fixture takes the new field and the switch off.

From the local branch exp/x86-asm-sync: 78c83308's third bullet.

Assisted-by: Claude Code
Member

Possibly related, for context:

  • issue #1505 Every event, queue and semaphore signal locks and wakes all waiters, and takes a global lock while any thread waits in a multi-wait — Open performance issue covers event signal/waiter wake paths.
Possibly related, for context: - issue #1505 Every event, queue and semaphore signal locks and wakes all waiters, and takes a global lock while any thread waits in a multi-wait — Open performance issue covers event signal/waiter wake paths.
BeanBot left a comment

Recommendation: Request changes. A timed waiter can take an earlier sleeper’s grant, and waking only one worker can stall an event on a busy virtual core. Both switches also have first-use races, and the fixture’s lower sleep counts leave the in-game benefit unproven.

With the experiment enabled, auto-clear event signals reserve a wake for a direct waiter. The branch also carries #1970’s Windows wait backend for events, semaphores and queues. The handoff reduces sleeps in one fixture run, but adds grant bookkeeping and changes which workers can make progress. The Windows backend adds a striped waiter list and depends on thread alerts shared with other Windows waits.

@fix will do (16)

  1. L1 host/nn_os_sync.c:974: A grant is available to anyone who has slept once, including a caller that started after the signal. (more on the line)
  2. L2 (its part an edit can do) host/nn_os_sync.c:890: Let eligible pre-signal sleepers on available virtual cores compete for the reservation, or preserve broadcast when the wake backend cannot select a runnable waiter.
  3. L3 host/nn_os_sync.c:875: Two first signals on different auto-clear events with sleepers can concurrently read and write the plain static on. (more on the line)
  4. L4 host/nn_os_sync.c:885: The signal path now reads autoclear unconditionally, including with the experiment off, while event_init writes it after the object is published. (more on the line)
  5. L5 host/hos_thread.c:2272: #1970’s first-use check allows two threads to bind park_wait and park_alert concurrently. (more on the line)
  6. L6 host/hos_thread.c:2323: STATUS_ALERTED is a thread-level notification. (more on the line)
  7. L7 host/hos_thread.c:2359: The park drops native API failures. (more on the line)
  8. L8 (minor) host/nn_os_sync.c:894: A successful handoff pokes WaitAny even though it creates no public readiness. (more on the line)
  9. L9 tests/test_event_handoff.py:85: The portable fixture forces the new behavior off, while the fixture that enables it skips Linux. (more on the line)
  10. L10 (minor) tests/test_event_park.py:95: The alert probes can miss the race they claim to catch. (more on the line)
  11. L11 tests/test_event_park.py:39: The new check hardcodes gcc in both its suite declaration and direct default, so setting CC to the supported llvm-mingw clang still leaves an LLVM-only Windows install unable to run it through the suite. (more on the line)
  12. L12 (minor) tests/test_event_park.py:252: The printed sleep counts use plain volatile reads while workers increment sleeps through InterlockedIncrement. (more on the line)
  13. L13 (minor) tests/test_event_park.py:230: release_all treats WAIT_FAILED as a successful join. (more on the line)
  14. L14 (minor) tests/test_event_park.py:331: Queue count and sum do not verify that nothing was lost or duplicated. (more on the line)
  15. L15 (its part an edit can do): Reuse the spin budget already read by spin_begin, avoid duplicate waiter checks on the off path, and avoid walking the whole park queue on every enqueue if contention shows up.
  16. L16 (minor): Testing says 87 checks ran, but 72 passed, 16 failed and 4 skipped total 92. Report the actual count and compiler invocation for the Windows results. The description also records that the PR opened before the required author game run. Record that run and any maintainer exception used to open early.

For the author (2) (a run, a measurement or a decision)

  • L2 host/nn_os_sync.c:890: Wake-one restores the busy-core stall covered by the existing event test. (more on the line)
  • L15 (minor): The required author game run, supported host builds, Tier 1 coverage and performance comparison are missing. Compare current main, the dependency alone, and the handoff on the same build and settings, using interleaved warmed runs. Include average FPS, 1% lows, median and p99 frame times, and signal-to-running latency. Cover low core counts and both OSes, including Steam Deck. The Windows park’s linear tail scan under a stripe lock and the default path’s added waiter check and spin-policy query are costs the sleep totals omit.

Checks: Still running: CI / Conventional Commit title (pull_request), CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request), CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request), CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request), CI / Workflow lint (actionlint) (pull_request), CI / Website build and checks (pull_request), Guide screenshots / Guide images and local UI source review (pull_request), Third-party parts / As their notes say (pull_request), Third-party parts / The same as upstream (pull_request), Website / Website on Cloudflare (pull_request).

More: does it help, other systems, switches, what was read

Does it help: Unproven. One Windows fixture run on rebased head 6353dc42 compared handoff on with unset on the same build. For 20,000 signals, sleeps fell from 30,052 to 19,999 on the native CV and from 32,860 to 19,999 on the park. For 5,000 timed signals, they fell from 7,185 to 5,007 and from 7,524 to 5,007, respectively. Those counts establish neither signal-to-running latency nor a frame-time gain. There are no interleaved game runs, current-base runtime comparisons, or Linux results. The park commit reports an earlier in-game comparison with no difference and supplies no numbers for it.

Other systems: The shared event path includes audio renderer notifications, display vsync, video and GPU completion events. Scheduler stalls can therefore affect pacing or audio on either OS. WaitAny still polls public readiness and cannot see a grant. The Windows park also affects manual-event, semaphore and queue waits, and its thread alerts can interact with the job barrier’s WaitOnAddress. Linux retains its futex-backed CV but uses the new handoff when enabled. Mutexes, guest CVs, timers and reader/writer locks still use their previous sleep and wake paths. There are no renderer, save, input, build or cache format changes. Runtime scheduling coverage is missing on AMD CPUs, lower-core systems, Linux and Steam Deck.

Switches: HOS_EVENT_HANDOFF is off by default on Windows and Linux. HOS_CV_PARK from #1970 is off by default, Windows-only, and falls back to the native CV if either export is missing. Both are read-once developer experiments with rows in the switch table and reference docs. Off is the safer interim default for both, but the repository requires a measured shipping decision. HOS_SPIN_US, HOS_SPIN_OTHER_US and HOS_SPIN_ALWAYS retain their defaults and now also influence whether the enabled park skips its own first spin.

Reviewed: 203ed5f289 against 6e1521a70a: Read AGENTS.md, CONTRIBUTING.md, docs/dev/standards.md and the Forgejo template in full. Reviewed all four commits, all 11 file diffs, the new park fixture and the portable event fixture in full, and the changed runtime and sibling-test functions with their surrounding code. Traced object reset, event producers, WaitAny, Linux CVs, virtual-core reacquisition, the job barrier, switch parsing and test-runner compiler selection. Skimmed unrelated portions of larger source and test files, and read the PR’s recorded Windows results and CI status. There were no comments, prior reviews, changed translations or generated files. The review checklist and adjacent Tooling checkout were absent.; nothing run (a code review).

Checked and fine:

  • All four commits carry Assisted-by trailers. The topic is disclosed and the title has the Conventional Commit form after the WIP prefix.
  • Both added switch rows and their reference entries are present. The Windows park launcher fixture clears inherited HOS_ and T_ variables before selecting its test modes.
  • Object initialization and reset zero the grant field. Grant count accesses in the wait and signal paths use the event lock, and the runtime park adds no per-thread heap allocation.
  • The diff contains no game-derived artifacts, build output, vendor SDKs, third-party edits, generated shader changes or renderer API changes. No API version bump applies.
  • The new fixture creates temporary source and executables for its stated checks. It adds no downloads or unrelated external actions.

Testing: Read-only review. git diff --check passed. The author reports Windows fixture passes on 6353dc42. No host build, game run or Linux result was supplied for 203ed5f2. The 16 listed failures reportedly reproduce on the earlier base, and the supplied CI checks are still running.

Overlaps and merge order: #1970 is the stated dependency. Its three commits end at 82d36ed1 and overlap the park implementation, switch docs and fixtures. Resolve its park findings there. If the PRs stay separate, merge #1970 first, rebase #1971 and repeat the handoff checks against that base. The repository standards also allow the track to be grouped into one PR.


Automated code review by BeanBot (gpt-6.2-sol), on the pull request's opening. Nothing was run; a maintainer decides.

**Recommendation: Request changes.** A timed waiter can take an earlier sleeper’s grant, and waking only one worker can stall an event on a busy virtual core. Both switches also have first-use races, and the fixture’s lower sleep counts leave the in-game benefit unproven. With the experiment enabled, auto-clear event signals reserve a wake for a direct waiter. The branch also carries #1970’s Windows wait backend for events, semaphores and queues. The handoff reduces sleeps in one fixture run, but adds grant bookkeeping and changes which workers can make progress. The Windows backend adds a striped waiter list and depends on thread alerts shared with other Windows waits. **`@fix` will do (16)** 1. **L1** `host/nn_os_sync.c:974`: A grant is available to anyone who has slept once, including a caller that started after the signal. (more on the line) 2. **L2** (its part an edit can do) `host/nn_os_sync.c:890`: Let eligible pre-signal sleepers on available virtual cores compete for the reservation, or preserve broadcast when the wake backend cannot select a runnable waiter. 3. **L3** `host/nn_os_sync.c:875`: Two first signals on different auto-clear events with sleepers can concurrently read and write the plain static on. (more on the line) 4. **L4** `host/nn_os_sync.c:885`: The signal path now reads autoclear unconditionally, including with the experiment off, while event_init writes it after the object is published. (more on the line) 5. **L5** `host/hos_thread.c:2272`: #1970’s first-use check allows two threads to bind park_wait and park_alert concurrently. (more on the line) 6. **L6** `host/hos_thread.c:2323`: STATUS_ALERTED is a thread-level notification. (more on the line) 7. **L7** `host/hos_thread.c:2359`: The park drops native API failures. (more on the line) 8. **L8** (minor) `host/nn_os_sync.c:894`: A successful handoff pokes WaitAny even though it creates no public readiness. (more on the line) 9. **L9** `tests/test_event_handoff.py:85`: The portable fixture forces the new behavior off, while the fixture that enables it skips Linux. (more on the line) 10. **L10** (minor) `tests/test_event_park.py:95`: The alert probes can miss the race they claim to catch. (more on the line) 11. **L11** `tests/test_event_park.py:39`: The new check hardcodes gcc in both its suite declaration and direct default, so setting CC to the supported llvm-mingw clang still leaves an LLVM-only Windows install unable to run it through the suite. (more on the line) 12. **L12** (minor) `tests/test_event_park.py:252`: The printed sleep counts use plain volatile reads while workers increment sleeps through InterlockedIncrement. (more on the line) 13. **L13** (minor) `tests/test_event_park.py:230`: release_all treats WAIT_FAILED as a successful join. (more on the line) 14. **L14** (minor) `tests/test_event_park.py:331`: Queue count and sum do not verify that nothing was lost or duplicated. (more on the line) 15. **L15** (its part an edit can do): Reuse the spin budget already read by spin_begin, avoid duplicate waiter checks on the off path, and avoid walking the whole park queue on every enqueue if contention shows up. 16. **L16** (minor): Testing says 87 checks ran, but 72 passed, 16 failed and 4 skipped total 92. Report the actual count and compiler invocation for the Windows results. The description also records that the PR opened before the required author game run. Record that run and any maintainer exception used to open early. **For the author (2)** (a run, a measurement or a decision) - **L2** `host/nn_os_sync.c:890`: Wake-one restores the busy-core stall covered by the existing event test. (more on the line) - **L15** (minor): The required author game run, supported host builds, Tier 1 coverage and performance comparison are missing. Compare current main, the dependency alone, and the handoff on the same build and settings, using interleaved warmed runs. Include average FPS, 1% lows, median and p99 frame times, and signal-to-running latency. Cover low core counts and both OSes, including Steam Deck. The Windows park’s linear tail scan under a stripe lock and the default path’s added waiter check and spin-policy query are costs the sleep totals omit. **Checks:** Still running: `CI / Conventional Commit title (pull_request)`, `CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request)`, `CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request)`, `CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request)`, `CI / Workflow lint (actionlint) (pull_request)`, `CI / Website build and checks (pull_request)`, `Guide screenshots / Guide images and local UI source review (pull_request)`, `Third-party parts / As their notes say (pull_request)`, `Third-party parts / The same as upstream (pull_request)`, `Website / Website on Cloudflare (pull_request)`. <details> <summary>More: does it help, other systems, switches, what was read</summary> **Does it help:** Unproven. One Windows fixture run on rebased head 6353dc42 compared handoff on with unset on the same build. For 20,000 signals, sleeps fell from 30,052 to 19,999 on the native CV and from 32,860 to 19,999 on the park. For 5,000 timed signals, they fell from 7,185 to 5,007 and from 7,524 to 5,007, respectively. Those counts establish neither signal-to-running latency nor a frame-time gain. There are no interleaved game runs, current-base runtime comparisons, or Linux results. The park commit reports an earlier in-game comparison with no difference and supplies no numbers for it. **Other systems:** The shared event path includes audio renderer notifications, display vsync, video and GPU completion events. Scheduler stalls can therefore affect pacing or audio on either OS. WaitAny still polls public readiness and cannot see a grant. The Windows park also affects manual-event, semaphore and queue waits, and its thread alerts can interact with the job barrier’s WaitOnAddress. Linux retains its futex-backed CV but uses the new handoff when enabled. Mutexes, guest CVs, timers and reader/writer locks still use their previous sleep and wake paths. There are no renderer, save, input, build or cache format changes. Runtime scheduling coverage is missing on AMD CPUs, lower-core systems, Linux and Steam Deck. **Switches:** HOS_EVENT_HANDOFF is off by default on Windows and Linux. HOS_CV_PARK from #1970 is off by default, Windows-only, and falls back to the native CV if either export is missing. Both are read-once developer experiments with rows in the switch table and reference docs. Off is the safer interim default for both, but the repository requires a measured shipping decision. HOS_SPIN_US, HOS_SPIN_OTHER_US and HOS_SPIN_ALWAYS retain their defaults and now also influence whether the enabled park skips its own first spin. **Reviewed:** 203ed5f28977 against 6e1521a70a32: Read AGENTS.md, CONTRIBUTING.md, docs/dev/standards.md and the Forgejo template in full. Reviewed all four commits, all 11 file diffs, the new park fixture and the portable event fixture in full, and the changed runtime and sibling-test functions with their surrounding code. Traced object reset, event producers, WaitAny, Linux CVs, virtual-core reacquisition, the job barrier, switch parsing and test-runner compiler selection. Skimmed unrelated portions of larger source and test files, and read the PR’s recorded Windows results and CI status. There were no comments, prior reviews, changed translations or generated files. The review checklist and adjacent Tooling checkout were absent.; nothing run (a code review). **Checked and fine:** - All four commits carry Assisted-by trailers. The topic is disclosed and the title has the Conventional Commit form after the WIP prefix. - Both added switch rows and their reference entries are present. The Windows park launcher fixture clears inherited HOS_ and T_ variables before selecting its test modes. - Object initialization and reset zero the grant field. Grant count accesses in the wait and signal paths use the event lock, and the runtime park adds no per-thread heap allocation. - The diff contains no game-derived artifacts, build output, vendor SDKs, third-party edits, generated shader changes or renderer API changes. No API version bump applies. - The new fixture creates temporary source and executables for its stated checks. It adds no downloads or unrelated external actions. **Testing:** Read-only review. git diff --check passed. The author reports Windows fixture passes on 6353dc42. No host build, game run or Linux result was supplied for 203ed5f2. The 16 listed failures reportedly reproduce on the earlier base, and the supplied CI checks are still running. **Overlaps and merge order:** #1970 is the stated dependency. Its three commits end at 82d36ed1 and overlap the park implementation, switch docs and fixtures. Resolve its park findings there. If the PRs stay separate, merge #1970 first, rebase #1971 and repeat the handoff checks against that base. The repository standards also allow the track to be grouped into one PR. </details> --- <sub>Automated code review by BeanBot (gpt-6.2-sol), on the pull request's opening. Nothing was run; a maintainer decides.</sub>
@ -2244,0 +2269,4 @@
v = 0;
if (env_on("HOS_CV_PARK")) {
const HMODULE nt = GetModuleHandleA("ntdll.dll");
park_wait = (ParkWaitFn)(void *)GetProcAddress(nt, "NtWaitForAlertByThreadId");
Member

L5: #1970’s first-use check allows two threads to bind park_wait and park_alert concurrently. One thread can publish on and start using the pointers while the other is still writing them. The release store and one-time logging guard do not protect those accesses. Bind and publish the mode and both pointers from one initializer.

**L5**: #1970’s first-use check allows two threads to bind park_wait and park_alert concurrently. One thread can publish on and start using the pointers while the other is still writing them. The release store and one-time logging guard do not protect those accesses. Bind and publish the mode and both pointers from one initializer.
@ -2244,0 +2320,4 @@
}
const LONG status = park_wait(cv, to);
if (status == 0x101 /* STATUS_ALERTED */) {
if (__atomic_load_n(&me.state, __ATOMIC_ACQUIRE) == PARK_WOKEN) break; /* (its wake's alert, taken) */
Member

L6: STATUS_ALERTED is a thread-level notification. If an unrelated alert arrives after the waker sets PARK_WOKEN but before it issues park_alert, the waiter can leave on the unrelated alert and the park’s own alert can reach a subsequent wait, including the job barrier. Use a waiter-specific wake primitive or a protocol that can establish that this wake’s notification has completed before the waiter leaves.

**L6**: STATUS_ALERTED is a thread-level notification. If an unrelated alert arrives after the waker sets PARK_WOKEN but before it issues park_alert, the waiter can leave on the unrelated alert and the park’s own alert can reach a subsequent wait, including the job barrier. Use a waiter-specific wake primitive or a protocol that can establish that this wake’s notification has completed before the waiter leaves.
@ -2244,0 +2356,4 @@
while (w) { /* (off the list: only this wake reads them; each is the waiter's stack until its state is PARK_WOKEN) */
Parked *next = all ? (Parked *)w->next : NULL;
const DWORD tid = w->tid;
if (__atomic_exchange_n(&w->state, PARK_WOKEN, __ATOMIC_SEQ_CST) == PARK_BLOCK) park_alert((HANDLE)(uintptr_t)tid);
Member

L7: The park drops native API failures. A resolved export that returns an error from park_wait makes even a finite wait retry indefinitely, and a failed park_alert leaves a detached waiter waiting for an alert that will never arrive. Handle unexpected statuses with a defined failure path that preserves waiter ownership, or use supported primitives. Falling back on an individual call would mix waiter representations in the same CV.

**L7**: The park drops native API failures. A resolved export that returns an error from park_wait makes even a finite wait retry indefinitely, and a failed park_alert leaves a detached waiter waiting for an alert that will never arrive. Handle unexpected statuses with a defined failure path that preserves waiter ownership, or use supported primitives. Falling back on an individual call would mix waiter representations in the same CV.
@ -851,0 +872,4 @@
Off: grants are never given, and event_wait is as without it */
static int event_handoff(void) {
static int on = -1;
if (on < 0) on = env_on("HOS_EVENT_HANDOFF");
Member

L3: Two first signals on different auto-clear events with sleepers can concurrently read and write the plain static on. This also occurs with the switch unset. Initialize the switch before producers start or publish its value atomically, as cv_signal_always does.

**L3**: Two first signals on different auto-clear events with sleepers can concurrently read and write the plain static on. This also occurs with the switch unset. Initialize the switch before producers start or publish its value atomically, as cv_signal_always does.
@ -855,6 +882,19 @@ static void event_signal(Obj *e) {
#endif
HOS_CHAIN(HOS_CHAIN_SIGNAL, e->key);
if (wake_lat_on()) __atomic_store_n(&e->sig_ns, hos_now_ns(), __ATOMIC_RELAXED);
if (e->autoclear && __atomic_load_n(&e->mwait, __ATOMIC_SEQ_CST) && event_handoff()) {
Member

L4: The signal path now reads autoclear unconditionally, including with the experiment off, while event_init writes it after the object is published. For example, the audio renderer thread can signal a reopened renderer event while OpenAudioRenderer is initializing it. Synchronize event mode initialization and access, and publish the audio notification key after event initialization.

**L4**: The signal path now reads autoclear unconditionally, including with the experiment off, while event_init writes it after the object is published. For example, the audio renderer thread can signal a reopened renderer event while OpenAudioRenderer is initializing it. Synchronize event mode initialization and access, and publish the audio notification key after event initialization.
@ -858,0 +887,4 @@
const int hand = e->mwait > e->grants;
if (hand) {
e->grants++;
hos_park_wake(&e->cv, 0);
Member

L2: Wake-one restores the busy-core stall covered by the existing event test. If the CV or FIFO selects A on a busy virtual core, B on a free core remains asleep even though it could complete the event. The portable test forces handoff off and the new fixture omits virtual cores. Exercise this schedule with handoff enabled and measure signal-to-running latency and frame-time tails.

**L2**: Wake-one restores the busy-core stall covered by the existing event test. If the CV or FIFO selects A on a busy virtual core, B on a free core remains asleep even though it could complete the event. The portable test forces handoff off and the new fixture omits virtual cores. Exercise this schedule with handoff enabled and measure signal-to-running latency and frame-time tails.
@ -858,0 +891,4 @@
}
ReleaseSRWLockExclusive(&e->lk);
if (hand) {
poke_multiwait(e);
Member

L8 (minor): A successful handoff pokes WaitAny even though it creates no public readiness. If a watched event is signaled repeatedly while direct sleepers are queued, WaitAny can wake and scan on every grant just to sleep again. Poke on transitions that can make the event ready to WaitAny.

**L8** (minor): A successful handoff pokes WaitAny even though it creates no public readiness. If a watched event is signaled repeatedly while direct sleepers are queued, WaitAny can wake and scan on every grant just to sleep again. Poke on transitions that can make the event ready to WaitAny.
@ -932,2 +972,3 @@
__atomic_add_fetch(&e->mwait, 1, __ATOMIC_SEQ_CST);
while (!(got = event_peek(e))) {
for (;;) {
if (slept && e->grants > 0) { /* (HOS_EVENT_HANDOFF: a signal gave it to a sleeper) */
Member

L1: A grant is available to anyone who has slept once, including a caller that started after the signal. Signal sleeping A, hold A before it retakes its core or the lock, and start timed waiter B. When B’s CV wait times out, B consumes A’s grant and reports success. A can then park indefinitely. Conversely, a timed waiter already removed from the CV can still count in mwait when the signal decides whether to leave the event set. Reserve the signal for an eligible pending wait and resolve timeout versus reservation before returning the outcome.

**L1**: A grant is available to anyone who has slept once, including a caller that started after the signal. Signal sleeping A, hold A before it retakes its core or the lock, and start timed waiter B. When B’s CV wait times out, B consumes A’s grant and reports success. A can then park indefinitely. Conversely, a timed waiter already removed from the CV can still count in mwait when the signal decides whether to leave the event set. Reserve the signal for an eligible pending wait and resolve timeout versus reservation before returning the outcome.
@ -81,0 +82,4 @@
// (the park, HOS_CV_PARK=1: host/hos_thread.c; its sleep and its wake of one or all are the condition variable's here)
void hos_park_sleep(std::condition_variable_any* cv, std::mutex* m, unsigned ms, int spun) { (void)spun; hos_cv_sleep(cv, m, ms); }
void hos_park_wake(std::condition_variable_any* cv, int all) { if (all) WakeAllConditionVariable(cv); else WakeConditionVariable(cv); }
int event_handoff() { return 0; } // (HOS_EVENT_HANDOFF=1, the experiment, is tests/test_event_park.py's)
Member

L9: The portable fixture forces the new behavior off, while the fixture that enables it skips Linux. Even on Windows, the two-signal case can pass with the old signaling path if the first sleeper consumes the event between the two signals, and its TryWait runs after both signals. Gate waiter completion until both signals have been issued, check TryWait while a reservation is pending, and exercise the late timed waiter on both platforms.

**L9**: The portable fixture forces the new behavior off, while the fixture that enables it skips Linux. Even on Windows, the two-signal case can pass with the old signaling path if the first sleeper consumes the event between the two signals, and its TryWait runs after both signals. Gate waiter completion until both signals have been issued, check TryWait while a reservation is pending, and exercise the late timed waiter on both platforms.
@ -0,0 +36,4 @@
import tempfile
from pathlib import Path
RUN_ALL_ARGS = ["--cc", "gcc"]
Member

L11: The new check hardcodes gcc in both its suite declaration and direct default, so setting CC to the supported llvm-mingw clang still leaves an LLVM-only Windows install unable to run it through the suite. Use compiler discovery that honors CC and allow --cc to override it.

**L11**: The new check hardcodes gcc in both its suite declaration and direct default, so setting CC to the supported llvm-mingw clang still leaves an LLVM-only Windows install unable to run it through the suite. Use compiler discovery that honors CC and allow --cc to override it.
@ -0,0 +92,4 @@
static void alert_left(void) {
LARGE_INTEGER now;
now.QuadPart = 0;
while (park_on() && park_wait((void *)&alerts_left, &now) != 0x102 /* STATUS_TIMEOUT */) InterlockedIncrement(&alerts_left);
Member

L10 (minor): The alert probes can miss the race they claim to catch. A delayed waker can send its alert after the final post-sleep probe, with no next probe to observe it. Gate the native alert call, force the competing timeout, and assert that the waiter cannot complete before the alert is sent and that it has no pending alert afterward.

**L10** (minor): The alert probes can miss the race they claim to catch. A delayed waker can send its alert after the final post-sleep probe, with no next probe to observe it. Gate the native alert call, force the competing timeout, and assert that the waiter cannot complete before the alert is sent and that it has no pending alert afterward.
@ -0,0 +227,4 @@
}
static void release_all(HANDLE *h, int n) { /* (stop set: the waiters still asleep are let out) */
__atomic_store_n(&stop, 1, __ATOMIC_SEQ_CST);
while (WaitForMultipleObjects(n, h, TRUE, 1) == WAIT_TIMEOUT) {
Member

L13 (minor): release_all treats WAIT_FAILED as a successful join. If one CreateThread call fails, the fixture can go on to replace ev and reset the counters while surviving workers still use them. Check allocation and thread creation, require a successful join, and close handles and free fixture objects afterward.

**L13** (minor): release_all treats WAIT_FAILED as a successful join. If one CreateThread call fails, the fixture can go on to replace ev and reset the counters while surviving workers still use them. Check allocation and thread creation, require a successful join, and close handles and free fixture objects afterward.
@ -0,0 +249,4 @@
wait_for(&taken, k + 1, "an auto-clear event's signal is taken");
if (rnd(&s) % 4 == 0) busy_ns(rnd(&s) % 30000);
}
const LONG slept = sleeps - s0;
Member

L12 (minor): The printed sleep counts use plain volatile reads while workers increment sleeps through InterlockedIncrement. The fixture also writes ev->signaled plainly at shutdown and between manual rounds while the real spin and peek paths read it atomically outside the lock. Use atomic accesses for these shared counters and event state.

**L12** (minor): The printed sleep counts use plain volatile reads while workers increment sleeps through InterlockedIncrement. The fixture also writes ev->signaled plainly at shutdown and between manual rounds while the real spin and peek paths read it atomically outside the lock. Use atomic accesses for these shared counters and event state.
@ -0,0 +328,4 @@
for (int i = 3; i < 5; i++) h[i] = CreateThread(NULL, 0, q_receiver, NULL, 0, NULL);
CHECK(WaitForMultipleObjects(5, h, TRUE, 30000) == WAIT_OBJECT_0);
const LONG64 want = (1 + 2 + 3) * 100000LL * 20000 + 3LL * 20000 * 20001 / 2;
CHECK(got_n == 60000 && got_sum == want && so->qlen == 0);
Member

L14 (minor): Queue count and sum do not verify that nothing was lost or duplicated. Dropping messages 1 and 4 while duplicating 2 and 3 preserves both checks. Verify each sent ID arrives exactly once.

**L14** (minor): Queue count and sum do not verify that nothing was lost or duplicated. Dropping messages 1 and 4 while duplicating 2 and 3 preserves both checks. Verify each sent ID arrives exactly once.
futondestroyer left a comment

Recommendation: Request changes. The handoff can award a signal to a timed waiter that arrived after it, and a single wake can leave a runnable guest worker asleep while the selected worker waits for a virtual core. Both defects are open on this head. Existing review

Does it help? Unproven. In one Windows fixture run at 6353dc42, enabling the handoff cut sleeps for 20,000 auto-clear signals from 30,052 to 19,999 with the native condition variable, and from 32,860 to 19,999 with the park. Those counts do not establish a frame-time or latency gain. A burst can also overwrite the single timestamp used by HOS_WAKE_LAT before either waiter returns. Use timing that associates each measured completion with the signal that released it. Compare current main, the park dependency alone, and handoff on and off in separate processes with the same Market route and settings. Report average FPS, 1% lows, median and p99 frame times, and the spread of interleaved runs.

Other systems. Event types share the handoff on Windows and Linux. Producers include audio, display vsync, video and GPU completion. WaitAny sees only public readiness. The Windows park dependency also serves semaphores and queues and shares thread alerts with other Windows waits.

Switches. HOS_EVENT_HANDOFF is a read-once experiment and defaults to off on both OSes. HOS_CV_PARK from the dependency defaults to off and uses native condition variables when unavailable. Both have switch-table rows and reference entries. Measurements are still needed for a shipping decision.

Reviewed. Head 203ed5f289776a0c945306f311ed38f5d0b696fe against merge base 6e1521a70a32603ed60a8263e4072625885669b7. Read all 11 changed-file diffs, the Windows park and event tests, the portable fixture, the affected runtime code, and the open review. Verified the source-only Graphify maps for both revisions and followed the event, wait, wake and virtual-core paths in source. No PR code ran.

Checked and fine. Grant increments and decrements hold the event lock. The new park and handoff switches are documented. The diff contains no renderer interface, generated shader, or third-party changes.

Blocking

  • host/nn_os_sync.c:974 checks whether a waiter has ever slept and whether any grant is pending. It does not establish that the waiter was eligible for the signal. Gate a portable test so an earlier sleeper is delayed, start a second timed wait after the signal, and verify that the earlier wait alone receives it. Resolve timeout and reservation under the same protocol. Existing finding
  • host/nn_os_sync.c:890 can select a thread pinned to a busy virtual core and leave a second thread on a free core asleep. Main broadcasts when several waiters can compete for the event. Preserve that progress guarantee and run the portable busy-core fixture with handoff enabled. Existing finding
  • host/nn_os_sync.c:875 caches the switch in a plain shared static that concurrent first callers can read and write even with the switch unset. host/nn_os_sync.c:885 also adds an unconditional read of the event mode while initialization can still be writing it. The renderer's audio-event producer can see the event key before initialization finishes. Synchronize switch initialization and event-mode publication. Switch finding · Event publication finding

Testing. Author-reported Windows fixture passes are from 6353dc42 on main 2e1f6c32. The PR reports patch parity across the rebase. Main 00e0dc93 changed test_crash_dialog.py and its implementation in the rebase interval, so the earlier inherited-failure result for that check needs verification on the current base and head. The description check passed on 203ed5f2, and the source checks are blocked by required conditions. Windows and Linux host builds, Linux handoff-enabled coverage, the author's Windows game run, and gameplay measurements are still open. Complete the deterministic event tests before hardware runs.

Overlaps and merge order. The first three commits are exactly the park dependency ending at 82d36ed1000c93c10526953b3c316782ff2c2409. Resolve the park review there and land it first, or group the dependent changes in one PR. The wake-latency diagnostic also conflicts in event_wait and needs a combined check if both changes land. Park dependency · Wake-latency diagnostic

**Recommendation: Request changes.** The handoff can award a signal to a timed waiter that arrived after it, and a single wake can leave a runnable guest worker asleep while the selected worker waits for a virtual core. Both defects are open on this head. [Existing review](https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/1971#issuecomment-62371) **Does it help?** Unproven. In one Windows fixture run at `6353dc42`, enabling the handoff cut sleeps for 20,000 auto-clear signals from 30,052 to 19,999 with the native condition variable, and from 32,860 to 19,999 with the park. Those counts do not establish a frame-time or latency gain. A burst can also overwrite the single timestamp used by `HOS_WAKE_LAT` before either waiter returns. Use timing that associates each measured completion with the signal that released it. Compare current main, the park dependency alone, and handoff on and off in separate processes with the same Market route and settings. Report average FPS, 1% lows, median and p99 frame times, and the spread of interleaved runs. **Other systems.** Event types share the handoff on Windows and Linux. Producers include audio, display vsync, video and GPU completion. WaitAny sees only public readiness. The Windows park dependency also serves semaphores and queues and shares thread alerts with other Windows waits. **Switches.** `HOS_EVENT_HANDOFF` is a read-once experiment and defaults to off on both OSes. `HOS_CV_PARK` from the dependency defaults to off and uses native condition variables when unavailable. Both have switch-table rows and reference entries. Measurements are still needed for a shipping decision. **Reviewed.** Head `203ed5f289776a0c945306f311ed38f5d0b696fe` against merge base `6e1521a70a32603ed60a8263e4072625885669b7`. Read all 11 changed-file diffs, the Windows park and event tests, the portable fixture, the affected runtime code, and the open review. Verified the source-only Graphify maps for both revisions and followed the event, wait, wake and virtual-core paths in source. No PR code ran. **Checked and fine.** Grant increments and decrements hold the event lock. The new park and handoff switches are documented. The diff contains no renderer interface, generated shader, or third-party changes. **Blocking** - `host/nn_os_sync.c:974` checks whether a waiter has ever slept and whether any grant is pending. It does not establish that the waiter was eligible for the signal. Gate a portable test so an earlier sleeper is delayed, start a second timed wait after the signal, and verify that the earlier wait alone receives it. Resolve timeout and reservation under the same protocol. [Existing finding](https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/1971#issuecomment-62357) - `host/nn_os_sync.c:890` can select a thread pinned to a busy virtual core and leave a second thread on a free core asleep. Main broadcasts when several waiters can compete for the event. Preserve that progress guarantee and run the portable busy-core fixture with handoff enabled. [Existing finding](https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/1971#issuecomment-62358) - `host/nn_os_sync.c:875` caches the switch in a plain shared static that concurrent first callers can read and write even with the switch unset. `host/nn_os_sync.c:885` also adds an unconditional read of the event mode while initialization can still be writing it. The renderer's audio-event producer can see the event key before initialization finishes. Synchronize switch initialization and event-mode publication. [Switch finding](https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/1971#issuecomment-62359) · [Event publication finding](https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/1971#issuecomment-62360) **Testing.** Author-reported Windows fixture passes are from `6353dc42` on main `2e1f6c32`. The PR reports patch parity across the rebase. Main `00e0dc93` changed `test_crash_dialog.py` and its implementation in the rebase interval, so the earlier inherited-failure result for that check needs verification on the current base and head. The description check passed on `203ed5f2`, and the source checks are blocked by required conditions. Windows and Linux host builds, Linux handoff-enabled coverage, the author's Windows game run, and gameplay measurements are still open. Complete the deterministic event tests before hardware runs. **Overlaps and merge order.** The first three commits are exactly the park dependency ending at `82d36ed1000c93c10526953b3c316782ff2c2409`. Resolve the park review there and land it first, or group the dependent changes in one PR. The wake-latency diagnostic also conflicts in `event_wait` and needs a combined check if both changes land. [Park dependency](https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/1970) · [Wake-latency diagnostic](https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/1967)
@ -858,0 +886,4 @@
AcquireSRWLockExclusive(&e->lk);
const int hand = e->mwait > e->grants;
if (hand) {
e->grants++;
Member

For the A/B, please verify signal-to-running latency with a timestamp for each grant. Two signals can grant two waiters before either returns. Both waiters then read the last event-wide sig_ns, so HOS_WAKE_LAT can understate the tail for the first signal.

For the A/B, please verify signal-to-running latency with a timestamp for each grant. Two signals can grant two waiters before either returns. Both waiters then read the last event-wide sig_ns, so HOS_WAKE_LAT can understate the tail for the first signal.
siantm force-pushed perf/event-handoff from 203ed5f289
Some checks failed
PR description / Description follows the template (pull_request_target) Successful in 15s
Website / Website on Cloudflare (pull_request_target) Has been skipped
CI / Conventional Commit title (pull_request) Has been cancelled
CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Has been cancelled
CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Has been cancelled
CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Has been cancelled
CI / Workflow lint (actionlint) (pull_request) Has been cancelled
CI / Website build and checks (pull_request) Has been cancelled
Guide screenshots / Guide images and local UI source review (pull_request) Has been cancelled
Third-party parts / As their notes say (pull_request) Has been cancelled
Third-party parts / The same as upstream (pull_request) Has been cancelled
Website / Website on Cloudflare (pull_request) Has been cancelled
to c438b91454
Some checks are pending
CI / Conventional Commit title (pull_request) Blocked by required conditions
CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Blocked by required conditions
CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Blocked by required conditions
CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Blocked by required conditions
CI / Workflow lint (actionlint) (pull_request) Blocked by required conditions
CI / Website build and checks (pull_request) Blocked by required conditions
Guide screenshots / Guide images and local UI source review (pull_request) Blocked by required conditions
Third-party parts / As their notes say (pull_request) Blocked by required conditions
Third-party parts / The same as upstream (pull_request) Blocked by required conditions
Website / Website on Cloudflare (pull_request) Blocked by required conditions
Website / Website on Cloudflare (pull_request_target) Has been skipped
beanstalk/approval waiting for a BigBean's 👍 on the request
PR description / Description follows the template (pull_request_target) Successful in 19s
2026-11-21 04:47:32 +00:00
Compare
Author
Contributor

Description updated for the push of c438b914 (on #1970's ff7e2362, main @ f1f2dc5b): the Summary describes the hand-off's eligibility, its wake (all of them when several sleep, as main), per-hand-off time and the atomic switch and event mode, Testing lists what ran in job fb3-1971-1 (including test_crash_dialog on main and the head, and the four switch combinations), and Measurements has the fixture's new sleep counts: with the wake of all kept, the hand-off now makes more sleeps than unset. Still depends on #1970.

Description updated for the push of c438b914 (on #1970's ff7e2362, `main` @ f1f2dc5b): the Summary describes the hand-off's eligibility, its wake (all of them when several sleep, as main), per-hand-off time and the atomic switch and event mode, Testing lists what ran in job fb3-1971-1 (including `test_crash_dialog` on `main` and the head, and the four switch combinations), and Measurements has the fixture's new sleep counts: with the wake of all kept, the hand-off now makes more sleeps than unset. Still depends on #1970.
Author
Contributor

@futondestroyer thanks. Pushed c438b914, rebased onto #1970's new head ff7e2362 (on main @ f1f2dc5b). This PR's commit 093e9e8a carries over with two conflicts, both from #1970's rewritten tests/test_event_park.py and its switch row: the hand-off's cases and row are added back unchanged. The test commit 140850be fails and the fix c438b914 passes, run through the PC's job queue (fb3-1971-1). The description still says it depends on #1970.

  • :974, a later timed waiter: sleepers are now numbered under e->lk as they come in (e->wseq). A hand-off records the last number (e->grant[], oldest first), and a sleeper takes only the oldest hand-off made while it was in. It looks before anything else each time it is back under e->lk, before its deadline is resolved there, so a hand-off made before that is its outcome and none is left behind. Check tests/test_event_handoff.py "late" (portable, gated): the sleeper handed the signal is held on its way back; a TryWait then fails, and a 20 ms timed wait started after the signal must time out. At 140850be that wait took the signal; at the head it times out and the held sleeper gets it. "timeout": a timed sleeper handed the signal before its deadline and held past it returns the signal, with no grant left.
  • :890, wake of one: the hand-off now wakes as event_signal does otherwise: one sleeper, or all of them when several sleep. The first back under e->lk takes it and the others sleep again. The portable busy-core case ("busy") now runs with the hand-off off and on: at 140850be, on, it stalled with the free-core sleeper asleep; at the head it passes both ways. This gives back part of the fixture's sleep saving when several sleep (in tests/test_event_park.py's races at the head, 44,441 sleeps for 20,000 auto-clear signals with the hand-off on against 29,794 unset (44,980 against 31,457 on the park); before this change, on was the lower one. The description says so under Measurements).
  • :875, the switch and :885, the event's mode: event_handoff reads and publishes the switch atomically, as cv_signal_always does. event_signal tests the switch first (off: the mode is not read there, as on main). event_init, the timer's init, event_peek and event_signal access autoclear atomically. OpenAudioRenderer initializes the event before it publishes the key to the renderer thread (atomic store/load), and withdraws the key while it makes the event again. These are data races with no behavioural test here (no ThreadSanitizer on Windows); the switch's values are covered as before.
  • Inline :889, one timestamp per grant: each hand-off carries its signal's time, and the sleeper that takes it reports that time to HOS_WAKE_LAT (a sleeper that took a set event still uses e->sig_ns). Check "lat": two signals 30 ms apart to two held sleepers. At 140850be both latencies were from the last signal (14 and 21 us); at the head, 27 us and 31,973 us.
  • Review, testing: test_crash_dialog fails the same way on main @ f1f2dc5b and at this head (job fb3-1971-1: a UnicodeDecodeError in its output reader, then TypeError: 'NoneType' object is not subscriptable at tests/test_crash_dialog.py:596), so the inherited-failure claim holds on the current base. run_all --changed origin/main at the head: 83 passed, 17 failed, 5 skipped, and all 17 fail on main too (run by name in the same job). The four combinations of HOS_CV_PARK and HOS_EVENT_HANDOFF on this head (tests/test_event_park.py, races: 20,000 auto-clear signals, manual-clear rounds, timed waits, semaphore, queue): unset, HOS_CV_PARK=1, HOS_EVENT_HANDOFF=1 and both all pass (job fb3-1971-1), with the hand-off's two-sleeper case in both of its runs and its no-grant control failing both ways. test_event_handoff's cases are portable and run on the condition-variable model, so they also cover Linux's path.
  • The A/B (main, #1970 alone, this PR with the switch off and on; the Market, same route and settings, interleaved runs; average fps, 1% lows, median and p99 frame times): pending, asked of HostLiftPRs.
  • Inherited park points (BeanBot L5-L7, L10-L14): fixed in #1970's commits; see the reply there.
  • Overlap with #1967 (event_wait and wake_lat_note): a read-only git merge-tree of #1967's new head 4d2ebc2a with this head conflicts in two places in event_wait: the wait's locals (missed, handed against #1967's naps) and the HOS_WAKE_LAT read (this PR's !handed, #1967's naps). Whichever lands second keeps #1967's sleep state and the park's first-sleep behaviour (missed && !slept) when it is rebased.
@futondestroyer thanks. Pushed c438b914, rebased onto #1970's new head ff7e2362 (on `main` @ f1f2dc5b). This PR's commit 093e9e8a carries over with two conflicts, both from #1970's rewritten `tests/test_event_park.py` and its switch row: the hand-off's cases and row are added back unchanged. The test commit 140850be fails and the fix c438b914 passes, run through the PC's job queue (fb3-1971-1). The description still says it depends on #1970. - [`:974`, a later timed waiter](https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/1971#issuecomment-62357): sleepers are now numbered under `e->lk` as they come in (`e->wseq`). A hand-off records the last number (`e->grant[]`, oldest first), and a sleeper takes only the oldest hand-off made while it was in. It looks before anything else each time it is back under `e->lk`, before its deadline is resolved there, so a hand-off made before that is its outcome and none is left behind. Check `tests/test_event_handoff.py` "late" (portable, gated): the sleeper handed the signal is held on its way back; a TryWait then fails, and a 20 ms timed wait started after the signal must time out. At 140850be that wait took the signal; at the head it times out and the held sleeper gets it. "timeout": a timed sleeper handed the signal before its deadline and held past it returns the signal, with no grant left. - [`:890`, wake of one](https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/1971#issuecomment-62358): the hand-off now wakes as `event_signal` does otherwise: one sleeper, or all of them when several sleep. The first back under `e->lk` takes it and the others sleep again. The portable busy-core case ("busy") now runs with the hand-off off and on: at 140850be, on, it stalled with the free-core sleeper asleep; at the head it passes both ways. This gives back part of the fixture's sleep saving when several sleep (in `tests/test_event_park.py`'s races at the head, 44,441 sleeps for 20,000 auto-clear signals with the hand-off on against 29,794 unset (44,980 against 31,457 on the park); before this change, on was the lower one. The description says so under Measurements). - [`:875`, the switch](https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/1971#issuecomment-62359) and [`:885`, the event's mode](https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/1971#issuecomment-62360): `event_handoff` reads and publishes the switch atomically, as `cv_signal_always` does. `event_signal` tests the switch first (off: the mode is not read there, as on `main`). `event_init`, the timer's init, `event_peek` and `event_signal` access `autoclear` atomically. `OpenAudioRenderer` initializes the event before it publishes the key to the renderer thread (atomic store/load), and withdraws the key while it makes the event again. These are data races with no behavioural test here (no ThreadSanitizer on Windows); the switch's values are covered as before. - [Inline `:889`, one timestamp per grant](https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/1971#issuecomment-62908): each hand-off carries its signal's time, and the sleeper that takes it reports that time to `HOS_WAKE_LAT` (a sleeper that took a set event still uses `e->sig_ns`). Check "lat": two signals 30 ms apart to two held sleepers. At 140850be both latencies were from the last signal (14 and 21 us); at the head, 27 us and 31,973 us. - [Review, testing](https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/1971#issuecomment-62909): `test_crash_dialog` fails the same way on `main` @ f1f2dc5b and at this head (job fb3-1971-1: a `UnicodeDecodeError` in its output reader, then `TypeError: 'NoneType' object is not subscriptable` at `tests/test_crash_dialog.py:596`), so the inherited-failure claim holds on the current base. `run_all --changed origin/main` at the head: 83 passed, 17 failed, 5 skipped, and all 17 fail on `main` too (run by name in the same job). The four combinations of `HOS_CV_PARK` and `HOS_EVENT_HANDOFF` on this head (`tests/test_event_park.py`, races: 20,000 auto-clear signals, manual-clear rounds, timed waits, semaphore, queue): unset, `HOS_CV_PARK=1`, `HOS_EVENT_HANDOFF=1` and both all pass (job fb3-1971-1), with the hand-off's two-sleeper case in both of its runs and its no-grant control failing both ways. `test_event_handoff`'s cases are portable and run on the condition-variable model, so they also cover Linux's path. - The A/B (`main`, #1970 alone, this PR with the switch off and on; the Market, same route and settings, interleaved runs; average fps, 1% lows, median and p99 frame times): pending, asked of HostLiftPRs. - Inherited park points (BeanBot L5-L7, L10-L14): fixed in #1970's commits; see [the reply there](https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/1970#issuecomment-63600). - Overlap with #1967 (`event_wait` and `wake_lat_note`): a read-only `git merge-tree` of #1967's new head 4d2ebc2a with this head conflicts in two places in `event_wait`: the wait's locals (`missed`, `handed` against #1967's `naps`) and the `HOS_WAKE_LAT` read (this PR's `!handed`, #1967's `naps`). Whichever lands second keeps #1967's sleep state and the park's first-sleep behaviour (`missed && !slept`) when it is rebased.
Author
Contributor

BeanBot's review, at c438b914 (test commit 140850be, fix c438b914; on #1970's ff7e2362):

  • L1 (a later timed waiter), L2 (wake of one, busy core), L3 (switch race), L4 (event mode during init): fixed; see the reply to futondestroyer's review for each change and its check. On L4: the audio renderer's key is now published after its event is initialised.
  • L5, L6, L7 (the park's first-use race, thread alerts, native failures): fixed in #1970 (e4b273b1) with its checks; see the reply there.
  • L8 (minor, WaitAny poke): a hand-off no longer pokes WaitAny.
  • L9 (the portable fixture forced the hand-off off): tests/test_event_handoff.py runs every case with it off and on. "late" holds the earlier sleeper until both the signal and a later timed wait are done, checks a TryWait during the reservation, and runs on both platforms.
  • L10-L14 (the park fixture: alert probes, gcc, atomics, joins, each queue message once): fixed in #1970's test commit 1e2adc46.
  • L15: the spin budget is read once (#1970 ff7e2362). With the switch off, the hand-off costs one atomic load of the switch before anything else, plus numbering the sleepers under e->lk. The park's list scan is not instrumented: a list holds the sleepers of one object, and the in-game A/B (pending with HostLiftPRs) measures the cost.
  • L16: Testing gives run_all's own totals now (83 passed, 17 failed, 5 skipped, each failure also failing on main). The game run is pending, asked of HostLiftPRs.
BeanBot's [review](https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/1971#issuecomment-62371), at c438b914 (test commit 140850be, fix c438b914; on #1970's ff7e2362): - L1 (a later timed waiter), L2 (wake of one, busy core), L3 (switch race), L4 (event mode during init): fixed; see the reply to futondestroyer's review for each change and its check. On L4: the audio renderer's key is now published after its event is initialised. - L5, L6, L7 (the park's first-use race, thread alerts, native failures): fixed in #1970 (e4b273b1) with its checks; see [the reply there](https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/1970#issuecomment-63603). - L8 (minor, WaitAny poke): a hand-off no longer pokes WaitAny. - L9 (the portable fixture forced the hand-off off): `tests/test_event_handoff.py` runs every case with it off and on. "late" holds the earlier sleeper until both the signal and a later timed wait are done, checks a TryWait during the reservation, and runs on both platforms. - L10-L14 (the park fixture: alert probes, gcc, atomics, joins, each queue message once): fixed in #1970's test commit 1e2adc46. - L15: the spin budget is read once (#1970 ff7e2362). With the switch off, the hand-off costs one atomic load of the switch before anything else, plus numbering the sleepers under `e->lk`. The park's list scan is not instrumented: a list holds the sleepers of one object, and the in-game A/B (pending with HostLiftPRs) measures the cost. - L16: Testing gives run_all's own totals now (83 passed, 17 failed, 5 skipped, each failure also failing on `main`). The game run is pending, asked of HostLiftPRs.
Author
Contributor

@Beanstalk This adds a switch, HOS_EVENT_HANDOFF, off by default: with HOS_EVENT_HANDOFF=1, the signal of an auto-clear guest event that finds a sleeper hands the event to it (host/nn_os_sync.c). It is on #1970, whose commits are in its diff. It has not run in the game yet. Please play Castle Town Market for two minutes, the PR against main on the same PC, on the PR's side with HOS_EVENT_HANDOFF=1 and with it unset, on a Windows PC with an AMD processor, a Linux PC with an AMD processor, a PC with 6 or fewer logical processors, and the Steam Deck. No stall, hang or crash, and no audio, vsync or movie trouble; set, frame time p99 equal to unset or better, within the run-to-run spread; unset, frame times as main's.

@Beanstalk This adds a switch, HOS_EVENT_HANDOFF, off by default: with HOS_EVENT_HANDOFF=1, the signal of an auto-clear guest event that finds a sleeper hands the event to it (host/nn_os_sync.c). It is on #1970, whose commits are in its diff. It has not run in the game yet. Please play Castle Town Market for two minutes, the PR against main on the same PC, on the PR's side with HOS_EVENT_HANDOFF=1 and with it unset, on a Windows PC with an AMD processor, a Linux PC with an AMD processor, a PC with 6 or fewer logical processors, and the Steam Deck. No stall, hang or crash, and no audio, vsync or movie trouble; set, frame time p99 equal to unset or better, within the run-to-run spread; unset, frame times as main's.
Member

@siantm only members of BigBeans can start the farm. One of them can approve this request by reacting 👍 to it or to this comment.

@siantm only members of `BigBeans` can start the farm. One of them can approve this request by reacting 👍 to it or to this comment.
Author
Contributor

The three renderer checks (test_present_queue_default, test_target_reuse_memory, test_vk_occlusion_queries) did not compile in the earlier run (no Vulkan headers on that job's path); rerun with them in job fb3-vk-1, at the head c438b914 and on main @ f1f2dc5b: all three fail at both with the same first error, so they fail on main too; Tested's sentence and counts corrected.

The three renderer checks (`test_present_queue_default`, `test_target_reuse_memory`, `test_vk_occlusion_queries`) did not compile in the earlier run (no Vulkan headers on that job's path); rerun with them in job fb3-vk-1, at the head c438b914 and on `main` @ f1f2dc5b: all three fail at both with the same first error, so they fail on `main` too; Tested's sentence and counts corrected.
futondestroyer left a comment

Recommendation: Request changes. The audio renderer can still deliver a stale notification across event teardown and reuse. The handoff also increases every synchronization object's footprint with the experiment off and inherits the Windows alert cleanup blocker from the park branch.

The new code reserves auto-clear signals for waiters already registered on the event. It wakes all sleepers when there is more than one, so a ready guest worker can compete even if another worker cannot retake its virtual core. TryWait and WaitAny can still consume a public signal when a signal has no sleeper it can grant to or when the reservation queue fills.

Does it help? Unproven. The author's current-head Windows fixture recorded 29,794 to 44,441 sleeps for 20,000 auto-clear signals with park unset (up 49%), and 31,457 to 44,980 with park enabled (up 43%). For 5,000 timed signals, the counts rose from 6,859 to 10,036 and from 7,440 to 9,327, respectively. These are single runs measuring sleeps. The reported 27 and 31,973 μs samples exercise signal attribution in a gated fixture. Gameplay frame times, CPU cost and memory costs are still unknown. Author's results

Other systems. Normal, light and system events share the handoff on Windows and Linux. Audio renderer notifications, display vsync, movie playback and GPU completion reach the system event path. The park dependency changes Windows waits for events, semaphores and queues and shares thread alerts with the game job barriers. Every synchronization object gets larger, including objects whose operations never use event handoffs.

Switches. HOS_EVENT_HANDOFF is documented, off by default on Windows and Linux and read on first use. HOS_CV_PARK is documented, off by default, and enabled only on Windows with working native entry points. Keep the experiments off pending correctness and measurements. Compare the exact dependency alone and this candidate with handoff unset because that candidate still adds allocation and wait-path costs.

Reviewed. Head c438b91454a40cd58c56d6164a693f38ebe68120 against merge base f1f2dc5bd2821cef061798ba40a043d281ac1707 of the head and official target 9dd41326e71094d8b774459dac5154eb5a3306d9. Prior reviewed head 203ed5f289776a0c945306f311ed38f5d0b696fe. The three handoff commits sit directly on ff7e23622b749cff8ac14e28323bb8818addd44d. Read the 13 changed-file diffs and the relevant event, audio, allocation, wait and test paths. Used local source-only Graphify maps for the head, base and dependency, verified their provenance and extraction logs, and checked HLE wrappers and native dispatch in source. Longshot guidance 9dd41326 and Tooling 0b0059ac. Source review only. Prior review

Checked and fine. The sequence cutoff excludes waiters who arrive after a grant. Oldest eligible grant first preserves a newer grant for a later waiter who missed the older one. Claims precede the timeout decision under the event lock, so a pre-deadline grant survives delayed virtual-core reacquisition. The portable busy-core case now exercises the restored broadcast with the switch enabled. The switch cache and event-mode scalar accesses are atomic. A fresh audio key is published after initialization. Each private grant carries its timestamp, and the grant path omits the WaitAny wake. The diff adds no generated shaders, vendor changes or renderer interfaces.

Blocking

  1. host/nn_audio.c:1496 can signal a reused event after the key was withdrawn. Pause the audio worker after it acquires the old key. Close the renderer and finish reopening at the same guest event address, then resume the worker. Its address-only lookup can signal or reserve a wake on the fully initialized new event even before the new renderer starts. Withdrawal leaves the address in the paused worker's local variable. Serialize the complete lookup and signal with retirement and replacement, or make the signal honor a lifetime token. Cover close and reopen at the same address with a gated producer. First-use publication fix
  2. host/nn_os_sync.c:46 adds 136 bytes to every x86-64 Obj by layout, including at the default setting. The eight timestamped grants grow the objects allocated and reset for mutexes, semaphores, queues and holders, and the pool retains spare allocations. Keep the reservation state on enabled events, or quantify the retained memory and runtime cost of both disabled and enabled arms against the park-only branch and account for the cost in Risks.
  3. The inherited park still discards the status of its final native-alert drain at host/hos_thread.c:2369. If the wait path fails after its peer sends a real alert, an alert can survive into the next job-barrier wait. Fix and verify the park dependency before rebasing this branch. Park review

Testing. The author reports that the portable cases failed the earlier late-waiter, busy-core and timestamp implementations and pass at this head, and that the Windows fixture passed all four switch combinations. The portable model replaces the virtual-core scheduler, park wait and switch reader. The Windows fixture tests the native park but leaves virtual-core reacquisition out. For the per-signal metric, the portable check currently accepts both samples using the first signal's timestamp. Give it a controlled clock and assert both latencies separately. Add an inverse timeout case: let the native wait expire while the guest core is unavailable, signal after the wall-clock deadline, and verify the intended arbitration for a timed waiter that has not yet returned under the event lock, including public readiness to TryWait or WaitAny. The existing timeout case only covers a signal sent before the deadline.

The author reports 83 passed, 17 failed and 5 skipped for the strict changed suite, with all 17 failures reproduced against f1f2dc5b, including three rerun with Vulkan headers. The description check passed in the Forgejo snapshot. CI and BigBean approval were pending. The full suite, Windows and Linux host builds, Linux execution, author gameplay and performance evidence remain due. Fixture and rebase notes · Strict-check follow-up

Overlaps and merge order. Correct and test the standalone park branch before rebasing this one. A three-way source check against current main shows conflicts in the spin-policy setup and the queue's NVN observation before sleep. Preserve the serialized spin switch from the merged fix, the park's first-sleep argument and the NVN observation. The wake-latency sibling also conflicts in two places in event_wait. Preserve its sleep count and the per-grant timestamp together. Re-run the affected fixtures with each relevant switch combination on the rebased head. Park dependency · Merged spin fix · Merged NVN change · Wake-latency sibling

**Recommendation: Request changes.** The audio renderer can still deliver a stale notification across event teardown and reuse. The handoff also increases every synchronization object's footprint with the experiment off and inherits the Windows alert cleanup blocker from the park branch. The new code reserves auto-clear signals for waiters already registered on the event. It wakes all sleepers when there is more than one, so a ready guest worker can compete even if another worker cannot retake its virtual core. TryWait and WaitAny can still consume a public signal when a signal has no sleeper it can grant to or when the reservation queue fills. **Does it help?** Unproven. The author's current-head Windows fixture recorded 29,794 to 44,441 sleeps for 20,000 auto-clear signals with park unset (up 49%), and 31,457 to 44,980 with park enabled (up 43%). For 5,000 timed signals, the counts rose from 6,859 to 10,036 and from 7,440 to 9,327, respectively. These are single runs measuring sleeps. The reported 27 and 31,973 μs samples exercise signal attribution in a gated fixture. Gameplay frame times, CPU cost and memory costs are still unknown. [Author's results](https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/1971#issuecomment-63605) **Other systems.** Normal, light and system events share the handoff on Windows and Linux. Audio renderer notifications, display vsync, movie playback and GPU completion reach the system event path. The park dependency changes Windows waits for events, semaphores and queues and shares thread alerts with the game job barriers. Every synchronization object gets larger, including objects whose operations never use event handoffs. **Switches.** `HOS_EVENT_HANDOFF` is documented, off by default on Windows and Linux and read on first use. `HOS_CV_PARK` is documented, off by default, and enabled only on Windows with working native entry points. Keep the experiments off pending correctness and measurements. Compare the exact dependency alone and this candidate with handoff unset because that candidate still adds allocation and wait-path costs. **Reviewed.** Head `c438b91454a40cd58c56d6164a693f38ebe68120` against merge base `f1f2dc5bd2821cef061798ba40a043d281ac1707` of the head and official target `9dd41326e71094d8b774459dac5154eb5a3306d9`. Prior reviewed head `203ed5f289776a0c945306f311ed38f5d0b696fe`. The three handoff commits sit directly on `ff7e23622b749cff8ac14e28323bb8818addd44d`. Read the 13 changed-file diffs and the relevant event, audio, allocation, wait and test paths. Used local source-only Graphify maps for the head, base and dependency, verified their provenance and extraction logs, and checked HLE wrappers and native dispatch in source. Longshot guidance `9dd41326` and Tooling `0b0059ac`. Source review only. [Prior review](https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/1971#issuecomment-62909) **Checked and fine.** The sequence cutoff excludes waiters who arrive after a grant. Oldest eligible grant first preserves a newer grant for a later waiter who missed the older one. Claims precede the timeout decision under the event lock, so a pre-deadline grant survives delayed virtual-core reacquisition. The portable busy-core case now exercises the restored broadcast with the switch enabled. The switch cache and event-mode scalar accesses are atomic. A fresh audio key is published after initialization. Each private grant carries its timestamp, and the grant path omits the WaitAny wake. The diff adds no generated shaders, vendor changes or renderer interfaces. **Blocking** 1. `host/nn_audio.c:1496` can signal a reused event after the key was withdrawn. Pause the audio worker after it acquires the old key. Close the renderer and finish reopening at the same guest event address, then resume the worker. Its address-only lookup can signal or reserve a wake on the fully initialized new event even before the new renderer starts. Withdrawal leaves the address in the paused worker's local variable. Serialize the complete lookup and signal with retirement and replacement, or make the signal honor a lifetime token. Cover close and reopen at the same address with a gated producer. [First-use publication fix](https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/1971#issuecomment-63605) 2. `host/nn_os_sync.c:46` adds 136 bytes to every x86-64 `Obj` by layout, including at the default setting. The eight timestamped grants grow the objects allocated and reset for mutexes, semaphores, queues and holders, and the pool retains spare allocations. Keep the reservation state on enabled events, or quantify the retained memory and runtime cost of both disabled and enabled arms against the park-only branch and account for the cost in Risks. 3. The inherited park still discards the status of its final native-alert drain at `host/hos_thread.c:2369`. If the wait path fails after its peer sends a real alert, an alert can survive into the next job-barrier wait. Fix and verify the park dependency before rebasing this branch. [Park review](https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/1970) **Testing.** The author reports that the portable cases failed the earlier late-waiter, busy-core and timestamp implementations and pass at this head, and that the Windows fixture passed all four switch combinations. The portable model replaces the virtual-core scheduler, park wait and switch reader. The Windows fixture tests the native park but leaves virtual-core reacquisition out. For the per-signal metric, the portable check currently accepts both samples using the first signal's timestamp. Give it a controlled clock and assert both latencies separately. Add an inverse timeout case: let the native wait expire while the guest core is unavailable, signal after the wall-clock deadline, and verify the intended arbitration for a timed waiter that has not yet returned under the event lock, including public readiness to TryWait or WaitAny. The existing timeout case only covers a signal sent before the deadline. The author reports 83 passed, 17 failed and 5 skipped for the strict changed suite, with all 17 failures reproduced against `f1f2dc5b`, including three rerun with Vulkan headers. The description check passed in the Forgejo snapshot. CI and BigBean approval were pending. The full suite, Windows and Linux host builds, Linux execution, author gameplay and performance evidence remain due. [Fixture and rebase notes](https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/1971#issuecomment-63605) · [Strict-check follow-up](https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/1971#issuecomment-64871) **Overlaps and merge order.** Correct and test the standalone park branch before rebasing this one. A three-way source check against current main shows conflicts in the spin-policy setup and the queue's NVN observation before sleep. Preserve the serialized spin switch from the merged fix, the park's first-sleep argument and the NVN observation. The wake-latency sibling also conflicts in two places in `event_wait`. Preserve its sleep count and the per-grant timestamp together. Re-run the affected fixtures with each relevant switch combination on the rebased head. [Park dependency](https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/1970) · [Merged spin fix](https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/2012) · [Merged NVN change](https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/1859) · [Wake-latency sibling](https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/1967)
@ -1494,2 +1494,3 @@
elapsed_frames++;
if (renderer_event) hos_system_event_signal(renderer_event);
const uint64_t ev = __atomic_load_n(&renderer_event, __ATOMIC_ACQUIRE);
if (ev) hos_system_event_signal(ev);
Member

Pause this worker just after it loads the key, then close and reopen the renderer at the same guest event address. When it resumes, the address-only lookup can signal the new event or reserve a wake for it using a frame from the old renderer. The release-store of zero cannot withdraw the address this worker already holds. Drain or serialize the complete lookup and signal with event replacement, or make signaling honor an event lifetime token. A gated close/reopen case can exercise this interleaving.

Pause this worker just after it loads the key, then close and reopen the renderer at the same guest event address. When it resumes, the address-only lookup can signal the new event or reserve a wake for it using a frame from the old renderer. The release-store of zero cannot withdraw the address this worker already holds. Drain or serialize the complete lookup and signal with event replacement, or make signaling honor an event lifetime token. A gated close/reopen case can exercise this interleaving.
@ -41,0 +43,4 @@
time (HOS_WAKE_LAT); at most 8 at once */
int grants;
unsigned wseq;
struct { unsigned seq; int64_t ns; } grant[8];
Member

The grant array, counter and sequence grow every x86-64 Obj by 136 bytes, even when the switch is unset and for mutex, queue and WaitAny objects. obj_lookup allocates and resets that larger layout, and the spare pool retains it. Keep the reservation state on enabled events, or report retained memory and disabled-path CPU/cache cost against the park-only branch and account for the cost under Risks.

The grant array, counter and sequence grow every x86-64 Obj by 136 bytes, even when the switch is unset and for mutex, queue and WaitAny objects. obj_lookup allocates and resets that larger layout, and the spare pool retains it. Keep the reservation state on enabled events, or report retained memory and disabled-path CPU/cache cost against the park-only branch and account for the cost under Risks.
@ -129,0 +197,4 @@
CHECK(lats.size()==2);
const int64_t hi=std::max(lats[0],lats[1]), lo=std::min(lats[0],lats[1]);
std::printf("lat: %lld and %lld us\n",(long long)(lo/1000),(long long)(hi/1000));
if (hi<25000000) FAIL("both sleepers' latencies are from the last signal (HOS_WAKE_LAT understates the first)");
Member

This passes if both samples use the first signal's timestamp, since the larger sample still exceeds 25 ms. Drive the two signals and gate release with a controlled clock and require the two distinct expected latencies, so the fixture rejects substituting either signal's timestamp for both.

This passes if both samples use the first signal's timestamp, since the larger sample still exceeds 25 ms. Drive the two signals and gate release with a controlled clock and require the two distinct expected latencies, so the fixture rejects substituting either signal's timestamp for both.
Some checks are pending
CI / Conventional Commit title (pull_request) Blocked by required conditions
CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Blocked by required conditions
CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Blocked by required conditions
CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Blocked by required conditions
CI / Workflow lint (actionlint) (pull_request) Blocked by required conditions
CI / Website build and checks (pull_request) Blocked by required conditions
Guide screenshots / Guide images and local UI source review (pull_request) Blocked by required conditions
Third-party parts / As their notes say (pull_request) Blocked by required conditions
Third-party parts / The same as upstream (pull_request) Blocked by required conditions
Website / Website on Cloudflare (pull_request) Blocked by required conditions
Website / Website on Cloudflare (pull_request_target) Has been skipped
beanstalk/approval waiting for a BigBean's 👍 on the request
PR description / Description follows the template (pull_request_target) Successful in 19s
This pull request has changes conflicting with the target branch.
  • host/nn_os_sync.c
Some workflows are waiting to be reviewed.
View command line instructions

Manual merge helper

Use this merge commit message when completing the merge manually.

Checkout

From your project repository, check out a new branch and test the changes.
git fetch -u 'perf/event-handoff:siantm-perf/event-handoff'
git switch 'siantm-perf/event-handoff'
Sign in to join this conversation.
No reviewers
No milestone
No project
No assignees
4 participants
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
MagicBeanDev/Longshot!1971
No description provided.