WIP: perf(host): notify message queue waiters on readiness transitions #1598

Draft
HumanBean wants to merge 1 commit from HumanBean/Longshot:perf/queue-readiness-transitions into main
Owner

Type: perf · Risk: medium · Tested on: CachyOS Linux, Ryzen 9 7900X, 32 GiB, GeForce RTX 3090, NVIDIA 620.70.06

TL;DR

Wake message queue waiters when a queue becomes readable or writable. A fill-and-drain burst now needs two readiness broadcasts in the synthetic test.

Why and what changed

Condition-variable and multiwait notifications follow the empty-to-readable and full-to-writable transitions. The regression covers competing consumers, blocking operations and transitions that satisfy a previously false predicate.

Measurements

The worker-only integration arm completed the headless route at 61.58 FPS in Market and 75.12 FPS in Field in its first full-profile run. Other runs varied with system load. Those rates describe that run and do not establish a general FPS improvement.

Testing

The relevant host path ran on Linux through the Market and Field route with player movement verified. The synthetic regression checks wait and wake behavior. A long-running multiwait stress pass and Windows coverage remain outstanding.

Risks and remaining work

Incorrect notification conditions could strand a waiter under contention. Keep this branch separate from the GPU buffer changes so it can be stress-tested independently.

Checklist

  • The changed path was exercised locally in the game.
  • The branch is based on main at 876dc360. Measurements were taken before the cold-install packaging change on that revision.
  • Project source, tests and measurement summaries only. Third-party sources are untouched.
  • Codex assistance recorded in the commit.
  • Complete standalone regression and scheduler stress runs on the publication base.
**Type:** perf · **Risk:** medium · **Tested on:** CachyOS Linux, Ryzen 9 7900X, 32 GiB, GeForce RTX 3090, NVIDIA 620.70.06 ## TL;DR Wake message queue waiters when a queue becomes readable or writable. A fill-and-drain burst now needs two readiness broadcasts in the synthetic test. ## Why and what changed Condition-variable and multiwait notifications follow the empty-to-readable and full-to-writable transitions. The regression covers competing consumers, blocking operations and transitions that satisfy a previously false predicate. ## Measurements The worker-only integration arm completed the headless route at 61.58 FPS in Market and 75.12 FPS in Field in its first full-profile run. Other runs varied with system load. Those rates describe that run and do not establish a general FPS improvement. ## Testing The relevant host path ran on Linux through the Market and Field route with player movement verified. The synthetic regression checks wait and wake behavior. A long-running multiwait stress pass and Windows coverage remain outstanding. ## Risks and remaining work Incorrect notification conditions could strand a waiter under contention. Keep this branch separate from the GPU buffer changes so it can be stress-tested independently. ## Checklist - [x] The changed path was exercised locally in the game. - [x] The branch is based on main at `876dc360`. Measurements were taken before the cold-install packaging change on that revision. - [x] Project source, tests and measurement summaries only. Third-party sources are untouched. - [x] Codex assistance recorded in the commit. - [ ] Complete standalone regression and scheduler stress runs on the publication base.
Author
Owner

Tested in game: Linux, Ryzen 9 7900X, RTX 3090, NVIDIA 620.70.06. The worker-only implementation completed Market and Field movement in the headless game. The marked starting locations and movement were checked.

Tested in game: Linux, Ryzen 9 7900X, RTX 3090, NVIDIA 620.70.06. The worker-only implementation completed Market and Field movement in the headless game. The marked starting locations and movement were checked.
Member

Possibly related, for context:

  • issue #1505 Every event, queue and semaphore signal locks and wakes all waiters, and takes a global lock while any thread waits in a multi-wait — Open issue proposing notification/waiter optimization in this host sync path
  • PR #1573 WIP: test(host): nn_os_sync.c compiled whole and driven from host threads — Open threaded test work exercises the same nn_os_sync message-queue code
Possibly related, for context: - issue #1505 Every event, queue and semaphore signal locks and wakes all waiters, and takes a global lock while any thread waits in a multi-wait — Open issue proposing notification/waiter optimization in this host sync path - PR #1573 WIP: test(host): nn_os_sync.c compiled whole and driven from host threads — Open threaded test work exercises the same nn_os_sync message-queue code
HumanBean force-pushed perf/queue-readiness-transitions from 6c745d779c
Some checks failed
CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Has been skipped
CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Has been skipped
CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Has been skipped
Third-party parts / The same as upstream (pull_request) Has been skipped
CI / Conventional Commit title (pull_request) Successful in 12s
PR description / Description follows the template (pull_request_target) Successful in 16s
Guide screenshots / Guide images and local UI source review (pull_request) Failing after 19s
CI / Workflow lint (actionlint) (pull_request) Successful in 22s
Third-party parts / As their notes say (pull_request) Successful in 23s
Website / Website on Cloudflare (pull_request_target) Has been skipped
to 299dac6538
Some checks reported errors
CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Has been skipped
CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Has been skipped
CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Has been skipped
Third-party parts / The same as upstream (pull_request) Has been skipped
CI / Conventional Commit title (pull_request) Successful in 11s
CI / Workflow lint (actionlint) (pull_request) Successful in 19s
Guide screenshots / Guide images and local UI source review (pull_request) Failing after 23s
Third-party parts / As their notes say (pull_request) Successful in 28s
PR description / Description follows the template (pull_request_target) Successful in 10s
Website / Website on Cloudflare (pull_request_target) Has been skipped
beanstalk/approval approved by HumanBean (reviewed)
beanstalk/any 0/1 pass, 2 error
2026-11-18 15:14:18 +00:00
Compare
Author
Owner

Corrected author and committer attribution to HumanBean. Verified identical file contents before and after the metadata change.

Corrected author and committer attribution to HumanBean. Verified identical file contents before and after the metadata change.
Author
Owner

@Beanstalk

@Beanstalk
Member

Beanstalk test farm

Commit 299dac6 · tier reviewed · approved by HumanBean · dashboard

What will be tested · AI plan (gpt-6.1-terra)

The plan compares movement and repeated scene loads in Market and Hyrule Field against main, taking about 12 minutes per tester. It measures frame times and load times and compares scene images; crashes, stalled loads, large performance regressions, or visual differences fail. This gameplay check cannot establish that all queue and multiwait contention cases are safe.

  • Where: any (any PC, 1-2 reports).
  • market-field: from the repository's save kokiri-start, then teleport to Market, move the stick, take the picture market to compare with main, measure market for 20 s (after 5 s warm-up), teleport to Hyrule Field, take the picture field to compare with main, measure field for 20 s (after 5 s warm-up), 3 times: teleport to Market, teleport to Hyrule Field; 1 run on each side.
  • Measured: crashes, frame times, stutters and hitches, load times.
  • The PR fails if: it crashes; or a picture differs from main by more than 5%; or longest load is more than 30% above main; or frame time (p99) is more than 20% above main.
  • Time: about 12 min of game runs per tester (2 game starts), plus the two builds.

Done (started 06:18 UTC, last report 06:46 UTC). Comment @Beanstalk to run it again.

Target Reports Crash Frame p99 (ms) vs main VRAM peak (MB) vs main Result
any · 1: windows+nvidia+nvidia-proprietary+dgpu NVIDIA GeForce RTX 3050 Ti Laptop GPU, 621.24.0; 2: linux+amd+radv+dgpu AMD Radeon RX 7900 XTX (RADV NAVI31), 26.2.7 2/1 1 2 none — — error

Measurements · any

Verdict: could not be judged on any (2/1 report).

Hardware: 1: windows+nvidia+nvidia-proprietary+dgpu NVIDIA GeForce RTX 3050 Ti Laptop GPU, 621.24.0 AMD Ryzen 7 5800H with Radeon Graphics, 16 GB RAM; 2: linux+amd+radv+dgpu AMD Radeon RX 7900 XTX (RADV NAVI31), 26.2.7 AMD Ryzen 7 5800X3D 8-Core Processor, 64 GB RAM
Scene: market-field, from the repository's save kokiri-start; each cell's settings and switches as the table names them
Method: the Beanstalk test farm: main and this PR built and run on the same PC in one session, game starts alternating (ABBA); market-field: 1 game start a side, market 20 s after 5 s warm-up, field 20 s after 5 s warm-up; each number the median of the runs, then of 2 reports
How to reproduce: the farm's plan above (@Beanstalk on this PR runs it again)

Nothing was measured: no metric came back.

Flags and settings

None: the plan varied no setting, switch or build option.

Markdown for the PR description
## Measurements

**Verdict:** could not be judged on any (2/1 report).

Hardware: 1: windows+nvidia+nvidia-proprietary+dgpu `NVIDIA GeForce RTX 3050 Ti Laptop GPU, 621.24.0` `AMD Ryzen 7 5800H with Radeon Graphics, 16 GB RAM`; 2: linux+amd+radv+dgpu `AMD Radeon RX 7900 XTX (RADV NAVI31), 26.2.7` `AMD Ryzen 7 5800X3D 8-Core Processor, 64 GB RAM`  
Scene: market-field, from the repository's save kokiri-start; each cell's settings and switches as the table names them  
Method: the Beanstalk test farm: main and this PR built and run on the same PC in one session, game starts alternating (ABBA); market-field: 1 game start a side, market 20 s after 5 s warm-up, field 20 s after 5 s warm-up; each number the median of the runs, then of 2 reports  
How to reproduce: the farm's plan above (`@Beanstalk` on this PR runs it again)

Nothing was measured: no metric came back.

### Flags and settings

None: the plan varied no setting, switch or build option.
<!-- beanstalk:sticky --> ### Beanstalk test farm Commit `299dac6` · tier **reviewed** · approved by HumanBean · [dashboard](https://beanstalk.magicbean.dev/prs/1598) **What will be tested** · AI plan (`gpt-6.1-terra`) The plan compares movement and repeated scene loads in Market and Hyrule Field against main, taking about 12 minutes per tester. It measures frame times and load times and compares scene images; crashes, stalled loads, large performance regressions, or visual differences fail. This gameplay check cannot establish that all queue and multiwait contention cases are safe. - **Where:** any (any PC, 1-2 reports). - **market-field:** from the repository's save `kokiri-start`, then teleport to Market, move the stick, take the picture market to compare with main, measure market for 20 s (after 5 s warm-up), teleport to Hyrule Field, take the picture field to compare with main, measure field for 20 s (after 5 s warm-up), 3 times: teleport to Market, teleport to Hyrule Field; 1 run on each side. - **Measured:** crashes, frame times, stutters and hitches, load times. - **The PR fails if:** it crashes; or a picture differs from main by more than 5%; or longest load is more than 30% above main; or frame time (p99) is more than 20% above main. - **Time:** about 12 min of game runs per tester (2 game starts), plus the two builds. Done (started 06:18 UTC, last report 06:46 UTC). Comment `@Beanstalk` to run it again. | Target | Reports | Crash | Frame p99 (ms) vs main | VRAM peak (MB) vs main | Result | | --- | --- | --- | --- | --- | --- | | any · 1: windows+nvidia+nvidia-proprietary+dgpu `NVIDIA GeForce RTX 3050 Ti Laptop GPU, 621.24.0`; 2: linux+amd+radv+dgpu `AMD Radeon RX 7900 XTX (RADV NAVI31), 26.2.7` | 2/1 [1](https://beanstalk.magicbean.dev/reports/rp_30419de4c875ac1c) [2](https://beanstalk.magicbean.dev/reports/rp_04697ff9021c21f4) | none | — | — | error | #### Measurements · any **Verdict:** could not be judged on any (2/1 report). Hardware: 1: windows+nvidia+nvidia-proprietary+dgpu `NVIDIA GeForce RTX 3050 Ti Laptop GPU, 621.24.0` `AMD Ryzen 7 5800H with Radeon Graphics, 16 GB RAM`; 2: linux+amd+radv+dgpu `AMD Radeon RX 7900 XTX (RADV NAVI31), 26.2.7` `AMD Ryzen 7 5800X3D 8-Core Processor, 64 GB RAM` Scene: market-field, from the repository's save kokiri-start; each cell's settings and switches as the table names them Method: the Beanstalk test farm: main and this PR built and run on the same PC in one session, game starts alternating (ABBA); market-field: 1 game start a side, market 20 s after 5 s warm-up, field 20 s after 5 s warm-up; each number the median of the runs, then of 2 reports How to reproduce: the farm's plan above (`@Beanstalk` on this PR runs it again) Nothing was measured: no metric came back. ### Flags and settings None: the plan varied no setting, switch or build option. <details><summary>Markdown for the PR description</summary> ```markdown ## Measurements **Verdict:** could not be judged on any (2/1 report). Hardware: 1: windows+nvidia+nvidia-proprietary+dgpu `NVIDIA GeForce RTX 3050 Ti Laptop GPU, 621.24.0` `AMD Ryzen 7 5800H with Radeon Graphics, 16 GB RAM`; 2: linux+amd+radv+dgpu `AMD Radeon RX 7900 XTX (RADV NAVI31), 26.2.7` `AMD Ryzen 7 5800X3D 8-Core Processor, 64 GB RAM` Scene: market-field, from the repository's save kokiri-start; each cell's settings and switches as the table names them Method: the Beanstalk test farm: main and this PR built and run on the same PC in one session, game starts alternating (ABBA); market-field: 1 game start a side, market 20 s after 5 s warm-up, field 20 s after 5 s warm-up; each number the median of the runs, then of 2 reports How to reproduce: the farm's plan above (`@Beanstalk` on this PR runs it again) Nothing was measured: no metric came back. ### Flags and settings None: the plan varied no setting, switch or build option. ``` </details>
Some checks reported errors
CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Has been skipped
CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Has been skipped
CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Has been skipped
Third-party parts / The same as upstream (pull_request) Has been skipped
CI / Conventional Commit title (pull_request) Successful in 11s
CI / Workflow lint (actionlint) (pull_request) Successful in 19s
Guide screenshots / Guide images and local UI source review (pull_request) Failing after 23s
Third-party parts / As their notes say (pull_request) Successful in 28s
PR description / Description follows the template (pull_request_target) Successful in 10s
Website / Website on Cloudflare (pull_request_target) Has been skipped
beanstalk/approval approved by HumanBean (reviewed)
beanstalk/any 0/1 pass, 2 error
This pull request has changes conflicting with the target branch.
  • host/nn_os_sync.c
View command line instructions

Manual merge helper

Use this merge commit message when completing the merge manually.

Checkout

From your project repository, check out a new branch and test the changes.
git fetch -u 'perf/queue-readiness-transitions:HumanBean-perf/queue-readiness-transitions'
git switch 'HumanBean-perf/queue-readiness-transitions'
Sign in to join this conversation.
No reviewers
No milestone
No project
No assignees
3 participants
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
MagicBeanDev/Longshot!1598
No description provided.