MagicBeanDev/Longshot
MagicBeanDev/Longshot
Watch
75
96
Fork
You've already forked Longshot
 99
Code Issues 229 Pull requests 255 Projects Releases 9 Packages Wiki Activity Actions 836

fix(vcores): a thread starved of its core gets it after 1 s, not an escape after 2 s (#262) #594

Open
mooki wants to merge 3 commits from mooki/Longshot:fix/vcore-escape-262b into main
pull from: mooki/Longshot:fix/vcore-escape-262b
merge into: MagicBeanDev:main
Conversation 3 Commits 3 Files changed 5 +123 -17
mooki commented 2026-11-14 22:01:56 +00:00
Member

Type: fix · Risk: medium · Tested on: Linux, RTX 4090, Core i9-13900K (windowless Kokiri Forest save loads; the game-free fixture with g++ and clang++); Windows only a syntax check of the fixture · Needs: Windows with an AMD or NVIDIA card (T-9R2R)

Why

While a save loads, some of the game's worker threads wait the full 2 seconds for the virtual core they are pinned to and then run without one, and the log says ran without a core after 2000 ms (#262). The core is not stuck: better threads take it tens of thousands of times a second and the waiting thread cannot ask any of them to give it up. This lets such a thread have the core after 1 second instead.

What is measured: the cause, by counting every take of the core during the waits (26 waits of 1 s or more in 12 starts, 24 of them this pattern and none a single long hold), and the fix, by 10 starts with it against 10 starts of the same build with the old rule (0 of 10 starts with an escape against 4 of 10; no core wait past 1.002 s against waits of 2.0 s). Not measured: Windows, any other GPU or CPU, play after loading, and how much work a starved thread gets done in its turn.

How to reproduce

Seen on: Linux, Core i9-13900K, 96 GB RAM, RTX 4090 24 GB with NVIDIA 615.78.08, Arch Linux 7.2.15, no window. The same escape line is in the lines posted on the issue (22 of 22), in a player's log of a 16 GB Linux PC with an RX 9070 XT (#91), and in rows in docs/tested-configurations.md from a Windows PC with a Ryzen 7 5700X3D, which list 3 and 4 core escapes in runs that loaded a save.
Settings and switches: Kokiri Forest (child), a copied save loaded from the title to play, 1920x1080, Custom preset, render 100%, the game's own upscaler, frame generation off, no frame cap, shaders prepared; the scheduler as on main, which is HOS_VCORE_STARVE_US=0.
Steps: copy the game folder, put the change's host/hos_thread.c on the commit under test, build, load a Kokiri Forest save from the title, quit once play begins, and count ran without a core in longshot_log.txt.
Expected: no wait of 2 seconds for a core and no ran without a core line: the 2 s escape is meant only as a deadlock breaker, and the core is not stuck.
Actual: while the save loads, some of the game's worker threads wait the full 2 seconds for the virtual core they are pinned to and run without one, and the log says vcores: bake::ModuleMgr (priority 28) ran without a core after 2000.2 ms: core 1: BaseProcCreateAndDeleteThread (23);. With the old rule 4 of 10 starts had an escape line (7 lines); main's own binary, ten starts earlier the same day, had 3 of 10 starts with an escape and 8 escape lines.

What changed

A thread that has waited HOS_VCORE_STARVE_US (1 s by default) for its virtual core is marked starved: it goes before every other waiter at the next release of a core it may use and asks the worst holder of those cores to give the core up, whatever the holder's priority. Below the starve time the order is unchanged, the 2 s escape and its two switches are unchanged, and HOS_VCORE_STARVE_US=0 turns the rule off, which is main's behaviour. The 5-second vcores: line now counts the grants to starved waiters, and tests/test_vcore_escape.py gets new cases in the harness that compiles the real virtual-core block.

  • host/hos_thread.c, vc_acquire: a waiter that has waited HOS_VCORE_STARVE_US (1 s by default) is marked starved. A starved waiter asks the worst holder of the cores it may use to give the core up, whatever the holder's priority, the way a better waiter already asks a worse holder.
  • vc_better (used by vc_grant): a starved waiter goes before every other waiter at the next release of a core it may use, the one waiting longest first among starved ones. Below the starve time the order is unchanged: lower priority value first, then the one waiting longest.
  • vc_yield: a holder asked to give up its core does so for a starved waiter, as it does for a better one. vc_take and the escape path clear the mark.
  • The 5-second vcores: line counts the grants to starved waiters (starved grants, before the escapes).
  • docs/development.md: the new switch HOS_VCORE_STARVE_US.
  • tests/test_vcore_escape.py: new cases in the harness that compiles the real virtual-core block (details under Testing). The existing cases are unchanged and pass. The fixture now defines _CRT_SECURE_NO_WARNINGS before its first include, because the block reads its switches with getenv and the Windows-target syntax check refuses the fixture without it (main's own fixture has the same fault).
  • docs/tested/2026-11-14-vcore-starve-mooki-run01.md and run02.md: the two arms of the comparison below, one row each.
  • Unchanged: the 2 s escape and its two switches (HOS_VCORE_ESCAPE_US, HOS_VCORE_ESCAPE_EQ_US), the time slice (HOS_VCORE_SLICE_US), HOS_VCORES, what happens after an escape, and every thread's priority and mask.

The chain of causes

  1. A guest thread holds one of the game's six virtual cores while it runs and gives it up where Horizon would switch threads (hos_cv_sleep, hos_sleep_ns, vc_yield at a checkpoint). A free core goes to the best waiter allowed on it (vc_grant).
  2. A waiter asks a holder to give the core up only when the holder is worse than itself, or as good after a 2 ms slice (vc_acquire). A worker pinned to one core with a priority of 25 to 29 (PreActorModelCompose, BakeProbe, bake::ModuleMgr, tera::FileResourceMgr, aglTextureDataMgr::Compaction and PlayReporter are the ones seen) can ask nobody on that core.
  3. During the load, that core changes hands between better threads at every block. In 12 starts of a diagnostic build (the same scheduler, plus a log of every take during a long wait), the waiter's single core was taken 42,593 to 53,473 times during each of 15 escapes (21,000 to 27,000 times a second), mostly by threads of priority 16 (21,214 to 22,957 takes) and 23 (16,378 to 20,220), and the holder at the escape had held it for 24 to 168 microseconds. No thread kept a core for the wait. In 11 waits of 1.22 to 1.97 s that ended by themselves, the granted core was taken 25,661 to 40,148 times; none was a take by a worse thread. The release always goes to a better waiter, so the pinned thread is never first.
  4. After 2 s vc_acquire lets the waiter run without a core, beside the holder, so two threads share the per-core state of that core. The escape is meant only as a deadlock breaker (see the comment above vc_lock); here it fires on a wait that is not a deadlock, and it is what the log line reports.

The holder named at an escape is better than the waiter in every line anyone has posted: 22 of 22 on the issue, 44 of 44 in a player's log of a 16 GB Linux PC with an RX 9070 XT (#91, the attachment longshot_log.txt, which also has vcores: 1352 takes, 866 waits (mean 64434 us, longest 2001.0 ms), 184 kicks, 172 yields, 13 escapes in 5 s; that log is of a machine under memory pressure and 1.7 frames a second, so it shows the same line and the same kind of holder, not the same count), and 8 of 8 in the first ten plain starts here. Rows in docs/tested-configurations.md from a Windows PC with a Ryzen 7 5700X3D also list 3 and 4 core escapes in runs that loaded a save.

Reviewer notes:

  • Start at vc_better, the starved branch of vc_acquire and vc_yield, then the cases at the end of tests/test_vcore_escape.py (starvation()).
  • Left alone on purpose: a protected time slice for a starved thread. After its turn a starved thread asks again from nothing and waits another second; the start-to-play times did not move, so it was not added.
  • Overlaps, from the open pull requests' diffs: #327 (mask 0 and the guest's core cache) changes the struct line int vc_held, vc_last, vc_queued, vc_escaped, vc_allowed, vc_free_for and the harness lines of tests/test_vcore_escape.py that this also edits: textual conflicts, and a mask-0 thread that may use every core is simply a starved waiter on all of them. #249 (HOS_VCORE_FIRST, work in progress) rewrites vc_better and adds vc_prio: a textual conflict there, and a first thread would rank below a starved waiter after 1 s. The pull requests for macOS (#288, #477, #490) have no hunk in the virtual-core block. A trial merge of this branch with main as of today (82886e89b) has no conflicts.

Measurements

Verdict: the escape is gone and no core wait lasted past 1.002 s with the change: 0 of 10 starts had a ran without a core line against 4 of 10 with the old rule (HOS_VCORE_STARVE_US=0 on the same binary), and the old rule had waits of 2.0 s in 4 of 10 starts. The count alone is weak (Fisher's exact test, p = 0.087 two-sided); the cap is the stronger evidence. No frame time or loading time is claimed (start to play did not move: median 43.1 s against 43.2 s, in ordinary runs on a shared PC). Not measured: Windows, any other GPU or CPU, and play after loading.

Branch state: on main 82886e89b970daf89a07535eaec083c6018e32ae; head 0e5fcecd5c66dd7733b42a051c3a6553d9c47465; whole game-free suite on this head: 552 passed, 9 failed, 16 skipped (main 82886e89b970daf89a07535eaec083c6018e32ae: 552 passed, 9 failed, 16 skipped). The 9 failing tests fail on main the same way: tests/test_appimage_player_build.py, tests/test_cpm_discovery.py, tests/test_env_on.py, tests/test_lift_cleanup_error.py, tests/test_linux_preflight.py, tests/test_make_game_cache.py, tests/test_readback_late.py, tests/test_vram_budget_row.py, tests/test_walk_key.py. The commits are unchanged by rebases apart from the results table.

main: 26a2eab003 (2026-11-14)
this PR: fix/vcore-escape-262-m543846eb @ 53a80e735
Hardware: Core i9-13900K, 96 GB RAM, RTX 4090 24 GB with NVIDIA 615.78.08, Arch Linux 7.2.15, no window
Scene: Kokiri Forest (child), a copied save loaded from the title to play, 1920x1080, Custom preset, render 100%, the game's own upscaler, frame generation off, no frame cap, shaders prepared
Method: 10 starts of each arm, interleaved start by start (three starts to one hold), 5 s standing after play began; the counts come from longshot_log.txt (ran without a core lines, the largest longest in the 5 s vcores: lines, starved grants). Ordinary runs on a shared PC with other work around (load 7 to 49 at the starts), so no frame time or loading time is claimed
How to reproduce: copy the game folder, put the change's host/hos_thread.c on the commit under test, build, load a Kokiri Forest save from the title, quit once play begins, and count ran without a core in longshot_log.txt; the second arm is the same folder with HOS_VCORE_STARVE_US=0
Baseline: there is no main column. The before column is the same build with HOS_VCORE_STARVE_US=0, which makes the scheduler decide as main does. It was used because the two arms could be interleaved start by start on one binary. The two builds are this change applied to main 26a2eab00 (not rebuilt on 53a80e735: host/hos_thread.c's scheduler block is the same on both bases). Main's own binary, ten starts earlier the same day: 3 of 10 starts with an escape, 8 escape lines.

Before, HOS_VCORE_STARVE_US=0: this PR @ 26a2eab00 After: this PR @ 53a80e735
starts with a ran without a core line 4 of 10 0 of 10
escape lines 7 (4, 1, 1, 1) 0
largest core wait in the 5 s lines, per start (min, median, max) 515, 1385, 2001 ms 532, 1000, 1002 ms
grants to a starved waiter 0 19 in 7 starts
start to play (ordinary runs, no claim) 40.6 to 45.1 s, median 43.2 41.0 to 45.8 s, median 43.1 
old rule, start 06:
[hos bake::ModuleMgr] vcores: bake::ModuleMgr (priority 28) ran without a core after 2000.2 ms: core 1: BaseProcCreateAndDeleteThread (23);
[hos BakeProbe] vcores: BakeProbe (priority 28) ran without a core after 2000.8 ms: core 1: BaseProcCreateAndDeleteThread (23);
[hos aglTextureDataMgr::Compaction] vcores: aglTextureDataMgr::Compaction (priority 25) ran without a core after 2000.2 ms: core 1: BaseProcCreateAndDeleteThread (23);
[hos PreActorModelCompose] vcores: PreActorModelCompose (priority 26) ran without a core after 2000.8 ms: core 1: BaseProcCreateAndDeleteThread (23);
this PR, start 06 (two of its three 5 s lines with grants):
[hos ModuleSystemWorker1] vcores: 390749 takes, 173795 waits (mean 71 us, longest 1000.4 ms), 57449 kicks, 54966 yields, 3 starved grants, 0 escapes in 5 s
[hos PreActorModelCompose] vcores: 486308 takes, 293613 waits (mean 105 us, longest 1001.0 ms), 124459 kicks, 116137 yields, 7 starved grants, 0 escapes in 5 s
test on main's host/hos_thread.c: FAIL: turn taking: the starved thread gets the core, no escape
test on this PR: ok

Fisher's exact test on starts with an escape (0 of 10 against 4 of 10) gives p = 0.087 two-sided, so that count alone is weak. The cap is the stronger evidence: every wait of every start of this PR ended by 1.002 s, and the old rule had waits of 2.0 s in 4 of 10 starts. The diagnostic build's counts above say why. Every run exited 0 with no fatal line and no kernel line. The first diagnostic build had 0 escapes in 12 starts and I cannot say why (its second version, with the same scheduler, had 15 in 5 of 12).

Screenshots

Nothing on screen. For the record, the last frame of one start of each arm (this change, 10 grants and no escape; old rule, 4 escapes): the same Kokiri Forest view, nothing different:

Kokiri Forest, the last frame of start 06 of each arm: 1. this change, 10 starved grants, no escape; 2. the old rule, 4 escapes

Flags and settings

HOS_VCORE_STARVE_US=<microseconds> is new, documented in docs/development.md. 0 turns the rule off, which is main's behaviour.

Flag What it is Before this PR After this PR Effect Helps / doesn't help / untested Recommendation
HOS_VCORE_STARVE_US New switch Absent (new) 1000000 (1 s) A thread that has waited this long for a virtual core goes first at the next release of a core it may use and asks the worst holder to give the core up Helps: Linux, NVIDIA RTX 4090, the load of a Kokiri Forest save. Doesn't help: A holder that never reaches a checkpoint (the 2 s escape still ends that wait). Untested: Windows, AMD, Intel, Steam Deck, any CPU but a Core i9-13900K. On by default (0 for the old rule, for comparisons)

Testing

Tested: the new cases in tests/test_vcore_escape.py both ways, with g++ and with clang++: on main's host/hos_thread.c four checks fail (the starved thread escapes after 2 s in both fixtures, so it neither gets the core nor takes the other one), on this branch the file prints ok; the same fixture with HOS_VCORE_STARVE_US=0 still escapes, which the test requires, and before the starve time the better waiter still goes first. Main's own test file passes unchanged against the new code. The whole game-free suite once on this head: 554 passed, 1 failed, 16 skipped in 153 s; the one failure, tests/test_walk_key.py (a link error, undefined reference to hos_oot_autoload_input), fails the same way on main 543846ebb and is unrelated to this change. The Windows-target syntax check of the changed fixture (clang from llvm-mingw 20261104) passes; the fixture was not run on Windows. In the game: 12 starts of a diagnostic build, 10 plain starts of main, and the 20 interleaved starts above, windowless on an RTX 4090 with NVIDIA 615.78.08 on Arch Linux. The last frame of one start of each arm shows the same Kokiri Forest view.

Built for Windows and Linux at the current head, not run: At 0e5fcecd5c66dd7733b42a051c3a6553d9c47465, the Windows x86-64 Vulkan renderer DLL and all 50 selected host translation units were cross-built on Linux with llvm-mingw 20261104 (Clang 23.1.5), using the project's production flags and pinned dependencies (renderer: 35 newly compiled / 170 reused; host: 50 newly compiled / 0 reused; lifted C: 0 newly compiled / 80 reused; all Windows units: 85 newly compiled / 564 reused from matching object-cache entries), the renderer DLL linked and the full game executable linked; nothing was run on Windows or under Wine, and this is neither the project's hosted builder run nor a Windows UTF-8 run-time path test.

The exact same head 0e5fcecd5c66dd7733b42a051c3a6553d9c47465 also compiled and linked its full native Linux executable and renderer with Clang 23.1.3 through the documented resume path (86 newly compiled units, 405 matching Linux-cache units, source stamp 781eb374), using the verified target-neutral C from lift 414f56e226d2be90a288ff2eb2fa172f7d4e6c93 rather than carrying the patch onto another commit; neither Linux binary was run.

CPU target (this revision builds the game for the build PC's own CPU, #524; the two builds differ on purpose): the Windows cross-build keeps the builder's pinned -march=x86-64-v3 -ffp-contract=off (488 of its 649 compile commands carry it; the other 161 carry no -march flag; none carries -march=native, -mtune or -DLONGSHOT_NATIVE_CPU), so the Windows binaries are not built for this PC's CPU. The native Linux build used the revision's default -march=native, which clang resolved on this PC (13th Gen Intel(R) Core(TM) i9-13900K) to raptorlake_raptorlake_5fe3e3650f (build log line: build: the game is built for this PC's CPU (raptorlake_raptorlake_5fe3e3650f)), so the Linux binaries are built for that CPU's instruction set and tuning; neither binary was run.

Renderer profile (this revision has the committed renderer play profile, #569, assets/profiles/longshot_vk.profdata.xz, 159448 bytes, git blob 01238f45954c): both runners chose the profile with the revision's own make_game.vk_profile_for, as the project's build does, and passed it as RX_VK_PGO. Windows renderer (llvm-mingw Clang): built with it: 43 of its 649 compile commands (the renderer's own sources, not Eden's recompiler or sirit) carry -fprofile-instr-use=<the unpacked profile> and -DLONGSHOT_VK_PROFILE_SHA1=d965e746b947f357, all exit 0; build log line: the Vulkan back end: built with the profile longshot_vk.shipped.profdata. Linux renderer (native Clang): built with it (build log line: the Vulkan back end: built with the profile longshot_vk.shipped.profdata). The game's own lifted code is built with the older longshot_pieces.prof profile as before; neither binary was run.

Not tested: Windows (rows in docs/tested-configurations.md from a Windows PC list core escapes too); AMD and Intel GPUs and other CPUs; a handheld; play after the save has loaded (the runs stop 5 s after play begins); how far a starved thread gets in its turn (a turn lasts until the holders ask for the core back, as for any worse thread); that nothing else slows down (only start to play was looked at, and it did not move); the effect on a real frame time, which needs the paired method and an exclusive hold.

Must be tested:

  • Windows with an AMD or NVIDIA card: load a save in Kokiri Forest ten times and search longshot_log.txt for ran without a core, expecting none with the default and some with HOS_VCORE_STARVE_US=0 (test request T-9R2R; by hand)

Risks

What could break: A starved thread now runs in a core that a better thread was using, so a better thread can wait for a starved thread to reach its next checkpoint (a few microseconds to a few milliseconds, once per second per starved waiter, and only after a wait of more than 1 s). The 2 s escape is unchanged and still ends a wait on a holder that never reaches a checkpoint.

Turning it off: HOS_VCORE_STARVE_US=0 turns the rule off; reverting the commit is enough.

Compatibility:

  • Saves, longshot_options.ini, the shader and pipeline caches and the vk ABI: the text says nothing; the diff changes only host/hos_thread.c, docs/development.md, the new docs/tested/ rows and tests/test_vcore_escape.py, so it adds no ini key (the new switch is an environment switch), touches no save or cache code and nothing under gpu/vk/.
  • What a player notices: a thread that waits more than 1 s for its virtual core now gets it instead of running without one after 2 s, and the 5-second vcores: line in longshot_log.txt gains a starved grants count; the 2 s escape and its switches are unchanged.
  • The builder's tools: the text says nothing; the diff touches none.

Issues

Fixes #262

Checklist

Everyone

  • One topic.
  • The title is a Conventional Commit (type(scope): description) of the status line's type.
  • Why gives the reason plainly and links the issues that fit (the tracker searched).
  • Measurements has a verdict, or one line saying why there is nothing to measure.
  • Testing names what was tested, what wasn't, and what must still be tested, where and how.
  • Rebased on current main.
  • python -I -B tests/run_all.py passes (any failure that also fails on main is named).
  • No game files, dumps, build output or generated files.
  • third_party/ untouched (a new folder with a PROVENANCE.md instead).
  • The license is OK: GPL-3.0-or-later, nothing incompatible and nothing decompiled.

When it applies

  • AI assistance recorded with Assisted-by: / Co-Authored-By: trailers.
  • Numbers from the same PC, scene and settings, at least 2 interleaved runs each, the median reported.
  • Switches documented (docs/development.md; the README for a menu row) and shipped at their recommended value.
  • A file in docs/tested/ for each new measured run (format in docs/tested-configurations.md; no appended row or assembled view).
  • A game-free test added for the fix, failing on main and passing here.
  • A check under tests/ that reads changed source updated in this PR, not weakened.
  • Host code builds on both Windows (llvm-mingw) and Linux, with UTF-8 paths. Built and linked for both at 0e5fcecd5c66dd7733b42a051c3a6553d9c47465 (under Testing); nothing was run on Windows at this head, so UTF-8 path behavior at run time is not claimed.
  • Docs updated.

New on this forge; not opened on GitHub.

**Type:** fix · **Risk:** medium · **Tested on:** Linux, RTX 4090, Core i9-13900K (windowless Kokiri Forest save loads; the game-free fixture with g++ and clang++); Windows only a syntax check of the fixture · **Needs:** Windows with an AMD or NVIDIA card (T-9R2R) ## Why While a save loads, some of the game's worker threads wait the full 2 seconds for the virtual core they are pinned to and then run without one, and the log says `ran without a core after 2000 ms` (https://git.magicbean.dev/MagicBeanDev/Longshot/issues/262). The core is not stuck: better threads take it tens of thousands of times a second and the waiting thread cannot ask any of them to give it up. This lets such a thread have the core after 1 second instead. **What is measured:** the cause, by counting every take of the core during the waits (26 waits of 1 s or more in 12 starts, 24 of them this pattern and none a single long hold), and the fix, by 10 starts with it against 10 starts of the same build with the old rule (0 of 10 starts with an escape against 4 of 10; no core wait past 1.002 s against waits of 2.0 s). **Not measured:** Windows, any other GPU or CPU, play after loading, and how much work a starved thread gets done in its turn. ## How to reproduce **Seen on:** Linux, Core i9-13900K, 96 GB RAM, RTX 4090 24 GB with NVIDIA 615.78.08, Arch Linux 7.2.15, no window. The same escape line is in the lines posted on the issue (22 of 22), in a player's log of a 16 GB Linux PC with an RX 9070 XT (https://git.magicbean.dev/MagicBeanDev/Longshot/issues/91), and in rows in `docs/tested-configurations.md` from a Windows PC with a Ryzen 7 5700X3D, which list 3 and 4 core escapes in runs that loaded a save. **Settings and switches:** Kokiri Forest (child), a copied save loaded from the title to play, 1920x1080, Custom preset, render 100%, the game's own upscaler, frame generation off, no frame cap, shaders prepared; the scheduler as on main, which is `HOS_VCORE_STARVE_US=0`. **Steps:** copy the game folder, put the change's `host/hos_thread.c` on the commit under test, build, load a Kokiri Forest save from the title, quit once play begins, and count `ran without a core` in `longshot_log.txt`. **Expected:** no wait of 2 seconds for a core and no `ran without a core` line: the 2 s escape is meant only as a deadlock breaker, and the core is not stuck. **Actual:** while the save loads, some of the game's worker threads wait the full 2 seconds for the virtual core they are pinned to and run without one, and the log says `vcores: bake::ModuleMgr (priority 28) ran without a core after 2000.2 ms: core 1: BaseProcCreateAndDeleteThread (23);`. With the old rule 4 of 10 starts had an escape line (7 lines); main's own binary, ten starts earlier the same day, had 3 of 10 starts with an escape and 8 escape lines. ## What changed A thread that has waited `HOS_VCORE_STARVE_US` (1 s by default) for its virtual core is marked starved: it goes before every other waiter at the next release of a core it may use and asks the worst holder of those cores to give the core up, whatever the holder's priority. Below the starve time the order is unchanged, the 2 s escape and its two switches are unchanged, and `HOS_VCORE_STARVE_US=0` turns the rule off, which is main's behaviour. The 5-second `vcores:` line now counts the grants to starved waiters, and `tests/test_vcore_escape.py` gets new cases in the harness that compiles the real virtual-core block. - `host/hos_thread.c`, `vc_acquire`: a waiter that has waited `HOS_VCORE_STARVE_US` (1 s by default) is marked starved. A starved waiter asks the worst holder of the cores it may use to give the core up, whatever the holder's priority, the way a better waiter already asks a worse holder. - `vc_better` (used by `vc_grant`): a starved waiter goes before every other waiter at the next release of a core it may use, the one waiting longest first among starved ones. Below the starve time the order is unchanged: lower priority value first, then the one waiting longest. - `vc_yield`: a holder asked to give up its core does so for a starved waiter, as it does for a better one. `vc_take` and the escape path clear the mark. - The 5-second `vcores:` line counts the grants to starved waiters (`starved grants`, before the escapes). - `docs/development.md`: the new switch `HOS_VCORE_STARVE_US`. - `tests/test_vcore_escape.py`: new cases in the harness that compiles the real virtual-core block (details under Testing). The existing cases are unchanged and pass. The fixture now defines `_CRT_SECURE_NO_WARNINGS` before its first include, because the block reads its switches with `getenv` and the Windows-target syntax check refuses the fixture without it (main's own fixture has the same fault). - `docs/tested/2026-11-14-vcore-starve-mooki-run01.md` and `run02.md`: the two arms of the comparison below, one row each. - Unchanged: the 2 s escape and its two switches (`HOS_VCORE_ESCAPE_US`, `HOS_VCORE_ESCAPE_EQ_US`), the time slice (`HOS_VCORE_SLICE_US`), `HOS_VCORES`, what happens after an escape, and every thread's priority and mask. ### The chain of causes 1. A guest thread holds one of the game's six virtual cores while it runs and gives it up where Horizon would switch threads (`hos_cv_sleep`, `hos_sleep_ns`, `vc_yield` at a checkpoint). A free core goes to the best waiter allowed on it (`vc_grant`). 2. A waiter asks a holder to give the core up only when the holder is worse than itself, or as good after a 2 ms slice (`vc_acquire`). A worker pinned to one core with a priority of 25 to 29 (PreActorModelCompose, BakeProbe, bake::ModuleMgr, tera::FileResourceMgr, aglTextureDataMgr::Compaction and PlayReporter are the ones seen) can ask nobody on that core. 3. During the load, that core changes hands between better threads at every block. In 12 starts of a diagnostic build (the same scheduler, plus a log of every take during a long wait), the waiter's single core was taken 42,593 to 53,473 times during each of 15 escapes (21,000 to 27,000 times a second), mostly by threads of priority 16 (21,214 to 22,957 takes) and 23 (16,378 to 20,220), and the holder at the escape had held it for 24 to 168 microseconds. No thread kept a core for the wait. In 11 waits of 1.22 to 1.97 s that ended by themselves, the granted core was taken 25,661 to 40,148 times; none was a take by a worse thread. The release always goes to a better waiter, so the pinned thread is never first. 4. After 2 s `vc_acquire` lets the waiter run without a core, beside the holder, so two threads share the per-core state of that core. The escape is meant only as a deadlock breaker (see the comment above `vc_lock`); here it fires on a wait that is not a deadlock, and it is what the log line reports. The holder named at an escape is better than the waiter in every line anyone has posted: 22 of 22 on the issue, 44 of 44 in a player's log of a 16 GB Linux PC with an RX 9070 XT (https://git.magicbean.dev/MagicBeanDev/Longshot/issues/91, the attachment `longshot_log.txt`, which also has `vcores: 1352 takes, 866 waits (mean 64434 us, longest 2001.0 ms), 184 kicks, 172 yields, 13 escapes in 5 s`; that log is of a machine under memory pressure and 1.7 frames a second, so it shows the same line and the same kind of holder, not the same count), and 8 of 8 in the first ten plain starts here. Rows in `docs/tested-configurations.md` from a Windows PC with a Ryzen 7 5700X3D also list 3 and 4 core escapes in runs that loaded a save. **Reviewer notes:** - Start at `vc_better`, the starved branch of `vc_acquire` and `vc_yield`, then the cases at the end of `tests/test_vcore_escape.py` (`starvation()`). - Left alone on purpose: a protected time slice for a starved thread. After its turn a starved thread asks again from nothing and waits another second; the start-to-play times did not move, so it was not added. - Overlaps, from the open pull requests' diffs: https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/327 (mask 0 and the guest's core cache) changes the struct line `int vc_held, vc_last, vc_queued, vc_escaped`, `vc_allowed`, `vc_free_for` and the harness lines of `tests/test_vcore_escape.py` that this also edits: textual conflicts, and a mask-0 thread that may use every core is simply a starved waiter on all of them. https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/249 (`HOS_VCORE_FIRST`, work in progress) rewrites `vc_better` and adds `vc_prio`: a textual conflict there, and a first thread would rank below a starved waiter after 1 s. The pull requests for macOS (https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/288, https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/477, https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/490) have no hunk in the virtual-core block. A trial merge of this branch with main as of today (82886e89b) has no conflicts. ## Measurements **Verdict:** the escape is gone and no core wait lasted past 1.002 s with the change: 0 of 10 starts had a `ran without a core` line against 4 of 10 with the old rule (`HOS_VCORE_STARVE_US=0` on the same binary), and the old rule had waits of 2.0 s in 4 of 10 starts. The count alone is weak (Fisher's exact test, p = 0.087 two-sided); the cap is the stronger evidence. No frame time or loading time is claimed (start to play did not move: median 43.1 s against 43.2 s, in ordinary runs on a shared PC). Not measured: Windows, any other GPU or CPU, and play after loading. <!-- branch-state --> Branch state: on main `82886e89b970daf89a07535eaec083c6018e32ae`; head `0e5fcecd5c66dd7733b42a051c3a6553d9c47465`; whole game-free suite on this head: `552 passed, 9 failed, 16 skipped` (main `82886e89b970daf89a07535eaec083c6018e32ae`: `552 passed, 9 failed, 16 skipped`). The 9 failing tests fail on main the same way: `tests/test_appimage_player_build.py`, `tests/test_cpm_discovery.py`, `tests/test_env_on.py`, `tests/test_lift_cleanup_error.py`, `tests/test_linux_preflight.py`, `tests/test_make_game_cache.py`, `tests/test_readback_late.py`, `tests/test_vram_budget_row.py`, `tests/test_walk_key.py`. The commits are unchanged by rebases apart from the results table. <!-- /branch-state --> main: 26a2eab003eeb18db69bcc41e1346dc4aec64b35 (2026-11-14) this PR: fix/vcore-escape-262-m543846eb @ 53a80e735 Hardware: Core i9-13900K, 96 GB RAM, RTX 4090 24 GB with NVIDIA 615.78.08, Arch Linux 7.2.15, no window Scene: Kokiri Forest (child), a copied save loaded from the title to play, 1920x1080, Custom preset, render 100%, the game's own upscaler, frame generation off, no frame cap, shaders prepared Method: 10 starts of each arm, interleaved start by start (three starts to one hold), 5 s standing after play began; the counts come from `longshot_log.txt` (`ran without a core` lines, the largest `longest` in the 5 s `vcores:` lines, `starved grants`). Ordinary runs on a shared PC with other work around (load 7 to 49 at the starts), so no frame time or loading time is claimed How to reproduce: copy the game folder, put the change's `host/hos_thread.c` on the commit under test, build, load a Kokiri Forest save from the title, quit once play begins, and count `ran without a core` in `longshot_log.txt`; the second arm is the same folder with `HOS_VCORE_STARVE_US=0` Baseline: there is no main column. The before column is the same build with `HOS_VCORE_STARVE_US=0`, which makes the scheduler decide as main does. It was used because the two arms could be interleaved start by start on one binary. The two builds are this change applied to main 26a2eab00 (not rebuilt on 53a80e735: `host/hos_thread.c`'s scheduler block is the same on both bases). Main's own binary, ten starts earlier the same day: 3 of 10 starts with an escape, 8 escape lines. | | Before, HOS_VCORE_STARVE_US=0: this PR @ 26a2eab00 | After: this PR @ 53a80e735 | |---|---|---| | starts with a `ran without a core` line | 4 of 10 | 0 of 10 | | escape lines | 7 (4, 1, 1, 1) | 0 | | largest core wait in the 5 s lines, per start (min, median, max) | 515, 1385, 2001 ms | 532, 1000, 1002 ms | | grants to a starved waiter | 0 | 19 in 7 starts | | start to play (ordinary runs, no claim) | 40.6 to 45.1 s, median 43.2 | 41.0 to 45.8 s, median 43.1 | ``` old rule, start 06: [hos bake::ModuleMgr] vcores: bake::ModuleMgr (priority 28) ran without a core after 2000.2 ms: core 1: BaseProcCreateAndDeleteThread (23); [hos BakeProbe] vcores: BakeProbe (priority 28) ran without a core after 2000.8 ms: core 1: BaseProcCreateAndDeleteThread (23); [hos aglTextureDataMgr::Compaction] vcores: aglTextureDataMgr::Compaction (priority 25) ran without a core after 2000.2 ms: core 1: BaseProcCreateAndDeleteThread (23); [hos PreActorModelCompose] vcores: PreActorModelCompose (priority 26) ran without a core after 2000.8 ms: core 1: BaseProcCreateAndDeleteThread (23); this PR, start 06 (two of its three 5 s lines with grants): [hos ModuleSystemWorker1] vcores: 390749 takes, 173795 waits (mean 71 us, longest 1000.4 ms), 57449 kicks, 54966 yields, 3 starved grants, 0 escapes in 5 s [hos PreActorModelCompose] vcores: 486308 takes, 293613 waits (mean 105 us, longest 1001.0 ms), 124459 kicks, 116137 yields, 7 starved grants, 0 escapes in 5 s test on main's host/hos_thread.c: FAIL: turn taking: the starved thread gets the core, no escape test on this PR: ok ``` Fisher's exact test on starts with an escape (0 of 10 against 4 of 10) gives p = 0.087 two-sided, so that count alone is weak. The cap is the stronger evidence: every wait of every start of this PR ended by 1.002 s, and the old rule had waits of 2.0 s in 4 of 10 starts. The diagnostic build's counts above say why. Every run exited 0 with no fatal line and no kernel line. The first diagnostic build had 0 escapes in 12 starts and I cannot say why (its second version, with the same scheduler, had 15 in 5 of 12). ### Screenshots Nothing on screen. For the record, the last frame of one start of each arm (this change, 10 grants and no escape; old rule, 4 escapes): the same Kokiri Forest view, nothing different: ![Kokiri Forest, the last frame of start 06 of each arm: 1. this change, 10 starved grants, no escape; 2. the old rule, 4 escapes](https://git.magicbean.dev/attachments/d862fb13-ef7a-4219-825f-12673c97b9e9) ### Flags and settings `HOS_VCORE_STARVE_US=<microseconds>` is new, documented in `docs/development.md`. 0 turns the rule off, which is main's behaviour. | Flag | What it is | Before this PR | After this PR | Effect | Helps / doesn't help / untested | Recommendation | |---|---|---|---|---|---|---| | `HOS_VCORE_STARVE_US` | New switch | Absent (new) | 1000000 (1 s) | A thread that has waited this long for a virtual core goes first at the next release of a core it may use and asks the worst holder to give the core up | Helps: Linux, NVIDIA RTX 4090, the load of a Kokiri Forest save. Doesn't help: A holder that never reaches a checkpoint (the 2 s escape still ends that wait). Untested: Windows, AMD, Intel, Steam Deck, any CPU but a Core i9-13900K. | On by default (0 for the old rule, for comparisons) | ## Testing **Tested:** the new cases in `tests/test_vcore_escape.py` both ways, with g++ and with clang++: on main's `host/hos_thread.c` four checks fail (the starved thread escapes after 2 s in both fixtures, so it neither gets the core nor takes the other one), on this branch the file prints `ok`; the same fixture with `HOS_VCORE_STARVE_US=0` still escapes, which the test requires, and before the starve time the better waiter still goes first. Main's own test file passes unchanged against the new code. The whole game-free suite once on this head: 554 passed, 1 failed, 16 skipped in 153 s; the one failure, `tests/test_walk_key.py` (a link error, `undefined reference to hos_oot_autoload_input`), fails the same way on main 543846ebb and is unrelated to this change. The Windows-target syntax check of the changed fixture (clang from llvm-mingw 20261104) passes; the fixture was not run on Windows. In the game: 12 starts of a diagnostic build, 10 plain starts of main, and the 20 interleaved starts above, windowless on an RTX 4090 with NVIDIA 615.78.08 on Arch Linux. The last frame of one start of each arm shows the same Kokiri Forest view. **Built for Windows and Linux at the current head, not run:** At `0e5fcecd5c66dd7733b42a051c3a6553d9c47465`, the Windows x86-64 Vulkan renderer DLL and all 50 selected host translation units were cross-built on Linux with llvm-mingw 20261104 (Clang 23.1.5), using the project's production flags and pinned dependencies (renderer: 35 newly compiled / 170 reused; host: 50 newly compiled / 0 reused; lifted C: 0 newly compiled / 80 reused; all Windows units: 85 newly compiled / 564 reused from matching object-cache entries), the renderer DLL linked and the full game executable linked; nothing was run on Windows or under Wine, and this is neither the project's hosted builder run nor a Windows UTF-8 run-time path test. The exact same head `0e5fcecd5c66dd7733b42a051c3a6553d9c47465` also compiled and linked its full native Linux executable and renderer with Clang 23.1.3 through the documented resume path (86 newly compiled units, 405 matching Linux-cache units, source stamp `781eb374`), using the verified target-neutral C from lift `414f56e226d2be90a288ff2eb2fa172f7d4e6c93` rather than carrying the patch onto another commit; neither Linux binary was run. CPU target (this revision builds the game for the build PC's own CPU, #524; the two builds differ on purpose): the Windows cross-build keeps the builder's pinned `-march=x86-64-v3 -ffp-contract=off` (488 of its 649 compile commands carry it; the other 161 carry no `-march` flag; none carries `-march=native`, `-mtune` or `-DLONGSHOT_NATIVE_CPU`), so the Windows binaries are not built for this PC's CPU. The native Linux build used the revision's default `-march=native`, which clang resolved on this PC (13th Gen Intel(R) Core(TM) i9-13900K) to `raptorlake_raptorlake_5fe3e3650f` (build log line: `build: the game is built for this PC's CPU (raptorlake_raptorlake_5fe3e3650f)`), so the Linux binaries are built for that CPU's instruction set and tuning; neither binary was run. Renderer profile (this revision has the committed renderer play profile, #569, `assets/profiles/longshot_vk.profdata.xz`, 159448 bytes, git blob `01238f45954c`): both runners chose the profile with the revision's own `make_game.vk_profile_for`, as the project's build does, and passed it as `RX_VK_PGO`. Windows renderer (llvm-mingw Clang): built with it: 43 of its 649 compile commands (the renderer's own sources, not Eden's recompiler or sirit) carry `-fprofile-instr-use=<the unpacked profile>` and `-DLONGSHOT_VK_PROFILE_SHA1=d965e746b947f357`, all exit 0; build log line: `the Vulkan back end: built with the profile longshot_vk.shipped.profdata`. Linux renderer (native Clang): built with it (build log line: `the Vulkan back end: built with the profile longshot_vk.shipped.profdata`). The game's own lifted code is built with the older `longshot_pieces.prof` profile as before; neither binary was run. **Not tested:** Windows (rows in `docs/tested-configurations.md` from a Windows PC list core escapes too); AMD and Intel GPUs and other CPUs; a handheld; play after the save has loaded (the runs stop 5 s after play begins); how far a starved thread gets in its turn (a turn lasts until the holders ask for the core back, as for any worse thread); that nothing else slows down (only start to play was looked at, and it did not move); the effect on a real frame time, which needs the paired method and an exclusive hold. **Must be tested:** - Windows with an AMD or NVIDIA card: load a save in Kokiri Forest ten times and search `longshot_log.txt` for `ran without a core`, expecting none with the default and some with `HOS_VCORE_STARVE_US=0` (test request T-9R2R; by hand) ## Risks **What could break:** A starved thread now runs in a core that a better thread was using, so a better thread can wait for a starved thread to reach its next checkpoint (a few microseconds to a few milliseconds, once per second per starved waiter, and only after a wait of more than 1 s). The 2 s escape is unchanged and still ends a wait on a holder that never reaches a checkpoint. **Turning it off:** `HOS_VCORE_STARVE_US=0` turns the rule off; reverting the commit is enough. **Compatibility:** - Saves, `longshot_options.ini`, the shader and pipeline caches and the vk ABI: the text says nothing; the diff changes only `host/hos_thread.c`, `docs/development.md`, the new `docs/tested/` rows and `tests/test_vcore_escape.py`, so it adds no ini key (the new switch is an environment switch), touches no save or cache code and nothing under `gpu/vk/`. - What a player notices: a thread that waits more than 1 s for its virtual core now gets it instead of running without one after 2 s, and the 5-second `vcores:` line in `longshot_log.txt` gains a `starved grants` count; the 2 s escape and its switches are unchanged. - The builder's tools: the text says nothing; the diff touches none. ## Issues Fixes https://git.magicbean.dev/MagicBeanDev/Longshot/issues/262 ## Checklist ### Everyone - [x] One topic. - [x] The title is a Conventional Commit (`type(scope): description`) of the status line's type. - [x] Why gives the reason plainly and links the issues that fit (the tracker searched). - [x] Measurements has a verdict, or one line saying why there is nothing to measure. - [x] Testing names what was tested, what wasn't, and what must still be tested, where and how. - [x] Rebased on current `main`. - [x] `python -I -B tests/run_all.py` passes (any failure that also fails on `main` is named). - [x] No game files, dumps, build output or generated files. - [x] `third_party/` untouched (a new folder with a `PROVENANCE.md` instead). - [x] The license is OK: GPL-3.0-or-later, nothing incompatible and nothing decompiled. ### When it applies - [x] AI assistance recorded with `Assisted-by:` / `Co-Authored-By:` trailers. - [x] Numbers from the same PC, scene and settings, at least 2 interleaved runs each, the median reported. - [x] Switches documented (docs/development.md; the README for a menu row) and shipped at their recommended value. - [x] A file in `docs/tested/` for each new measured run (format in `docs/tested-configurations.md`; no appended row or assembled view). - [x] A game-free test added for the fix, failing on `main` and passing here. - [x] A check under `tests/` that reads changed source updated in this PR, not weakened. - [x] Host code builds on both Windows (llvm-mingw) and Linux, with UTF-8 paths. Built and linked for both at `0e5fcecd5c66dd7733b42a051c3a6553d9c47465` (under Testing); nothing was run on Windows at this head, so UTF-8 path behavior at run time is not claimed. - [x] Docs updated. --- New on this forge; not opened on GitHub.
hookshot-vcores-starved-thread-last-frames.jpg
135 KiB 
tests/test_vcore_escape.py compiles the real virtual-core block with a single-threaded
harness. New cases (starvation()): a thread of priority 28 pinned to core 1 waits while
two better threads take the core in turns at every block, and while a better thread runs
on and answers requests at its checkpoints. It must get the core after 1 s, not run
without one after 2 s; before 1 s the better waiter still goes first. The same fixture
with HOS_VCORE_STARVE_US=0 (the old rule) must still escape, so the harness shows the
fault on its own.

On main's host/hos_thread.c the new cases fail (4 checks); this commit alone is that
state, the next one is the fix.

Assisted-by: Claude Sonnet 5.5
Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
While a save loads, worker threads pinned to one core (PreActorModelCompose, BakeProbe,
bake::ModuleMgr, tera::FileResourceMgr, aglTextureDataMgr::Compaction, PlayReporter, ...)
wait the whole 2 s of vc_acquire's escape for it and then run without one. The core they
wait for is taken tens of thousands of times a second by better threads (priority 16 and
23 mostly) in turns; a waiter asks only holders worse than itself to give a core up, so
it can ask none of them, and vc_grant hands each release to the best waiter.

A waiter that has waited HOS_VCORE_STARVE_US (1000000, 1 s; 0 turns it off) is starved:
vc_better puts it before every other waiter, vc_yield counts it as a reason to give the
core up whatever the priorities, and it asks the worst holder of the cores it may use to
do so, as a better waiter does for a worse holder. Its wait ends in a grant instead of
the escape; the escape stays the deadlock breaker for a holder that never reaches a
checkpoint. The 5-s vcores line counts the grants (starved grants).

HOS_VCORE_STARVE_US is documented in docs/development.md.

Assisted-by: Claude Sonnet 5.5
Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
docs(tested): ten starts with and ten without the starvation rule, Kokiri Forest on an RTX 4090 (#262)
All checks were successful
CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Has been skipped
Details
CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Has been skipped
Details
CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Has been skipped
Details
Third-party parts / The same as upstream (pull_request) Has been skipped
Details
Website / Website on Cloudflare (pull_request) Has been skipped
Details
Website / Website on Cloudflare (pull_request_target) Has been skipped
Details
CI / Conventional Commit title (pull_request) Successful in 14s
Details
CI / Workflow lint (actionlint) (pull_request) Successful in 20s
Details
Third-party parts / As their notes say (pull_request) Successful in 29s
Details
PR description / Description follows the template (pull_request_target) Successful in 17s
Details
53a80e7351 
Two run files for the interleaved comparison of the same build, HOS_VCORE_STARVE_US at its
default and at 0: escapes, the largest core wait and the starved grants per start. Counts
and log lines of ordinary runs; no frame time is claimed.

Assisted-by: Claude Sonnet 5.5
Co-Authored-By: Claude Sonnet 5.5 <[email protected]>
mooki commented 2026-11-14 22:11:03 +00:00
Author
Member

Carried onto current main 82886e89b (main changed host/hos_thread.c since in #522; the change applies on top of it). Head 0e5fcecd5; whole game-free suite on it: 552 passed, 9 failed, 16 skipped. The builds for both systems follow.

Carried onto current main `82886e89b` (main changed `host/hos_thread.c` since in #522; the change applies on top of it). Head `0e5fcecd5`; whole game-free suite on it: 552 passed, 9 failed, 16 skipped. The builds for both systems follow.
mooki force-pushed fix/vcore-escape-262b from 53a80e7351
All checks were successful
CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Has been skipped
Details
CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Has been skipped
Details
CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Has been skipped
Details
Third-party parts / The same as upstream (pull_request) Has been skipped
Details
Website / Website on Cloudflare (pull_request) Has been skipped
Details
Website / Website on Cloudflare (pull_request_target) Has been skipped
Details
CI / Conventional Commit title (pull_request) Successful in 14s
Details
CI / Workflow lint (actionlint) (pull_request) Successful in 20s
Details
Third-party parts / As their notes say (pull_request) Successful in 29s
Details
PR description / Description follows the template (pull_request_target) Successful in 17s
Details
to 0e5fcecd5c
All checks were successful
CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Has been skipped
Details
CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Has been skipped
Details
CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Has been skipped
Details
Third-party parts / The same as upstream (pull_request) Has been skipped
Details
Website / Website on Cloudflare (pull_request) Has been skipped
Details
CI / Conventional Commit title (pull_request) Successful in 9s
Details
CI / Workflow lint (actionlint) (pull_request) Successful in 12s
Details
Third-party parts / As their notes say (pull_request) Successful in 19s
Details
PR description / Description follows the template (pull_request_target) Successful in 4s
Details
2026-11-14 22:11:05 +00:00 Compare
mooki changed title from WIP: fix(vcores): a thread starved of its core gets it after 1 s, not an escape after 2 s (#262) to fix(vcores): a thread starved of its core gets it after 1 s, not an escape after 2 s (#262) 2026-11-14 22:26:11 +00:00
mooki commented 2026-11-14 22:26:42 +00:00
Author
Member

Built at the current head 0e5fcecd5c66dd7733b42a051c3a6553d9c47465 for Windows (llvm-mingw cross-build) and Linux (native); both linked without errors, nothing was run. The build box is ticked for this head and the details are under Testing. Every box is done, so this is ready for review.

Built at the current head `0e5fcecd5c66dd7733b42a051c3a6553d9c47465` for Windows (llvm-mingw cross-build) and Linux (native); both linked without errors, nothing was run. The build box is ticked for this head and the details are under Testing. Every box is done, so this is ready for review.
DuckNintendo commented 2026-11-15 00:11:24 +00:00
First-time contributor

I ran the new fixture and the local evidence. The mechanism reproduces exactly:

  • Against main 6c9edffe's vcore block it exits with the four new failures (turn taking: the starved thread gets the core, no escape, and not before it has waited 1 s, a holder that runs on gives the core to the starved thread, and takes the other core after it, not before 1 s); on this branch it prints ok / ok, including the HOS_VCORE_STARVE_US=0 arm. Same under clang++ and g++.
  • The fixture embeds the real vc block, and dropping the starved pick, the vc_better clause, or the vc_yield clause each breaks it. The test is genuine, not a reimplementation.

Two evidence points from local runs:

  • At the 2 s default our 2 s arm logs have ten escape lines, all with a better holder (from three starts). A separate local developer log from another branch has a 2 s escape with a worse holder: AudioFrameworkThread (4) ... ran without a core after 2000.2 ms: core 2: BootupBackgroundWorkThread (30) asked. It is not one of the posted issue logs, so it does not refute the posted “every line” claim literally; it does show why that sentence should be scoped if it is meant beyond those issue logs. The 1 s rule does not reach that case: the existing worse-holder ask already applies and the holder did not yield; the risk section already says a holder that never reaches a checkpoint still escapes.
  • The 5 s line counts only vc_grant. A starved waiter that takes a free core directly in vc_acquire via vc_take is not counted; with fixed masks that is near-unreachable, so 19 is probably exact there. It needs a mask change to all-cores while the waiter is queued (or a full queue) to become reachable, not just a static mask-0. “Starved handoffs” would be exact.

One risk I would call out: the starved holder scan has no vc_lock_depth filter. With the held-mutex deferral work, the other holder scans skip locked holders and this one does not; the kick is deferred to the outermost unlock, so the 1 s cap is not guaranteed and the 2 s escape can fire. Guard it like the other scans, or state the locked-holder case in the risks.

Test gaps: removing t->vc_starved = 0 in vc_take or on the escape path still leaves the fixture green, and the starved grants line is not asserted. A reset/stat case would close that.

Minor: the table's this PR @ 53a80e735 is not in the PR history (head 0e5fcecd, fix f2f24114); could you point it at a reachable commit? And because the bug is CPU contention, a Steam Deck/4-core load-and-play run with a frame-time check would exercise the priority inversion; T-9R2R is Windows/GPU only.

I merged this with #327 locally: the resolved host compiles and the mask-0 and starvation fixtures pass; the starved + mask-0 + TLS-cell case is the one still untested.

I ran the new fixture and the local evidence. The mechanism reproduces exactly: - Against main `6c9edffe`'s vcore block it exits with the four new failures (`turn taking: the starved thread gets the core, no escape`, `and not before it has waited 1 s`, `a holder that runs on gives the core to the starved thread`, `and takes the other core after it, not before 1 s`); on this branch it prints `ok` / `ok`, including the `HOS_VCORE_STARVE_US=0` arm. Same under clang++ and g++. - The fixture embeds the real vc block, and dropping the starved pick, the `vc_better` clause, or the `vc_yield` clause each breaks it. The test is genuine, not a reimplementation. Two evidence points from local runs: - At the 2 s default our 2 s arm logs have ten escape lines, all with a better holder (from three starts). A separate local developer log from another branch has a 2 s escape with a worse holder: `AudioFrameworkThread (4) ... ran without a core after 2000.2 ms: core 2: BootupBackgroundWorkThread (30) asked`. It is not one of the posted issue logs, so it does not refute the posted “every line” claim literally; it does show why that sentence should be scoped if it is meant beyond those issue logs. The 1 s rule does not reach that case: the existing worse-holder ask already applies and the holder did not yield; the risk section already says a holder that never reaches a checkpoint still escapes. - The 5 s line counts only `vc_grant`. A starved waiter that takes a free core directly in `vc_acquire` via `vc_take` is not counted; with fixed masks that is near-unreachable, so `19` is probably exact there. It needs a mask change to all-cores while the waiter is queued (or a full queue) to become reachable, not just a static mask-0. “Starved handoffs” would be exact. One risk I would call out: the starved holder scan has no `vc_lock_depth` filter. With the held-mutex deferral work, the other holder scans skip locked holders and this one does not; the kick is deferred to the outermost unlock, so the 1 s cap is not guaranteed and the 2 s escape can fire. Guard it like the other scans, or state the locked-holder case in the risks. Test gaps: removing `t->vc_starved = 0` in `vc_take` or on the escape path still leaves the fixture green, and the `starved grants` line is not asserted. A reset/stat case would close that. Minor: the table's `this PR @ 53a80e735` is not in the PR history (head `0e5fcecd`, fix `f2f24114`); could you point it at a reachable commit? And because the bug is CPU contention, a Steam Deck/4-core load-and-play run with a frame-time check would exercise the priority inversion; T-9R2R is Windows/GPU only. I merged this with #327 locally: the resolved host compiles and the mask-0 and starvation fixtures pass; the starved + mask-0 + TLS-cell case is the one still untested.
All checks were successful
CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Has been skipped
Details
CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Has been skipped
Details
CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Has been skipped
Details
Third-party parts / The same as upstream (pull_request) Has been skipped
Details
Website / Website on Cloudflare (pull_request) Has been skipped
Details
CI / Conventional Commit title (pull_request) Successful in 9s
Details
CI / Workflow lint (actionlint) (pull_request) Successful in 12s
Details
Third-party parts / As their notes say (pull_request) Successful in 19s
Details
PR description / Description follows the template (pull_request_target) Successful in 4s
Details
This pull request has changes conflicting with the target branch.
  • docs/development.md
View command line instructions

Manual merge helper

Use this merge commit message when completing the merge manually.

Checkout

From your project repository, check out a new branch and test the changes.
git fetch -u 'fix/vcore-escape-262b:mooki-fix/vcore-escape-262b'
git switch 'mooki-fix/vcore-escape-262b'
Sign in to join this conversation.
Reviewers
No reviewers
Labels
Clear labels   
area/accessibility
Barriers for people with disabilities

  
area/audio
Sound and music

  
area/input
Controllers, keyboard, mouse

  
area/install
Installing, updating, building from your game

  
area/menu
In-game menu and settings

  
area/renderer
Vulkan renderer, shaders, effects

  
area/saves
Save files

  
area/upscaling
DLSS, FSR, frame generation

  
beanstalk: approved @ 1697603
Set by Beanstalk for show; the farm's database is what counts

  
beanstalk: approved @ 2c5a62e
Set by Beanstalk for show; the farm's database is what counts

  
beanstalk: approved @ 4fd82b7
Set by Beanstalk for show; the farm's database is what counts

  
beanstalk: approved @ 64d6d09
Set by Beanstalk for show; the farm's database is what counts

  
beanstalk: approved @ 6f91fae
Set by Beanstalk for show; the farm's database is what counts

  
beanstalk: approved @ 70a78f9
Set by Beanstalk for show; the farm's database is what counts

  
beanstalk: approved @ 8015c9a
Set by Beanstalk for show; the farm's database is what counts

  
beanstalk: approved @ 8d361dc
Set by Beanstalk for show; the farm's database is what counts

  
beanstalk: approved @ 9d8356e
Set by Beanstalk for show; the farm's database is what counts

  
beanstalk: approved @ abf2227
Set by Beanstalk for show; the farm's database is what counts

  
beanstalk: approved @ d82f202
Set by Beanstalk for show; the farm's database is what counts

  
beanstalk: approved @ e5f1e40
Set by Beanstalk for show; the farm's database is what counts

  
beanstalk: approved @ ee8ab5f
Set by Beanstalk for show; the farm's database is what counts

  
ci: approved
A maintainer approved CircleCI for this fork PR (sticky; removed after 14 quiet days)

  
good first issue
A good start for a new contributor

  
gpu/amd
AMD GPUs

  
gpu/igpu
Integrated GPUs

  
gpu/intel
Intel GPUs

  
gpu/nvidia
NVIDIA GPUs

  
help wanted
Extra hands or knowledge wanted

  
kind
bug
 Something doesn't work as it should

  
kind
chore
 Build, CI, dependencies, tidying

  
kind
docs
 Documentation

  
kind
feature
 A new feature or an improvement

  
kind
performance
 Frame rate, frame time, loading, memory

  
kind
question
 A question rather than a change

  
needs measurements
Waiting for before/after measurements

  
platform/linux
Affects Linux (a change for every system carries both Windows and Linux)

  
platform/macos
macOS (Apple silicon or Intel Macs)

  
platform/steam-deck
Steam Deck

  
platform/windows
Affects Windows (a change for every system carries both Windows and Linux)

  
preview
Maintainers: deploy a preview of this PR's website (read its changes first). Removed after the run.

  
priority
critical
 Crashes, data loss, or blocks most players

  
priority
high
 Important: next in line

  
priority
low
 Nice to have

  
regression
Worked in an earlier build, broken in a newer one: highest priority, above other bugs

  
status
blocked
 Waits on something else first

  
status
confirmed
 Reproduced or agreed on

  
status
duplicate
 Already reported or proposed

  
status
invalid
 Not a problem in this project

  
status
needs-info
 Waiting for more information from the reporter

  
status
wontfix
 Will not be worked on

  
testing wanted
Wanted: a test on other hardware (posts a test request in #test-requests)

  
ci: approved
No labels
area/accessibility
area/audio
area/input
area/install
area/menu
area/renderer
area/saves
area/upscaling
beanstalk: approved @ 1697603
beanstalk: approved @ 2c5a62e
beanstalk: approved @ 4fd82b7
beanstalk: approved @ 64d6d09
beanstalk: approved @ 6f91fae
beanstalk: approved @ 70a78f9
beanstalk: approved @ 8015c9a
beanstalk: approved @ 8d361dc
beanstalk: approved @ 9d8356e
beanstalk: approved @ abf2227
beanstalk: approved @ d82f202
beanstalk: approved @ e5f1e40
beanstalk: approved @ ee8ab5f
ci: approved
good first issue
gpu/amd
gpu/igpu
gpu/intel
gpu/nvidia
help wanted
kind
bug
kind
chore
kind
docs
kind
feature
kind
performance
kind
question
needs measurements
platform/linux
platform/macos
platform/steam-deck
platform/windows
preview
priority
critical
priority
high
priority
low
regression
status
blocked
status
confirmed
status
duplicate
status
invalid
status
needs-info
status
wontfix
testing wanted
ci: approved
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
2 participants
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
MagicBeanDev/Longshot!594
No description provided.