perf(host): the game's short sleeps wait on the timer where the process has six logical processors or fewer #2263

Merged

klimovich008 merged 3 commits from perf/short-sleep-timer-few-threads into main

2026-11-11 18:26:34 +00:00

klimovich008 commented

2026-11-11 12:29:21 +00:00

(Migrated from github.com)

Why it happens

OoT's job workers (ModuleSystemWorker*, OptimizeChangeBanc), with nothing to do, sleep 20 us and look at their queues again: about 7,600 SleepThread calls a second in Hyrule Field.
Windows cannot sleep 20 us: its high-resolution timer wakes about 0.5 ms late. So sleep_ns in host/hos_thread.c answers a sleep under a millisecond by yielding until its time (SwitchToThread in a loop). That is exact, and it keeps a processor busy for every sleeping thread.
With 16 logical processors that costs CPU time only. With 4 or 6, which is no more than the game's six cores, the yielding loops run instead of the threads that make the frame.

What changes

host/hos_thread.c: hos_short_sleep_timer decides once how sleeps under a millisecond wait. With six logical processors or fewer in the process's affinity mask, the game's own sleeps (nn::os::SleepThread and the kernel's SleepThread, both through hos_sleep_logged) wait on the timer. With more, and when the count is not known (the call failed, or the PC has more than one processor group, where the mask is one group's), they yield as before.
The host's own short sleeps (hos_sleep_ns: the frame limiter in nn_vi.c, the audio renderer, timer events) keep yielding in both cases, so pacing is not made half a millisecond late.
HOS_SHORT_SLEEP_TIMER has three states. Not set: the rule above. Set: it follows env_on's rule from #2042 (on: the timer for every short sleep, which is what the switch did before; 0, false, off, no or empty: yielding for every one).
The log says once when the timer is chosen, with the processor count.
The choice by processors is made on Windows only, where it was measured. Linux and the web build are as they were (the switch works there by its value).
tests/test_short_sleep_timer.py: the real hos_short_sleep_timer, sleep_away, hos_sleep_ns, sleep_ns and hos_sleep_logged, cut from the source, over stubs that count yields and timer waits, one process a case: 26 cases of processor count and switch value, the processor-group case, and six broken variants that must fail (the processors not looked at, the switch read by presence, the switch read as a number instead of by env_on's rule, the game's sleeps passed on as the host's, the host's sleeps put on the timer, one group's mask taken for the process's).

Tested

Windows 11, Ryzen 7 5700X3D (8 cores, 16 threads), Radeon RX 9070 XT (Vulkan 1.4, driver 0x8001a0), 2560x720 window (32:9), frame rate cap 60.

Fewer processors are emulated: the game is started with an affinity mask (inherited from its first instruction), on this one CPU. No PC with a 4- or 6-thread CPU was tested; such a CPU also has slower cores and less cache, and the renderer still sized its pools for 16 threads here.

Each cell is 4 pairs of runs on one binary, control HOS_SHORT_SLEEP_TIMER=0 (all yield, as main does when the switch is not set) against the switch unset (this PR's rule), alternating. A run measures 20 s from 3 s after the save has loaded: Hyrule Field with the camera turning (a player's save), Kokiri Forest standing (the default save). The log line confirms the rule in all 16 variant runs and in none of the control runs. The binary is main d8ce191 with this PR's change to hos_thread.c as it was before the merge with #2042 (so the PC's prepared shader caches matched; the merge changed only how a set switch's value is read). On the PR's head, merged with main b744b1ec, I ran the tests and a whole-file syntax compile of host/*.c, not the game.

logical processors	scene	fps, yielding to the rule	CPU time a frame	p99 frame	worst frame of each run, yielding (ms)	with the rule (ms)
4 (mask 0x55: 4 cores)	Hyrule Field	47.0 to 51.3 (+9.2 %, 4 of 4 pairs)	64.7 to 55.6 ms (-14.0 %)	42.8 to 28.4 ms	279, 55, 729, 60	59, 62, 53, 53
	Kokiri Forest	58.0 to 58.9 (+1.7 %, 4 of 4)	50.1 to 45.6 ms (-8.5 %)	27.4 to 25.3 ms	422, 78, 53, 55	30, 30, 30, 30
6 (mask 0x555: 6 cores)	Hyrule Field	52.1 to 52.2 (+0.4 %, 4 of 4)	65.5 to 59.7 ms (-9.1 %)	26.3 to 25.0 ms	50, 55, 64, 49	51, 50, 54, 57
	Kokiri Forest	60.0 to 60.0	53.1 to 48.2 ms (-9.2 %)	22.1 to 21.8 ms	432, 26, 44, 31	28, 25, 25, 27

Frames over 250 ms: in 4 of the 16 yielding runs (2 of 4 in Hyrule Field on 4 processors), in none of the 16 runs with the rule.
Why the edge is at 6 and not higher, measured before this change with the old switch (HOS_SHORT_SLEEP_TIMER=1, every short sleep on the timer, release binary of main d8ce191, 4 pairs a cell):
- 4 processors, Hyrule Field: 50.4 to 52.8 fps (4 of 4); with yielding a frame over 250 ms in 5 of 16 runs, with the timer in none of 8.
- 6 processors: fps the same, CPU time a frame -12 %, frames over 100 ms in 2 of 4 yielding Hyrule runs and in none with the timer.
- 8 processors laid out as the host lays out a 4-core, 8-thread CPU (the Presentation Thread alone on the last core): no frame over 100 ms either way; Hyrule Field 45.5 to 46.1 fps, Kokiri Forest 59.5 to 59.3 fps (lower in 4 of 4 pairs). Left as it is.
- 16 processors: fps the same, CPU time a frame -5 to -13 %. Left as it is: nothing to gain in frame rate, and in Hyrule Field 3 of 32 runs with the timer (most of them with a shorter spin too) had a frame over 250 ms against 0 of 38 without (too few to tell).
A shorter spin before a wait (HOS_SPIN_US=5) was tried beside it on 4 processors: no effect. Not part of this PR.
Content, checked separately by a second agent (Codex): it compared this exact binary with the rule against =0 on 4 processors in Temple of Time, Hyrule Field and Kokiri Forest (frames captured every 120 to 600 frames, validation layers off): world and HUD intact in both arms, the only differences animation timing (the fairy, a title fading). Saves unchanged, no fatal line.
On the merged head: python -I -B tests/test_short_sleep_timer.py --cc gcc (MinGW gcc 16.2) and with clang, and tests/test_env_on.py: pass. The full tests/run_all.py on the merged head (Windows, MinGW gcc): 324 passed, 6 failed, 58 skipped. None of the six is this PR's: test_vcore_fs_open and test_vcore_video_acquire fail the same way on unchanged main b744b1ec (fixed on main since by #2380), and test_fpx_sticky, test_pipeline_play_cache, nx2/test_nup_paths and test_precompile_compute_guard failed the same way on unchanged main ebbba2ea on this PC.
Independent review (gpt-6.1-sol, read-only): one finding on the first version (processor groups), fixed with its own test case; none on the final head, and none on the merge with main.

Not covered

A real 4- or 6-thread CPU, Intel hybrid CPUs, and laptops: only an affinity mask on a 5700X3D.
Linux and the Steam Deck: the choice is not made there. The POSIX layer's timer is not Windows' and was not measured.
Longer play and other places than the two scenes; loading screens were not measured.
8 to 12 logical processors, where the timer for the game's sleeps alone was not measured (only the old all-sleeps switch).
NVIDIA and Intel GPUs (the change is on the CPU side; all runs were on one AMD card).

Kind of change: performance on small CPUs, a behaviour change in scheduling there (who runs when), default on only where measured, with the switch to turn it off (HOS_SHORT_SLEEP_TIMER=0). Related: #995 (low fps with the GPU not fully used); no report from that issue was reproduced.

🤖 Generated with Claude Code

On a PC with few logical processors the game's idle job workers take processor time the frame's threads need: the frame rate is lower and some frames take a quarter of a second or more. This makes those workers really sleep there. Nothing changes with 8 logical processors or more. ## Why it happens - OoT's job workers (`ModuleSystemWorker*`, `OptimizeChangeBanc`), with nothing to do, sleep 20 us and look at their queues again: about 7,600 `SleepThread` calls a second in Hyrule Field. - Windows cannot sleep 20 us: its high-resolution timer wakes about 0.5 ms late. So `sleep_ns` in `host/hos_thread.c` answers a sleep under a millisecond by yielding until its time (`SwitchToThread` in a loop). That is exact, and it keeps a processor busy for every sleeping thread. - With 16 logical processors that costs CPU time only. With 4 or 6, which is no more than the game's six cores, the yielding loops run instead of the threads that make the frame. ## What changes - `host/hos_thread.c`: `hos_short_sleep_timer` decides once how sleeps under a millisecond wait. With six logical processors or fewer in the process's affinity mask, the game's own sleeps (`nn::os::SleepThread` and the kernel's `SleepThread`, both through `hos_sleep_logged`) wait on the timer. With more, and when the count is not known (the call failed, or the PC has more than one processor group, where the mask is one group's), they yield as before. - The host's own short sleeps (`hos_sleep_ns`: the frame limiter in `nn_vi.c`, the audio renderer, timer events) keep yielding in both cases, so pacing is not made half a millisecond late. - `HOS_SHORT_SLEEP_TIMER` has three states. Not set: the rule above. Set: it follows `env_on`'s rule from #2042 (on: the timer for every short sleep, which is what the switch did before; `0`, `false`, `off`, `no` or empty: yielding for every one). - The log says once when the timer is chosen, with the processor count. - The choice by processors is made on Windows only, where it was measured. Linux and the web build are as they were (the switch works there by its value). - `tests/test_short_sleep_timer.py`: the real `hos_short_sleep_timer`, `sleep_away`, `hos_sleep_ns`, `sleep_ns` and `hos_sleep_logged`, cut from the source, over stubs that count yields and timer waits, one process a case: 26 cases of processor count and switch value, the processor-group case, and six broken variants that must fail (the processors not looked at, the switch read by presence, the switch read as a number instead of by `env_on`'s rule, the game's sleeps passed on as the host's, the host's sleeps put on the timer, one group's mask taken for the process's). ## Tested Windows 11, Ryzen 7 5700X3D (8 cores, 16 threads), Radeon RX 9070 XT (Vulkan 1.4, driver 0x8001a0), 2560x720 window (32:9), frame rate cap 60. **Fewer processors are emulated:** the game is started with an affinity mask (inherited from its first instruction), on this one CPU. No PC with a 4- or 6-thread CPU was tested; such a CPU also has slower cores and less cache, and the renderer still sized its pools for 16 threads here. Each cell is 4 pairs of runs on one binary, control `HOS_SHORT_SLEEP_TIMER=0` (all yield, as main does when the switch is not set) against the switch unset (this PR's rule), alternating. A run measures 20 s from 3 s after the save has loaded: Hyrule Field with the camera turning (a player's save), Kokiri Forest standing (the default save). The log line confirms the rule in all 16 variant runs and in none of the control runs. The binary is main d8ce191 with this PR's change to `hos_thread.c` as it was before the merge with #2042 (so the PC's prepared shader caches matched; the merge changed only how a set switch's value is read). On the PR's head, merged with main b744b1ec, I ran the tests and a whole-file syntax compile of `host/*.c`, not the game. | logical processors | scene | fps, yielding to the rule | CPU time a frame | p99 frame | worst frame of each run, yielding (ms) | with the rule (ms) | |---|---|---|---|---|---|---| | 4 (mask 0x55: 4 cores) | Hyrule Field | 47.0 to 51.3 (+9.2 %, 4 of 4 pairs) | 64.7 to 55.6 ms (-14.0 %) | 42.8 to 28.4 ms | 279, 55, 729, 60 | 59, 62, 53, 53 | | | Kokiri Forest | 58.0 to 58.9 (+1.7 %, 4 of 4) | 50.1 to 45.6 ms (-8.5 %) | 27.4 to 25.3 ms | 422, 78, 53, 55 | 30, 30, 30, 30 | | 6 (mask 0x555: 6 cores) | Hyrule Field | 52.1 to 52.2 (+0.4 %, 4 of 4) | 65.5 to 59.7 ms (-9.1 %) | 26.3 to 25.0 ms | 50, 55, 64, 49 | 51, 50, 54, 57 | | | Kokiri Forest | 60.0 to 60.0 | 53.1 to 48.2 ms (-9.2 %) | 22.1 to 21.8 ms | 432, 26, 44, 31 | 28, 25, 25, 27 | - Frames over 250 ms: in 4 of the 16 yielding runs (2 of 4 in Hyrule Field on 4 processors), in none of the 16 runs with the rule. - Why the edge is at 6 and not higher, measured before this change with the old switch (`HOS_SHORT_SLEEP_TIMER=1`, every short sleep on the timer, release binary of main d8ce191, 4 pairs a cell): - 4 processors, Hyrule Field: 50.4 to 52.8 fps (4 of 4); with yielding a frame over 250 ms in 5 of 16 runs, with the timer in none of 8. - 6 processors: fps the same, CPU time a frame -12 %, frames over 100 ms in 2 of 4 yielding Hyrule runs and in none with the timer. - 8 processors laid out as the host lays out a 4-core, 8-thread CPU (the Presentation Thread alone on the last core): no frame over 100 ms either way; Hyrule Field 45.5 to 46.1 fps, Kokiri Forest 59.5 to 59.3 fps (lower in 4 of 4 pairs). Left as it is. - 16 processors: fps the same, CPU time a frame -5 to -13 %. Left as it is: nothing to gain in frame rate, and in Hyrule Field 3 of 32 runs with the timer (most of them with a shorter spin too) had a frame over 250 ms against 0 of 38 without (too few to tell). - A shorter spin before a wait (`HOS_SPIN_US=5`) was tried beside it on 4 processors: no effect. Not part of this PR. - Content, checked separately by a second agent (Codex): it compared this exact binary with the rule against `=0` on 4 processors in Temple of Time, Hyrule Field and Kokiri Forest (frames captured every 120 to 600 frames, validation layers off): world and HUD intact in both arms, the only differences animation timing (the fairy, a title fading). Saves unchanged, no fatal line. - On the merged head: `python -I -B tests/test_short_sleep_timer.py --cc gcc` (MinGW gcc 16.2) and with clang, and `tests/test_env_on.py`: pass. The full `tests/run_all.py` on the merged head (Windows, MinGW gcc): 324 passed, 6 failed, 58 skipped. None of the six is this PR's: `test_vcore_fs_open` and `test_vcore_video_acquire` fail the same way on unchanged main b744b1ec (fixed on main since by #2380), and `test_fpx_sticky`, `test_pipeline_play_cache`, `nx2/test_nup_paths` and `test_precompile_compute_guard` failed the same way on unchanged main ebbba2ea on this PC. - Independent review (gpt-6.1-sol, read-only): one finding on the first version (processor groups), fixed with its own test case; none on the final head, and none on the merge with main. ## Not covered - A real 4- or 6-thread CPU, Intel hybrid CPUs, and laptops: only an affinity mask on a 5700X3D. - Linux and the Steam Deck: the choice is not made there. The POSIX layer's timer is not Windows' and was not measured. - Longer play and other places than the two scenes; loading screens were not measured. - 8 to 12 logical processors, where the timer for the game's sleeps alone was not measured (only the old all-sleeps switch). - NVIDIA and Intel GPUs (the change is on the CPU side; all runs were on one AMD card). Kind of change: performance on small CPUs, a behaviour change in scheduling there (who runs when), default on only where measured, with the switch to turn it off (`HOS_SHORT_SLEEP_TIMER=0`). Related: #995 (low fps with the GPU not fully used); no report from that issue was reproduced. 🤖 Generated with [Claude Code](https://claude.com/claude-code)

klimovich008 commented

2026-11-11 17:08:17 +00:00

(Migrated from github.com)

Merged main (b744b1ec) to resolve the conflict with #2042 in host/hos_thread.c. HOS_SHORT_SLEEP_TIMER keeps its third state (not set: the processor count decides); when it is set, its value now follows env_on's rule, where this branch read it as a number (so =on would have been off). HOS_SLEEP_LOG is main's env_on. The test has a broken variant for that (the switch read as a number must fail), and tests/test_env_on.py passes. The description is updated.

🤖 Generated with Claude Code

Merged main (b744b1ec) to resolve the conflict with #2042 in `host/hos_thread.c`. `HOS_SHORT_SLEEP_TIMER` keeps its third state (not set: the processor count decides); when it is set, its value now follows `env_on`'s rule, where this branch read it as a number (so `=on` would have been off). `HOS_SLEEP_LOG` is main's `env_on`. The test has a broken variant for that (the switch read as a number must fail), and `tests/test_env_on.py` passes. The description is updated. _🤖 Generated with [Claude Code](https://claude.com/claude-code)_

hckrman101 (Migrated from github.com) approved these changes

2026-11-11 17:36:00 +00:00

hckrman101 (Migrated from github.com) left a comment

What I checked:

Timer fallback: if CreateWaitableTimerExW(..., CREATE_WAITABLE_TIMER_HIGH_RESOLUTION, ...) fails (Windows older than 10 1803), a short sleep falls through to the same SwitchToThread loop, so nothing breaks and you just keep today's behaviour (see the inline nit about the log). timeBeginPeriod(1) in hos_main.c doesn't matter for this path, since the high-resolution timer doesn't depend on it.
short_timer is a plain static that several threads may start concurrently, but they all compute the same value, and the log line is guarded by InterlockedExchange. That's fine.
Both guest entries (nn::os::SleepThread and svc 0x0B) go through hos_sleep_logged → sleep_away(ns, 1). hos_sleep_ns stays host-side.
The processor-group check (GetActiveProcessorGroupCount() == 1) is right: on a multi-group machine the count is left unknown and sleeps yield as before.
CI: test_short_sleep_timer.py and test_env_on.py pass on Windows and Ubuntu. The red checks aren't caused by this PR: test_native_astc_texture_count fails the same way on main's latest source run (71096934), and the appimage job timed out installing packages.

Linux parity and #2313: the two PRs rewrite the same short_timer lines in sleep_ns, so whichever lands second needs a rebase. #2313 is also based on main from before #2042 (it reads the switch by presence and atoi, not env_on). They don't overlap in intent, though. This PR leaves Linux yielding unless the switch is set, while #2313 makes the timer the default on Linux, where clock_nanosleep wakes within ~50 us (measured on the Deck: kernel time per worker 0.4-0.6 s/s → 0.04). I'd land this one first, then rebase #2313 onto hos_short_sleep_timer: on non-Windows, an unset switch would return 2 (all sleeps on the timer) instead of 0, and the new test would need one case for it. @Kolkka FYI.

Nits only. Approving.

Looks good to me. The split is the right one: only the game's own SleepThread waits go on the timer, the host's frame limiter and audio keep the exact yield, and the rule only kicks in at 6 logical processors or fewer, with `HOS_SHORT_SLEEP_TIMER` as an escape hatch both ways. The numbers back it up: the 250+ ms frames go away on 4 threads, and fps and CPU time a frame either improve or stay the same on 6. What I checked: - Timer fallback: if `CreateWaitableTimerExW(..., CREATE_WAITABLE_TIMER_HIGH_RESOLUTION, ...)` fails (Windows older than 10 1803), a short sleep falls through to the same `SwitchToThread` loop, so nothing breaks and you just keep today's behaviour (see the inline nit about the log). `timeBeginPeriod(1)` in hos_main.c doesn't matter for this path, since the high-resolution timer doesn't depend on it. - `short_timer` is a plain static that several threads may start concurrently, but they all compute the same value, and the log line is guarded by `InterlockedExchange`. That's fine. - Both guest entries (`nn::os::SleepThread` and svc 0x0B) go through `hos_sleep_logged` → `sleep_away(ns, 1)`. `hos_sleep_ns` stays host-side. - The processor-group check (`GetActiveProcessorGroupCount() == 1`) is right: on a multi-group machine the count is left unknown and sleeps yield as before. - CI: `test_short_sleep_timer.py` and `test_env_on.py` pass on Windows and Ubuntu. The red checks aren't caused by this PR: `test_native_astc_texture_count` fails the same way on main's latest source run (71096934), and the appimage job timed out installing packages. Linux parity and #2313: the two PRs rewrite the same `short_timer` lines in `sleep_ns`, so whichever lands second needs a rebase. #2313 is also based on main from before #2042 (it reads the switch by presence and `atoi`, not `env_on`). They don't overlap in intent, though. This PR leaves Linux yielding unless the switch is set, while #2313 makes the timer the default on Linux, where `clock_nanosleep` wakes within ~50 us (measured on the Deck: kernel time per worker 0.4-0.6 s/s → 0.04). I'd land this one first, then rebase #2313 onto `hos_short_sleep_timer`: on non-Windows, an unset switch would return 2 (all sleeps on the timer) instead of 0, and the new test would need one case for it. @Kolkka FYI. Nits only. Approving.

host/hos_thread.c

Nit: this logs "wait on the timer" even when this thread's CreateWaitableTimerExW(... HIGH_RESOLUTION ...) ends up NULL (Windows before 10 1803). In that case a short sleep still goes through the yield loop at the bottom of sleep_ns, so the log line overstates what happens. That's harmless, but if someone reports "the log says timer but CPU use didn't drop", this is the reason. A word in the comment would be enough.

Nit: this logs "wait on the timer" even when this thread's `CreateWaitableTimerExW(... HIGH_RESOLUTION ...)` ends up NULL (Windows before 10 1803). In that case a short sleep still goes through the yield loop at the bottom of `sleep_ns`, so the log line overstates what happens. That's harmless, but if someone reports "the log says timer but CPU use didn't drop", this is the reason. A word in the comment would be enough.

klimovich008 (Migrated from github.com) reviewed

2026-11-11 18:11:27 +00:00

host/hos_thread.c

Fixed in d106855e. The log line now reads "wait on the timer where Windows has the high-resolution one", and the comment above it says the line is written when the choice is made, before any thread has created its timer: where CreateWaitableTimerExW(HIGH_RESOLUTION) fails (Windows before 10 1803) a short sleep falls through to the yield loop at the end of sleep_ns and nothing changes.

🤖 Addressed by Claude Code

Fixed in d106855e. The log line now reads "wait on the timer where Windows has the high-resolution one", and the comment above it says the line is written when the choice is made, before any thread has created its timer: where `CreateWaitableTimerExW(HIGH_RESOLUTION)` fails (Windows before 10 1803) a short sleep falls through to the yield loop at the end of `sleep_ns` and nothing changes. _🤖 Addressed by [Claude Code](https://claude.com/claude-code)_

hckrman101 (Migrated from github.com) approved these changes

2026-11-11 18:24:08 +00:00

hckrman101 (Migrated from github.com) left a comment

Re-approving at d106855. The only change since my approval is the nit fix: the log line now says "where Windows has the high-resolution one" and the comment above it explains the pre-1803 fallback to the yield loop. That covers it, nothing new.

CI is green except Windows part 1, where only tests/test_native_astc_texture_count.py fails. That test is unrelated to this change (it doesn't touch host/hos_thread.c), so I'd treat it as main's flake or an existing failure. Worth a re-run before merging.

Re-approving at d106855. The only change since my approval is the nit fix: the log line now says "where Windows has the high-resolution one" and the comment above it explains the pre-1803 fallback to the yield loop. That covers it, nothing new. CI is green except Windows part 1, where only tests/test_native_astc_texture_count.py fails. That test is unrelated to this change (it doesn't touch host/hos_thread.c), so I'd treat it as main's flake or an existing failure. Worth a re-run before merging.