fix(host): a thread lowered right after it is created runs lowered on Linux #496
No reviewers
Labels
No labels
area/accessibility
area/audio
area/input
area/install
area/menu
area/renderer
area/saves
area/upscaling
beanstalk: approved @ 1697603
beanstalk: approved @ 2c5a62e
beanstalk: approved @ 4fd82b7
beanstalk: approved @ 64d6d09
beanstalk: approved @ 6f91fae
beanstalk: approved @ 70a78f9
beanstalk: approved @ 8d361dc
beanstalk: approved @ 9d8356e
beanstalk: approved @ abf2227
beanstalk: approved @ d82f202
beanstalk: approved @ e5f1e40
beanstalk: approved @ ee8ab5f
ci: approved
good first issue
gpu/amd
gpu/igpu
gpu/intel
gpu/nvidia
help wanted
kind
bug
kind
chore
kind
docs
kind
feature
kind
performance
kind
question
needs measurements
platform/linux
platform/macos
platform/steam-deck
platform/windows
preview
priority
critical
priority
high
priority
low
regression
status
blocked
status
confirmed
status
duplicate
status
invalid
status
needs-info
status
wontfix
testing wanted
ci: approved
No milestone
No project
No assignees
2 participants
Notifications
Due date
No due date set.
Dependencies
No dependencies set
Reference
MagicBeanDev/Longshot!496
Loading…
Reference in a new issue
No description provided.
Delete branch "zeldaootr/Longshot:posix-thread-priority-before-start"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Type: fix · Risk: low · Tested on: Linux x86-64 (game-free); Steam Deck OLED, SteamOS 3.9.5, RADV (a tester's run) · Needs: none
Why
On Linux and the Steam Deck, a thread whose priority is lowered right after it is created can keep the normal priority. That is how the game lowers OoT's background
OptimizeChangeBankthread (and anyHOS_PRIOlowering), so when it happens that thread competes with the renderer and the driver for the CPU, which the lowering exists to stop.How to reproduce
Seen on: Linux x86-64 (Ubuntu 24.04), game-free; not seen in the game (see Testing)
Settings and switches: defaults
Steps: on
mainaa95d35fwith this PR's test added (cc676b23), runpython -I -B tests/test_posix_thread_priority.py(not undernice): it builds the realhost/posix/win32_posix.cand lowers 20 threads withTHREAD_PRIORITY_LOWESTstraight afterCreateThreadExpected: each thread runs at nice 10
Actual:
FAIL: 20 of 20 threads lowered right after CreateThread kept the normal priorityWhat changed
SetThreadPriorityon Linux now waits for a just-created thread to get its kernel id before lowering it, instead of dropping the lowering, so a thread lowered at creation runs lowered.SetThreadPriority()inhost/posix/win32_posix.cskippedsetprioritywhile the thread had no kernel id yet (t->tid == 0). A thread fresh fromCreateThreadhas none until it first runs, andhos_thread_start->hos_thread_pin(host/hos_thread.c) lowers it straight away, so the lowering was dropped without a word. It now yields until the id is there (or the thread has ended), asGetThreadId()already does, when there is a lowering to apply.tests/test_posix_thread_priority.pybuilds the realhost/posix/win32_posix.cwith a fixture that creates a thread, lowers it at once withTHREAD_PRIORITY_LOWESTand has the thread read its own nice value, 20 times. It skips itself when the test process already runs at nice 10 or more.Measurements
Verdict: fixed in the game-free test (0 of 20 threads lowered on
main, 20 of 20 with this PR, two interleaved rounds); no change on a tester's Steam Deck, where the race was not lost without the PR either (nice 10 both ways, fps within the run-to-run spread).Hardware: Linux x86-64 server (Ubuntu 24.04), no GPU or game used; gcc 13.3, clang 23.1.5
Scene: no game: the game-free check
Method:
python3 -I -B tests/test_posix_thread_priority.pyat the test-only commitcc676b23(mainaa95d35fplus the test) and at6218c00b, twice each, alternating, not underniceHow to reproduce: check out each commit and run the command above
aa95d35f(with the test,cc676b23)6218c00bCreateThreadthat run at nice 10 (both rounds the same)In the game, dukenheimer's Steam Deck run for test request T-YHUH (pass; #496 (comment)): this PR's head
7fb4e2c1(on main543846eb, the same change) against the same build with this PR's merge reverted, Steam Deck OLED (SteamOS 3.9.5, Mesa 26.2.0-devel, stock clocks, Game Mode), a moving Kokiri route, two runs each, interleaved.OptimizeChangeBankran at nice 10 in every sample with and without this PR: in both starts without it the new thread already had its id whenSetThreadPriorityran. Real fps 13.1 and 12.8 with it, 13.0 and 13.5 without (medians 12.95 and 13.25, inside the 12.8 to 13.5 spread of the series).Flags and settings
None
Testing
Tested: Linux x86-64, game-free. The new test fails on the test-only commit
cc676b23and passes at6218c00b(this branch rebased ontomainaa95d35f), twice each, alternating, and undertests/run_all.pywith g++ 13.3 and with clang 23.1.5 asCXXat both commits. At the tip the 11 checks that readhost/posix/win32_posix.cor the new test: 10 passed, andtest_mods_size_tablefailed as it does onmainaa95d35f. Before the rebase the same at9f641552and27113be7(main7ff5256e). Steam Deck: dukenheimer's run for T-YHUH, in Measurements: nice 10 with and without this PR, fps unchanged within the spread.Not tested: A start where the race is lost in the game: none was seen (2 of 2 starts on the tester's Deck kept the lowering without this PR), so how often a player hits it is unknown; a slower or busier start is where it would show. A Linux desktop run (welcome, not required).
HOS_PRIOlowerings take the same path and were not tried. Windows is not affected: the file is the POSIX layer, not built on Windows.Must be tested: None
Risks
What could break:
SetThreadPrioritynow yields until a new thread has its id, asGetThreadIddoes; a thread that never starts would have hung that call already (CREATE_SUSPENDEDis not supported and starts the thread at once). Only lowerings wait. A lowered background thread gets less CPU on Linux when the race would have been lost: the intended behaviour and what Windows already does, but on a Deck it could slow whatever that thread does if it matters for frame pacing.Turning it off:
HOS_OCB_NORMAL=1keepsOptimizeChangeBankat normal priority; otherwise revert the commit.Compatibility: Nothing changes: no save, option or cache is touched.
Issues
None filed. Note for reviewers: this PR's own test skips itself in a process at nice 10 or more (a
run_all.pyundernice -n 10reports it as passed while it skipped); the results above come from runs withoutnice.Checklist
Everyone
type(scope): description) of the status line's type.main. Onto mainaa95d35f, no conflict; the test and fix commits are the same changes as before (git patch-id).python -I -B tests/run_all.pypasses (any failure that also fails onmainis named). The full suite on mainaa95d35f(Linux, g++ 13.3, the Vulkan headers set): 528 passed, 15 failed, 50 skipped; on this branch the new test and the checks that read the changed file ran, with no failure thatmaindoes not have (see Testing). The 15 that fail on main:test_appimage_player_build,test_bake_portability,test_cpm_discovery,test_crash_dialog,test_env_on,test_lift_cache,test_lift_cleanup_error,test_make_game_cache,test_mods_size_table,test_options_budget_once,test_pipeline_tour,test_readback_late,test_vk_pgo,test_walk_keyandnx2/test_texture_init_barrier.py.third_party/untouched (a new folder with aPROVENANCE.mdinstead).When it applies
Assisted-by:/Co-Authored-By:trailers.mainand passing here.SetThreadPriorityright afterCreateThreadlowers the thread on Linux ea76cc841dSetThreadPrioritywaits for a new thread's id instead of dropping the loweringTest request T-YHUH: pass
Hardware: Steam Deck OLED (AMD Custom APU 0932, RADV VANGOGH, 16 GB), Mesa 26.2.0-devel git-5c65d4fc31, SteamOS 3.9.5
(build 20261023.100), kernel 7.2.7-valve2. Built from this PR's head
7fb4e2c1, merged into a Deck user build (details below). Thecomparison is the same build with this PR's merge reverted, not plain main.
What I checked
ps -L -o ni,comm, every 30 s (7 to 9 samples a run).Result
OptimizeChangethread10 OptimizeChangeBin every sample10 OptimizeChangeBin every sampleThe thread was at nice 10 without this PR too. In both starts without the PR the new thread already had its id when
SetThreadPriorityran, so the lowering that this PR keeps from being lost was not lost on main's code either. Both arms lowered thesame other threads: 5 at nice 19 (
hooksho:disk$0-$3) and 2 at nice 5 (mkw_vk); 102 stayed at 0.fps. The medians are 12.95 with the PR and 13.25 without, inside the run-to-run spread (12.8 to 13.5 over all 8 runs of the
series). The priorities were the same in both arms, so this PR can't account for the difference. The p95 gap follows the run order:
by position in the series p95 was 73.4, 78.3, 79.4, 95.2, 100.7, 99.7, 79.1, 73.6 ms, rising toward the middle whatever ran there.
Two starts are few for a race. All this shows is that the lowering wasn't lost in 2 of 2 starts without the PR on this Deck at these
settings; with the PR it no longer depends on that.
Setup
Mesa 26.2.0-devel git-5c65d4fc31. Stock clocks. Game Mode, docked: the game gets a 1280x720 surface.
543846eb+ #2270fc2e8c9(with #225) + #22352a61f40+ #2323ccb48a8+ #303995aab15, with#384
a9b0c880, #468e5889e76, #488abf2227fand #4967fb4e2c1merged on top. All four merged clean. Linux suite: 544 passed,main's 5 known failures. Built with x86-64-v3 and the release play profile, LLVM 23.1.5 (glibc 2.35).
git revert -m 1), built the same way. Only that file is swappedin one warm test folder: the host
longshotfor #496,longshot_vk.sofor #384 and #468.upscaler, frame generation 3x, frame rate cap 60, vsync on.
RX_VK_READBACK_EARLY=1(#303) andHOS_VIDEO_LOG=1.In runs 2 (-#496) and 4 (-#468) the scripted route went into a house after its start, so most of those two windows is an interior.
7fb4e2c17127113be72eRebased onto main
7ff5256e, and dukenheimer's Steam Deck result (T-YHUH: pass) is in the description: the thread ran at nice 10 with and without this PR in 2 of 2 starts there, so on that Deck it changed nothing measured; it removes a race the game-free test loses 19 or 20 times of 20. Thank you for the run.WIP: fix(host): a thread lowered right after it is created runs lowered on Linuxto fix(host): a thread lowered right after it is created runs lowered on LinuxRebased onto
mainaa95d35fwith no conflict (the same two changes,git patch-id;host/posix/win32_posix.cis untouched bymainsince7ff5256e) and the description moved to the current template; the stale "not measured in the game yet" line is gone, since dukenheimer's T-YHUH Deck run is in.The game-free test was run again on the new base (20 of 20 kept the normal priority at the test-only commit, 20 of 20 at nice 10 at the tip, two rounds, g++ and clang); the Deck numbers still hold: since that build (main
543846eb)mainchanged the changed file not at all andhost/hos_thread.conly in which core is kept for the replay (#522), not in the priority path.27113be72e6218c00b8aView command line instructions
Checkout
From your project repository, check out a new branch and test the changes.