fix(vk): on Linux the texture helpers started inside a replay run on every core again #499
No reviewers
Labels
No labels
area/accessibility
area/audio
area/input
area/install
area/menu
area/renderer
area/saves
area/tooling
area/upscaling
beanstalk: approved
beanstalk: failed
beanstalk: needs approval
beanstalk: running
beanstalk: tested
ci: approved
good first issue
gpu/amd
gpu/igpu
gpu/intel
gpu/nvidia
help wanted
kind
bug
kind
chore
kind
docs
kind
feature
kind
performance
kind
question
needs measurements
platform/linux
platform/macos
platform/steam-deck
platform/windows
preview
priority
critical
priority
high
priority
low
regression
risk
high
risk
low
risk
medium
status
blocked
status
confirmed
status
duplicate
status
invalid
status
needs-info
status
wontfix
ci: approved
No milestone
No project
No assignees
2 participants
Notifications
Due date
No due date set.
Dependencies
No dependencies set
Reference
MagicBeanDev/Longshot!499
Loading…
Reference in a new issue
No description provided.
Delete branch "jarynclouatre/Longshot:fix/replay-helpers-every-core"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Type: fix · Risk: low · Tested on: Linux x86-64 (game-free checks only) · Needs: none
Why
On Linux a new thread inherits its maker's affinity. Since #445 the replaying thread runs on the reserved logical processor inside a
ReplayPin(gpu/vk/vk_replay.cpp; the synchronous replay is the default). The helper threads it starts to gather a big sparse texture (guest_bytes), scan one for zeros, or unswizzle one (gpu/vk/vk_texture.cpp) then all run on that one logical processor. So does the pipeline cache save thatdo_presentstarts every 300 presents. None of them registers withhost/posix/win32_posix.c, which is what gives a new thread the process's affinity (its own comment says a std::thread that never registers keeps what it inherited).A thread that registers that way had a second problem:
thread_selfallocates a record that is never given back when a std::thread ends (thread_finishruns only for CreateThread's threads and ExitThread).None filed. Found reading the threads
guest_bytesstarts after #445.How to reproduce
Seen on: found reading the code; not seen in the game
Settings and switches: Default synchronous replay inside a
ReplayPin.Steps: On Linux, let a replay start helpers to gather a big sparse texture, scan one for zeros, or unswizzle one; the pipeline cache save starts every 300 presents. Check the helpers' affinity and the records left by threads that register and end.
Expected: Helpers use the process's affinity, and registered threads give their records back when they end.
Actual: The helpers inherit the replay's one logical processor.
What changed
On Linux, texture helpers and the pipeline cache save thread now use the process's affinity instead of staying on the replay's core. Threads that register with the POSIX shim also give their records back when they end.
gpu/vk/vk_texture.cpp,vk_texture.h):helper_thread_start(), called first on a thread the replaying thread may have made. On Linux it registers the thread (GetCurrentThreadId), which gives it the process's affinity; on Windows it does nothing, since a new thread there starts with the process's affinity. Each spawned helper (t >= 1) of the three pools calls it;part(0), run by the replaying thread itself, stays on its core.gpu/vk/vk_draw.cpp,pipeline_cache_save): the save thread calls it too.host/posix/win32_posix.c): a thread that registers throughthread_selfgives its record back when it ends, through a pthread key's destructor (marks it done and drops its reference).thread_finishclears the key first, so a registered thread that calls ExitThread gives its record back once. CreateThread's threads leave the key clear, as before. The main thread keeps its record. The std::thread pools that already register (vk_draw.cpp's prepare workers, which callSetThreadPriority) stop leaving one record per thread as well.gpu/vk(vk_progcache.cpp,vk_draw.cpp'sprecompile_at_create,precompile_runandpredict_run,vk_main.cpp's start-up threads) run at start-up or while preparing, outside a replay: not changed.tests/test_reserved_core.py): on Linux, real threads overhost/posix/win32_posix.cas it is, with its records counted (thread_new's allocations againstthread_unref's frees), under AddressSanitizer when the compiler has it. Inside the realReplayPin, the realguest_bytesover agpu_readspy: every helper reads with the process's mask, the replaying thread with the pin, and the pin is undone after. 200 std::threads that register and end: 200 records made, 200 given back. A CreateThread thread, and a registered thread that calls ExitThread: one record each, given back once.tests/pipeline_cache_save_test.cppgets a stand-in forhelper_thread_start.Measurements
Verdict: the new checks fail on main and pass here; the suite has no new failures.
Rebased onto main
6c9edffe(now6517f152) for a conflict in a test stub only; the numbers below are fromb6143fc2on9c9d69ed.Hardware: none used (game-free checks on a Linux x86-64 PC, gcc 15.2, Python 3.14)
Scene: none, the game was not run
Method: 2026-11-14.
python3 -I -B tests/run_all.py --jobs 4, two runs each. The new checks also ran with main'sgpu/vk/vk_texture.cpp, then with main'shost/posix/win32_posix.c. Branch:fix/replay-helpers-every-core.How to reproduce: the commands above in a checkout of each commit
9c9d69edb6143fc2tests/run_all.py(run 1, run 2)test_bake_portability.py,test_walk_key.pytests/test_reserved_core.py, main'svk_texture.cpphelper 43892 read with mask 0x8000, not the process's 0xfffftests/test_reserved_core.py, main'swin32_posix.c200 registered threads ended: 200 records made, 0 given backtest_bake_portability.pyfails here for a missing Python module (zstandard).test_walk_key.py's fixture doesn't link on main (undefined reference to hos_oot_autoload_input).Flags and settings
None.
Testing
Tested: the game-free suite on a Linux PC, as in Measurements.
vk_texture.cppandvk_draw.cpppass-fsyntax-onlywith llvm-mingw 20261104's clang++, for Windows and for x86_64-linux-gnu (the build's defines and include folders).Not tested: the game on Linux (no Linux game run here), and how long a big texture's gather takes on the replay's core against every core. A Linux player with 8 or more logical processors would see it in a scene with big textures.
Must be tested: None.
Risks
What could break: #490 (macOS, WIP) adds its own thread key for Darwin in the same function; whichever lands second needs a small rebase.
Turning it off: Reverting the commit puts them back on the replay's core and the records back to being kept.
Compatibility: On Linux the helpers now share every logical processor with the game's other threads, as on Windows.
Checklist
Everyone
type(scope): description) of the status line's type.main.python -I -B tests/run_all.pypasses (any failure that also fails onmainis named).third_party/untouched (a new folder with aPROVENANCE.mdinstead).When it applies
mainand passing here.tests/that reads changed source updated in this PR, not weakened.WIP: fix(vk): on Linux the texture helpers started inside a replay run on every core againto fix(vk): on Linux the texture helpers started inside a replay run on every core againb6143fc2396517f15273Rebased onto main
6c9edffeafter a conflict intests/pipeline_cache_save_test.cpp(now6517f152): #404 added the serial builds' stubs on the line where this PR adds itshelper_thread_startstub; both are kept. Nothing else changed. On the rebased branchtests/test_pipeline_cache_save.pyandtests/test_reserved_core.pypass.Read the diff.
helper_thread_start()callsGetCurrentThreadId(), which goes throughthread_self(), so it registers the thread and applies the process affinity; it is a no-op on Windows, and the spawned helpers (t >= 1) are the only ones wrapped whilepart(0)stays on the replay core. The pthread-key destructor gives the record back:thread_finishclears the key beforethread_done, so ExitThread does not free twice, and CreateThread's threads never set the key.vk_texture.his included byvk_draw.cpp. The test stub inpipeline_cache_save_test.cppand the checks that fail on main fit the description. Linux was not run in game, as the PR says; the change is small and contained. Approving.