fix(vk): on Linux the texture helpers started inside a replay run on every core again #499

Merged
HumanBean merged 1 commit from jarynclouatre/Longshot:fix/replay-helpers-every-core into main 2026-11-15 19:31:19 +00:00
Contributor

Type: fix · Risk: low · Tested on: Linux x86-64 (game-free checks only) · Needs: none

Why

On Linux a new thread inherits its maker's affinity. Since #445 the replaying thread runs on the reserved logical processor inside a ReplayPin (gpu/vk/vk_replay.cpp; the synchronous replay is the default). The helper threads it starts to gather a big sparse texture (guest_bytes), scan one for zeros, or unswizzle one (gpu/vk/vk_texture.cpp) then all run on that one logical processor. So does the pipeline cache save that do_present starts every 300 presents. None of them registers with host/posix/win32_posix.c, which is what gives a new thread the process's affinity (its own comment says a std::thread that never registers keeps what it inherited).

A thread that registers that way had a second problem: thread_self allocates a record that is never given back when a std::thread ends (thread_finish runs only for CreateThread's threads and ExitThread).

None filed. Found reading the threads guest_bytes starts after #445.

How to reproduce

Seen on: found reading the code; not seen in the game
Settings and switches: Default synchronous replay inside a ReplayPin.
Steps: On Linux, let a replay start helpers to gather a big sparse texture, scan one for zeros, or unswizzle one; the pipeline cache save starts every 300 presents. Check the helpers' affinity and the records left by threads that register and end.
Expected: Helpers use the process's affinity, and registered threads give their records back when they end.
Actual: The helpers inherit the replay's one logical processor.

What changed

On Linux, texture helpers and the pipeline cache save thread now use the process's affinity instead of staying on the replay's core. Threads that register with the POSIX shim also give their records back when they end.

  • renderer (gpu/vk/vk_texture.cpp, vk_texture.h): helper_thread_start(), called first on a thread the replaying thread may have made. On Linux it registers the thread (GetCurrentThreadId), which gives it the process's affinity; on Windows it does nothing, since a new thread there starts with the process's affinity. Each spawned helper (t >= 1) of the three pools calls it; part(0), run by the replaying thread itself, stays on its core.
  • renderer (gpu/vk/vk_draw.cpp, pipeline_cache_save): the save thread calls it too.
  • POSIX shim (host/posix/win32_posix.c): a thread that registers through thread_self gives its record back when it ends, through a pthread key's destructor (marks it done and drops its reference). thread_finish clears the key first, so a registered thread that calls ExitThread gives its record back once. CreateThread's threads leave the key clear, as before. The main thread keeps its record. The std::thread pools that already register (vk_draw.cpp's prepare workers, which call SetThreadPriority) stop leaving one record per thread as well.
  • The other raw thread pools in gpu/vk (vk_progcache.cpp, vk_draw.cpp's precompile_at_create, precompile_run and predict_run, vk_main.cpp's start-up threads) run at start-up or while preparing, outside a replay: not changed.
  • tests (tests/test_reserved_core.py): on Linux, real threads over host/posix/win32_posix.c as it is, with its records counted (thread_new's allocations against thread_unref's frees), under AddressSanitizer when the compiler has it. Inside the real ReplayPin, the real guest_bytes over a gpu_read spy: every helper reads with the process's mask, the replaying thread with the pin, and the pin is undone after. 200 std::threads that register and end: 200 records made, 200 given back. A CreateThread thread, and a registered thread that calls ExitThread: one record each, given back once. tests/pipeline_cache_save_test.cpp gets a stand-in for helper_thread_start.

Measurements

Verdict: the new checks fail on main and pass here; the suite has no new failures.

Rebased onto main 6c9edffe (now 6517f152) for a conflict in a test stub only; the numbers below are from b6143fc2 on 9c9d69ed.

Hardware: none used (game-free checks on a Linux x86-64 PC, gcc 15.2, Python 3.14)
Scene: none, the game was not run
Method: 2026-11-14. python3 -I -B tests/run_all.py --jobs 4, two runs each. The new checks also ran with main's gpu/vk/vk_texture.cpp, then with main's host/posix/win32_posix.c. Branch: fix/replay-helpers-every-core.
How to reproduce: the commands above in a checkout of each commit

Before: main @ 9c9d69ed After: this PR @ b6143fc2
tests/run_all.py (run 1, run 2) 485 passed, 2 failed, 83 skipped (both) 485 passed, 2 failed, 83 skipped (both)
failing checks test_bake_portability.py, test_walk_key.py the same two
tests/test_reserved_core.py, main's vk_texture.cpp helper 43892 read with mask 0x8000, not the process's 0xffff pass
tests/test_reserved_core.py, main's win32_posix.c 200 registered threads ended: 200 records made, 0 given back pass

test_bake_portability.py fails here for a missing Python module (zstandard). test_walk_key.py's fixture doesn't link on main (undefined reference to hos_oot_autoload_input).

Flags and settings

None.

Testing

Tested: the game-free suite on a Linux PC, as in Measurements. vk_texture.cpp and vk_draw.cpp pass -fsyntax-only with llvm-mingw 20261104's clang++, for Windows and for x86_64-linux-gnu (the build's defines and include folders).

Not tested: the game on Linux (no Linux game run here), and how long a big texture's gather takes on the replay's core against every core. A Linux player with 8 or more logical processors would see it in a scene with big textures.

Must be tested: None.

Risks

What could break: #490 (macOS, WIP) adds its own thread key for Darwin in the same function; whichever lands second needs a small rebase.

Turning it off: Reverting the commit puts them back on the replay's core and the records back to being kept.

Compatibility: On Linux the helpers now share every logical processor with the game's other threads, as on Windows.

Checklist

Everyone

  • One topic.
  • The title is a Conventional Commit (type(scope): description) of the status line's type.
  • Why gives the reason plainly and links the issues that fit (the tracker searched).
  • Measurements has a verdict, or one line saying why there is nothing to measure.
  • Testing names what was tested, what wasn't, and what must still be tested, where and how.
  • Rebased on current main.
  • python -I -B tests/run_all.py passes (any failure that also fails on main is named).
  • No game files, dumps, build output or generated files.
  • third_party/ untouched (a new folder with a PROVENANCE.md instead).
  • The license is OK: GPL-3.0-or-later, nothing incompatible and nothing decompiled.

When it applies

  • Numbers from the same PC, scene and settings, at least 2 interleaved runs each, the median reported.
  • Host code builds on both Windows (llvm-mingw) and Linux, with UTF-8 paths.
  • A game-free test added for the fix, failing on main and passing here.
  • A check under tests/ that reads changed source updated in this PR, not weakened.
**Type:** fix · **Risk:** low · **Tested on:** Linux x86-64 (game-free checks only) · **Needs:** none ## Why On Linux a new thread inherits its maker's affinity. Since #445 the replaying thread runs on the reserved logical processor inside a `ReplayPin` (`gpu/vk/vk_replay.cpp`; the synchronous replay is the default). The helper threads it starts to gather a big sparse texture (`guest_bytes`), scan one for zeros, or unswizzle one (`gpu/vk/vk_texture.cpp`) then all run on that one logical processor. So does the pipeline cache save that `do_present` starts every 300 presents. None of them registers with `host/posix/win32_posix.c`, which is what gives a new thread the process's affinity (its own comment says a std::thread that never registers keeps what it inherited). A thread that registers that way had a second problem: `thread_self` allocates a record that is never given back when a std::thread ends (`thread_finish` runs only for CreateThread's threads and ExitThread). None filed. Found reading the threads `guest_bytes` starts after #445. ## How to reproduce **Seen on:** found reading the code; not seen in the game **Settings and switches:** Default synchronous replay inside a `ReplayPin`. **Steps:** On Linux, let a replay start helpers to gather a big sparse texture, scan one for zeros, or unswizzle one; the pipeline cache save starts every 300 presents. Check the helpers' affinity and the records left by threads that register and end. **Expected:** Helpers use the process's affinity, and registered threads give their records back when they end. **Actual:** The helpers inherit the replay's one logical processor. ## What changed On Linux, texture helpers and the pipeline cache save thread now use the process's affinity instead of staying on the replay's core. Threads that register with the POSIX shim also give their records back when they end. - renderer (`gpu/vk/vk_texture.cpp`, `vk_texture.h`): `helper_thread_start()`, called first on a thread the replaying thread may have made. On Linux it registers the thread (`GetCurrentThreadId`), which gives it the process's affinity; on Windows it does nothing, since a new thread there starts with the process's affinity. Each spawned helper (`t >= 1`) of the three pools calls it; `part(0)`, run by the replaying thread itself, stays on its core. - renderer (`gpu/vk/vk_draw.cpp`, `pipeline_cache_save`): the save thread calls it too. - POSIX shim (`host/posix/win32_posix.c`): a thread that registers through `thread_self` gives its record back when it ends, through a pthread key's destructor (marks it done and drops its reference). `thread_finish` clears the key first, so a registered thread that calls ExitThread gives its record back once. CreateThread's threads leave the key clear, as before. The main thread keeps its record. The std::thread pools that already register (`vk_draw.cpp`'s prepare workers, which call `SetThreadPriority`) stop leaving one record per thread as well. - The other raw thread pools in `gpu/vk` (`vk_progcache.cpp`, `vk_draw.cpp`'s `precompile_at_create`, `precompile_run` and `predict_run`, `vk_main.cpp`'s start-up threads) run at start-up or while preparing, outside a replay: not changed. - tests (`tests/test_reserved_core.py`): on Linux, real threads over `host/posix/win32_posix.c` as it is, with its records counted (`thread_new`'s allocations against `thread_unref`'s frees), under AddressSanitizer when the compiler has it. Inside the real `ReplayPin`, the real `guest_bytes` over a `gpu_read` spy: every helper reads with the process's mask, the replaying thread with the pin, and the pin is undone after. 200 std::threads that register and end: 200 records made, 200 given back. A CreateThread thread, and a registered thread that calls ExitThread: one record each, given back once. `tests/pipeline_cache_save_test.cpp` gets a stand-in for `helper_thread_start`. ## Measurements **Verdict:** the new checks fail on main and pass here; the suite has no new failures. Rebased onto main 6c9edffe (now 6517f152) for a conflict in a test stub only; the numbers below are from b6143fc2 on 9c9d69ed. Hardware: none used (game-free checks on a Linux x86-64 PC, gcc 15.2, Python 3.14) Scene: none, the game was not run Method: 2026-11-14. `python3 -I -B tests/run_all.py --jobs 4`, two runs each. The new checks also ran with main's `gpu/vk/vk_texture.cpp`, then with main's `host/posix/win32_posix.c`. Branch: `fix/replay-helpers-every-core`. How to reproduce: the commands above in a checkout of each commit | | Before: main @ 9c9d69ed | After: this PR @ b6143fc2 | |---|---|---| | `tests/run_all.py` (run 1, run 2) | 485 passed, 2 failed, 83 skipped (both) | 485 passed, 2 failed, 83 skipped (both) | | failing checks | `test_bake_portability.py`, `test_walk_key.py` | the same two | | `tests/test_reserved_core.py`, main's `vk_texture.cpp` | `helper 43892 read with mask 0x8000, not the process's 0xffff` | pass | | `tests/test_reserved_core.py`, main's `win32_posix.c` | `200 registered threads ended: 200 records made, 0 given back` | pass | `test_bake_portability.py` fails here for a missing Python module (zstandard). `test_walk_key.py`'s fixture doesn't link on main (`undefined reference to hos_oot_autoload_input`). ### Flags and settings None. ## Testing **Tested:** the game-free suite on a Linux PC, as in Measurements. `vk_texture.cpp` and `vk_draw.cpp` pass `-fsyntax-only` with llvm-mingw 20261104's clang++, for Windows and for x86_64-linux-gnu (the build's defines and include folders). **Not tested:** the game on Linux (no Linux game run here), and how long a big texture's gather takes on the replay's core against every core. A Linux player with 8 or more logical processors would see it in a scene with big textures. **Must be tested:** None. ## Risks **What could break:** #490 (macOS, WIP) adds its own thread key for Darwin in the same function; whichever lands second needs a small rebase. **Turning it off:** Reverting the commit puts them back on the replay's core and the records back to being kept. **Compatibility:** On Linux the helpers now share every logical processor with the game's other threads, as on Windows. ## Checklist ### Everyone - [x] One topic. - [x] The title is a Conventional Commit (`type(scope): description`) of the status line's type. - [x] Why gives the reason plainly and links the issues that fit (the tracker searched). - [x] Measurements has a verdict, or one line saying why there is nothing to measure. - [x] Testing names what was tested, what wasn't, and what must still be tested, where and how. - [x] Rebased on current `main`. - [x] `python -I -B tests/run_all.py` passes (any failure that also fails on `main` is named). - [x] No game files, dumps, build output or generated files. - [x] `third_party/` untouched (a new folder with a `PROVENANCE.md` instead). - [x] The license is OK: GPL-3.0-or-later, nothing incompatible and nothing decompiled. ### When it applies - [x] Numbers from the same PC, scene and settings, at least 2 interleaved runs each, the median reported. - [x] Host code builds on both Windows (llvm-mingw) and Linux, with UTF-8 paths. - [x] A game-free test added for the fix, failing on `main` and passing here. - [x] A check under `tests/` that reads changed source updated in this PR, not weakened.
fix(vk): on Linux the texture helpers started inside a replay run on every core again
Some checks failed
Website / Website on Cloudflare (pull_request_target) Has been skipped
PR description / Description follows the template (pull_request_target) Successful in 5s
beanstalk/approval Test farm not requested
CI / Conventional Commit title (pull_request) Has been cancelled
CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Has been cancelled
CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Has been cancelled
CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Has been cancelled
CI / Workflow lint (actionlint) (pull_request) Has been cancelled
Third-party parts / As their notes say (pull_request) Has been cancelled
Third-party parts / The same as upstream (pull_request) Has been cancelled
b6143fc239
On Linux a new thread inherits its maker's affinity. Inside a ReplayPin the
replaying thread is on the reserved logical processor, so the helper threads
it starts to gather, scan or unswizzle a big texture all ran there too, and
so did the pipeline cache save that do_present starts every 300 presents.
None of them registers with the POSIX shim, which is what gives a new thread
the process's affinity.

Each spawned helper of the three texture pools, and the cache save thread,
now calls helper_thread_start first. On Linux that registers the thread
(GetCurrentThreadId); on Windows it does nothing, since a new thread there
starts with the process's affinity. The replaying thread's own share stays
on its core.

A thread that registers this way used to keep its record after it ended.
The shim now gives it back through a pthread key's destructor; thread_finish
clears the key first, so a registered thread that calls ExitThread gives its
record back once. The std::thread pools that already call SetThreadPriority
on Linux stop leaving one record per thread as well.
jarynclouatre changed title from WIP: fix(vk): on Linux the texture helpers started inside a replay run on every core again to fix(vk): on Linux the texture helpers started inside a replay run on every core again 2026-11-14 16:51:48 +00:00
jarynclouatre force-pushed fix/replay-helpers-every-core from b6143fc239
Some checks failed
Website / Website on Cloudflare (pull_request_target) Has been skipped
PR description / Description follows the template (pull_request_target) Successful in 5s
beanstalk/approval Test farm not requested
CI / Conventional Commit title (pull_request) Has been cancelled
CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Has been cancelled
CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Has been cancelled
CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Has been cancelled
CI / Workflow lint (actionlint) (pull_request) Has been cancelled
Third-party parts / As their notes say (pull_request) Has been cancelled
Third-party parts / The same as upstream (pull_request) Has been cancelled
to 6517f15273
Some checks are pending
CI / Conventional Commit title (pull_request) Blocked by required conditions
CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Blocked by required conditions
CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Blocked by required conditions
CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Blocked by required conditions
CI / Workflow lint (actionlint) (pull_request) Blocked by required conditions
Third-party parts / As their notes say (pull_request) Blocked by required conditions
Third-party parts / The same as upstream (pull_request) Blocked by required conditions
Website / Website on Cloudflare (pull_request) Blocked by required conditions
PR description / Description follows the template (pull_request_target) Successful in 14s
2026-11-14 23:57:12 +00:00
Compare
Author
Contributor

Rebased onto main 6c9edffe after a conflict in tests/pipeline_cache_save_test.cpp (now 6517f152): #404 added the serial builds' stubs on the line where this PR adds its helper_thread_start stub; both are kept. Nothing else changed. On the rebased branch tests/test_pipeline_cache_save.py and tests/test_reserved_core.py pass.

Rebased onto main 6c9edffe after a conflict in `tests/pipeline_cache_save_test.cpp` (now 6517f152): #404 added the serial builds' stubs on the line where this PR adds its `helper_thread_start` stub; both are kept. Nothing else changed. On the rebased branch `tests/test_pipeline_cache_save.py` and `tests/test_reserved_core.py` pass.
HumanBean approved these changes 2026-11-15 19:03:48 +00:00
HumanBean left a comment

Read the diff. helper_thread_start() calls GetCurrentThreadId(), which goes through thread_self(), so it registers the thread and applies the process affinity; it is a no-op on Windows, and the spawned helpers (t >= 1) are the only ones wrapped while part(0) stays on the replay core. The pthread-key destructor gives the record back: thread_finish clears the key before thread_done, so ExitThread does not free twice, and CreateThread's threads never set the key. vk_texture.h is included by vk_draw.cpp. The test stub in pipeline_cache_save_test.cpp and the checks that fail on main fit the description. Linux was not run in game, as the PR says; the change is small and contained. Approving.

Read the diff. `helper_thread_start()` calls `GetCurrentThreadId()`, which goes through `thread_self()`, so it registers the thread and applies the process affinity; it is a no-op on Windows, and the spawned helpers (`t >= 1`) are the only ones wrapped while `part(0)` stays on the replay core. The pthread-key destructor gives the record back: `thread_finish` clears the key before `thread_done`, so ExitThread does not free twice, and CreateThread's threads never set the key. `vk_texture.h` is included by `vk_draw.cpp`. The test stub in `pipeline_cache_save_test.cpp` and the checks that fail on main fit the description. Linux was not run in game, as the PR says; the change is small and contained. Approving.
HumanBean deleted branch fix/replay-helpers-every-core 2026-11-15 19:31:20 +00:00
Some checks are pending
CI / Conventional Commit title (pull_request) Blocked by required conditions
CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Blocked by required conditions
CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Blocked by required conditions
CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Blocked by required conditions
CI / Workflow lint (actionlint) (pull_request) Blocked by required conditions
Third-party parts / As their notes say (pull_request) Blocked by required conditions
Third-party parts / The same as upstream (pull_request) Blocked by required conditions
Website / Website on Cloudflare (pull_request) Blocked by required conditions
PR description / Description follows the template (pull_request_target) Successful in 14s
Some workflows are waiting to be reviewed.
Sign in to join this conversation.
No reviewers
No milestone
No project
No assignees
2 participants
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
MagicBeanDev/Longshot!499
No description provided.