fix(vk): on Linux the texture helpers started inside a replay run on every core again #499

Open

jarynclouatre wants to merge 1 commit from jarynclouatre/Longshot:fix/replay-helpers-every-core into main

pull from: jarynclouatre/Longshot:fix/replay-helpers-every-core

merge into: MagicBeanDev:main

MagicBeanDev:main

Conversation 0 Commits 1 Files changed 6 +209 -12

jarynclouatre commented 2026-11-14 16:50:33 +00:00

Contributor

• Copy link
• Copy Markdown

Why

On Linux a new thread inherits its maker's affinity. Since #445 the replaying thread runs on the reserved logical processor inside a ReplayPin (gpu/vk/vk_replay.cpp; the synchronous replay is the default). The helper threads it starts to gather a big sparse texture (guest_bytes), scan one for zeros, or unswizzle one (gpu/vk/vk_texture.cpp) then all run on that one logical processor. So does the pipeline cache save that do_present starts every 300 presents. None of them registers with host/posix/win32_posix.c, which is what gives a new thread the process's affinity (its own comment says a std::thread that never registers keeps what it inherited).

A thread that registers that way had a second problem: thread_self allocates a record that is never given back when a std::thread ends (thread_finish runs only for CreateThread's threads and ExitThread). Registering five helpers per big texture would leak five records each time.

What changed

• renderer (gpu/vk/vk_texture.cpp, vk_texture.h): helper_thread_start(), called first on a thread the replaying thread may have made. On Linux it registers the thread (GetCurrentThreadId), which gives it the process's affinity; on Windows it does nothing, since a new thread there starts with the process's affinity. Each spawned helper (t >= 1) of the three pools calls it; part(0), run by the replaying thread itself, stays on its core.
• renderer (gpu/vk/vk_draw.cpp, pipeline_cache_save): the save thread calls it too.
• POSIX shim (host/posix/win32_posix.c): a thread that registers through thread_self gives its record back when it ends, through a pthread key's destructor (marks it done and drops its reference). thread_finish clears the key first, so a registered thread that calls ExitThread gives its record back once. CreateThread's threads leave the key clear, as before. The main thread keeps its record. The std::thread pools that already register (vk_draw.cpp's prepare workers, which call SetThreadPriority) stop leaving one record per thread as well.
• The other raw thread pools in gpu/vk (vk_progcache.cpp, vk_draw.cpp's precompile_at_create, precompile_run and predict_run, vk_main.cpp's start-up threads) run at start-up or while preparing, outside a replay: not changed.
• tests (tests/test_reserved_core.py): on Linux, real threads over host/posix/win32_posix.c as it is, with its records counted (thread_new's allocations against thread_unref's frees), under AddressSanitizer when the compiler has it. Inside the real ReplayPin, the real guest_bytes over a gpu_read spy: every helper reads with the process's mask, the replaying thread with the pin, and the pin is undone after. 200 std::threads that register and end: 200 records made, 200 given back. A CreateThread thread, and a registered thread that calls ExitThread: one record each, given back once. tests/pipeline_cache_save_test.cpp gets a stand-in for helper_thread_start.

Flags and settings

None.

Measurements

main: 9c9d69ed (2026-11-14)
this PR: fix/replay-helpers-every-core @ b6143fc2
Hardware: none used (game-free checks on a Linux x86-64 PC, gcc 15.2, Python 3.14)
Scene: none, the game was not run
Method: python3 -I -B tests/run_all.py --jobs 4, two runs each. The new checks also ran with main's gpu/vk/vk_texture.cpp, then with main's host/posix/win32_posix.c.
How to reproduce: the commands above in a checkout of each commit

	main @ 9c9d69ed	this PR @ b6143fc2
tests/run_all.py (run 1, run 2)	485 passed, 2 failed, 83 skipped (both)	485 passed, 2 failed, 83 skipped (both)
failing checks	test_bake_portability.py, test_walk_key.py	the same two
tests/test_reserved_core.py, main's vk_texture.cpp	helper 43892 read with mask 0x8000, not the process's 0xffff	pass
tests/test_reserved_core.py, main's win32_posix.c	200 registered threads ended: 200 records made, 0 given back	pass

test_bake_portability.py fails here for a missing Python module (zstandard). test_walk_key.py's fixture doesn't link on main (undefined reference to hos_oot_autoload_input).

Testing

Tested: the game-free suite on a Linux PC, as in Measurements. vk_texture.cpp and vk_draw.cpp pass -fsyntax-only with llvm-mingw 20261104's clang++, for Windows and for x86_64-linux-gnu (the build's defines and include folders).

Not tested: the game on Linux (no Linux game run here), and how long a big texture's gather takes on the replay's core against every core. A Linux player with 8 or more logical processors would see it in a scene with big textures.

Needs other testers: None.

Screenshots

Nothing on screen.

Risk and rollback

On Linux the helpers now share every logical processor with the game's other threads, as on Windows. Reverting the commit puts them back on the replay's core and the records back to being kept. #490 (macOS, WIP) adds its own thread key for Darwin in the same function; whichever lands second needs a small rebase.

Issues

None filed. Found reading the threads guest_bytes starts after #445.

Checklist

For everyone

• One topic.
• The title is a Conventional Commit (type(scope): description).
• Rebased on current main.
• python -I -B tests/run_all.py run and passing (any failure that also fails on main is named above).
• Tested in the game, or what wasn't is named under Testing.
• Testing filled in: what was tested, what wasn't, and what needs other testers (with test requests).
• Measured: main and this PR at named commits, on the same PC, scene and settings, at least 2 interleaved runs each, the median reported.
• No game files, dumps, build output or generated files.
• third_party/ untouched (a new folder with a PROVENANCE.md instead).
• The license is OK: GPL-3.0-or-later, nothing incompatible and nothing decompiled.

If applicable

• Host code builds on both Windows (llvm-mingw) and Linux, with UTF-8 paths.
• A game-free test added for the fix, failing on main and passing here.
• A check under tests/ that reads changed source updated in this PR, not weakened.

## Why On Linux a new thread inherits its maker's affinity. Since #445 the replaying thread runs on the reserved logical processor inside a `ReplayPin` (`gpu/vk/vk_replay.cpp`; the synchronous replay is the default). The helper threads it starts to gather a big sparse texture (`guest_bytes`), scan one for zeros, or unswizzle one (`gpu/vk/vk_texture.cpp`) then all run on that one logical processor. So does the pipeline cache save that `do_present` starts every 300 presents. None of them registers with `host/posix/win32_posix.c`, which is what gives a new thread the process's affinity (its own comment says a std::thread that never registers keeps what it inherited). A thread that registers that way had a second problem: `thread_self` allocates a record that is never given back when a std::thread ends (`thread_finish` runs only for CreateThread's threads and ExitThread). Registering five helpers per big texture would leak five records each time. ## What changed - renderer (`gpu/vk/vk_texture.cpp`, `vk_texture.h`): `helper_thread_start()`, called first on a thread the replaying thread may have made. On Linux it registers the thread (`GetCurrentThreadId`), which gives it the process's affinity; on Windows it does nothing, since a new thread there starts with the process's affinity. Each spawned helper (`t >= 1`) of the three pools calls it; `part(0)`, run by the replaying thread itself, stays on its core. - renderer (`gpu/vk/vk_draw.cpp`, `pipeline_cache_save`): the save thread calls it too. - POSIX shim (`host/posix/win32_posix.c`): a thread that registers through `thread_self` gives its record back when it ends, through a pthread key's destructor (marks it done and drops its reference). `thread_finish` clears the key first, so a registered thread that calls ExitThread gives its record back once. CreateThread's threads leave the key clear, as before. The main thread keeps its record. The std::thread pools that already register (`vk_draw.cpp`'s prepare workers, which call `SetThreadPriority`) stop leaving one record per thread as well. - The other raw thread pools in `gpu/vk` (`vk_progcache.cpp`, `vk_draw.cpp`'s `precompile_at_create`, `precompile_run` and `predict_run`, `vk_main.cpp`'s start-up threads) run at start-up or while preparing, outside a replay: not changed. - tests (`tests/test_reserved_core.py`): on Linux, real threads over `host/posix/win32_posix.c` as it is, with its records counted (`thread_new`'s allocations against `thread_unref`'s frees), under AddressSanitizer when the compiler has it. Inside the real `ReplayPin`, the real `guest_bytes` over a `gpu_read` spy: every helper reads with the process's mask, the replaying thread with the pin, and the pin is undone after. 200 std::threads that register and end: 200 records made, 200 given back. A CreateThread thread, and a registered thread that calls ExitThread: one record each, given back once. `tests/pipeline_cache_save_test.cpp` gets a stand-in for `helper_thread_start`. ## Flags and settings None. ## Measurements main: 9c9d69ed (2026-11-14) this PR: fix/replay-helpers-every-core @ b6143fc2 Hardware: none used (game-free checks on a Linux x86-64 PC, gcc 15.2, Python 3.14) Scene: none, the game was not run Method: `python3 -I -B tests/run_all.py --jobs 4`, two runs each. The new checks also ran with main's `gpu/vk/vk_texture.cpp`, then with main's `host/posix/win32_posix.c`. How to reproduce: the commands above in a checkout of each commit | | main @ 9c9d69ed | this PR @ b6143fc2 | |---|---|---| | `tests/run_all.py` (run 1, run 2) | 485 passed, 2 failed, 83 skipped (both) | 485 passed, 2 failed, 83 skipped (both) | | failing checks | `test_bake_portability.py`, `test_walk_key.py` | the same two | | `tests/test_reserved_core.py`, main's `vk_texture.cpp` | `helper 43892 read with mask 0x8000, not the process's 0xffff` | pass | | `tests/test_reserved_core.py`, main's `win32_posix.c` | `200 registered threads ended: 200 records made, 0 given back` | pass | `test_bake_portability.py` fails here for a missing Python module (zstandard). `test_walk_key.py`'s fixture doesn't link on main (`undefined reference to hos_oot_autoload_input`). ## Testing **Tested:** the game-free suite on a Linux PC, as in Measurements. `vk_texture.cpp` and `vk_draw.cpp` pass `-fsyntax-only` with llvm-mingw 20261104's clang++, for Windows and for x86_64-linux-gnu (the build's defines and include folders). **Not tested:** the game on Linux (no Linux game run here), and how long a big texture's gather takes on the replay's core against every core. A Linux player with 8 or more logical processors would see it in a scene with big textures. **Needs other testers:** None. ## Screenshots Nothing on screen. ## Risk and rollback On Linux the helpers now share every logical processor with the game's other threads, as on Windows. Reverting the commit puts them back on the replay's core and the records back to being kept. #490 (macOS, WIP) adds its own thread key for Darwin in the same function; whichever lands second needs a small rebase. ## Issues None filed. Found reading the threads `guest_bytes` starts after #445. ## Checklist ### For everyone - [x] One topic. - [x] The title is a Conventional Commit (`type(scope): description`). - [x] Rebased on current `main`. - [x] `python -I -B tests/run_all.py` run and passing (any failure that also fails on `main` is named above). - [x] Tested in the game, or what wasn't is named under Testing. - [x] Testing filled in: what was tested, what wasn't, and what needs other testers (with test requests). - [x] Measured: main and this PR at named commits, on the same PC, scene and settings, at least 2 interleaved runs each, the median reported. - [x] No game files, dumps, build output or generated files. - [x] `third_party/` untouched (a new folder with a `PROVENANCE.md` instead). - [x] The license is OK: GPL-3.0-or-later, nothing incompatible and nothing decompiled. ### If applicable - [x] Host code builds on both Windows (llvm-mingw) and Linux, with UTF-8 paths. - [x] A game-free test added for the fix, failing on `main` and passing here. - [x] A check under `tests/` that reads changed source updated in this PR, not weakened.

jarynclouatre added 1 commit 2026-11-14 16:50:33 +00:00

fix(vk): on Linux the texture helpers started inside a replay run on every core again ... b6143fc239

On Linux a new thread inherits its maker's affinity. Inside a ReplayPin the
replaying thread is on the reserved logical processor, so the helper threads
it starts to gather, scan or unswizzle a big texture all ran there too, and
so did the pipeline cache save that do_present starts every 300 presents.
None of them registers with the POSIX shim, which is what gives a new thread
the process's affinity.

Each spawned helper of the three texture pools, and the cache save thread,
now calls helper_thread_start first. On Linux that registers the thread
(GetCurrentThreadId); on Windows it does nothing, since a new thread there
starts with the process's affinity. The replaying thread's own share stays
on its core.

A thread that registers this way used to keep its record after it ended.
The shim now gives it back through a pthread key's destructor; thread_finish
clears the key first, so a registered thread that calls ExitThread gives its
record back once. The std::thread pools that already call SetThreadPriority
on Linux stop leaving one record per thread as well.

jarynclouatre changed title from WIP: fix(vk): on Linux the texture helpers started inside a replay run on every core again to fix(vk): on Linux the texture helpers started inside a replay run on every core again 2026-11-14 16:51:48 +00:00

Some checks are pending

Hide checks

CI / Conventional Commit title (pull_request) Blocked by required conditions

Details

CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Blocked by required conditions

Details

CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Blocked by required conditions

Details

CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Blocked by required conditions

Details

CI / Workflow lint (actionlint) (pull_request) Blocked by required conditions

Details

Third-party parts / As their notes say (pull_request) Blocked by required conditions

Details

Third-party parts / The same as upstream (pull_request) Blocked by required conditions

Details

PR description / Description follows the template (pull_request_target) Successful in 5s

Details

This pull request doesn't have enough approvals yet. 0 of 1 approvals granted.

Some workflows are waiting to be reviewed.

This branch is out-of-date with the base branch

You are not authorized to merge this pull request.

View command line instructions

Checkout

From your project repository, check out a new branch and test the changes.

git fetch -u 'fix/replay-helpers-every-core:jarynclouatre-fix/replay-helpers-every-core'

git switch 'jarynclouatre-fix/replay-helpers-every-core'

Sign in to join this conversation.

Reviewers

No reviewers

Labels

Clear labels   

area/accessibility

Barriers for people with disabilities

  

area/audio

Sound and music

  

area/input

Controllers, keyboard, mouse

  

area/install

Installing, updating, building from your game

  

area/menu

In-game menu and settings

  

area/renderer

Vulkan renderer, shaders, effects

  

area/saves

Save files

  

area/upscaling

DLSS, FSR, frame generation

  

good first issue

A good start for a new contributor

  

gpu/amd

AMD GPUs

  

gpu/igpu

Integrated GPUs

  

gpu/intel

Intel GPUs

  

gpu/nvidia

NVIDIA GPUs

  

help wanted

Extra hands or knowledge wanted

  

kind

bug

Something doesn't work as it should

  

kind

chore

Build, CI, dependencies, tidying

  

kind

docs

Documentation

  

kind

feature

A new feature or an improvement

  

kind

performance

Frame rate, frame time, loading, memory

  

kind

question

A question rather than a change

  

needs measurements

Waiting for before/after measurements

  

platform/linux

Linux only

  

platform/steam-deck

Steam Deck

  

platform/windows

Windows only

  

preview

Maintainers: deploy a preview of this PR's website (read its changes first). Removed after the run.

  

priority

critical

Crashes, data loss, or blocks most players

  

priority

high

Important: next in line

  

priority

low

Nice to have

  

status

blocked

Waits on something else first

  

status

confirmed

Reproduced or agreed on

  

status

duplicate

Already reported or proposed

  

status

invalid

Not a problem in this project

  

status

needs-info

Waiting for more information from the reporter

  

status

wontfix

Will not be worked on

  

testing wanted

Wanted: a test on other hardware (posts a test request in #test-requests)

No labels

area/accessibility

area/audio

area/input

area/install

area/menu

area/renderer

area/saves

area/upscaling

good first issue

gpu/amd

gpu/igpu

gpu/intel

gpu/nvidia

help wanted

kind

bug

kind

chore

kind

docs

kind

feature

kind

performance

kind

question

needs measurements

platform/linux

platform/steam-deck

platform/windows

preview

priority

critical

priority

high

priority

low

status

blocked

status

confirmed

status

duplicate

status

invalid

status

needs-info

status

wontfix

testing wanted

Milestone

Clear milestone

No items

No milestone

Projects

Clear projects

No items

No project

Assignees

Clear assignees

No assignees

1 participant

Notifications

Subscribe

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference

MagicBeanDev/Longshot!499

No description provided.