WIP: fix(os): a thread's name is never written over: a rename publishes a kept name #2020

Draft
siantm wants to merge 5 commits from siantm/Longshot:fix/os-thread-names-kept into main
Contributor

Summary

A guest thread's name can change while other threads read it. hos_thread_set_name (the guest's SetThreadName, which
may name any thread) wrote the new name with snprintf into the thread's name buffer, the same buffer every name
reader gets from hos_thread_name (the log's [hos <thread>] prefix, Tracy, the profiler's thread list, the
HOS_CHAIN file, the vcore lines, the waits' diagnostics). A reader during a rename could read half of the old name and
half of the new, and a name it held changed under it.

Names are now never written over. name_kept keeps one copy of each different name for the rest of the run (in 256
lists by a hash, under a lock taken only at a thread's creation and at a rename; games name a few dozen threads), and a
thread's name is a pointer to one: set when the thread is made and, at a rename, published with one atomic release
store; hos_thread_name loads it with acquire. Every reader in host/hos_thread.c goes through hos_thread_name, and
readers elsewhere already did. The result: a reader on any thread reads a whole name, the old one or the new, and a name
it took stays as it was.

After BeanBot's review of this PR (64191):

  • A kept name holds its role (one of the frame's serial threads: serial_name), so a thread's name and its role are
    always one rename's; before, two renames at once could leave a serial thread's name with a worker's spin.
  • A thread's start and its renames give the system its name (SetThreadDescription, HOS_PIN / HOS_PRIO through
    hos_thread_pin) under one lock, the start storing the handle and a rename publishing its name under it: the system
    gets the names in the order they were published (before, a start could give it a stale name after a rename).
  • A name is at most 47 bytes cut before a character they would cut, with '?' for a byte that is no part of a valid
    UTF-8 character (before, a cut could leave a lead byte, which the Linux layer's MultiByteToWideChar read past).
  • A rename reads the guest's name 47 bytes at most, never further, and none whose 47 bytes are not all in the guest
    window (it logs the address and keeps the name).
  • A new name's copy is allocated without the lock; the log says how many different names are kept at 64, 128, 256, ...
  • A grant's long-wait cause (vc_by, HOS_VCORE_LOG_MS's lines) keeps the releasing thread's kept name, not a copy.
  • tests/test_vcore_escape.py and tests/test_vcore_mask0.py, which had not compiled since #1553 and #980, build and
    run again, without the caller's HOS_ switches; the escape fixture checks the grant's cause across a rename.

How to see it on main: python -I -B tests/test_thread_rename.py: a reader on another thread during a rename halted
halfway reads a mix of the two names, a name taken before a rename reads the new one after it, two renames at once leave
one's name with the other's role, a rename during a start leaves the system the start's stale name, a cut name ends in
a lead byte, and a name before an unreadable page faults (see Testing). In the game it would show as a garbled thread
name in a log line or a profile, rarely, or a frame thread with a worker's short spin.

Found in #1953's review: BeanBot's Blocking 2
("Concurrent SetThreadName can also race with the Tracy and long-wait name reads"), which #1953 left on main for a
separate change (its description and siantm's reply there name it).

Read first: name_kept, os_name, hos_thread_name, hos_thread_start and hos_thread_set_name in
host/hos_thread.c; tests/test_thread_rename.py.

Overlap: #1967 (feat/wake-lat-resleeps @ 4d2ebc2a) takes a lock around the rename's write and adds
hos_thread_name_copy, a copy taken under that lock, for its own rows. That makes its one reader safe; this makes every
reader safe, so the lock is no longer needed. The two conflict next to hos_thread_name and in hos_thread_set_name
(git merge-tree on this head). Proposed order: this first; #1967 is then rebased, dropping its lock (its copy can read
hos_thread_name, or keep the pointer, which lasts). If #1967 lands first, this PR is rebased and removes that lock
instead. #1953 reads names with hos_thread_name and merges with this without a conflict. #2019 is stacked on this PR:
its recorder keeps the kept name pointers hos_thread_name gives.

Left as on main: Tracy only takes a name a thread gives itself (HT_THREAD_NAME at its start and at its own renames),
so a rename from another thread after the start never reaches Tracy; the lock here does not change that.

Testing

Tested: Windows 11, i7-9700, RTX 4060 (NVIDIA 610.88), llvm-mingw clang 23.1.5 and MSYS2's MinGW-w64 GCC 14.2.0;
game-free checks only. This has NOT run in the game on this PC, nor on Linux (Still to test). Rebased onto main @
b62932a9 without a conflict, its first two commits unchanged (git range-diff: now a1491edf and eeee7e72). Each check
ran in a hidden console of its own, in build jobs of the PC's job queue: fb2-2020-1 (test commit 02a0276d, fix
008b1771) and fb2-2020-2 (test commit 1ac90685, head c29a9378: 008b1771's host code, with publish renaming on until
the reader has seen new names, after fb2-2020-1's suite run showed it could see none on a busy PC).

  • python -I -B tests/test_thread_rename.py, with clang and with GCC: at the head "PASS: every case (mid, held,
    publish, two, start, utf8, bounded, names)"; fb2-2020-2 ran it four times at once too, each passing, and the PR's six checks through run_all (8 at once): all pass. At the test commit (eeee7e72's code) two, start, utf8 and
    bounded fail with both compilers: "two: the name is "Presentation Thread" but its role is another thread's",
    "start: the name is "Renamed at start" but the system's is "Thread_7100"", six of the ten utf8 names kept wrong (46
    'A' and a lone 0xF0 byte, invalid bytes kept), and bounded ended with an access violation (0xC0000005). On main
    (fb2-2020-1, the check copied into a main worktree) mid ("NewName_012345ghijkl"), held, two, start, utf8
    and bounded fail. publish passed everywhere here without ThreadSanitizer (not available for Windows targets): its
    control did not run.
  • tests/test_vcore_escape.py and tests/test_vcore_mask0.py, with clang++ and with g++: "ok" at the head and at the
    test commit (with vc_by still a copy); on main they stop at their compile errors (hos_fps_timing redefined;
    env_on_or undeclared).
  • tests/test_spin_by_role.py (clang, GCC) and tests/test_log_levels.py: pass.
  • host/hos_thread.c compiled (-fsyntax-only) with the host build's own flags, with and without Tracy: no error.
  • python -I -B tests/run_all.py --strict --changed origin/main on 008b1771 (57 checks read the changed files; the
    Vulkan headers found through RX_CPM): 46 passed, 9 failed, 2 skipped (test_posix_thread_priority.py,
    test_tracy_funcs.py: Linux only). Of the 9, test_thread_rename.py's publish read no new name with 8 checks
    running at once (fixed in c29a9378: fb2-2020-2 ran it four at once, each passing) and test_invalidate_vcore.py's compile
    hit its own 60 s limit under that load (fb2-2020-2: passes); the other 7 (test_crash_record.py, test_dev_packs.py,
    test_host_guest_writes.py, test_libc_hooks.py, test_mods_list.py, test_profiling_mode.py,
    test_slow_frames.py) fail on main @ b62932a9 too, run by name from a main worktree in the same job, with the
    same messages.
  • Not run: mise run lint (ruff and pre-commit are not installed on this PC) and the whole suite (mise run test).

Still to test:

  • Windows 11, this PC (i7-9700, RTX 4060 (NVIDIA 610.88)): start the game to Castle Town's market and play two minutes,
    this PR against main, expecting the same thread names in longshot_log.txt's [hos <thread>] prefixes (MainThread,
    the game's own names after it names its threads), no "threads: N different thread names kept" line below 64 names,
    no stall, and median and tail frame times within the runs' spread over repeated matched runs (by hand)
  • Windows 11, this PC: the same with HOS_PROFILE=1 and with LONGSHOT_TRACY built in, expecting the profile's and
    Tracy's thread names as on main (by hand)
  • Windows 11, this PC: a session of an hour, expecting the log's "threads: N different thread names kept" lines (if
    any) to say how many different names the game gives (by hand)
  • Linux: build the host and run python3 -I -B tests/test_thread_rename.py --cc clang and --cc gcc, expecting it to
    pass, publish under ThreadSanitizer included and its control failing (by hand)
  • Linux with an AMD CPU and the Steam Deck: Castle Town's market, two minutes, this PR against main, expecting the same
    thread names in the log and frame times within the runs' spread (the Test Farm: play Castle Town's market for two
    minutes on Linux with an AMD CPU and the Steam Deck, this PR against main, comparing the log's thread names and frame
    times)

Measurements

No game measurement yet: the game runs are in Still to test. The fix's before-and-after proof is the game-free
fixture tests/test_thread_rename.py, run on this PC (i7-9700, Windows 11) in the queue jobs named in Tested, with
clang and with GCC:

case Before: main @ b62932a9 Before: this PR @ eeee7e72 (before the review) After: this PR @ c29a9378
mid: a reader during a rename FAIL ("NewName_012345ghijkl") pass pass
held: a name taken before a rename FAIL pass pass
two: two renames at once FAIL FAIL pass
start: a rename during the start FAIL FAIL pass
utf8: ten names cut at 47 bytes FAIL FAIL pass
bounded: a name before an unreadable page FAIL (access violation) FAIL (access violation) pass
names pass pass pass
publish (without ThreadSanitizer) pass pass pass

(main with clang only.) What the change adds at run time, not measured: reading a name is one atomic load (as at
eeee7e72), and a thread's role one more load from its kept name, on every wait's spin (hos_spin_ns); a start and a
rename hold a lock over a few system calls; a grant stores a pointer instead of copying 48 bytes. Whether any of it
shows in frame times is for the game runs in Still to test.

Verdict: every case that failed on main or at eeee7e72 passes at c29a9378 (game-free fixture); the game's frame
times are not measured yet.

Risks

A name is now a pointer: a name that is never set would be NULL, but every thread gets one when it is made (MainThread,
Thread_<entry>), as on main. Each different name stays in memory for the run, 64 bytes each (a game that gave every
new thread a new name would keep one per thread it made; the log counts them from 64 on). A name with bytes that are not
UTF-8 is kept with '?' for them, in the log and the system's name alike. The start and a rename now hold a lock while
they call SetThreadDescription and hos_thread_pin (a few system calls): a rename of another thread waits for them.
No switch: a revert gives back main's code. Players notice nothing after updating.

Checklist

Ready for review

  • One topic, on current main; the title is a Conventional Commit.
  • Tested on my own PC, in the game for a change the game runs, as Testing says.
  • mise run test passes (any failure that also fails on main is named).
  • Measured against main, or Measurements says why there is nothing to measure.
  • No game files, build output or hand-edited generated files; third_party/ untouched; nothing GPL-incompatible.

When it applies

  • A fix: a game-free test that fails on main and passes here.
  • A check under tests/ that reads changed source is updated, not weakened.
  • Host code builds on Windows (llvm-mingw) and Linux, with UTF-8 paths.
  • AI assistance recorded with Assisted-by: / Co-Authored-By: trailers.

Ready to merge

  • Everything in Still to test has run, its results in Tested.
  • Tier 1 hardware it can affect is tested (docs/dev/standards.md, "Hardware tiers").
  • Approved in review, every comment answered.
  • CI passes on the head; on current main, measured again or a comment says why the numbers still hold.
## Summary A guest thread's name can change while other threads read it. `hos_thread_set_name` (the guest's `SetThreadName`, which may name any thread) wrote the new name with `snprintf` into the thread's name buffer, the same buffer every name reader gets from `hos_thread_name` (the log's `[hos <thread>]` prefix, Tracy, the profiler's thread list, the `HOS_CHAIN` file, the vcore lines, the waits' diagnostics). A reader during a rename could read half of the old name and half of the new, and a name it held changed under it. Names are now never written over. `name_kept` keeps one copy of each different name for the rest of the run (in 256 lists by a hash, under a lock taken only at a thread's creation and at a rename; games name a few dozen threads), and a thread's name is a pointer to one: set when the thread is made and, at a rename, published with one atomic release store; `hos_thread_name` loads it with acquire. Every reader in host/hos_thread.c goes through `hos_thread_name`, and readers elsewhere already did. The result: a reader on any thread reads a whole name, the old one or the new, and a name it took stays as it was. After BeanBot's review of this PR ([64191](https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/2020#issuecomment-64191)): - A kept name holds its role (one of the frame's serial threads: `serial_name`), so a thread's name and its role are always one rename's; before, two renames at once could leave a serial thread's name with a worker's spin. - A thread's start and its renames give the system its name (`SetThreadDescription`, HOS_PIN / HOS_PRIO through `hos_thread_pin`) under one lock, the start storing the handle and a rename publishing its name under it: the system gets the names in the order they were published (before, a start could give it a stale name after a rename). - A name is at most 47 bytes cut before a character they would cut, with '?' for a byte that is no part of a valid UTF-8 character (before, a cut could leave a lead byte, which the Linux layer's `MultiByteToWideChar` read past). - A rename reads the guest's name 47 bytes at most, never further, and none whose 47 bytes are not all in the guest window (it logs the address and keeps the name). - A new name's copy is allocated without the lock; the log says how many different names are kept at 64, 128, 256, ... - A grant's long-wait cause (`vc_by`, HOS_VCORE_LOG_MS's lines) keeps the releasing thread's kept name, not a copy. - `tests/test_vcore_escape.py` and `tests/test_vcore_mask0.py`, which had not compiled since #1553 and #980, build and run again, without the caller's HOS_ switches; the escape fixture checks the grant's cause across a rename. How to see it on `main`: `python -I -B tests/test_thread_rename.py`: a reader on another thread during a rename halted halfway reads a mix of the two names, a name taken before a rename reads the new one after it, two renames at once leave one's name with the other's role, a rename during a start leaves the system the start's stale name, a cut name ends in a lead byte, and a name before an unreadable page faults (see Testing). In the game it would show as a garbled thread name in a log line or a profile, rarely, or a frame thread with a worker's short spin. Found in #1953's review: BeanBot's [Blocking 2](https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/1953#issuecomment-61950) ("Concurrent SetThreadName can also race with the Tracy and long-wait name reads"), which #1953 left on `main` for a separate change (its description and siantm's reply there name it). Read first: `name_kept`, `os_name`, `hos_thread_name`, `hos_thread_start` and `hos_thread_set_name` in `host/hos_thread.c`; `tests/test_thread_rename.py`. Overlap: #1967 (`feat/wake-lat-resleeps` @ 4d2ebc2a) takes a lock around the rename's write and adds `hos_thread_name_copy`, a copy taken under that lock, for its own rows. That makes its one reader safe; this makes every reader safe, so the lock is no longer needed. The two conflict next to `hos_thread_name` and in `hos_thread_set_name` (`git merge-tree` on this head). Proposed order: this first; #1967 is then rebased, dropping its lock (its copy can read `hos_thread_name`, or keep the pointer, which lasts). If #1967 lands first, this PR is rebased and removes that lock instead. #1953 reads names with `hos_thread_name` and merges with this without a conflict. #2019 is stacked on this PR: its recorder keeps the kept name pointers `hos_thread_name` gives. Left as on `main`: Tracy only takes a name a thread gives itself (`HT_THREAD_NAME` at its start and at its own renames), so a rename from another thread after the start never reaches Tracy; the lock here does not change that. ## Testing **Tested:** Windows 11, i7-9700, RTX 4060 (NVIDIA 610.88), llvm-mingw clang 23.1.5 and MSYS2's MinGW-w64 GCC 14.2.0; game-free checks only. This has NOT run in the game on this PC, nor on Linux (Still to test). Rebased onto `main` @ b62932a9 without a conflict, its first two commits unchanged (`git range-diff`: now a1491edf and eeee7e72). Each check ran in a hidden console of its own, in build jobs of the PC's job queue: fb2-2020-1 (test commit 02a0276d, fix 008b1771) and fb2-2020-2 (test commit 1ac90685, head c29a9378: 008b1771's host code, with `publish` renaming on until the reader has seen new names, after fb2-2020-1's suite run showed it could see none on a busy PC). - `python -I -B tests/test_thread_rename.py`, with clang and with GCC: at the head "PASS: every case (mid, held, publish, two, start, utf8, bounded, names)"; fb2-2020-2 ran it four times at once too, each passing, and the PR's six checks through run_all (8 at once): all pass. At the test commit (eeee7e72's code) `two`, `start`, `utf8` and `bounded` fail with both compilers: "two: the name is "Presentation Thread" but its role is another thread's", "start: the name is "Renamed at start" but the system's is "Thread_7100"", six of the ten utf8 names kept wrong (46 'A' and a lone 0xF0 byte, invalid bytes kept), and `bounded` ended with an access violation (0xC0000005). On `main` (fb2-2020-1, the check copied into a `main` worktree) `mid` ("NewName_012345ghijkl"), `held`, `two`, `start`, `utf8` and `bounded` fail. `publish` passed everywhere here without ThreadSanitizer (not available for Windows targets): its control did not run. - `tests/test_vcore_escape.py` and `tests/test_vcore_mask0.py`, with clang++ and with g++: "ok" at the head and at the test commit (with `vc_by` still a copy); on `main` they stop at their compile errors (`hos_fps_timing` redefined; `env_on_or` undeclared). - `tests/test_spin_by_role.py` (clang, GCC) and `tests/test_log_levels.py`: pass. - `host/hos_thread.c` compiled (`-fsyntax-only`) with the host build's own flags, with and without Tracy: no error. - `python -I -B tests/run_all.py --strict --changed origin/main` on 008b1771 (57 checks read the changed files; the Vulkan headers found through `RX_CPM`): 46 passed, 9 failed, 2 skipped (`test_posix_thread_priority.py`, `test_tracy_funcs.py`: Linux only). Of the 9, `test_thread_rename.py`'s `publish` read no new name with 8 checks running at once (fixed in c29a9378: fb2-2020-2 ran it four at once, each passing) and `test_invalidate_vcore.py`'s compile hit its own 60 s limit under that load (fb2-2020-2: passes); the other 7 (`test_crash_record.py`, `test_dev_packs.py`, `test_host_guest_writes.py`, `test_libc_hooks.py`, `test_mods_list.py`, `test_profiling_mode.py`, `test_slow_frames.py`) fail on `main` @ b62932a9 too, run by name from a `main` worktree in the same job, with the same messages. - Not run: `mise run lint` (ruff and pre-commit are not installed on this PC) and the whole suite (`mise run test`). **Still to test:** - Windows 11, this PC (i7-9700, RTX 4060 (NVIDIA 610.88)): start the game to Castle Town's market and play two minutes, this PR against `main`, expecting the same thread names in `longshot_log.txt`'s `[hos <thread>]` prefixes (MainThread, the game's own names after it names its threads), no "threads: N different thread names kept" line below 64 names, no stall, and median and tail frame times within the runs' spread over repeated matched runs (by hand) - Windows 11, this PC: the same with `HOS_PROFILE=1` and with `LONGSHOT_TRACY` built in, expecting the profile's and Tracy's thread names as on `main` (by hand) - Windows 11, this PC: a session of an hour, expecting the log's "threads: N different thread names kept" lines (if any) to say how many different names the game gives (by hand) - Linux: build the host and run `python3 -I -B tests/test_thread_rename.py --cc clang` and `--cc gcc`, expecting it to pass, `publish` under ThreadSanitizer included and its control failing (by hand) - Linux with an AMD CPU and the Steam Deck: Castle Town's market, two minutes, this PR against `main`, expecting the same thread names in the log and frame times within the runs' spread (the Test Farm: play Castle Town's market for two minutes on Linux with an AMD CPU and the Steam Deck, this PR against main, comparing the log's thread names and frame times) ## Measurements No game measurement yet: the game runs are in Still to test. The fix's before-and-after proof is the game-free fixture `tests/test_thread_rename.py`, run on this PC (i7-9700, Windows 11) in the queue jobs named in Tested, with clang and with GCC: | case | Before: main @ b62932a9 | Before: this PR @ eeee7e72 (before the review) | After: this PR @ c29a9378 | |---|---|---|---| | mid: a reader during a rename | FAIL ("NewName_012345ghijkl") | pass | pass | | held: a name taken before a rename | FAIL | pass | pass | | two: two renames at once | FAIL | FAIL | pass | | start: a rename during the start | FAIL | FAIL | pass | | utf8: ten names cut at 47 bytes | FAIL | FAIL | pass | | bounded: a name before an unreadable page | FAIL (access violation) | FAIL (access violation) | pass | | names | pass | pass | pass | | publish (without ThreadSanitizer) | pass | pass | pass | (`main` with clang only.) What the change adds at run time, not measured: reading a name is one atomic load (as at eeee7e72), and a thread's role one more load from its kept name, on every wait's spin (`hos_spin_ns`); a start and a rename hold a lock over a few system calls; a grant stores a pointer instead of copying 48 bytes. Whether any of it shows in frame times is for the game runs in Still to test. Verdict: every case that failed on `main` or at eeee7e72 passes at c29a9378 (game-free fixture); the game's frame times are not measured yet. ## Risks A name is now a pointer: a name that is never set would be NULL, but every thread gets one when it is made (`MainThread`, `Thread_<entry>`), as on `main`. Each different name stays in memory for the run, 64 bytes each (a game that gave every new thread a new name would keep one per thread it made; the log counts them from 64 on). A name with bytes that are not UTF-8 is kept with '?' for them, in the log and the system's name alike. The start and a rename now hold a lock while they call `SetThreadDescription` and `hos_thread_pin` (a few system calls): a rename of another thread waits for them. No switch: a revert gives back `main`'s code. Players notice nothing after updating. ## Checklist ### Ready for review - [x] One topic, on current `main`; the title is a Conventional Commit. - [ ] Tested on my own PC, in the game for a change the game runs, as Testing says. - [ ] `mise run test` passes (any failure that also fails on `main` is named). - [x] Measured against `main`, or Measurements says why there is nothing to measure. - [x] No game files, build output or hand-edited generated files; `third_party/` untouched; nothing GPL-incompatible. ### When it applies - [x] A fix: a game-free test that fails on `main` and passes here. - [x] A check under `tests/` that reads changed source is updated, not weakened. - [ ] Host code builds on Windows (llvm-mingw) and Linux, with UTF-8 paths. - [x] AI assistance recorded with `Assisted-by:` / `Co-Authored-By:` trailers. ### Ready to merge - [ ] Everything in Still to test has run, its results in Tested. - [ ] Tier 1 hardware it can affect is tested (docs/dev/standards.md, "Hardware tiers"). - [ ] Approved in review, every comment answered. - [ ] CI passes on the head; on current `main`, measured again or a comment says why the numbers still hold.
tests/test_thread_rename.py runs host/hos_thread.c's hos_thread_set_name, hos_thread_name and serial_name, with the
HosThread name field as declared, cut out with tests/srccut.py. Its snprintf writes in order as the C library's does
and, in the rename under test, stops halfway until a reader on another thread has read the name. The reader must read
the old name or the new one whole, and a name taken before a rename must still read the same after it. On main the
rename writes the new name over the old one in place: the reader reads half of each, and a held name changes.

Assisted-by: Claude Code
fix(os): a thread's name is never written over: a rename publishes a kept name
Some checks failed
PR description / Description follows the template (pull_request_target) Successful in 19s
Website / Website on Cloudflare (pull_request_target) Has been skipped
CI / Conventional Commit title (pull_request) Has been cancelled
CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Has been cancelled
CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Has been cancelled
CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Has been cancelled
CI / Workflow lint (actionlint) (pull_request) Has been cancelled
CI / Website build and checks (pull_request) Has been cancelled
Guide screenshots / Guide images and local UI source review (pull_request) Has been cancelled
Third-party parts / As their notes say (pull_request) Has been cancelled
Third-party parts / The same as upstream (pull_request) Has been cancelled
67baca9690
hos_thread_set_name (the guest's SetThreadName, which may name another thread) wrote the new name with snprintf into
the HosThread's name buffer, which every name reader (logs, Tracy, the profiler, HOS_CHAIN's file, the vcore lines, the
waits' diagnostics) read through hos_thread_name at the same time: a reader could read half of the old name and half
of the new, and a name it held changed under it.

Names are now kept: name_kept gives one copy of each different name (under a lock, in a list kept for the run; games
name a few dozen threads), and a HosThread's name is a pointer to one, set at creation and, on a rename, published with
one atomic release store; hos_thread_name loads it with acquire. Every reader in hos_thread.c goes through
hos_thread_name. vc_grant copies the name with strncpy (a name is at most 47 characters). The checks that read the
changed code follow it: test_log_levels, test_vcore_escape and test_vcore_mask0 define hos_thread_name for the code they
cut (the vcore ones give their threads names that last, as the host's do), and test_spin_by_role looks for the serial
flag set from the kept name.

Assisted-by: Claude Code
BeanBot left a comment

Recommendation: Request changes. Immutable names fix torn guest-name reads, but concurrent renames can leave a thread permanently using the wrong wait policy or OS name. The Linux regression fixture also fails to link.

A rename gives logs, profilers, and wait diagnostics a complete name, and a name already handed to a reader remains valid. The design keeps reads lock-free and adds a process-lifetime name table, a lock on creation and rename, and work under the virtual-core lock.

@fix will do (13)

  1. L1 host/hos_thread.c:764: The name and its serial role can settle on different renames. (more on the line)
  2. L2 host/hos_thread.c:655: Start and rename can apply host-side effects in reverse order. (more on the line)
  3. L3 host/hos_thread.c:770: Byte truncation can split valid UTF-8. (more on the line)
  4. L4 host/hos_chain.c:66: A chain recording can outlive a recorded worker. The recorder stores the worker's HosThread *, destruction frees it, and the dump later loads the name pointer from freed storage. The kept string only helps once the pointer has been acquired. Publish a kept name into the recorder's slot while the thread is live and preserve it after destruction. Test recording a worker that exits before the dump.
  5. L5 (minor) host/hos_thread.c:1413: The rename still passes a raw guest pointer to %s. (more on the line)
  6. L6 (minor) host/hos_thread.c:114: A failed name allocation holds the global table lock through up to five seconds of retries and potentially an OOM dialog. (more on the line)
  7. L7 (its part an edit can do) host/hos_thread.c:112: Hash the interned names to avoid linear scans. If retained memory needs a bound, move readers that retain names to owned copies or handles before reclaiming entries.
  8. L8 tests/test_thread_rename.py:242: The Linux fixture calls Sleep in both halves of the rendezvous. (more on the line)
  9. L9 tests/test_thread_rename.py:133: The cross-thread check reads only during formatting, before the new pointer is published. (more on the line)
  10. L10 (minor) tests/test_thread_rename.py:150: A missed rendezvous can pass. (more on the line)
  11. L11 (minor) tests/test_vcore_escape.py:268: The adapted vcore fixtures still provide no runtime coverage of the changed name path. The reported baseline compile failures are a duplicate hos_fps_timing here and a missing env_on_or in the mask fixture. Once built, the escape fixture also inherits switches such as HOS_VCORE_STARVE_US=0 in its default case. Repair the harnesses, clear inherited HOS switches before each case, and assert that the grant keeps the releasing thread's name across a rename.
  12. L12 (its part an edit can do) host/hos_thread.c:1984: Keep the grant-time name pointer in vc_by and use a null sentinel. The name already outlives the grant, so this removes the string scan and padding under the scheduler lock.
  13. L13 (minor): Measurements says there is nothing to measure, but the policy asks for a fix's before-and-after proof in that section, and the patch changes the grant path as well as name reads. Put the fixture results with their tested commits in Measurements, add the head and game results when available, and record the required lint and full-suite outcomes.

For the author (2) (a run, a measurement or a decision)

  • L7 (minor) host/hos_thread.c:112: A session that generates a fresh name each time grows the table for the rest of the process and scans all prior names for each new one. (more on the line)
  • L12 (minor) host/hos_thread.c:1984: The required in-game run on the author's PC is missing, and the Windows fixture is from before the rebase. (more on the line)

Checks: Still running: CI / Conventional Commit title (pull_request), CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request), CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request), CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request), CI / Workflow lint (actionlint) (pull_request), CI / Website build and checks (pull_request), Guide screenshots / Guide images and local UI source review (pull_request), Third-party parts / As their notes say (pull_request), Third-party parts / The same as upstream (pull_request).

More: does it help, other systems, switches, what was read

Does it help: Yes for the reproduced torn and changing names. The reported Windows fixture failed 2 of 3 cases at test-only da86ea40 and passed all 3 at 3551fad2, including 1,000 alternating renames. Those runs predate the rebase to 67baca96. The in-game result, Linux validation, and frame-time cost are unproven.

Other systems: The change reaches game logs and crash reports, Windows and Linux profilers, Tracy, HOS_CHAIN, virtual-core grants, and name-based spin, affinity, and priority. The renderer's optional frame trace still queries by name through the existing callback. There is no direct renderer, driver, save, audio, or input change. The scheduling path is shared across GPU vendors and Steam Deck, and there is no game run or frame-time comparison on any of them yet.

Switches: No switch is added or removed. The existing HOS_TEST_ALLOC_REFUSE diagnostic, off when unset, now also matches the label native: thread names. Existing name-based HOS_PIN and HOS_PRIO settings and spin defaults are unchanged in configuration, but their application can race as described below. No switch-table row is needed.

Reviewed: 67baca9690 against 40bfceb055: Read all seven diffs and both commits, all five changed Python tests, the changed declarations and surrounding host functions, and the relevant callers, locks, allocation code, POSIX shims, and Tracy naming code. Read AGENTS.md, CONTRIBUTING.md, the PR template, standards, and the test-writing rules. No review-checklist file or adjacent Tooling checkout was present. Skimmed recorded-run references and found no record for this head. Unrelated portions of the host files and third-party code were skimmed. No translations or generated files changed. The earlier tested branch commits and the overlapping branch tip were unavailable locally.; nothing run (a code review).

Checked and fine:

  • Every direct production read of the guest-name field now goes through the accessor. Initial names are installed before thread publication, and the table lock protects complete entries before release/acquire publishes them.
  • The log-level fixture keeps its original assertions. The spin-by-role fixture still isolates its HOS settings and exercises both spin budgets.
  • The change stays on one topic. Both commits carry Assisted-by trailers. No game-derived artifacts, vendor SDKs, generated files, third-party changes, menu keys, or renderer API changes appear in the diff.

Testing: Read-only review. No builds or tests were run. The author reports Windows fixture and syntax-check results with and without Tracy on the earlier branch, plus nine baseline failures and two Linux-only skips. The listed CI jobs are still running.

Overlaps and merge order: The PR reports conflicts with #1967 at the getter and rename. Land #2020 first after fixes, then rebase #1967 to use the kept-name accessor and drop the redundant reader lock. The PR reports #1953 can merge independently. The #1967 tip was unavailable in the checkout.


Automated code review by BeanBot (gpt-6.2-sol), on the pull request's opening. Nothing was run; a maintainer decides.

**Recommendation: Request changes.** Immutable names fix torn guest-name reads, but concurrent renames can leave a thread permanently using the wrong wait policy or OS name. The Linux regression fixture also fails to link. A rename gives logs, profilers, and wait diagnostics a complete name, and a name already handed to a reader remains valid. The design keeps reads lock-free and adds a process-lifetime name table, a lock on creation and rename, and work under the virtual-core lock. **`@fix` will do (13)** 1. **L1** `host/hos_thread.c:764`: The name and its serial role can settle on different renames. (more on the line) 2. **L2** `host/hos_thread.c:655`: Start and rename can apply host-side effects in reverse order. (more on the line) 3. **L3** `host/hos_thread.c:770`: Byte truncation can split valid UTF-8. (more on the line) 4. **L4** `host/hos_chain.c:66`: A chain recording can outlive a recorded worker. The recorder stores the worker's `HosThread *`, destruction frees it, and the dump later loads the name pointer from freed storage. The kept string only helps once the pointer has been acquired. Publish a kept name into the recorder's slot while the thread is live and preserve it after destruction. Test recording a worker that exits before the dump. 5. **L5** (minor) `host/hos_thread.c:1413`: The rename still passes a raw guest pointer to `%s`. (more on the line) 6. **L6** (minor) `host/hos_thread.c:114`: A failed name allocation holds the global table lock through up to five seconds of retries and potentially an OOM dialog. (more on the line) 7. **L7** (its part an edit can do) `host/hos_thread.c:112`: Hash the interned names to avoid linear scans. If retained memory needs a bound, move readers that retain names to owned copies or handles before reclaiming entries. 8. **L8** `tests/test_thread_rename.py:242`: The Linux fixture calls `Sleep` in both halves of the rendezvous. (more on the line) 9. **L9** `tests/test_thread_rename.py:133`: The cross-thread check reads only during formatting, before the new pointer is published. (more on the line) 10. **L10** (minor) `tests/test_thread_rename.py:150`: A missed rendezvous can pass. (more on the line) 11. **L11** (minor) `tests/test_vcore_escape.py:268`: The adapted vcore fixtures still provide no runtime coverage of the changed name path. The reported baseline compile failures are a duplicate `hos_fps_timing` here and a missing `env_on_or` in the mask fixture. Once built, the escape fixture also inherits switches such as `HOS_VCORE_STARVE_US=0` in its default case. Repair the harnesses, clear inherited HOS switches before each case, and assert that the grant keeps the releasing thread's name across a rename. 12. **L12** (its part an edit can do) `host/hos_thread.c:1984`: Keep the grant-time name pointer in `vc_by` and use a null sentinel. The name already outlives the grant, so this removes the string scan and padding under the scheduler lock. 13. **L13** (minor): Measurements says there is nothing to measure, but the policy asks for a fix's before-and-after proof in that section, and the patch changes the grant path as well as name reads. Put the fixture results with their tested commits in Measurements, add the head and game results when available, and record the required lint and full-suite outcomes. **For the author (2)** (a run, a measurement or a decision) - **L7** (minor) `host/hos_thread.c:112`: A session that generates a fresh name each time grows the table for the rest of the process and scans all prior names for each new one. (more on the line) - **L12** (minor) `host/hos_thread.c:1984`: The required in-game run on the author's PC is missing, and the Windows fixture is from before the rebase. (more on the line) **Checks:** Still running: `CI / Conventional Commit title (pull_request)`, `CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request)`, `CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request)`, `CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request)`, `CI / Workflow lint (actionlint) (pull_request)`, `CI / Website build and checks (pull_request)`, `Guide screenshots / Guide images and local UI source review (pull_request)`, `Third-party parts / As their notes say (pull_request)`, `Third-party parts / The same as upstream (pull_request)`. <details> <summary>More: does it help, other systems, switches, what was read</summary> **Does it help:** Yes for the reproduced torn and changing names. The reported Windows fixture failed 2 of 3 cases at test-only da86ea40 and passed all 3 at 3551fad2, including 1,000 alternating renames. Those runs predate the rebase to 67baca96. The in-game result, Linux validation, and frame-time cost are unproven. **Other systems:** The change reaches game logs and crash reports, Windows and Linux profilers, Tracy, HOS_CHAIN, virtual-core grants, and name-based spin, affinity, and priority. The renderer's optional frame trace still queries by name through the existing callback. There is no direct renderer, driver, save, audio, or input change. The scheduling path is shared across GPU vendors and Steam Deck, and there is no game run or frame-time comparison on any of them yet. **Switches:** No switch is added or removed. The existing HOS_TEST_ALLOC_REFUSE diagnostic, off when unset, now also matches the label `native: thread names`. Existing name-based HOS_PIN and HOS_PRIO settings and spin defaults are unchanged in configuration, but their application can race as described below. No switch-table row is needed. **Reviewed:** 67baca9690e1 against 40bfceb0552e: Read all seven diffs and both commits, all five changed Python tests, the changed declarations and surrounding host functions, and the relevant callers, locks, allocation code, POSIX shims, and Tracy naming code. Read AGENTS.md, CONTRIBUTING.md, the PR template, standards, and the test-writing rules. No review-checklist file or adjacent Tooling checkout was present. Skimmed recorded-run references and found no record for this head. Unrelated portions of the host files and third-party code were skimmed. No translations or generated files changed. The earlier tested branch commits and the overlapping branch tip were unavailable locally.; nothing run (a code review). **Checked and fine:** - Every direct production read of the guest-name field now goes through the accessor. Initial names are installed before thread publication, and the table lock protects complete entries before release/acquire publishes them. - The log-level fixture keeps its original assertions. The spin-by-role fixture still isolates its HOS settings and exercises both spin budgets. - The change stays on one topic. Both commits carry Assisted-by trailers. No game-derived artifacts, vendor SDKs, generated files, third-party changes, menu keys, or renderer API changes appear in the diff. **Testing:** Read-only review. No builds or tests were run. The author reports Windows fixture and syntax-check results with and without Tracy on the earlier branch, plus nine baseline failures and two Linux-only skips. The listed CI jobs are still running. **Overlaps and merge order:** The PR reports conflicts with #1967 at the getter and rename. Land #2020 first after fixes, then rebase #1967 to use the kept-name accessor and drop the redundant reader lock. The PR reports #1953 can merge independently. The #1967 tip was unavailable in the checkout. </details> --- <sub>Automated code review by BeanBot (gpt-6.2-sol), on the pull request's opening. Nothing was run; a maintainer decides.</sub>
@ -101,0 +109,4 @@
snprintf(s, sizeof s, "%s", name);
AcquireSRWLockExclusive(&kept_names_lock);
KeptName *k = kept_names;
while (k && strcmp(k->s, s)) k = k->next;
Member

L7 (minor): A session that generates a fresh name each time grows the table for the rest of the process and scans all prior names for each new one. At 10,000 distinct names that is about 0.53 MiB before allocator overhead and about 50 million cumulative comparisons under one lock. Confirm the expected distinct-name count in a longer session and decide whether to retain the process-lifetime pointer contract.

**L7** (minor): A session that generates a fresh name each time grows the table for the rest of the process and scans all prior names for each new one. At 10,000 distinct names that is about 0.53 MiB before allocator overhead and about 50 million cumulative comparisons under one lock. Confirm the expected distinct-name count in a longer session and decide whether to retain the process-lifetime pointer contract.
@ -101,0 +111,4 @@
KeptName *k = kept_names;
while (k && strcmp(k->s, s)) k = k->next;
if (!k) {
k = (KeptName *)hos_xmalloc(sizeof *k, "native: thread names");
Member

L6 (minor): A failed name allocation holds the global table lock through up to five seconds of retries and potentially an OOM dialog. A different thread trying to use an already interned name waits behind it. Look up under the lock, allocate on a miss outside it, and recheck before inserting.

**L6** (minor): A failed name allocation holds the global table lock through up to five seconds of retries and potentially an OOM dialog. A different thread trying to use an already interned name waits behind it. Look up under the lock, allocate on a miss outside it, and recheck before inserting.
@ -630,3 +654,2 @@
t->handle = CreateThread(NULL, 64 << 20, thread_main, t, STACK_SIZE_PARAM_IS_A_RESERVATION, NULL);
if (!t->handle) hos_fatal(NULL, "CreateThread failed for %s", t->name);
if (t->name[0]) { /* (named before it started: the OS name too, for profilers and HOS_SAMPLE) */
const char *name = hos_thread_name(t);
Member

L2: Start and rename can apply host-side effects in reverse order. Start can snapshot the default name, the new thread can rename itself and apply the new description and priority, and start can then apply the stale snapshot. Two renamers can do the same, and the plain handle read in rename can race with its assignment in start. Serialize handle publication and name-dependent host updates per target, including Tracy startup. On Linux this also prevents concurrent guest naming calls from writing the same POSIX description buffer.

**L2**: Start and rename can apply host-side effects in reverse order. Start can snapshot the default name, the new thread can rename itself and apply the new description and priority, and start can then apply the stale snapshot. Two renamers can do the same, and the plain handle read in rename can race with its assignment in start. Serialize handle publication and name-dependent host updates per target, including Tracy startup. On Linux this also prevents concurrent guest naming calls from writing the same POSIX description buffer.
@ -1385,7 +1411,7 @@ HLE(_ZN2nn2os13DestroyThreadEPNS0_10ThreadTypeE) {
HLE(_ZN2nn2os13SetThreadNameEPNS0_10ThreadTypeEPKc) {
HosThread *t = thread_arg(c, ARG(0));
hos_thread_set_name(t, ARG(1) ? (const char *)GP(ARG(1)) : "");
Member

L5 (minor): The rename still passes a raw guest pointer to %s. A name next to an uncommitted page can fault if its terminator lies beyond the readable page, because snprintf scans the whole input even when it only writes 47 bytes. Copy the bounded prefix from readable guest memory with an explicit terminator and handle an invalid address before interning.

**L5** (minor): The rename still passes a raw guest pointer to `%s`. A name next to an uncommitted page can fault if its terminator lies beyond the readable page, because snprintf scans the whole input even when it only writes 47 bytes. Copy the bounded prefix from readable guest memory with an explicit terminator and handle an invalid address before interning.
@ -738,1 +762,3 @@
__atomic_store_n(&t->serial, serial_name(t->name), __ATOMIC_RELAXED); /* (atomic: SetThreadName may name another thread) */
const char *kept = name_kept(name);
__atomic_store_n(&t->name, kept, __ATOMIC_RELEASE); /* (release: a reader that loads it reads the whole name) */
__atomic_store_n(&t->serial, serial_name(kept), __ATOMIC_RELAXED); /* (atomic: SetThreadName may name another thread) */
Member

L1: The name and its serial role can settle on different renames. A publishes Worker, B publishes Presentation Thread and stores serial=1, then A stores serial=0. The final name is Presentation Thread, but its waits use the worker policy. Compute the role when interning the name and publish the name and its role through one immutable entry. Add a two-writer test.

**L1**: The name and its serial role can settle on different renames. A publishes `Worker`, B publishes `Presentation Thread` and stores serial=1, then A stores serial=0. The final name is `Presentation Thread`, but its waits use the worker policy. Compute the role when interning the name and publish the name and its role through one immutable entry. Add a two-writer test.
@ -742,3 +768,3 @@
if (t->handle) {
wchar_t w[48];
MultiByteToWideChar(CP_UTF8, 0, t->name, -1, w, 48);
MultiByteToWideChar(CP_UTF8, 0, kept, -1, w, 48);
Member

L3: Byte truncation can split valid UTF-8. If a four-byte character starts at byte 46, the kept name ends with its lead byte. Linux's MultiByteToWideChar with length -1 consumes the terminator as a continuation and can read past the 48-byte name allocation. Truncate at a complete UTF-8 character, handle malformed input, and check conversion before applying the OS name. Exercise that boundary on both platforms.

**L3**: Byte truncation can split valid UTF-8. If a four-byte character starts at byte 46, the kept name ends with its lead byte. Linux's `MultiByteToWideChar` with length -1 consumes the terminator as a continuation and can read past the 48-byte name allocation. Truncate at a complete UTF-8 character, handle malformed input, and check conversion before applying the OS name. Exercise that boundary on both platforms.
@ -1956,3 +1982,3 @@
if (by) { /* (the long-wait line's cause: vc_long_wait) */
const int64_t now = hos_now_ns();
memcpy(best->vc_by, by->name, sizeof best->vc_by);
strncpy(best->vc_by, hos_thread_name(by), sizeof best->vc_by); /* (a name is at most 47 characters) */
Member

L12 (minor): The required in-game run on the author's PC is missing, and the Windows fixture is from before the rebase. Run base and fixed head on Windows, Linux, and Steam Deck as planned, including profiling and Tracy. Compare median and tail frame times in repeated matched runs. Every named virtual-core grant now does a string copy under the scheduler lock, even when long-wait logging is off.

**L12** (minor): The required in-game run on the author's PC is missing, and the Windows fixture is from before the rebase. Run base and fixed head on Windows, Linux, and Steam Deck as planned, including profiling and Tracy. Compare median and tail frame times in repeated matched runs. Every named virtual-core grant now does a string copy under the scheduler lock, even when long-wait logging is off.
@ -0,0 +130,4 @@
(void)arg;
const uint64_t t0 = now_ms();
while (!__atomic_load_n(&halfway, __ATOMIC_SEQ_CST) && now_ms() - t0 < 1000) Sleep(1);
read_name(hos_thread_name(&th), seen, sizeof seen);
Member

L9: The cross-thread check reads only during formatting, before the new pointer is published. The other cases read on the writer thread. Replacing the release store and acquire load with plain pointer accesses can therefore pass all three cases. Add a concurrent reader of newly published names with no separate handoff that orders the pointer access. Verify with a race-detector control that reverting the atomic access fails.

**L9**: The cross-thread check reads only during formatting, before the new pointer is published. The other cases read on the writer thread. Replacing the release store and acquire load with plain pointer accesses can therefore pass all three cases. Add a concurrent reader of newly published names with no separate handoff that orders the pointer access. Verify with a race-detector control that reverting the atomic access fails.
@ -0,0 +147,4 @@
hos_thread_set_name(&th, NEW);
pthread_join(r, NULL);
#endif
if (strcmp(seen, OLD) && strcmp(seen, NEW)) {
Member

L10 (minor): A missed rendezvous can pass. If CI delays the reader beyond the writer's one-second timeout, it can see the whole new name and report that it read during the rename. Fail when either side misses the rendezvous, check thread creation and join results, and close the Windows thread handle.

**L10** (minor): A missed rendezvous can pass. If CI delays the reader beyond the writer's one-second timeout, it can see the whole new name and report that it read during the rename. Fail when either side misses the rendezvous, check thread creation and join results, and close the Windows thread handle.
@ -0,0 +239,4 @@
c.write_text(fixture_source(), encoding="utf-8")
cmd = [cc, "-std=gnu11", "-O2", "-Wall", "-Wextra", "-Wno-unused-function"]
if os.name != "nt":
cmd += ["-I", str(ROOT / "host" / "posix"), "-pthread"]
Member

L8: The Linux fixture calls Sleep in both halves of the rendezvous. The POSIX header only declares it, and the build links the fixture and pthreads without the shim that defines it. Use a local POSIX sleep or a pthread rendezvous, then verify the fixture with both Clang and GCC.

**L8**: The Linux fixture calls `Sleep` in both halves of the rendezvous. The POSIX header only declares it, and the build links the fixture and pthreads without the shim that defines it. Use a local POSIX sleep or a pthread rendezvous, then verify the fixture with both Clang and GCC.
siantm force-pushed fix/os-thread-names-kept from 67baca9690
Some checks failed
PR description / Description follows the template (pull_request_target) Successful in 19s
Website / Website on Cloudflare (pull_request_target) Has been skipped
CI / Conventional Commit title (pull_request) Has been cancelled
CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Has been cancelled
CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Has been cancelled
CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Has been cancelled
CI / Workflow lint (actionlint) (pull_request) Has been cancelled
CI / Website build and checks (pull_request) Has been cancelled
Guide screenshots / Guide images and local UI source review (pull_request) Has been cancelled
Third-party parts / As their notes say (pull_request) Has been cancelled
Third-party parts / The same as upstream (pull_request) Has been cancelled
to c29a937856
Some checks are pending
CI / Conventional Commit title (pull_request) Blocked by required conditions
CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Blocked by required conditions
CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Blocked by required conditions
CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Blocked by required conditions
CI / Workflow lint (actionlint) (pull_request) Blocked by required conditions
CI / Website build and checks (pull_request) Blocked by required conditions
Guide screenshots / Guide images and local UI source review (pull_request) Blocked by required conditions
Third-party parts / As their notes say (pull_request) Blocked by required conditions
Third-party parts / The same as upstream (pull_request) Blocked by required conditions
Website / Website on Cloudflare (pull_request) Blocked by required conditions
PR description / Description follows the template (pull_request_target) Successful in 4s
beanstalk/approval waiting for a BigBean's 👍 on the request
Website / Website on Cloudflare (pull_request_target) Has been skipped
2026-11-21 09:32:52 +00:00
Compare
Author
Contributor

Pushed c29a9378 for BeanBot's review (64191), rebased onto main @ b62932a9 (no conflict; the first two commits unchanged, now a1491edf and eeee7e72). Three commits are new: 3c8dba82 makes tests/test_vcore_escape.py and tests/test_vcore_mask0.py build and run again (L11), 1ac90685 adds the two, start, utf8, bounded and publish cases and fixes the fixture's rendezvous (L1-L3, L5, L8-L10), and c29a9378 fixes the host (L1-L3, L5-L7, L12). The description's Summary, Testing, Still to test, Measurements and Risks say what changed and what ran; each point has its own reply.

Pushed c29a9378 for BeanBot's review ([64191](https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/2020#issuecomment-64191)), rebased onto `main` @ b62932a9 (no conflict; the first two commits unchanged, now a1491edf and eeee7e72). Three commits are new: 3c8dba82 makes `tests/test_vcore_escape.py` and `tests/test_vcore_mask0.py` build and run again (L11), 1ac90685 adds the `two`, `start`, `utf8`, `bounded` and `publish` cases and fixes the fixture's rendezvous (L1-L3, L5, L8-L10), and c29a9378 fixes the host (L1-L3, L5-L7, L12). The description's Summary, Testing, Still to test, Measurements and Risks say what changed and what ran; each point has its own reply.
Author
Contributor

Re L1: right, fixed in c29a9378. A kept name now holds its role (name_kept computes serial_name once, before the name can be published), the thread has no serial flag of its own, and hos_thread_serial reads the role of the name it loads, so the name and its role are always one rename's. The new two case in tests/test_thread_rename.py (1ac90685) halts the first of two renames once its role is computed until the second is done: at the test commit the name was "Presentation Thread" with a worker's role and the first rename's system name, at c29a9378 all three are one rename's.

Re [L1](https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/2020#issuecomment-64179): right, fixed in c29a9378. A kept name now holds its role (`name_kept` computes `serial_name` once, before the name can be published), the thread has no serial flag of its own, and `hos_thread_serial` reads the role of the name it loads, so the name and its role are always one rename's. The new `two` case in `tests/test_thread_rename.py` (1ac90685) halts the first of two renames once its role is computed until the second is done: at the test commit the name was "Presentation Thread" with a worker's role and the first rename's system name, at c29a9378 all three are one rename's.
Author
Contributor

Re L2: right, fixed in c29a9378. hos_thread_start stores the handle (atomic release), reads the name and gives the system its name (description, hos_thread_pin, the web's list) under one lock, os_names_lock, and a rename publishes its name and names the thread under the same lock, so the system gets the names in the order they were published and a rename reads the handle under the lock its start stored it in; on Linux that also serialises the POSIX description writes. The new start case halts a start after it converted the name while another thread renames: at the test commit the system kept the start's stale name, at c29a9378 it has the rename's; two checks two renamers the same way. Tracy is left as on main: it only takes a name a thread gives itself, so a rename from another thread after the start never reaches it with or without the lock (thread_main's naming loads the published name atomically); that is noted as a separate follow-up.

Re [L2](https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/2020#issuecomment-64181): right, fixed in c29a9378. `hos_thread_start` stores the handle (atomic release), reads the name and gives the system its name (description, `hos_thread_pin`, the web's list) under one lock, `os_names_lock`, and a rename publishes its name and names the thread under the same lock, so the system gets the names in the order they were published and a rename reads the handle under the lock its start stored it in; on Linux that also serialises the POSIX description writes. The new `start` case halts a start after it converted the name while another thread renames: at the test commit the system kept the start's stale name, at c29a9378 it has the rename's; `two` checks two renamers the same way. Tracy is left as on `main`: it only takes a name a thread gives itself, so a rename from another thread after the start never reaches it with or without the lock (`thread_main`'s naming loads the published name atomically); that is noted as a separate follow-up.
Author
Contributor

Re L3: right, fixed in c29a9378. name_kept cuts before a character its 47 bytes would cut and keeps each byte that is no part of a valid UTF-8 character (bad lead, wrong continuation, overlong, surrogate, past U+10FFFF) as '?', so a kept name is always whole UTF-8 and the Linux MultiByteToWideChar never meets a lead byte before the terminator; the conversion's result is checked before SetThreadDescription. The new utf8 case runs ten names through the real code (a four-byte character from byte 46, one ending at byte 47, two- and three-byte cuts, malformed bytes) and compares the kept bytes with the expected ones in Python: it fails at the test commit and passes at c29a9378 with llvm-mingw clang 23.1.5 and MinGW-w64 GCC 14.2 (Windows targets). Linux itself was not run here (no Linux machine on this PC): pending, in Still to test.

Re [L3](https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/2020#issuecomment-64183): right, fixed in c29a9378. `name_kept` cuts before a character its 47 bytes would cut and keeps each byte that is no part of a valid UTF-8 character (bad lead, wrong continuation, overlong, surrogate, past U+10FFFF) as '?', so a kept name is always whole UTF-8 and the Linux `MultiByteToWideChar` never meets a lead byte before the terminator; the conversion's result is checked before `SetThreadDescription`. The new `utf8` case runs ten names through the real code (a four-byte character from byte 46, one ending at byte 47, two- and three-byte cuts, malformed bytes) and compares the kept bytes with the expected ones in Python: it fails at the test commit and passes at c29a9378 with llvm-mingw clang 23.1.5 and MinGW-w64 GCC 14.2 (Windows targets). Linux itself was not run here (no Linux machine on this PC): pending, in Still to test.
Author
Contributor

Re L4 (review 64191, host/hos_chain.c:66): right, and it is on main too: the recorder keeps each noting thread's HosThread * and the dump reads its name after the thread may have been destroyed (hos_chain.c:43 and :66 on main @ b62932a9; this PR does not change hos_chain.c). It is the same point as #2019's L1, and #2019, now stacked on this PR, fixes it there (its fix commit 75107f9e, test 4ef154f7, on c29a9378): the recorder stores the noting thread's kept name pointer (hos_thread_name, which this PR keeps valid for the run) in its own table while the thread is live, so the dump never touches a HosThread; its test records a worker that exits before the dump.

Re L4 (review [64191](https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/2020#issuecomment-64191), `host/hos_chain.c:66`): right, and it is on `main` too: the recorder keeps each noting thread's `HosThread *` and the dump reads its name after the thread may have been destroyed (`hos_chain.c:43` and `:66` on `main` @ b62932a9; this PR does not change `hos_chain.c`). It is the same point as #2019's L1, and #2019, now stacked on this PR, fixes it there (its fix commit 75107f9e, test 4ef154f7, on c29a9378): the recorder stores the noting thread's kept name pointer (`hos_thread_name`, which this PR keeps valid for the run) in its own table while the thread is live, so the dump never touches a `HosThread`; its test records a worker that exits before the dump.
Author
Contributor

Re L5: right, fixed in c29a9378. name_kept copies the guest's bytes itself and reads 47 at most, never past them, and the SetThreadName HLE first checks with hos_guest_range that those 47 bytes lie in the guest window (else it logs the address and leaves the name as it was). The new bounded case puts 47 bytes with no terminator at the end of a readable page before a PAGE_NOACCESS / PROT_NONE one: at the test commit the rename faulted, at c29a9378 it keeps the 47 bytes. An address inside the window that the game never mapped is read as the game's other HLE pointer arguments are (on Windows it is committed on touch, on Linux it faults); hos_mem_mapped is not used because it would refuse names in the system region, where the main thread's stack is (hos_sys_alloc).

Re [L5](https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/2020#issuecomment-64184): right, fixed in c29a9378. `name_kept` copies the guest's bytes itself and reads 47 at most, never past them, and the SetThreadName HLE first checks with `hos_guest_range` that those 47 bytes lie in the guest window (else it logs the address and leaves the name as it was). The new `bounded` case puts 47 bytes with no terminator at the end of a readable page before a `PAGE_NOACCESS` / `PROT_NONE` one: at the test commit the rename faulted, at c29a9378 it keeps the 47 bytes. An address inside the window that the game never mapped is read as the game's other HLE pointer arguments are (on Windows it is committed on touch, on Linux it faults); `hos_mem_mapped` is not used because it would refuse names in the system region, where the main thread's stack is (`hos_sys_alloc`).
Author
Contributor

Re L6: right, fixed in c29a9378. name_kept looks the name up under the shared lock, allocates a new one's copy with no lock held, and looks again under the exclusive lock before it inserts (a copy another thread inserted meanwhile wins, ours is freed), so an allocation that retries never holds the table. The two case halts one rename after that allocation, before it takes the exclusive lock, and a second rename runs to its end meanwhile.

Re [L6](https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/2020#issuecomment-64185): right, fixed in c29a9378. `name_kept` looks the name up under the shared lock, allocates a new one's copy with no lock held, and looks again under the exclusive lock before it inserts (a copy another thread inserted meanwhile wins, ours is freed), so an allocation that retries never holds the table. The `two` case halts one rename after that allocation, before it takes the exclusive lock, and a second rename runs to its end meanwhile.
Author
Contributor

Re L7: the edit part is in c29a9378: the kept names are in 256 lists by an FNV-1a hash instead of one, and the log says how many different names are kept at 64, 128, 256, ... ("threads: N different thread names kept"), so a game that makes up names without end shows in a player's log. The process-lifetime contract stays: every reader holds the pointer without a lock, and #2019 (stacked on this) keeps it in its recorder; reclaiming would need owned copies in every reader. The distinct-name count in a long session needs a game run: pending (requested on this PC; the description lists it in Still to test).

Re [L7](https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/2020#issuecomment-64186): the edit part is in c29a9378: the kept names are in 256 lists by an FNV-1a hash instead of one, and the log says how many different names are kept at 64, 128, 256, ... ("threads: N different thread names kept"), so a game that makes up names without end shows in a player's log. The process-lifetime contract stays: every reader holds the pointer without a lock, and #2019 (stacked on this) keeps it in its recorder; reclaiming would need owned copies in every reader. The distinct-name count in a long session needs a game run: pending (requested on this PC; the description lists it in Still to test).
Author
Contributor

Re L8: right, fixed in 1ac90685. The fixture no longer calls Sleep: it naps with nanosleep on POSIX (Sleep only on Windows), and its other POSIX dependencies are pthreads, clock_gettime, mmap and the posix header's inline SRW locks. Here it was built and run with llvm-mingw clang and MSYS2's mingw-w64 GCC 14.2.0 (Windows targets); there is no Linux machine on this PC, so the Linux build with Clang and GCC is still to test (listed in the description).

Re [L8](https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/2020#issuecomment-64187): right, fixed in 1ac90685. The fixture no longer calls `Sleep`: it naps with `nanosleep` on POSIX (`Sleep` only on Windows), and its other POSIX dependencies are pthreads, `clock_gettime`, `mmap` and the posix header's inline SRW locks. Here it was built and run with llvm-mingw clang and MSYS2's mingw-w64 GCC 14.2.0 (Windows targets); there is no Linux machine on this PC, so the Linux build with Clang and GCC is still to test (listed in the description).
Author
Contributor

Re L9: right, added in 1ac90685. The new publish case reads the name in a loop on one thread while another renames it, each time to a name never used before (2000 names, and on until the reader has read 100 new ones), with nothing but the name pointer between them; built with -fsanitize=thread where the compiler has it, the case must pass and a control built from the same source with the release store and acquire load replaced by plain accesses must report a data race. ThreadSanitizer is not available for Windows targets, so here publish ran without it (it passed with clang and GCC, four runs at once included) and the control did not run; the Linux run with the control is pending, in Still to test.

Re [L9](https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/2020#issuecomment-64188): right, added in 1ac90685. The new `publish` case reads the name in a loop on one thread while another renames it, each time to a name never used before (2000 names, and on until the reader has read 100 new ones), with nothing but the name pointer between them; built with `-fsanitize=thread` where the compiler has it, the case must pass and a control built from the same source with the release store and acquire load replaced by plain accesses must report a data race. ThreadSanitizer is not available for Windows targets, so here `publish` ran without it (it passed with clang and GCC, four runs at once included) and the control did not run; the Linux run with the control is pending, in Still to test.
Author
Contributor

Re L10: right, fixed in 1ac90685. Every rendezvous a case waits for (both halves of mid, two, start, publish's start) fails the case when it does not come within 5 s, thread creation and joins are checked (CreateThread/WaitForSingleObject == WAIT_OBJECT_0/CloseHandle, pthread_create/pthread_join), and the Windows handles are closed.

Re [L10](https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/2020#issuecomment-64189): right, fixed in 1ac90685. Every rendezvous a case waits for (both halves of `mid`, `two`, `start`, `publish`'s start) fails the case when it does not come within 5 s, thread creation and joins are checked (`CreateThread`/`WaitForSingleObject` == `WAIT_OBJECT_0`/`CloseHandle`, `pthread_create`/`pthread_join`), and the Windows handles are closed.
Author
Contributor

@Beanstalk This keeps guest thread names (the log's prefixes, the profiler's and Tracy's names) from being written over when a thread is renamed, and touches the serial threads' wait spin (host/hos_thread.c). No switch: a revert gives back main's code. It has not run in the game on these PCs yet. Please start from the market-entrance save and play Castle Town Market for two minutes, the PR against main on the same PC, on a Linux PC with an AMD processor, on the Steam Deck and on a Windows PC. No stall or crash; the same thread names in longshot_log.txt as on main; frame time p99 equal to main's, within the run-to-run spread.

@Beanstalk This keeps guest thread names (the log's prefixes, the profiler's and Tracy's names) from being written over when a thread is renamed, and touches the serial threads' wait spin (host/hos_thread.c). No switch: a revert gives back main's code. It has not run in the game on these PCs yet. Please start from the market-entrance save and play Castle Town Market for two minutes, the PR against main on the same PC, on a Linux PC with an AMD processor, on the Steam Deck and on a Windows PC. No stall or crash; the same thread names in longshot_log.txt as on main; frame time p99 equal to main's, within the run-to-run spread.
Member

@siantm only members of BigBeans can start the farm. One of them can approve this request by reacting 👍 to it or to this comment.

@siantm only members of `BigBeans` can start the farm. One of them can approve this request by reacting 👍 to it or to this comment.
Author
Contributor

Re L11 (review 64191, tests/test_vcore_escape.py:268): right, fixed in 3c8dba82 and c29a9378. The escape fixture declares hos_fps_timing once (it had not compiled since #1553) and the mask-0 fixture includes longshot_env.h for env_on_or (not since #980); both now run with none of the caller's HOS_ switches (the old-rule case sets HOS_VCORE_STARVE_US=0 itself), and the escape fixture checks that a grant's long-wait line names the thread that let the core go as it was named then, though it is renamed right after (see L12). Both print "ok" with clang++ and g++ at the head and at the test commit; on main they still stop at those two compile errors.

Re L11 (review [64191](https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/2020#issuecomment-64191), `tests/test_vcore_escape.py:268`): right, fixed in 3c8dba82 and c29a9378. The escape fixture declares `hos_fps_timing` once (it had not compiled since #1553) and the mask-0 fixture includes `longshot_env.h` for `env_on_or` (not since #980); both now run with none of the caller's `HOS_` switches (the old-rule case sets `HOS_VCORE_STARVE_US=0` itself), and the escape fixture checks that a grant's long-wait line names the thread that let the core go as it was named then, though it is renamed right after (see L12). Both print "ok" with clang++ and g++ at the head and at the test commit; on `main` they still stop at those two compile errors.
Some checks are pending
CI / Conventional Commit title (pull_request) Blocked by required conditions
CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Blocked by required conditions
CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Blocked by required conditions
CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Blocked by required conditions
CI / Workflow lint (actionlint) (pull_request) Blocked by required conditions
CI / Website build and checks (pull_request) Blocked by required conditions
Guide screenshots / Guide images and local UI source review (pull_request) Blocked by required conditions
Third-party parts / As their notes say (pull_request) Blocked by required conditions
Third-party parts / The same as upstream (pull_request) Blocked by required conditions
Website / Website on Cloudflare (pull_request) Blocked by required conditions
PR description / Description follows the template (pull_request_target) Successful in 4s
beanstalk/approval waiting for a BigBean's 👍 on the request
Website / Website on Cloudflare (pull_request_target) Has been skipped
This pull request is marked as a work in progress.
Some workflows are waiting to be reviewed.
This branch is out-of-date with the base branch
View command line instructions

Checkout

From your project repository, check out a new branch and test the changes.
git fetch -u 'fix/os-thread-names-kept:siantm-fix/os-thread-names-kept'
git switch 'siantm-fix/os-thread-names-kept'
Sign in to join this conversation.
No reviewers
No milestone
No project
No assignees
3 participants
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
MagicBeanDev/Longshot!2020
No description provided.