perf(vk): the recorder thread's slot resets without the wait and an idle that waits on its ring (RX_VK_RECORDER_OWN, RX_VK_RECORDER_IDLE=wait; opt-in) #162

Merged
HumanBean merged 6 commits from klimovich008/Longshot:perf/recorder-own-idle into main 2026-11-14 02:11:20 +00:00
Contributor

Why

With RX_VK_RECORDER=thread (Hookshot #1473) the replay thread still waits for the recorder thread at every recording slot's
reuse, for a descriptor pool reset nothing in the recorder's ring can touch, and the recorder spins up to 200 us each time it runs
dry: in a census of an older Hookshot build (Hyrule Field, about 60 fps) about 25-32 such waits a frame (~0.13 ms of the replay thread) and about half a core of spin. This adds two
opt-in switches that take each away. Both are off by default; with both unset nothing changes.

What changed

  • recorder: RX_VK_RECORDER_OWN=1 (gpu/vk/vk_recorder.h: OwnSlot, slot_begin, slot_done, produced, and one line in
    boundary()). A recording slot's own calls at its reuse skip the replay thread's wait for the recorder: begin_recording's two
    vkResetCommandPool and two vkBeginCommandBuffer (gpu/vk/vk_replay.cpp), and draw_begin_recording's
    vkResetDescriptorPool and the destruction of the extra pools it grew (gpu/vk/vk_draw.cpp). Nothing still in the ring can name
    them: the slot's last recording ended with a wait for the recorder (vkEndCommandBuffer), was submitted (each submit waits too)
    and its submit is done (recording_wait). The calls stay direct on the replay thread, which goes on to allocate and write
    descriptor sets from that pool; only the wait goes. slot_begin checks the recorder has run past the slot's last end
    (rec_end_pos, set after the two ends in submit_recording); if not, or with the switch off, the calls wait as before (logged
    once). Every other reset, begin and destruction keeps its wait: the frame's command buffer, ImGui, NGX, FSR.
  • recorder: RX_VK_RECORDER_IDLE=wait (Recorder::idle, wake, start). With nothing to run the recorder spins 20 us, then
    waits on its ring counter W with WaitOnAddress (compare against the last W it saw, 2 ms timeout), and the replay thread's
    publish wakes it with WakeByAddressSingle; unset or spin, the adaptive 20-200 us spin and the condition variable as before.
    On Windows the two functions come from kernelbase.dll at run time (no Synchronization.lib in the link); on Linux, host/posix's
    futex. Without them the switch turns itself off at the recorder's start, logged.
  • tests: tests/test_recorder_own_idle.py and tests/recorder_own_idle_test.cpp (new); the source-cutting fixtures of
    test_draw_stream_visibility, test_fsr_uniform_ownership, test_fsr_image_retirement and test_recording_admission get stubs
    for the new marks.
  • docs: both switches in docs/development.md (The GPU).

Flags and settings

Flag What it does Default now Recommendation Helps on Doesn't help on Untested on
RX_VK_RECORDER_OWN (set) With RX_VK_RECORDER=thread: a slot's own pool resets and begins at its reuse without the wait for the recorder off off by default for now; on with the recorder after more testing (other GPUs, Linux) RX 9070 XT, Windows: +2.0 % fps, 3.7 ms less CPU a frame than the recorder alone (n=2, diagnostic) not known NVIDIA, Intel, Linux, Steam Deck
RX_VK_RECORDER_IDLE=wait With RX_VK_RECORDER=thread: the recorder's idle waits on its ring after a 20 us spin, instead of spinning up to 200 us off off: it costs fps here - RX 9070 XT, Windows: -1.2 % fps (n=2, diagnostic); the replay thread waits longer for the woken recorder NVIDIA, Intel, Linux, Steam Deck

Measurements

main: 839e85ff (2026-11-13)
this PR: perf/recorder-own-idle, its commits on 839e85ff @ 9ac9b714 (the same diff as this branch's)
Hardware: RX 9070 XT, AMD 26.10.2 (Vulkan driver 0x8001a0), Ryzen 7 5700X3D, 64 GB, Windows 11
Scene: Hyrule Field, the standard test: the save loaded, the camera turning at full speed, frame rate unlocked
(HOS_FPS_MAX=240), 1280x720 window, 2048x1152 output (16:9), no frame generation
Method: a 20 s window after the load, warm caches, 3 alternating pairs of the four arms; iteration 1 was rejected whole (the
thread arm's clock read 0.908 over frames of 407 and 486 ms), so 2 runs an arm. fps and CPU ms a frame from the present log and
the process times; the replay's times from the log's "a frame's replay time (ms)" lines and the recorder's from its 10 s stats
line, both inside the window.
How to reproduce: the build above with RX_VK_RECORDER=thread, then RX_VK_RECORDER_OWN=1 or RX_VK_RECORDER_IDLE=wait;
our standard-test script with one arm each, alternated.

Baseline: Longshot 839e85ff + this PR @ 9ac9b714, RX_VK_RECORDER=thread

Main was not the comparison: both switches are options of the recorder thread, which is itself opt-in. With them unset the code
is main's, so the recorder-off and recorder-on columns are main's two paths on this build.

Measured on main 839e85ff; main is at 514591a3 since. Its 36 commits in between add the occlusion query reports (#197, #260) and the Damage taken row, among others, and change none of the code this PR changes (vk_recorder.h, begin_recording and submit_recording in vk_replay.cpp, draw_begin_recording in vk_draw.cpp): the fps on today's main will differ, the difference between the arms is this PR's own. A measurement on the next base will replace these.

Longshot 839e85ff + this PR @ 9ac9b714, recorder off Longshot 839e85ff + this PR @ 9ac9b714, RX_VK_RECORDER=thread Longshot 839e85ff + this PR @ 9ac9b714, thread + RX_VK_RECORDER_OWN=1 Longshot 839e85ff + this PR @ 9ac9b714, thread + RX_VK_RECORDER_IDLE=wait
fps 66.52 75.04 76.57 (+2.0 %) 74.12 (-1.2 %)
CPU ms a frame 59.07 68.65 64.91 (-3.7) 66.82 (-1.8)
p99 frame ms 22.10 19.93 19.01 20.17
worst frame ms 57.4 67.8 47.3 69.5
replay thread's waits for the recorder, ms a second 14.97 9.34 22.10
of them at vkResetDescriptorPool, a second 1,693 (7.65 ms) 0 1,688 (15.01 ms)
recorder spinning idle, of a core 0.44 0.45 0.32
replay's submit time, ms a frame 11.46 9.23 9.07 9.61
replay's present time, ms a frame 0.10 0.11 0.10 0.12

Each difference is under 5 % with 2 runs an arm: diagnostic. OWN removes the replay thread's waits at the descriptor pool reset
(1,693 a second to none) and its waits for the recorder fall by about 5.6 ms a second. The present did not take up what the
replay saved: the replay's present time is ~0.1 ms a frame in every arm. IDLE=wait spins less (0.32 of a core against 0.44) but
the replay thread waits longer for the recorder it woke (22.1 against 15.0 ms a second), and fps drops: as measured here it does
not pay.

std_test, per run (fps, CPU ms a frame; iterations 0 and 2)
off       66.59 57.95 | 66.44 60.20
thread    75.54 67.33 | 74.54 69.96
own       76.79 64.78 | 76.34 65.04
idle      74.79 66.12 | 73.44 67.52
vk: recorder: ... the replay waited for it N times, T ms in all; ... vkResetDescriptorPool N/T ms   (10 s lines in the window)
vk: a frame's replay time (ms): submit S, present P, ...                                             (in the window)

Testing

Tested: game-free, on Windows with llvm-mingw clang. tests/test_recorder_own_idle.py builds a fixture with the real
vk_work.h and vk_recorder.h and a fake Vulkan, the recorder thread held in a gated packet (no timing: a test hook lets the
packet run when the replay thread starts to wait, and a 3 s watchdog turns a wrong wait into a failure):

  • the switches read as the renderer reads them (RX_VK_RECORDER_OWN by env_on; 0 and off are off), and named in the
    recorder's log line;
  • OWN: the slot's descriptor pool reset within OwnSlot reaches the driver while a packet is in flight; a call that is not the
    slot's own (vkEndCommandBuffer) within it, the reset outside it, the switch off, and a slot whose last end the recorder has
    not run (the guard, logged once) all wait for the packet;
  • IDLE: unset, WaitOnAddress is never called; =wait without the functions is turned off and logged; the recorder waiting on W
    is woken by a publish; a publish between sleeping set and the wait, with the wake made to do nothing, makes the wait return
    at once on the changed W;
  • the sources: OwnSlot only at the slot's own resets and begins, after slot_begin, and the end position kept after both ends.
    Controls: vk_recorder.h with vkEndCommandBuffer among the slot's own calls, the guard removed, the switch ignored, OwnSlot
    ignoring slot_begin, the wait's compare taken from W instead of the last W seen, or the wake removed, each fails its case; the
    source check finds an OwnSlot elsewhere and slot_begin after it. On the head d958b46a (main 514591a3 merged in): the renderer
    DLL builds and links, and tests/run_all.py gives 496 passed, 4 failed, 34 skipped. The four fail on main 514591a3 too
    (nx2/test_nup_paths.py, tests/test_check_third_party.py, tests/test_release.py, and tests/test_menu_i18n.py, checked
    on a clean main checkout). Since rebased linearly onto main 76699aad (head 5c7342c9, the merge dropped, no conflicts: the code commits are
    unchanged); the four runs' rows moved to docs/tested/. run_all was not run again on this head here.

Tested in the game: the four arms above on the RX 9070 XT (Windows 11), Hyrule Field; each switch named in the recorder's
start line, the guard's line never logged.

Not tested: Vulkan validation with either switch. Other scenes (the standard test only, by our rule). Not built on Linux.
With both switches unset the code paths are main's, the recorder's log line included (a Steam Deck profile that turns
RX_VK_RECORDER=thread on is not changed by this PR).

Needs other testers:

  • NVIDIA on Windows: RX_VK_RECORDER=thread with each switch, in the standard test (Hyrule Field, the camera turning fast), no new errors with Vulkan validation and the
    recorder's stats line, expecting fewer waits at "vkResetDescriptorPool" with OWN and less spin with IDLE (T-NSLN)
  • Linux, RADV: as above, and the RX_VK_RECORDER_IDLE=wait futex path (T-GTM9)

Screenshots

Nothing on screen.

Risk and rollback

Both off unless set, and only with RX_VK_RECORDER=thread, itself opt-in. OWN could let a reset race a recorded command if a
later change queued slot ends or let a slot be reused before its end ran: the guard then makes the calls wait, logged once.
IDLE=wait could leave the recorder asleep through a publish it missed: the 2 ms timeout and the replay thread's kick while it
waits remain as before. Unsetting a switch turns it off; reverting the commits is enough.

Reviewer notes

These are steps 1 and 2 of a design that went through four rounds of review; its step 3, moving the submits to the recorder, is
not part of this PR and is a separate decision. In practice step 1 is the descriptor pool reset: the
command pool resets and begins already find the ring empty.

Duplicate check (2026-11-13): Longshot's open and closed PRs and issues, #agent-updates, and NotRexed/Hookshot's open PRs
searched for the recorder, replay thread, parallel replay, dirty pages, descriptors, drains, futex/WaitOnAddress and spin. No one
implements either switch. #101 and #60 here touch tests/test_recorder_sync.py, which this PR does not change.
Hookshot #2912 measured the recorder thread itself (rows only).

Issues

None.

Checklist

For everyone

  • One topic.
  • The title is a Conventional Commit (type(scope): description).
  • Rebased on current main.
  • python -I -B tests/run_all.py run and passing (any failure that also fails on main is named above).
  • Tested in the game, or what wasn't is named under Testing.
  • Testing filled in: what was tested, what wasn't, and what needs other testers (with test requests).
  • Measured: main and this PR at named commits, on the same PC, scene and settings, at least 2 interleaved runs each, the median reported.
  • No game files, dumps, build output or generated files.
  • third_party/ untouched (a new folder with a PROVENANCE.md instead).
  • The license is OK: GPL-3.0-or-later, nothing incompatible and nothing decompiled.

If applicable

  • AI assistance recorded with Assisted-by: / Co-Authored-By: trailers.
  • Flags noted, and the switch reference updated (docs/development.md, the README).
  • A file in docs/tested/ for each new measured run (format in docs/tested-configurations.md; no appended row or assembled view).
  • A game-free test added for the fix, failing on main and passing here.
  • A check under tests/ that reads changed source updated in this PR, not weakened.
  • Docs updated.

🤖 Generated with Claude Code

## Why With `RX_VK_RECORDER=thread` ([Hookshot #1473](https://github.com/NotRexed/Hookshot/pull/1473)) the replay thread still waits for the recorder thread at every recording slot's reuse, for a descriptor pool reset nothing in the recorder's ring can touch, and the recorder spins up to 200 us each time it runs dry: in a census of an older Hookshot build (Hyrule Field, about 60 fps) about 25-32 such waits a frame (~0.13 ms of the replay thread) and about half a core of spin. This adds two opt-in switches that take each away. Both are off by default; with both unset nothing changes. ## What changed - recorder: `RX_VK_RECORDER_OWN=1` (`gpu/vk/vk_recorder.h`: `OwnSlot`, `slot_begin`, `slot_done`, `produced`, and one line in `boundary()`). A recording slot's own calls at its reuse skip the replay thread's wait for the recorder: `begin_recording`'s two `vkResetCommandPool` and two `vkBeginCommandBuffer` (`gpu/vk/vk_replay.cpp`), and `draw_begin_recording`'s `vkResetDescriptorPool` and the destruction of the extra pools it grew (`gpu/vk/vk_draw.cpp`). Nothing still in the ring can name them: the slot's last recording ended with a wait for the recorder (`vkEndCommandBuffer`), was submitted (each submit waits too) and its submit is done (`recording_wait`). The calls stay direct on the replay thread, which goes on to allocate and write descriptor sets from that pool; only the wait goes. `slot_begin` checks the recorder has run past the slot's last end (`rec_end_pos`, set after the two ends in `submit_recording`); if not, or with the switch off, the calls wait as before (logged once). Every other reset, begin and destruction keeps its wait: the frame's command buffer, ImGui, NGX, FSR. - recorder: `RX_VK_RECORDER_IDLE=wait` (`Recorder::idle`, `wake`, `start`). With nothing to run the recorder spins 20 us, then waits on its ring counter W with `WaitOnAddress` (compare against the last W it saw, 2 ms timeout), and the replay thread's publish wakes it with `WakeByAddressSingle`; unset or `spin`, the adaptive 20-200 us spin and the condition variable as before. On Windows the two functions come from kernelbase.dll at run time (no Synchronization.lib in the link); on Linux, host/posix's futex. Without them the switch turns itself off at the recorder's start, logged. - tests: `tests/test_recorder_own_idle.py` and `tests/recorder_own_idle_test.cpp` (new); the source-cutting fixtures of `test_draw_stream_visibility`, `test_fsr_uniform_ownership`, `test_fsr_image_retirement` and `test_recording_admission` get stubs for the new marks. - docs: both switches in `docs/development.md` (The GPU). ## Flags and settings | Flag | What it does | Default now | Recommendation | Helps on | Doesn't help on | Untested on | |---|---|---|---|---|---|---| | `RX_VK_RECORDER_OWN` (set) | With `RX_VK_RECORDER=thread`: a slot's own pool resets and begins at its reuse without the wait for the recorder | off | off by default for now; on with the recorder after more testing (other GPUs, Linux) | RX 9070 XT, Windows: +2.0 % fps, 3.7 ms less CPU a frame than the recorder alone (n=2, diagnostic) | not known | NVIDIA, Intel, Linux, Steam Deck | | `RX_VK_RECORDER_IDLE=wait` | With `RX_VK_RECORDER=thread`: the recorder's idle waits on its ring after a 20 us spin, instead of spinning up to 200 us | off | off: it costs fps here | - | RX 9070 XT, Windows: -1.2 % fps (n=2, diagnostic); the replay thread waits longer for the woken recorder | NVIDIA, Intel, Linux, Steam Deck | ## Measurements main: 839e85ff (2026-11-13) this PR: perf/recorder-own-idle, its commits on 839e85ff @ 9ac9b714 (the same diff as this branch's) Hardware: RX 9070 XT, AMD 26.10.2 (Vulkan driver 0x8001a0), Ryzen 7 5700X3D, 64 GB, Windows 11 Scene: Hyrule Field, the standard test: the save loaded, the camera turning at full speed, frame rate unlocked (`HOS_FPS_MAX=240`), 1280x720 window, 2048x1152 output (16:9), no frame generation Method: a 20 s window after the load, warm caches, 3 alternating pairs of the four arms; iteration 1 was rejected whole (the thread arm's clock read 0.908 over frames of 407 and 486 ms), so 2 runs an arm. fps and CPU ms a frame from the present log and the process times; the replay's times from the log's "a frame's replay time (ms)" lines and the recorder's from its 10 s stats line, both inside the window. How to reproduce: the build above with `RX_VK_RECORDER=thread`, then `RX_VK_RECORDER_OWN=1` or `RX_VK_RECORDER_IDLE=wait`; our standard-test script with one arm each, alternated. Baseline: Longshot 839e85ff + this PR @ 9ac9b714, RX_VK_RECORDER=thread Main was not the comparison: both switches are options of the recorder thread, which is itself opt-in. With them unset the code is main's, so the recorder-off and recorder-on columns are main's two paths on this build. Measured on main 839e85ff; main is at 514591a3 since. Its 36 commits in between add the occlusion query reports (#197, #260) and the Damage taken row, among others, and change none of the code this PR changes (`vk_recorder.h`, `begin_recording` and `submit_recording` in `vk_replay.cpp`, `draw_begin_recording` in `vk_draw.cpp`): the fps on today's main will differ, the difference between the arms is this PR's own. A measurement on the next base will replace these. | | Longshot 839e85ff + this PR @ 9ac9b714, recorder off | Longshot 839e85ff + this PR @ 9ac9b714, RX_VK_RECORDER=thread | Longshot 839e85ff + this PR @ 9ac9b714, thread + RX_VK_RECORDER_OWN=1 | Longshot 839e85ff + this PR @ 9ac9b714, thread + RX_VK_RECORDER_IDLE=wait | |---|---|---|---|---| | fps | 66.52 | 75.04 | 76.57 (+2.0 %) | 74.12 (-1.2 %) | | CPU ms a frame | 59.07 | 68.65 | 64.91 (-3.7) | 66.82 (-1.8) | | p99 frame ms | 22.10 | 19.93 | 19.01 | 20.17 | | worst frame ms | 57.4 | 67.8 | 47.3 | 69.5 | | replay thread's waits for the recorder, ms a second | | 14.97 | 9.34 | 22.10 | | of them at vkResetDescriptorPool, a second | | 1,693 (7.65 ms) | 0 | 1,688 (15.01 ms) | | recorder spinning idle, of a core | | 0.44 | 0.45 | 0.32 | | replay's submit time, ms a frame | 11.46 | 9.23 | 9.07 | 9.61 | | replay's present time, ms a frame | 0.10 | 0.11 | 0.10 | 0.12 | Each difference is under 5 % with 2 runs an arm: diagnostic. OWN removes the replay thread's waits at the descriptor pool reset (1,693 a second to none) and its waits for the recorder fall by about 5.6 ms a second. The present did not take up what the replay saved: the replay's present time is ~0.1 ms a frame in every arm. IDLE=wait spins less (0.32 of a core against 0.44) but the replay thread waits longer for the recorder it woke (22.1 against 15.0 ms a second), and fps drops: as measured here it does not pay. ``` std_test, per run (fps, CPU ms a frame; iterations 0 and 2) off 66.59 57.95 | 66.44 60.20 thread 75.54 67.33 | 74.54 69.96 own 76.79 64.78 | 76.34 65.04 idle 74.79 66.12 | 73.44 67.52 vk: recorder: ... the replay waited for it N times, T ms in all; ... vkResetDescriptorPool N/T ms (10 s lines in the window) vk: a frame's replay time (ms): submit S, present P, ... (in the window) ``` ## Testing **Tested:** game-free, on Windows with llvm-mingw clang. `tests/test_recorder_own_idle.py` builds a fixture with the real `vk_work.h` and `vk_recorder.h` and a fake Vulkan, the recorder thread held in a gated packet (no timing: a test hook lets the packet run when the replay thread starts to wait, and a 3 s watchdog turns a wrong wait into a failure): - the switches read as the renderer reads them (`RX_VK_RECORDER_OWN` by `env_on`; 0 and off are off), and named in the recorder's log line; - OWN: the slot's descriptor pool reset within `OwnSlot` reaches the driver while a packet is in flight; a call that is not the slot's own (`vkEndCommandBuffer`) within it, the reset outside it, the switch off, and a slot whose last end the recorder has not run (the guard, logged once) all wait for the packet; - IDLE: unset, `WaitOnAddress` is never called; `=wait` without the functions is turned off and logged; the recorder waiting on W is woken by a publish; a publish between `sleeping` set and the wait, with the wake made to do nothing, makes the wait return at once on the changed W; - the sources: `OwnSlot` only at the slot's own resets and begins, after `slot_begin`, and the end position kept after both ends. Controls: `vk_recorder.h` with `vkEndCommandBuffer` among the slot's own calls, the guard removed, the switch ignored, `OwnSlot` ignoring `slot_begin`, the wait's compare taken from W instead of the last W seen, or the wake removed, each fails its case; the source check finds an `OwnSlot` elsewhere and `slot_begin` after it. On the head d958b46a (main 514591a3 merged in): the renderer DLL builds and links, and `tests/run_all.py` gives 496 passed, 4 failed, 34 skipped. The four fail on main 514591a3 too (`nx2/test_nup_paths.py`, `tests/test_check_third_party.py`, `tests/test_release.py`, and `tests/test_menu_i18n.py`, checked on a clean main checkout). Since rebased linearly onto main 76699aad (head 5c7342c9, the merge dropped, no conflicts: the code commits are unchanged); the four runs' rows moved to `docs/tested/`. `run_all` was not run again on this head here. **Tested in the game:** the four arms above on the RX 9070 XT (Windows 11), Hyrule Field; each switch named in the recorder's start line, the guard's line never logged. **Not tested:** Vulkan validation with either switch. Other scenes (the standard test only, by our rule). Not built on Linux. With both switches unset the code paths are main's, the recorder's log line included (a Steam Deck profile that turns `RX_VK_RECORDER=thread` on is not changed by this PR). **Needs other testers:** - NVIDIA on Windows: `RX_VK_RECORDER=thread` with each switch, in the standard test (Hyrule Field, the camera turning fast), no new errors with Vulkan validation and the recorder's stats line, expecting fewer waits at "vkResetDescriptorPool" with OWN and less spin with IDLE (T-NSLN) - Linux, RADV: as above, and the `RX_VK_RECORDER_IDLE=wait` futex path (T-GTM9) ## Screenshots Nothing on screen. ## Risk and rollback Both off unless set, and only with `RX_VK_RECORDER=thread`, itself opt-in. OWN could let a reset race a recorded command if a later change queued slot ends or let a slot be reused before its end ran: the guard then makes the calls wait, logged once. IDLE=wait could leave the recorder asleep through a publish it missed: the 2 ms timeout and the replay thread's kick while it waits remain as before. Unsetting a switch turns it off; reverting the commits is enough. ## Reviewer notes These are steps 1 and 2 of a design that went through four rounds of review; its step 3, moving the submits to the recorder, is not part of this PR and is a separate decision. In practice step 1 is the descriptor pool reset: the command pool resets and begins already find the ring empty. Duplicate check (2026-11-13): Longshot's open and closed PRs and issues, #agent-updates, and NotRexed/Hookshot's open PRs searched for the recorder, replay thread, parallel replay, dirty pages, descriptors, drains, futex/WaitOnAddress and spin. No one implements either switch. #101 and #60 here touch `tests/test_recorder_sync.py`, which this PR does not change. [Hookshot #2912](https://github.com/NotRexed/Hookshot/pull/2912) measured the recorder thread itself (rows only). ## Issues None. ## Checklist ### For everyone - [x] One topic. - [x] The title is a Conventional Commit (`type(scope): description`). - [x] Rebased on current `main`. - [x] `python -I -B tests/run_all.py` run and passing (any failure that also fails on `main` is named above). - [x] Tested in the game, or what wasn't is named under Testing. - [x] Testing filled in: what was tested, what wasn't, and what needs other testers (with test requests). - [x] Measured: main and this PR at named commits, on the same PC, scene and settings, at least 2 interleaved runs each, the median reported. - [x] No game files, dumps, build output or generated files. - [x] `third_party/` untouched (a new folder with a `PROVENANCE.md` instead). - [x] The license is OK: GPL-3.0-or-later, nothing incompatible and nothing decompiled. ### If applicable - [x] AI assistance recorded with `Assisted-by:` / `Co-Authored-By:` trailers. - [x] Flags noted, and the switch reference updated (docs/development.md, the README). - [x] A file in `docs/tested/` for each new measured run (format in `docs/tested-configurations.md`; no appended row or assembled view). - [x] A game-free test added for the fix, failing on `main` and passing here. - [x] A check under `tests/` that reads changed source updated in this PR, not weakened. - [x] Docs updated. 🤖 Generated with [Claude Code](https://claude.com/claude-code)
Both opt-in, for RX_VK_RECORDER=thread. OWN: a recording slot's own command
pool resets and begins and its descriptor pool reset, at the slot's reuse, no
longer wait for the recorder (OwnSlot at those call sites; slot_begin() checks
the recorder ran the slot's last end). IDLE=wait: after a 20 us spin the
recorder waits on its ring counter (WaitOnAddress, a futex on Linux), and a
publish wakes it. Unset, both behave as before.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
The OWN cases let the gated packet run when the producer starts waiting
(a test hook in wait_idle), not after 20 ms; the race case wakes the parked
recorder directly instead of through a packet the producer would wait for;
the switches printed after mode() reads them; the source check takes the
OwnSlot's whole block.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
draw_stream_visibility, fsr_uniform_ownership, fsr_image_retirement and
recording_admission cut begin_recording, submit_recording or
draw_begin_recording from the source: they get rec::slot_begin, slot_done,
produced, OwnSlot and rec_end_pos as the recorder-off renderer has them. The
OWN skip in boundary() is its own line, so test_recorder_sync's unordered()
variant finds its anchor.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Co-Authored-By: Claude Opus 5.5 <[email protected]>
docs(development): RX_VK_RECORDER_OWN and RX_VK_RECORDER_IDLE in The GPU
Some checks failed
PR description / Description follows the template (pull_request_target) Successful in 4s
CI / Conventional Commit title (pull_request) Has been cancelled
CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Has been cancelled
CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Has been cancelled
CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Has been cancelled
CI / Workflow lint (actionlint) (pull_request) Has been cancelled
Third-party parts / As their notes say (pull_request) Has been cancelled
Third-party parts / The same as upstream (pull_request) Has been cancelled
cb5719d270
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Author
Contributor

Measurement plan: the standard Hyrule test only (no Kokiri or Temple route), 3 alternating pairs per comparison on the Longshot base at 16:9; under 5 % from 3 pairs reported as diagnostic. The tester request names the same scene.

Measurement plan: the standard Hyrule test only (no Kokiri or Temple route), 3 alternating pairs per comparison on the Longshot base at 16:9; under 5 % from 3 pairs reported as diagnostic. The tester request names the same scene.
Author
Contributor

Measured on Longshot (base 839e85ff + this PR as a pack of the same patch, Hyrule standard test at 16:9, 2 valid pairs of 3 on an RX 9070 XT): recorder off 66.52 fps, RX_VK_RECORDER=thread 75.04, + RX_VK_RECORDER_OWN=1 76.57 (+2.0 % over the thread alone, about 40 % fewer replay waits for the recorder), + RX_VK_RECORDER_IDLE=wait 74.12 (-1.2 %). The third pair was rejected for stalls in a thread-only run. Diagnostic; both switches stay off. Description updated; merging current main next.

Measured on Longshot (base 839e85ff + this PR as a pack of the same patch, Hyrule standard test at 16:9, 2 valid pairs of 3 on an RX 9070 XT): recorder off 66.52 fps, RX_VK_RECORDER=thread 75.04, + RX_VK_RECORDER_OWN=1 76.57 (+2.0 % over the thread alone, about 40 % fewer replay waits for the recorder), + RX_VK_RECORDER_IDLE=wait 74.12 (-1.2 %). The third pair was rejected for stalls in a thread-only run. Diagnostic; both switches stay off. Description updated; merging current main next.
Author
Contributor

Measured on Longshot 839e85ff + this PR (9ac9b714, same diff), Hyrule standard test at 16:9, 2 valid runs an arm: the recorder thread 75.04 fps; with RX_VK_RECORDER_OWN=1 76.57 (+2.0 %, 3.7 ms less CPU a frame, no waits left at vkResetDescriptorPool); with RX_VK_RECORDER_IDLE=wait 74.12 (-1.2 %: the replay waits longer for the woken recorder). Diagnostic. The present took up none of it (~0.1 ms a frame in every arm). Rows added to docs/tested-configurations.md (8d rows commit).

Measured on Longshot 839e85ff + this PR (9ac9b714, same diff), Hyrule standard test at 16:9, 2 valid runs an arm: the recorder thread 75.04 fps; with RX_VK_RECORDER_OWN=1 76.57 (+2.0 %, 3.7 ms less CPU a frame, no waits left at vkResetDescriptorPool); with RX_VK_RECORDER_IDLE=wait 74.12 (-1.2 %: the replay waits longer for the woken recorder). Diagnostic. The present took up none of it (~0.1 ms a frame in every arm). Rows added to docs/tested-configurations.md (8d rows commit).
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Co-Authored-By: Claude Opus 5.5 <[email protected]>
docs(tested): RX 9070 XT, the standard test with the recorder thread and each of its new switches
Some checks failed
PR description / Description follows the template (pull_request_target) Successful in 5s
CI / Conventional Commit title (pull_request) Has been cancelled
CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Has been cancelled
CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Has been cancelled
CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Has been cancelled
CI / Workflow lint (actionlint) (pull_request) Has been cancelled
Third-party parts / As their notes say (pull_request) Has been cancelled
Third-party parts / The same as upstream (pull_request) Has been cancelled
Website / Website on Cloudflare (pull_request) Has been cancelled
0d934b045f
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Author
Contributor

Says why the 839e85ff numbers hold on today's main: main's later commits do not touch the recorder or the slot functions.

Says why the 839e85ff numbers hold on today's main: main's later commits do not touch the recorder or the slot functions.
docs(tested): one set of rows for the 839e85ff A/B (the per-arm ones)
Some checks failed
PR description / Description follows the template (pull_request_target) Successful in 3s
Website / Website on Cloudflare (pull_request_target) Has been skipped
CI / Conventional Commit title (pull_request) Has been cancelled
CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Has been cancelled
CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Has been cancelled
CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Has been cancelled
CI / Workflow lint (actionlint) (pull_request) Has been cancelled
Third-party parts / As their notes say (pull_request) Has been cancelled
Third-party parts / The same as upstream (pull_request) Has been cancelled
d958b46a28
Co-Authored-By: Claude Opus 5.5 <[email protected]>
Author
Contributor

Test requests open: T-NSLN (NVIDIA RTX, Windows) and T-GTM9 (AMD or NVIDIA, Linux). They can be claimed once a maintainer adds the 'testing wanted' label.

Test requests open: T-NSLN (NVIDIA RTX, Windows) and T-GTM9 (AMD or NVIDIA, Linux). They can be claimed once a maintainer adds the 'testing wanted' label.
Author
Contributor

run_all on d958b46a (main 514591a3 merged in): 496 passed, 4 failed, all four failing on main 514591a3 too (test_menu_i18n checked on a clean main checkout); the renderer DLL builds.

run_all on d958b46a (main 514591a3 merged in): 496 passed, 4 failed, all four failing on main 514591a3 too (test_menu_i18n checked on a clean main checkout); the renderer DLL builds.
klimovich008 changed title from WIP: perf(vk): the recorder thread's slot resets without the wait and an idle that waits on its ring (RX_VK_RECORDER_OWN, RX_VK_RECORDER_IDLE=wait; opt-in) to perf(vk): the recorder thread's slot resets without the wait and an idle that waits on its ring (RX_VK_RECORDER_OWN, RX_VK_RECORDER_IDLE=wait; opt-in) 2026-11-13 20:21:00 +00:00
Contributor

Tested on NVIDIA, Windows (test request T-NSLN): both switches work as described. OWN removes every wait at vkResetDescriptorPool. IDLE=wait spins less, but the replay waits longer. Neither switch shows a new validation error.

Build: this PR's head d958b46a (longshot v3.0.0-rc.3-34-gd958b46a), built with make_game.py --update (llvm-mingw).
Hardware: RTX 5070 Ti 16 GB, driver 621.01 (Vulkan 0x9b404000), Ryzen 7 7800X3D, 32 GB, Windows 11 Pro 26200.
Scene: Hyrule Field from a save, the right stick held fully right from the moment play starts. Frame rate unlocked (HOS_FPS_MAX=240), HOS_FPS_LOG=1, a 1280x720 window (1920x1080 at 150 % scaling), 2048x1152 output, Quality preset Custom with Render resolution 75 %, no frame generation, audio off.
Method: one warm-up leg was dropped. Then 3 rounds of the four arms, in alternating order (off, thread, own, idle, then reversed). Each leg waits 5 s and measures 30 s. fps and frame times come from PresentMon 2.6. CPU ms a frame is the process CPU time over the window divided by its frames. The recorder rows come from its 10 s stats lines in the window, and the replay rows from its "a frame's replay time" lines. The game clock read 1.00 in every leg.

median of 3 (each run) recorder off RX_VK_RECORDER=thread thread + RX_VK_RECORDER_OWN=1 thread + RX_VK_RECORDER_IDLE=wait
fps 113.80 (113.97 / 113.31 / 113.80) 116.14 (115.72 / 116.14 / 117.19) 117.00 (117.31 / 117.00 / 115.95), +0.7 % 115.18 (115.18 / 115.00 / 116.35), -0.8 %
CPU ms a frame 36.90 40.15 40.40 40.40
p99 frame ms 11.31 10.98 10.98 11.08
worst frame ms 15.74 14.27 15.52 13.09
replay thread's waits for the recorder, ms a second 16.04 13.44 19.59
of them at vkResetDescriptorPool, a second 2,475 (3.77 ms) 0 2,445 (6.99 ms)
recorder spinning idle, of a core 0.42 0.42 0.30
replay's submit, ms a frame 6.31 5.19 5.21 5.29
replay's present, ms a frame 0.08 0.08 0.07 0.08

The fps differences are under 1 % and the runs overlap, so they are diagnostic only. The mechanism reproduces the AMD numbers:

  • OWN takes the descriptor-pool waits from about 2,475 a second to none, and the replay's total wait for the recorder falls by about 2.6 ms a second.
  • The CPU cost a frame does not drop here (40.15 against 40.40 ms). On the RX 9070 XT it dropped 3.7 ms.
  • IDLE=wait cuts the recorder's idle spin from 0.42 to 0.30 of a core. The replay then waits longer for it (19.6 against 16.0 ms a second), and fps does not improve. That matches the PR's "off: it costs fps here".
  • The OWN guard line ("a recording slot reused before the recorder ran its last end") was never logged in any OWN leg.

Vulkan validation (RX_VK_VALIDATION=1, Khronos layer from SDK 1.4.341.1, confirmed loaded in the process): one leg each of thread, thread+OWN and thread+IDLE=wait, same scene.

  • No threading, command-pool-in-use or descriptor-pool-in-use errors in any arm.
  • All three arms show the same shader-creation VUIDs: VkShaderModuleCreateInfo-pCode-08740, RuntimeSpirv-MeshEXT-10883, RuntimeSpirv-shaderDenormFlushToZeroFloat32-06300 and -06297. These are unrelated to this PR.
  • The plain thread arm also logged vkCmdDrawIndexed-format-07753 once: the sampler fs_tex2_a8 expects UINT but is bound to a B10G11R11_UFLOAT view. That is main's path and not this PR's.

Not tested: Linux, Intel, other scenes, and both switches together.

Tested on NVIDIA, Windows (test request T-NSLN): both switches work as described. OWN removes every wait at `vkResetDescriptorPool`. IDLE=wait spins less, but the replay waits longer. Neither switch shows a new validation error. **Build:** this PR's head d958b46a (`longshot v3.0.0-rc.3-34-gd958b46a`), built with `make_game.py --update` (llvm-mingw). **Hardware:** RTX 5070 Ti 16 GB, driver 621.01 (Vulkan 0x9b404000), Ryzen 7 7800X3D, 32 GB, Windows 11 Pro 26200. **Scene:** Hyrule Field from a save, the right stick held fully right from the moment play starts. Frame rate unlocked (`HOS_FPS_MAX=240`), `HOS_FPS_LOG=1`, a 1280x720 window (1920x1080 at 150 % scaling), 2048x1152 output, Quality preset Custom with Render resolution 75 %, no frame generation, audio off. **Method:** one warm-up leg was dropped. Then 3 rounds of the four arms, in alternating order (off, thread, own, idle, then reversed). Each leg waits 5 s and measures 30 s. fps and frame times come from PresentMon 2.6. CPU ms a frame is the process CPU time over the window divided by its frames. The recorder rows come from its 10 s stats lines in the window, and the replay rows from its "a frame's replay time" lines. The game clock read 1.00 in every leg. | median of 3 (each run) | recorder off | `RX_VK_RECORDER=thread` | thread + `RX_VK_RECORDER_OWN=1` | thread + `RX_VK_RECORDER_IDLE=wait` | |---|---|---|---|---| | fps | 113.80 (113.97 / 113.31 / 113.80) | 116.14 (115.72 / 116.14 / 117.19) | 117.00 (117.31 / 117.00 / 115.95), +0.7 % | 115.18 (115.18 / 115.00 / 116.35), -0.8 % | | CPU ms a frame | 36.90 | 40.15 | 40.40 | 40.40 | | p99 frame ms | 11.31 | 10.98 | 10.98 | 11.08 | | worst frame ms | 15.74 | 14.27 | 15.52 | 13.09 | | replay thread's waits for the recorder, ms a second | | 16.04 | 13.44 | 19.59 | | of them at vkResetDescriptorPool, a second | | 2,475 (3.77 ms) | 0 | 2,445 (6.99 ms) | | recorder spinning idle, of a core | | 0.42 | 0.42 | 0.30 | | replay's submit, ms a frame | 6.31 | 5.19 | 5.21 | 5.29 | | replay's present, ms a frame | 0.08 | 0.08 | 0.07 | 0.08 | The fps differences are under 1 % and the runs overlap, so they are diagnostic only. The mechanism reproduces the AMD numbers: - OWN takes the descriptor-pool waits from about 2,475 a second to none, and the replay's total wait for the recorder falls by about 2.6 ms a second. - The CPU cost a frame does not drop here (40.15 against 40.40 ms). On the RX 9070 XT it dropped 3.7 ms. - IDLE=wait cuts the recorder's idle spin from 0.42 to 0.30 of a core. The replay then waits longer for it (19.6 against 16.0 ms a second), and fps does not improve. That matches the PR's "off: it costs fps here". - The OWN guard line ("a recording slot reused before the recorder ran its last end") was never logged in any OWN leg. **Vulkan validation** (`RX_VK_VALIDATION=1`, Khronos layer from SDK 1.4.341.1, confirmed loaded in the process): one leg each of thread, thread+OWN and thread+IDLE=wait, same scene. - No threading, command-pool-in-use or descriptor-pool-in-use errors in any arm. - All three arms show the same shader-creation VUIDs: `VkShaderModuleCreateInfo-pCode-08740`, `RuntimeSpirv-MeshEXT-10883`, `RuntimeSpirv-shaderDenormFlushToZeroFloat32-06300` and `-06297`. These are unrelated to this PR. - The plain thread arm also logged `vkCmdDrawIndexed-format-07753` once: the sampler `fs_tex2_a8` expects UINT but is bound to a B10G11R11_UFLOAT view. That is main's path and not this PR's. **Not tested:** Linux, Intel, other scenes, and both switches together.
Contributor

Test request T-GTM9: mixed

Hardware: RTX 3070 Ti 8 GB (it also drives the desktop), NVIDIA 595.104.02 (Vulkan driver 0x94da0080), i7-12700K, 16 GB, Pop!_OS 24.04 (kernel 7.2.15), Linux. Built d958b46a (the PR head, longshot v3.0.0-rc.3-34-gd958b46a, source c1dc3c0b) from source with clang (make_game.py, 18 min 52 s).

In short: on Linux both switches do what the PR says. OWN takes the replay's waits at vkResetDescriptorPool from about 1,250 a second to none. IDLE=wait takes the futex path (no "no WaitOnAddress" line), spins less and makes the replay wait longer. Neither switch adds a validation error. Two things stop me calling it a pass: the fps comparison came out inconclusive here, and under the Khronos layer this PC has, every recorder-thread run crashes inside the layer during the save's load screen, the plain RX_VK_RECORDER=thread arm included. So play was never validated with either switch.

What I checked: Hyrule Field from a save (child Link by the stone sign), the right stick held fully right from before the load ends, HOS_FPS_MAX=240, the game's own 2048x1152 output (16:9), headless (no window), audio off. One warm-up per arm first. Then 3 exclusive GPU holds of 4 runs each, alternated: thread/OWN/thread/OWN, OWN/thread/OWN/thread, thread/IDLE/thread/IDLE. That gives 6 thread, 4 OWN and 2 IDLE runs. Each run measures 30 s, starting 5 s after the save's load gap. fps comes from the renderer's frame log (headless, so CPU intervals after submission, not display timing), and CPU ms a frame from the process's CPU time over the window. Other processes used 0.04–0.24 cores in every window. The recorder rows are the 10 s vk: recorder: lines inside the window. The options file was the one Auto (this PC) wrote on the first start, tuned once and then unchanged through all the runs (checksum checked after each hold).

Recorder (each run, in the window):

hold, run fps CPU ms a frame GPU state replay waits for the recorder, a second of them at vkResetDescriptorPool, a second recorder spinning idle, of a core
A thread 49.85 69.7 fast 3,119 (8.40 ms) 1,260 (1.69 ms) 0.35
A OWN 49.76 69.7 fast 1,861 (6.80 ms) 0 0.35
A thread 40.35 81.5 slow 2,497 (8.77 ms) 1,009 (1.88 ms) 0.31
A OWN 53.66 66.8 fast 1,996 (6.97 ms) 0 0.36
B OWN 35.93 90.4 slow 1,361 (7.10 ms) 0 0.30
B thread 49.96 69.9 fast 3,129 (8.43 ms) 1,264 (1.74 ms) 0.35
B OWN 37.07 87.5 slow 1,394 (7.40 ms) 0 0.30
B thread 49.65 70.6 fast 3,104 (8.57 ms) 1,254 (1.76 ms) 0.35
C thread 37.07 86.6 slow 2,300 (8.56 ms) 929 (1.87 ms) 0.30
C IDLE 49.61 68.6 fast 3,105 (10.43 ms) 1,256 (2.87 ms) 0.26
C thread 49.73 69.2 fast 3,102 (8.01 ms) 1,253 (1.67 ms) 0.35
C IDLE 49.28 68.7 fast 3,065 (10.05 ms) 1,239 (2.74 ms) 0.26
  • OWN: the waits at vkResetDescriptorPool go to none in all 4 runs. The replay's total wait time for the recorder falls by about 1.5 ms a second (7.0 against 8.5). The guard line ("a recording slot reused before the recorder ran its last end") was never logged.
  • IDLE=wait (futex): the recorder's idle spin falls from 0.35 to 0.26 of a core. The replay waits longer for it (10.2 against 8.5 ms a second, and 2.8 against 1.7 ms a second at the descriptor pool reset). That is the same direction as on the RX 9070 XT and the RTX 5070 Ti.
  • On this PC the descriptor-pool waits are small to begin with (about 1.7 ms a second, against 7.65 on the RX 9070 XT), so OWN has little to win here.

fps: inconclusive. Runs fall into two GPU states regardless of arm. In the "fast" state GPU time is about 11 ms a frame and the longest submission 3.5 ms. In the "slow" state the draw counts and VRAM use are the same (5.3–5.5 of 6.8 GB), but GPU time is 15.5 ms and the longest submission 11–12 ms. The slow state hit 2 of 6 thread runs, 2 of 4 OWN and 0 of 2 IDLE, so the repeats of an arm differ by far more than 5 % and no pair decides. Comparing only fast-state runs: thread 49.65–49.96 fps (4 runs), OWN 49.76 and 53.66, IDLE 49.28 and 49.61. Nothing lower outside noise, but too few runs to claim anything. I didn't look further into what flips the GPU state; it shows in every arm, so it isn't this PR.

Vulkan validation (RX_VK_VALIDATION=1, Ubuntu's vulkan-validationlayers 1.3.275, its default reporting; the shader precompile skipped with RX_VK_PRECOMPILE=0 because it doesn't finish under the layer), 150 s runs:

arm ran validation errors which
recorder off the full 150 s 122 VkShaderModuleCreateInfo-pCode-08740, RuntimeSpirv-shaderDenormFlushToZeroFloat32-06300
thread crashed in the load (~30 s) 16,186 the two above, plus vkCmdDrawIndexed-None-07627/07628/07629 (5,032 each) and vkCmdDrawMeshTasksEXT-None-07627/07628/07629 (133 each)
thread + OWN crashed in the load (~30 s) 16,178 the same, same counts
thread + IDLE=wait crashed in the load (~30 s) 16,178 the same, same counts
thread, again crashed in the load (~30 s) 16,178 the same, same counts
  • The crash is a segfault inside the layer on the recorder thread (CRASH: exception 0xc0000005 (read 0x1a8) at libVkLayer_khronos_validation.so+0x141e7e9, called from mkwvk::rec::Recorder::loop()). An earlier round without RX_VK_PRECOMPILE=0 crashed the same way.
  • The 07627–07629 errors say vkCmdSetColorBlendEnableEXT, ...EquationEXT and ...WriteMaskEXT were "not called in the current command buffer" before a draw. They appear only with the recorder thread on, and the same in every recorder arm.
  • Neither switch adds an error: same VUIDs, same counts as the plain thread arm up to the crash. But the crash comes during the save's load screen (about 22 s into the renderer's clock, before the field appears), so play itself was never validated with the recorder on: what was validated is the title, the file select and the start of the load.
  • The Windows RTX 5070 Ti test above used SDK 1.4.341 and saw neither the crash nor these VUIDs. So this is most likely the old 1.3.275 layer mishandling extended dynamic state 3 recorded from a second thread, not the recorder. Either way it's main's RX_VK_RECORDER=thread path, not this PR's.

Game-free: tests/test_recorder_own_idle.py builds and passes on Linux with clang 23 (all cases, and all 7 broken variants fail as they should). That covers the fixture, not host/posix's futex WaitOnAddress itself, which the IDLE runs above use.

Not tested: both switches together. Another scene. A newer validation layer on Linux. A windowed (swapchain) run. The cause of the GPU's slow state.

The scene: Hyrule Field, the camera turning (warm-up run, thread arm, every 720 presents)

🤖 Generated with Claude Code

Test request T-GTM9: mixed Hardware: RTX 3070 Ti 8 GB (it also drives the desktop), NVIDIA 595.104.02 (Vulkan driver 0x94da0080), i7-12700K, 16 GB, Pop!_OS 24.04 (kernel 7.2.15), Linux. Built d958b46a (the PR head, `longshot v3.0.0-rc.3-34-gd958b46a`, source c1dc3c0b) from source with clang (`make_game.py`, 18 min 52 s). **In short:** on Linux both switches do what the PR says. OWN takes the replay's waits at `vkResetDescriptorPool` from about 1,250 a second to none. IDLE=wait takes the futex path (no "no WaitOnAddress" line), spins less and makes the replay wait longer. Neither switch adds a validation error. Two things stop me calling it a pass: the fps comparison came out inconclusive here, and under the Khronos layer this PC has, every recorder-thread run crashes inside the layer during the save's load screen, the plain `RX_VK_RECORDER=thread` arm included. So play was never validated with either switch. **What I checked:** Hyrule Field from a save (child Link by the stone sign), the right stick held fully right from before the load ends, `HOS_FPS_MAX=240`, the game's own 2048x1152 output (16:9), headless (no window), audio off. One warm-up per arm first. Then 3 exclusive GPU holds of 4 runs each, alternated: thread/OWN/thread/OWN, OWN/thread/OWN/thread, thread/IDLE/thread/IDLE. That gives 6 thread, 4 OWN and 2 IDLE runs. Each run measures 30 s, starting 5 s after the save's load gap. fps comes from the renderer's frame log (headless, so CPU intervals after submission, not display timing), and CPU ms a frame from the process's CPU time over the window. Other processes used 0.04–0.24 cores in every window. The recorder rows are the 10 s `vk: recorder:` lines inside the window. The options file was the one Auto (this PC) wrote on the first start, tuned once and then unchanged through all the runs (checksum checked after each hold). **Recorder (each run, in the window):** | hold, run | fps | CPU ms a frame | GPU state | replay waits for the recorder, a second | of them at vkResetDescriptorPool, a second | recorder spinning idle, of a core | |---|---|---|---|---|---|---| | A thread | 49.85 | 69.7 | fast | 3,119 (8.40 ms) | 1,260 (1.69 ms) | 0.35 | | A OWN | 49.76 | 69.7 | fast | 1,861 (6.80 ms) | 0 | 0.35 | | A thread | 40.35 | 81.5 | slow | 2,497 (8.77 ms) | 1,009 (1.88 ms) | 0.31 | | A OWN | 53.66 | 66.8 | fast | 1,996 (6.97 ms) | 0 | 0.36 | | B OWN | 35.93 | 90.4 | slow | 1,361 (7.10 ms) | 0 | 0.30 | | B thread | 49.96 | 69.9 | fast | 3,129 (8.43 ms) | 1,264 (1.74 ms) | 0.35 | | B OWN | 37.07 | 87.5 | slow | 1,394 (7.40 ms) | 0 | 0.30 | | B thread | 49.65 | 70.6 | fast | 3,104 (8.57 ms) | 1,254 (1.76 ms) | 0.35 | | C thread | 37.07 | 86.6 | slow | 2,300 (8.56 ms) | 929 (1.87 ms) | 0.30 | | C IDLE | 49.61 | 68.6 | fast | 3,105 (10.43 ms) | 1,256 (2.87 ms) | 0.26 | | C thread | 49.73 | 69.2 | fast | 3,102 (8.01 ms) | 1,253 (1.67 ms) | 0.35 | | C IDLE | 49.28 | 68.7 | fast | 3,065 (10.05 ms) | 1,239 (2.74 ms) | 0.26 | - **OWN:** the waits at `vkResetDescriptorPool` go to none in all 4 runs. The replay's total wait time for the recorder falls by about 1.5 ms a second (7.0 against 8.5). The guard line ("a recording slot reused before the recorder ran its last end") was never logged. - **IDLE=wait (futex):** the recorder's idle spin falls from 0.35 to 0.26 of a core. The replay waits longer for it (10.2 against 8.5 ms a second, and 2.8 against 1.7 ms a second at the descriptor pool reset). That is the same direction as on the RX 9070 XT and the RTX 5070 Ti. - On this PC the descriptor-pool waits are small to begin with (about 1.7 ms a second, against 7.65 on the RX 9070 XT), so OWN has little to win here. **fps: inconclusive.** Runs fall into two GPU states regardless of arm. In the "fast" state GPU time is about 11 ms a frame and the longest submission 3.5 ms. In the "slow" state the draw counts and VRAM use are the same (5.3–5.5 of 6.8 GB), but GPU time is 15.5 ms and the longest submission 11–12 ms. The slow state hit 2 of 6 thread runs, 2 of 4 OWN and 0 of 2 IDLE, so the repeats of an arm differ by far more than 5 % and no pair decides. Comparing only fast-state runs: thread 49.65–49.96 fps (4 runs), OWN 49.76 and 53.66, IDLE 49.28 and 49.61. Nothing lower outside noise, but too few runs to claim anything. I didn't look further into what flips the GPU state; it shows in every arm, so it isn't this PR. **Vulkan validation** (`RX_VK_VALIDATION=1`, Ubuntu's `vulkan-validationlayers` 1.3.275, its default reporting; the shader precompile skipped with `RX_VK_PRECOMPILE=0` because it doesn't finish under the layer), 150 s runs: | arm | ran | validation errors | which | |---|---|---|---| | recorder off | the full 150 s | 122 | `VkShaderModuleCreateInfo-pCode-08740`, `RuntimeSpirv-shaderDenormFlushToZeroFloat32-06300` | | thread | crashed in the load (~30 s) | 16,186 | the two above, plus `vkCmdDrawIndexed-None-07627/07628/07629` (5,032 each) and `vkCmdDrawMeshTasksEXT-None-07627/07628/07629` (133 each) | | thread + OWN | crashed in the load (~30 s) | 16,178 | the same, same counts | | thread + IDLE=wait | crashed in the load (~30 s) | 16,178 | the same, same counts | | thread, again | crashed in the load (~30 s) | 16,178 | the same, same counts | - The crash is a segfault inside the layer on the recorder thread (`CRASH: exception 0xc0000005 (read 0x1a8) at libVkLayer_khronos_validation.so+0x141e7e9`, called from `mkwvk::rec::Recorder::loop()`). An earlier round without `RX_VK_PRECOMPILE=0` crashed the same way. - The 07627–07629 errors say `vkCmdSetColorBlendEnableEXT`, `...EquationEXT` and `...WriteMaskEXT` were "not called in the current command buffer" before a draw. They appear only with the recorder thread on, and the same in every recorder arm. - **Neither switch adds an error:** same VUIDs, same counts as the plain thread arm up to the crash. But the crash comes during the save's load screen (about 22 s into the renderer's clock, before the field appears), so play itself was never validated with the recorder on: what was validated is the title, the file select and the start of the load. - The Windows RTX 5070 Ti test above used SDK 1.4.341 and saw neither the crash nor these VUIDs. So this is most likely the old 1.3.275 layer mishandling extended dynamic state 3 recorded from a second thread, not the recorder. Either way it's main's `RX_VK_RECORDER=thread` path, not this PR's. **Game-free:** `tests/test_recorder_own_idle.py` builds and passes on Linux with clang 23 (all cases, and all 7 broken variants fail as they should). That covers the fixture, not host/posix's futex `WaitOnAddress` itself, which the IDLE runs above use. **Not tested:** both switches together. Another scene. A newer validation layer on Linux. A windowed (swapchain) run. The cause of the GPU's slow state. ![The scene: Hyrule Field, the camera turning (warm-up run, thread arm, every 720 presents)](https://git.magicbean.dev/attachments/8ecce0bf-d914-4013-8278-3be1d372485e) 🤖 Generated with [Claude Code](https://claude.com/claude-code)
Author
Contributor

Crash test (play, not a measurement): a person played the game with the recorder thread and this PR's own slot resets, RX_VK_RECORDER=thread and RX_VK_RECORDER_OWN=1, to see whether the GPU crashes. RX_VK_RECORDER_IDLE=wait was not set, so this covers OWN only. (Edited: it said "both of this PR's switches" before; thanks, mooki.)

Build: this PR's commits on main 839e85ff (9ac9b714, the same patch as the measured pack), renderer DLL only
Hardware: RX 9070 XT, AMD 26.10.2, Ryzen 7 5700X3D, 64 GB, Windows 11
Play: about 25 minutes in two runs (5 and 20 minutes), from a save in normal play, keyboard and mouse, the runtime's options file as it comes (the game ran at 60 fps)

Result: no crash. No device lost, no Vulkan error, no exception or abort in either log, and the game exited normally when its window was closed. It held 60 fps for most of the play; the frames over 100 ms (25 of 230 ten-second stretches) came while pipelines were still being prepared in play.

Not tested: the default (both switches off) in the same play; a controller (it gave no input in this build, also with Steam closed; this PR does not touch input, so it is not from it); the PR's current head d958b46a (main 514591a3 merged in) in play.

Crash test (play, not a measurement): a person played the game with the recorder thread and this PR's own slot resets, `RX_VK_RECORDER=thread` and `RX_VK_RECORDER_OWN=1`, to see whether the GPU crashes. `RX_VK_RECORDER_IDLE=wait` was not set, so this covers OWN only. (Edited: it said "both of this PR's switches" before; thanks, mooki.) Build: this PR's commits on main 839e85ff (9ac9b714, the same patch as the measured pack), renderer DLL only Hardware: RX 9070 XT, AMD 26.10.2, Ryzen 7 5700X3D, 64 GB, Windows 11 Play: about 25 minutes in two runs (5 and 20 minutes), from a save in normal play, keyboard and mouse, the runtime's options file as it comes (the game ran at 60 fps) Result: no crash. No device lost, no Vulkan error, no exception or abort in either log, and the game exited normally when its window was closed. It held 60 fps for most of the play; the frames over 100 ms (25 of 230 ten-second stretches) came while pipelines were still being prepared in play. Not tested: the default (both switches off) in the same play; a controller (it gave no input in this build, also with Steam closed; this PR does not touch input, so it is not from it); the PR's current head d958b46a (main 514591a3 merged in) in play.
Member

Test request T-NSLN: mixed — both mechanisms reproduce, no new validation errors; no demonstrated FPS benefit on this PC.

Hardware: RTX 4070 12 GB, NVIDIA 621.01, Core i7-13700F, 32 GB RAM, Windows 11 Home 26H2 (26300.9974).

Build: renderer-only dev pack from PR head d958b46a28 (renderer source fingerprint c1dc3c0b), on the existing main 514591a30d base. Host remains main's v2.1.0-rc.3-413-g514591a3. The pack contains only longshot_vk.dll. Renderer compile/link succeeded, 0 compiler failures. All arms use that same host and renderer; only the recorder switches differ.

Scene/settings: Hyrule Field, child Link outside Lon Lon Ranch, separate copy of the same save each launch. Right stick fully right, left stick neutral. 1280x720 window (bridge-confirmed), 2048x1152 output, Custom quality, render resolution 100%, SDR, DLSS/FG/VSync off, HOS_FPS_MAX=240, real-time game speed, audio off. Camera rotation begins once the field is reached, followed by 12 s settling and a 30 s measurement.

Method: excluded first shader-preparation/warm-up run, then three rounds: thread/OWN/IDLE, IDLE/OWN/thread, thread/OWN/IDLE. Full present trace from HOS_PRESENT_TRACE, process CPU time divided by measured frames, and recorder 10 s stats inside each window. All nine measured windows passed background-load checks (mean <3 other cores, 5 s peak <=6). Validation runs are separate and excluded from these numbers. These are game-present-return intervals, not PresentMon/display scanout timings.

Median of 3 runs thread thread + OWN thread + IDLE=wait
FPS 82.59 81.41 82.09
CPU ms/frame 61.06 61.71 60.46
p99 frame ms 16.72 16.91 16.55
Worst frame ms (median of per-run maxima) 33.05 31.68 31.47
Replay waits for recorder, ms/second 12.40 9.64 15.40
Descriptor-pool waits/second 2248 0 2255
Descriptor-pool wait ms/second 2.80 0.00 3.89
Recorder idle spin, cores 0.463 0.453 0.338

Per-run data (all retained, no timed run rejected):

Round Arm FPS CPU ms/frame p99 ms Worst ms Frames >100 ms
1 thread 82.59 60.81 16.72 36.18 0
1 own 83.02 60.76 16.91 29.54 0
1 idle 83.18 59.86 16.16 31.47 0
2 thread 83.58 61.06 15.48 33.05 0
2 own 81.41 61.71 16.78 32.44 0
2 idle 82.09 60.57 16.55 29.15 0
3 thread 81.94 61.36 17.42 28.37 0
3 own 80.23 62.80 18.69 31.68 0
3 idle 81.03 60.46 17.61 31.59 0

OWN removes the logged waits at vkResetDescriptorPool in all three runs (about 2,248/second to zero). Total recorder wait time falls from 12.40 to 9.64 ms/second. The slot-reuse guard never logged. IDLE=wait reduces spin from 0.463 to 0.338 of a core, while increasing recorder wait time to 15.40 ms/second; the Windows wait functions were available (no fallback log).

FPS medians are OWN -1.43% and IDLE -0.60% relative to thread. Three short rounds, under-5% differences and overlapping ranges are diagnostic, not proof of a speedup or regression. The requested “FPS not lower” outcome is not established here, hence mixed rather than pass. CPU/frame did not improve with OWN. No frames >100 ms in any non-validation measured window. No crash/device-loss line in any run, and the inspected screenshots show the expected field and HUD.

Vulkan validation: official Khronos portable Windows nightly, manifest API 1.4.368, archive SHA-256 cfcb5fe29f248c295256366dfbf399511af7d8b5f03b14dd7c9563840d9cd39a. RX_VK_VALIDATION=1; RX_VK_PRECOMPILE=0 (the documented switch) in all three arms. The layer DLL was confirmed loaded in each game process. Each arm reached Hyrule Field, rotated the camera and completed a 30 s validation window, then exited normally.

All three arms report the same VUID set: VkShaderModuleCreateInfo-pCode-08740, RuntimeSpirv-shaderDenormFlushToZeroFloat32-06300 and RuntimeSpirv-shaderDenormPreserveFloat32-06297. No additional VUID with either switch, and no threading, command-pool-in-use or descriptor-pool-in-use error. This is a comparison against recorder-only's existing errors, not a clean validation pass.

game-free-bundled: 497 passed, 4 failed, 33 skipped in 121 s (windows, 24 at once)

main-failures: 4 passed, 4 failed, 0 skipped in 10 s (windows, 24 at once)

The four failing checks are nx2/test_nup_paths.py, tests/test_check_third_party.py, tests/test_menu_i18n.py and tests/test_release.py. All four also fail when rerun on clean main 514591a3 with the same interpreter/environment. The new recorder own/idle test and all changed-source fixture tests pass. Final suite uses the project's bundled Python 3.14.7, llvm-mingw and the existing header cache, without the optional NGX build variable leaking into game-free fixtures. An earlier system-Python run had additional missing-package/environment failures and is superseded by this run.

Recorder-off follow-up (same verified binaries and settings; fresh interleaved comparison, kept separate from the earlier nine runs):

The recorder is fully disabled in OFF: RX_VK_RECORDER, RX_VK_RECORDER_OWN and RX_VK_RECORDER_IDLE are all unset. THREAD sets only RX_VK_RECORDER=thread. One OFF warm-up was excluded, then three pairs in OFF/THREAD, THREAD/OFF, OFF/THREAD order. Same copied save, camera rotation, 12 s settling and 30 s measurement per run. All six windows passed the same background-load checks.

Median of 3 fresh runs Recorder OFF Recorder THREAD
FPS 79.28 81.26
CPU ms/frame 56.16 63.03
p99 frame ms 18.68 16.72
Worst frame ms (median of per-run maxima) 37.68 30.92
Pair Arm FPS CPU ms/frame p99 ms Worst ms Frames >100 ms
1 off 80.05 55.22 18.75 30.23 0
1 thread 81.26 63.20 17.82 30.92 0
2 off 77.89 57.53 17.22 37.68 0
2 thread 81.11 62.89 16.63 44.23 0
3 off 79.28 56.16 18.68 41.60 0
3 thread 82.13 63.03 16.72 21.94 0

Relative to OFF, THREAD median FPS is +2.50% and CPU cost is +6.86 ms/frame. These short three-pair results are diagnostic; the prior OWN/IDLE data are not pooled with this later comparison.

Recorder-OFF Vulkan validation also reached Hyrule Field, rotated the camera for a 30 s window and exited normally. Same portable Khronos layer and RX_VK_PRECOMPILE=0 as before; the layer DLL was confirmed loaded. VUIDs: RuntimeSpirv-shaderDenormFlushToZeroFloat32-06300, RuntimeSpirv-shaderDenormPreserveFloat32-06297, VkShaderModuleCreateInfo-pCode-08740. The VUID set matches the earlier THREAD/OWN/IDLE validation runs.

Not tested: both switches together, Linux/AMD/Intel, other scenes, long play sessions or a full host rebuild of the PR. The test request was unapproved when read, so this was a direct PR test without a request claim.

Tested with Codex following the repository's testing workflow. Game files, saves, runtimes and raw traces remain local.

Test request T-NSLN: mixed — both mechanisms reproduce, no new validation errors; no demonstrated FPS benefit on this PC. Hardware: RTX 4070 12 GB, NVIDIA 621.01, Core i7-13700F, 32 GB RAM, Windows 11 Home 26H2 (26300.9974). Build: renderer-only dev pack from PR head d958b46a28a5242971968f4eb0d0cfc180d87c3a (renderer source fingerprint c1dc3c0b), on the existing main 514591a30d707fe9f3ad31eae148c3cc1e750bff base. Host remains main's v2.1.0-rc.3-413-g514591a3. The pack contains only longshot_vk.dll. Renderer compile/link succeeded, 0 compiler failures. All arms use that same host and renderer; only the recorder switches differ. Scene/settings: Hyrule Field, child Link outside Lon Lon Ranch, separate copy of the same save each launch. Right stick fully right, left stick neutral. 1280x720 window (bridge-confirmed), 2048x1152 output, Custom quality, render resolution 100%, SDR, DLSS/FG/VSync off, HOS_FPS_MAX=240, real-time game speed, audio off. Camera rotation begins once the field is reached, followed by 12 s settling and a 30 s measurement. Method: excluded first shader-preparation/warm-up run, then three rounds: thread/OWN/IDLE, IDLE/OWN/thread, thread/OWN/IDLE. Full present trace from HOS_PRESENT_TRACE, process CPU time divided by measured frames, and recorder 10 s stats inside each window. All nine measured windows passed background-load checks (mean <3 other cores, 5 s peak <=6). Validation runs are separate and excluded from these numbers. These are game-present-return intervals, not PresentMon/display scanout timings. | Median of 3 runs | thread | thread + OWN | thread + IDLE=wait | |---|---:|---:|---:| | FPS | 82.59 | 81.41 | 82.09 | | CPU ms/frame | 61.06 | 61.71 | 60.46 | | p99 frame ms | 16.72 | 16.91 | 16.55 | | Worst frame ms (median of per-run maxima) | 33.05 | 31.68 | 31.47 | | Replay waits for recorder, ms/second | 12.40 | 9.64 | 15.40 | | Descriptor-pool waits/second | 2248 | 0 | 2255 | | Descriptor-pool wait ms/second | 2.80 | 0.00 | 3.89 | | Recorder idle spin, cores | 0.463 | 0.453 | 0.338 | Per-run data (all retained, no timed run rejected): | Round | Arm | FPS | CPU ms/frame | p99 ms | Worst ms | Frames >100 ms | |---|---|---:|---:|---:|---:|---:| | 1 | thread | 82.59 | 60.81 | 16.72 | 36.18 | 0 | | 1 | own | 83.02 | 60.76 | 16.91 | 29.54 | 0 | | 1 | idle | 83.18 | 59.86 | 16.16 | 31.47 | 0 | | 2 | thread | 83.58 | 61.06 | 15.48 | 33.05 | 0 | | 2 | own | 81.41 | 61.71 | 16.78 | 32.44 | 0 | | 2 | idle | 82.09 | 60.57 | 16.55 | 29.15 | 0 | | 3 | thread | 81.94 | 61.36 | 17.42 | 28.37 | 0 | | 3 | own | 80.23 | 62.80 | 18.69 | 31.68 | 0 | | 3 | idle | 81.03 | 60.46 | 17.61 | 31.59 | 0 | OWN removes the logged waits at vkResetDescriptorPool in all three runs (about 2,248/second to zero). Total recorder wait time falls from 12.40 to 9.64 ms/second. The slot-reuse guard never logged. IDLE=wait reduces spin from 0.463 to 0.338 of a core, while increasing recorder wait time to 15.40 ms/second; the Windows wait functions were available (no fallback log). FPS medians are OWN -1.43% and IDLE -0.60% relative to thread. Three short rounds, under-5% differences and overlapping ranges are diagnostic, not proof of a speedup or regression. The requested “FPS not lower” outcome is not established here, hence mixed rather than pass. CPU/frame did not improve with OWN. No frames >100 ms in any non-validation measured window. No crash/device-loss line in any run, and the inspected screenshots show the expected field and HUD. Vulkan validation: official Khronos portable Windows nightly, manifest API 1.4.368, archive SHA-256 cfcb5fe29f248c295256366dfbf399511af7d8b5f03b14dd7c9563840d9cd39a. RX_VK_VALIDATION=1; RX_VK_PRECOMPILE=0 (the documented switch) in all three arms. The layer DLL was confirmed loaded in each game process. Each arm reached Hyrule Field, rotated the camera and completed a 30 s validation window, then exited normally. All three arms report the same VUID set: VkShaderModuleCreateInfo-pCode-08740, RuntimeSpirv-shaderDenormFlushToZeroFloat32-06300 and RuntimeSpirv-shaderDenormPreserveFloat32-06297. No additional VUID with either switch, and no threading, command-pool-in-use or descriptor-pool-in-use error. This is a comparison against recorder-only's existing errors, not a clean validation pass. game-free-bundled: 497 passed, 4 failed, 33 skipped in 121 s (windows, 24 at once) main-failures: 4 passed, 4 failed, 0 skipped in 10 s (windows, 24 at once) The four failing checks are nx2/test_nup_paths.py, tests/test_check_third_party.py, tests/test_menu_i18n.py and tests/test_release.py. All four also fail when rerun on clean main 514591a3 with the same interpreter/environment. The new recorder own/idle test and all changed-source fixture tests pass. Final suite uses the project's bundled Python 3.14.7, llvm-mingw and the existing header cache, without the optional NGX build variable leaking into game-free fixtures. An earlier system-Python run had additional missing-package/environment failures and is superseded by this run. Recorder-off follow-up (same verified binaries and settings; fresh interleaved comparison, kept separate from the earlier nine runs): The recorder is fully disabled in OFF: RX_VK_RECORDER, RX_VK_RECORDER_OWN and RX_VK_RECORDER_IDLE are all unset. THREAD sets only RX_VK_RECORDER=thread. One OFF warm-up was excluded, then three pairs in OFF/THREAD, THREAD/OFF, OFF/THREAD order. Same copied save, camera rotation, 12 s settling and 30 s measurement per run. All six windows passed the same background-load checks. | Median of 3 fresh runs | Recorder OFF | Recorder THREAD | |---|---:|---:| | FPS | 79.28 | 81.26 | | CPU ms/frame | 56.16 | 63.03 | | p99 frame ms | 18.68 | 16.72 | | Worst frame ms (median of per-run maxima) | 37.68 | 30.92 | | Pair | Arm | FPS | CPU ms/frame | p99 ms | Worst ms | Frames >100 ms | |---|---|---:|---:|---:|---:|---:| | 1 | off | 80.05 | 55.22 | 18.75 | 30.23 | 0 | | 1 | thread | 81.26 | 63.20 | 17.82 | 30.92 | 0 | | 2 | off | 77.89 | 57.53 | 17.22 | 37.68 | 0 | | 2 | thread | 81.11 | 62.89 | 16.63 | 44.23 | 0 | | 3 | off | 79.28 | 56.16 | 18.68 | 41.60 | 0 | | 3 | thread | 82.13 | 63.03 | 16.72 | 21.94 | 0 | Relative to OFF, THREAD median FPS is +2.50% and CPU cost is +6.86 ms/frame. These short three-pair results are diagnostic; the prior OWN/IDLE data are not pooled with this later comparison. Recorder-OFF Vulkan validation also reached Hyrule Field, rotated the camera for a 30 s window and exited normally. Same portable Khronos layer and RX_VK_PRECOMPILE=0 as before; the layer DLL was confirmed loaded. VUIDs: RuntimeSpirv-shaderDenormFlushToZeroFloat32-06300, RuntimeSpirv-shaderDenormPreserveFloat32-06297, VkShaderModuleCreateInfo-pCode-08740. The VUID set matches the earlier THREAD/OWN/IDLE validation runs. Not tested: both switches together, Linux/AMD/Intel, other scenes, long play sessions or a full host rebuild of the PR. The test request was unapproved when read, so this was a direct PR test without a request claim. Tested with Codex following the repository's testing workflow. Game files, saves, runtimes and raw traces remain local.
mooki left a comment

Read-only review of head d958b46a against main 514591a3. I read the patch, the recording and descriptor code around it, the tests and the whole discussion. Nothing was built or run by me. I found no correctness problem; the points below are about coverage and about fitting in with other open work.

What is good

  • The reset still belongs to the replay thread. OwnSlot removes the wait for the recorder to drain, not the Vulkan calls or who makes them. Descriptor allocation and updates stay on the replay thread (gpu/vk/vk_replay.cpp, gpu/vk/vk_draw.cpp).
  • Two separate completion conditions are kept apart. recording_wait covers the slot's previous use by the GPU; slot_begin compares the recorder's consumed position with the position saved after both command buffers ended. Neither stands in for the other.
  • The exemption is narrow: the slot's command-pool resets, command-buffer begins, descriptor-pool reset and the destruction of extra descriptor pools. Ends, submissions, other destruction and the opaque recording through ImGui, NGX and FSR keep their boundaries.
  • The idle wait compares against the last consumed position rather than a fresh look at the published one just before sleeping, and the timeout and the producer's kicks remain (gpu/vk/vk_recorder.h).
  • Both switches are opt-in, and with them unset the old ordering and the old spin-then-wait path remain. The measurements are labelled as diagnostic, and the description says plainly that RX_VK_RECORDER_IDLE=wait did not raise the frame rate.

Suggestions

  1. A test that would catch the wrong pool. In tests/recorder_own_idle_test.cpp the stand-in for vkResetDescriptorPool records when it ran but ignores which pool it was given, so the case cannot tell resetting this slot's pool from resetting another's. The command-pool reset, command-buffer begin and extra-pool destruction are not exercised with identities either. A case with distinct slots, buffers and pools, a delayed recorder and a delayed GPU completion, that fails when the wrong pool is reset or a slot is reused early, would pin the ownership rule the patch relies on. The source-placement checks are useful but do not establish it.
  2. The real wait call and shutdown. The test replaces the operating system's wait with a condition variable and leaves the timeout out. That finds a missing wake, but not a broken Windows wait, a broken Linux futex path, timeout recovery, or stop while parked. Passing cases also leave through _Exit, so an orderly shutdown in the new idle mode is never run. Cases through the real adapter for a publish while parked, a publish just before the wait, and a stop while parked would cover it, as would both switches together (the combined-environment case exits before the recorder starts).
  3. This will meet #60. tests/test_recorder_own_idle.py copies vk_work.h, vk_recorder.h and the test loader into its own include folder. #60 makes vk_work.h include a new header, gpu_wait_stats.h, and updates the older recorder fixture's copy list, but it cannot update this new one. As I read the include paths, whichever of the two lands second has to add that header to this fixture or it will not compile. #288 carries the same change. This is not a fault of this head against main.
  4. Result rows. docs/tested-configurations.md gains four rows at the top of the shared table, where every pull request with results collides. #45 and #96 propose freezing that table and keeping new runs in their own files. Main still asks for rows today, so this is only a heads-up to keep the results in the description too until that settles.
  5. The description is behind the testing. It still says Linux was not built and NVIDIA was not tested. Since then: Windows with an RTX 5070 Ti (#162 (comment)), Linux with an RTX 3070 Ti (#162 (comment)) and Windows with an RTX 4070 (#162 (comment)), all on this head. They support fewer waits at the descriptor reset and less idle spinning, and none shows a frame-rate gain. The Linux report's caveat deserves to stay visible: its runs under the validation layer failed during loading even without the switches, so play under validation is not established there.

Small

  • The play report (#162 (comment)) says both of this pull request's switches were on, but names RX_VK_RECORDER=thread and RX_VK_RECORDER_OWN=1, not RX_VK_RECORDER_IDLE=wait. Worth saying which were set before it is counted as covering both.

Not checked

Nothing was built or run here, the fixtures included. I traced the ownership argument by reading and found no path that binds an old descriptor set from a slot being recycled, but that argument rests on where the calls sit today: OwnSlot itself does not check which handles it is used for, which is why suggestion 1 matters.

Read-only review of head `d958b46a` against main `514591a3`. I read the patch, the recording and descriptor code around it, the tests and the whole discussion. Nothing was built or run by me. I found no correctness problem; the points below are about coverage and about fitting in with other open work. ### What is good - **The reset still belongs to the replay thread.** `OwnSlot` removes the wait for the recorder to drain, not the Vulkan calls or who makes them. Descriptor allocation and updates stay on the replay thread (`gpu/vk/vk_replay.cpp`, `gpu/vk/vk_draw.cpp`). - **Two separate completion conditions are kept apart.** `recording_wait` covers the slot's previous use by the GPU; `slot_begin` compares the recorder's consumed position with the position saved after both command buffers ended. Neither stands in for the other. - **The exemption is narrow:** the slot's command-pool resets, command-buffer begins, descriptor-pool reset and the destruction of extra descriptor pools. Ends, submissions, other destruction and the opaque recording through ImGui, NGX and FSR keep their boundaries. - **The idle wait compares against the last consumed position** rather than a fresh look at the published one just before sleeping, and the timeout and the producer's kicks remain (`gpu/vk/vk_recorder.h`). - **Both switches are opt-in,** and with them unset the old ordering and the old spin-then-wait path remain. The measurements are labelled as diagnostic, and the description says plainly that `RX_VK_RECORDER_IDLE=wait` did not raise the frame rate. ### Suggestions 1. **A test that would catch the wrong pool.** In `tests/recorder_own_idle_test.cpp` the stand-in for `vkResetDescriptorPool` records when it ran but ignores which pool it was given, so the case cannot tell resetting this slot's pool from resetting another's. The command-pool reset, command-buffer begin and extra-pool destruction are not exercised with identities either. A case with distinct slots, buffers and pools, a delayed recorder and a delayed GPU completion, that fails when the wrong pool is reset or a slot is reused early, would pin the ownership rule the patch relies on. The source-placement checks are useful but do not establish it. 2. **The real wait call and shutdown.** The test replaces the operating system's wait with a condition variable and leaves the timeout out. That finds a missing wake, but not a broken Windows wait, a broken Linux futex path, timeout recovery, or stop while parked. Passing cases also leave through `_Exit`, so an orderly shutdown in the new idle mode is never run. Cases through the real adapter for a publish while parked, a publish just before the wait, and a stop while parked would cover it, as would both switches together (the combined-environment case exits before the recorder starts). 3. **This will meet https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/60.** `tests/test_recorder_own_idle.py` copies `vk_work.h`, `vk_recorder.h` and the test loader into its own include folder. https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/60 makes `vk_work.h` include a new header, `gpu_wait_stats.h`, and updates the older recorder fixture's copy list, but it cannot update this new one. As I read the include paths, whichever of the two lands second has to add that header to this fixture or it will not compile. https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/288 carries the same change. This is not a fault of this head against main. 4. **Result rows.** `docs/tested-configurations.md` gains four rows at the top of the shared table, where every pull request with results collides. https://git.magicbean.dev/MagicBeanDev/Longshot/issues/45 and https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/96 propose freezing that table and keeping new runs in their own files. Main still asks for rows today, so this is only a heads-up to keep the results in the description too until that settles. 5. **The description is behind the testing.** It still says Linux was not built and NVIDIA was not tested. Since then: Windows with an RTX 5070 Ti (https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/162#issuecomment-15252), Linux with an RTX 3070 Ti (https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/162#issuecomment-15291) and Windows with an RTX 4070 (https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/162#issuecomment-15482), all on this head. They support fewer waits at the descriptor reset and less idle spinning, and none shows a frame-rate gain. The Linux report's caveat deserves to stay visible: its runs under the validation layer failed during loading even without the switches, so play under validation is not established there. ### Small - The play report (https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/162#issuecomment-15345) says both of this pull request's switches were on, but names `RX_VK_RECORDER=thread` and `RX_VK_RECORDER_OWN=1`, not `RX_VK_RECORDER_IDLE=wait`. Worth saying which were set before it is counted as covering both. ### Not checked Nothing was built or run here, the fixtures included. I traced the ownership argument by reading and found no path that binds an old descriptor set from a slot being recycled, but that argument rests on where the calls sit today: `OwnSlot` itself does not check which handles it is used for, which is why suggestion 1 matters.
HumanBean force-pushed perf/recorder-own-idle from d958b46a28
Some checks failed
PR description / Description follows the template (pull_request_target) Successful in 3s
Website / Website on Cloudflare (pull_request_target) Has been skipped
CI / Conventional Commit title (pull_request) Has been cancelled
CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Has been cancelled
CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Has been cancelled
CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Has been cancelled
CI / Workflow lint (actionlint) (pull_request) Has been cancelled
Third-party parts / As their notes say (pull_request) Has been cancelled
Third-party parts / The same as upstream (pull_request) Has been cancelled
to 5c7342c997
All checks were successful
CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Has been skipped
CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Has been skipped
CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Has been skipped
Third-party parts / The same as upstream (pull_request) Has been skipped
Website / Website on Cloudflare (pull_request) Has been skipped
CI / Conventional Commit title (pull_request) Successful in 5s
CI / Workflow lint (actionlint) (pull_request) Successful in 8s
Third-party parts / As their notes say (pull_request) Successful in 10s
PR description / Description follows the template (pull_request_target) Successful in 4s
Website / Website on Cloudflare (pull_request_target) Has been skipped
2026-11-14 02:08:03 +00:00
Compare
Owner

Rebased onto current main 76699aad so the PR merges again. The new head is 5c7342c9.

  • The merge of main 514591a3 is dropped. The five code and docs commits (038c70a9..cb5719d2) were replayed in order onto 76699aad. They are still klimovich008's, and none of them conflicted. Their own +/- lines against the new base are identical to the original ones (714 lines, compared line for line). occ_collect() from main stays at the top of begin_recording, outside the OwnSlot. rec::slot_begin / OwnSlot still sit at the slot's own vkResetCommandPool/vkBeginCommandBuffer, and in draw_begin_recording at vkResetDescriptorPool. rec_end_pos is still set after both ends in submit_recording. Main added no slot reset or begin elsewhere. The source-cutting fixtures still find their anchors. Main's own new stubs (occ_collect, Writes& wr) are there next to the PR's rec stubs. No newer check compiles the cut regions without them: test_vk_occlusion_queries only reads begin_recording as text.
  • The tested-configurations rows are moved to docs/tested/. Main merged #96, which freezes docs/tested-configurations.md. So the three commits that added and edited rows there (0806b1af, 0d934b04, d958b46a) are dropped. The four per-arm rows of the 839e85ff A/B are now one file per run, with the author's data unchanged and "(iterations 0 and 2 of the A/B)" added to each Result:
    • docs/tested/2026-11-13-recorder-off-klimovich008-run01.md
    • docs/tested/2026-11-13-recorder-thread-klimovich008-run02.md
    • docs/tested/2026-11-13-recorder-own-klimovich008-run03.md
    • docs/tested/2026-11-13-recorder-idle-klimovich008-run04.md
  • The description is updated. The checklist box now uses the new template wording, and Testing notes the rebase.

Nothing was built or run here: no game, no run_all, not even the tested-configurations check. The file format was checked by reading scripts/tested_configurations.py.

Rebased onto current main [76699aad](https://git.magicbean.dev/MagicBeanDev/Longshot/commit/76699aad3a6b) so the PR merges again. The new head is 5c7342c9. - **The merge of main 514591a3 is dropped.** The five code and docs commits (038c70a9..cb5719d2) were replayed in order onto 76699aad. They are still klimovich008's, and none of them conflicted. Their own +/- lines against the new base are identical to the original ones (714 lines, compared line for line). `occ_collect()` from main stays at the top of `begin_recording`, outside the `OwnSlot`. `rec::slot_begin` / `OwnSlot` still sit at the slot's own `vkResetCommandPool`/`vkBeginCommandBuffer`, and in `draw_begin_recording` at `vkResetDescriptorPool`. `rec_end_pos` is still set after both ends in `submit_recording`. Main added no slot reset or begin elsewhere. The source-cutting fixtures still find their anchors. Main's own new stubs (`occ_collect`, `Writes& wr`) are there next to the PR's `rec` stubs. No newer check compiles the cut regions without them: `test_vk_occlusion_queries` only reads `begin_recording` as text. - **The tested-configurations rows are moved to `docs/tested/`.** Main merged [#96](https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/96), which freezes `docs/tested-configurations.md`. So the three commits that added and edited rows there (0806b1af, 0d934b04, d958b46a) are dropped. The four per-arm rows of the 839e85ff A/B are now one file per run, with the author's data unchanged and "(iterations 0 and 2 of the A/B)" added to each Result: - `docs/tested/2026-11-13-recorder-off-klimovich008-run01.md` - `docs/tested/2026-11-13-recorder-thread-klimovich008-run02.md` - `docs/tested/2026-11-13-recorder-own-klimovich008-run03.md` - `docs/tested/2026-11-13-recorder-idle-klimovich008-run04.md` - **The description is updated.** The checklist box now uses the new template wording, and Testing notes the rebase. Nothing was built or run here: no game, no `run_all`, not even the tested-configurations check. The file format was checked by reading `scripts/tested_configurations.py`.
HumanBean deleted branch perf/recorder-own-idle 2026-11-14 02:11:23 +00:00
Author
Contributor

Sorry: at about 02:33Z I pushed review commits to this branch, replacing @HumanBean's rebase (5c7342c9), which had already been merged here at 02:11Z. Main has exactly 5c7342c9's tree in the squash 98e2d95f, so nothing on main changed. The branch is restored to 5c7342c9, and the test changes below are in #342 on current main instead.

Thanks, @mooki, for the careful read. Point by point (#342, tests only; the production code is unchanged):

  1. Identities. New own_slots cases in tests/recorder_own_idle_test.cpp: two slots, each with its own command pool, command
    buffer, descriptor pool, extra pool and a set from each pool. Slot 0 is used and ended. Slot 1 is begun and its two binds are
    held in the recorder. Then slot 0 comes round, its GPU work completing 20 ms late (standing in for recording_wait), and makes
    its own calls: command pool reset, begin, descriptor pool reset and extra pool destruction. The fakes know which object each
    held packet names, and record a fault for a reset, begin or destruction of an object a held packet still uses, or of a slot
    whose GPU work has not completed. The case passes only with no fault, slot 1's packets still held when slot 0's calls return
    (none waited), and no watchdog. Two self-checks show the fakes catch each fault: own_slots_wrong resets slot 1's pool, and
    own_slots_early reuses slot 0 before its GPU work completes. Since OwnSlot itself takes no handles, the source check now
    also pins what the fixture can't see: in begin_recording, recording_wait(r) comes before rec::slot_begin(), and the
    OwnSlot blocks call only the slot's own handles (r.pool, r.upload_pool, r.cmd, r.upload; res[slot]'s pool and
    extra pools, with draw_begin_recording(r.slot, ...) and slot_begin(rec_end_pos[r.slot])). The source check reads the
    code with comments left out, and has controls for recording_wait commented out and for another slot's descriptor pools.

  2. The real wait and shutdown. New real_ cases go through vk_replay.cpp's own adapter: the test cuts the
    rec_wait_hooks body from the source, so it is kernelbase's WaitOnAddress. The cases:

    • a publish while the recorder is inside the OS wait wakes it. This wait gets 10 s instead of 2 ms, past the 3 s the case
      allows, so only the wake can end it in time. A run counts only when the wait that returned began before the publish and
      blocked 15 ms or more, so a compare at its start can't stand in for the wake; otherwise it is retried (up to 20 attempts);
    • with nothing published and the wake turned off, the 2 ms timeout ends each wait (counted only as FALSE with
      ERROR_TIMEOUT, since WaitOnAddress may also return early), and brings the recorder back to a later publish;
    • a publish just before a 10 s wait, the wake off, is found by the compare;
    • a stop while it is inside the wait returns, through vkDestroyDevice's teardown;
    • both switches together: the slot's own reset while a packet is held, then a parked publish.

    Each real_ case ends with the recorder stopped and joined through vkDestroyDevice, its watchdog threads joined, then a
    normal return through exit(), no _Exit. Three new broken variants must fail: the wait's timeout made infinite fails the timeout case, and the adapter with
    a wake that does nothing, or with a wait comparing against W as it is now, fails the parked or the race case. Since the OS
    wait may return early, a broken real-wait build counts as caught when any of three runs fails it; the correct code passes
    every run. The real_ cases run on
    Windows. The Linux futex path is host/posix's and isn't built by this test; tonesi's Linux run exercised it in the game (no
    "no WaitOnAddress" line, less spin). The older fake cases still leave through _Exit, because their recorder waits on the
    fixture's own condition variable, which exit() would destroy.

  3. #60 and #288. In #342 the fixture copies every header vk_work.h and vk_recorder.h include from gpu/vk, so gpu_wait_stats.h
    comes along whichever lands first. Said in the reviewer notes too.

  4. Result rows. The results are in the description as well as the rows, which @HumanBean's rebase moved to docs/tested/.

  5. The testing since, in one place now that the PR is merged. On d958b46a, the same code as merged:

    • RTX 5070 Ti, Windows (15252): OWN took the waits at vkResetDescriptorPool from 2,475 a second to none. IDLE=wait spun 0.30 of a
      core against 0.42, and the replay waited longer. fps was within 1 %. Under validation, no new error in any arm.
    • RTX 3070 Ti, Linux (15291): OWN took the waits from ~1,250 a second to none. IDLE=wait took the futex path and spun less. fps
      was inconclusive. Under the old 1.3.275 layer every recorder-thread run crashed inside the layer during the load, the plain
      RX_VK_RECORDER=thread arm included, so play under validation is not established on Linux.
    • RTX 4070, Windows (15482): OWN took the waits from 2,248 a second to none. IDLE=wait spun 0.338 of a core against 0.463. fps
      was within 1.5 %. Same VUID set in all arms.
    • RX 9070 XT, Windows (15345): about 25 minutes of play with OWN, no crash.

    So OWN removes the waits it targets everywhere tested, but gained fps only on the RX 9070 XT; IDLE=wait cost fps everywhere,
    and both stay off. Intel and the Steam Deck are untested.

Small: the play report covered RX_VK_RECORDER_OWN=1 only. It was corrected (thanks).

Checks, in #342 on main cf605c85: the new test passes its 22 cases and 9 broken variants three times in a row on each commit, and
tests/run_all.py passes all but five tests that fail the same on a clean main. Our own second-opinion review went
through eleven rounds on these tests; each found timing or lifetime gaps in them, fixed in #342, and the last
round is clean.

Sorry: at about 02:33Z I pushed review commits to this branch, replacing @HumanBean's rebase (5c7342c9), which had already been merged here at 02:11Z. Main has exactly 5c7342c9's tree in the squash 98e2d95f, so nothing on main changed. The branch is restored to 5c7342c9, and the test changes below are in #342 on current main instead. Thanks, @mooki, for the careful read. Point by point (#342, tests only; the production code is unchanged): 1. **Identities.** New `own_slots` cases in `tests/recorder_own_idle_test.cpp`: two slots, each with its own command pool, command buffer, descriptor pool, extra pool and a set from each pool. Slot 0 is used and ended. Slot 1 is begun and its two binds are held in the recorder. Then slot 0 comes round, its GPU work completing 20 ms late (standing in for `recording_wait`), and makes its own calls: command pool reset, begin, descriptor pool reset and extra pool destruction. The fakes know which object each held packet names, and record a fault for a reset, begin or destruction of an object a held packet still uses, or of a slot whose GPU work has not completed. The case passes only with no fault, slot 1's packets still held when slot 0's calls return (none waited), and no watchdog. Two self-checks show the fakes catch each fault: `own_slots_wrong` resets slot 1's pool, and `own_slots_early` reuses slot 0 before its GPU work completes. Since `OwnSlot` itself takes no handles, the source check now also pins what the fixture can't see: in `begin_recording`, `recording_wait(r)` comes before `rec::slot_begin()`, and the `OwnSlot` blocks call only the slot's own handles (`r.pool`, `r.upload_pool`, `r.cmd`, `r.upload`; `res[slot]`'s pool and extra pools, with `draw_begin_recording(r.slot, ...)` and `slot_begin(rec_end_pos[r.slot])`). The source check reads the code with comments left out, and has controls for `recording_wait` commented out and for another slot's descriptor pools. 2. **The real wait and shutdown.** New `real_` cases go through `vk_replay.cpp`'s own adapter: the test cuts the `rec_wait_hooks` body from the source, so it is kernelbase's `WaitOnAddress`. The cases: - a publish while the recorder is inside the OS wait wakes it. This wait gets 10 s instead of 2 ms, past the 3 s the case allows, so only the wake can end it in time. A run counts only when the wait that returned began before the publish and blocked 15 ms or more, so a compare at its start can't stand in for the wake; otherwise it is retried (up to 20 attempts); - with nothing published and the wake turned off, the 2 ms timeout ends each wait (counted only as FALSE with `ERROR_TIMEOUT`, since `WaitOnAddress` may also return early), and brings the recorder back to a later publish; - a publish just before a 10 s wait, the wake off, is found by the compare; - a stop while it is inside the wait returns, through `vkDestroyDevice`'s teardown; - both switches together: the slot's own reset while a packet is held, then a parked publish. Each `real_` case ends with the recorder stopped and joined through `vkDestroyDevice`, its watchdog threads joined, then a normal return through `exit()`, no `_Exit`. Three new broken variants must fail: the wait's timeout made infinite fails the timeout case, and the adapter with a wake that does nothing, or with a wait comparing against W as it is now, fails the parked or the race case. Since the OS wait may return early, a broken real-wait build counts as caught when any of three runs fails it; the correct code passes every run. The `real_` cases run on Windows. The Linux futex path is `host/posix`'s and isn't built by this test; tonesi's Linux run exercised it in the game (no "no WaitOnAddress" line, less spin). The older fake cases still leave through `_Exit`, because their recorder waits on the fixture's own condition variable, which `exit()` would destroy. 3. **#60 and #288.** In #342 the fixture copies every header `vk_work.h` and `vk_recorder.h` include from `gpu/vk`, so `gpu_wait_stats.h` comes along whichever lands first. Said in the reviewer notes too. 4. **Result rows.** The results are in the description as well as the rows, which @HumanBean's rebase moved to `docs/tested/`. 5. **The testing since,** in one place now that the PR is merged. On d958b46a, the same code as merged: - RTX 5070 Ti, Windows (15252): OWN took the waits at vkResetDescriptorPool from 2,475 a second to none. IDLE=wait spun 0.30 of a core against 0.42, and the replay waited longer. fps was within 1 %. Under validation, no new error in any arm. - RTX 3070 Ti, Linux (15291): OWN took the waits from ~1,250 a second to none. IDLE=wait took the futex path and spun less. fps was inconclusive. Under the old 1.3.275 layer every recorder-thread run crashed inside the layer during the load, the plain `RX_VK_RECORDER=thread` arm included, so play under validation is not established on Linux. - RTX 4070, Windows (15482): OWN took the waits from 2,248 a second to none. IDLE=wait spun 0.338 of a core against 0.463. fps was within 1.5 %. Same VUID set in all arms. - RX 9070 XT, Windows (15345): about 25 minutes of play with OWN, no crash. So OWN removes the waits it targets everywhere tested, but gained fps only on the RX 9070 XT; IDLE=wait cost fps everywhere, and both stay off. Intel and the Steam Deck are untested. **Small:** the play report covered `RX_VK_RECORDER_OWN=1` only. It was corrected (thanks). Checks, in #342 on main cf605c85: the new test passes its 22 cases and 9 broken variants three times in a row on each commit, and `tests/run_all.py` passes all but five tests that fail the same on a clean main. Our own second-opinion review went through eleven rounds on these tests; each found timing or lifetime gaps in them, fixed in #342, and the last round is clean.
Sign in to join this conversation.
No reviewers
No milestone
No project
No assignees
6 participants
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
MagicBeanDev/Longshot!162
No description provided.