fix(vk): the replay's own buffer writes stay in command order when unaligned or split #440

Open

mooki wants to merge 1 commit from mooki/Longshot:fix/replay-write-order into main

pull from: mooki/Longshot:fix/replay-write-order

merge into: MagicBeanDev:main

MagicBeanDev:main

Conversation 0 Commits 1 Files changed 5 +676 -41

mooki commented 2026-11-14 14:01:18 +00:00

Member

• Copy link
• Copy Markdown

Type: fix · Risk: low · Tested on: macOS 26.7.1, Apple M4 Max (game-free checks, arm64 and x86_64 object compiles; no game or GPU run) · Needs: Windows or Linux, an NVIDIA, AMD or Intel GPU

Why

The replay writes guest memory itself in three places: an event signal, the stream's uniform updates, and
nvnCommandBufferClearBuffer. Those writes go on the GPU in command order only when the bytes are 4-byte aligned and one
import holds them. Otherwise they were written on the CPU when the replay recorded them, which is before the GPU ran the
draws recorded ahead of them. Those draws could then read the new value, and an earlier GPU write of the same bytes could
land over it. That happens to an unaligned write and to a range that crosses from one game memory pool into the next. It
affects every GPU. Nobody has measured how often the game does this.

How to reproduce

Seen on: found by reading the code; not seen in a game. The game-free test shows it on macOS 26.7.1, Apple M4 Max,
against main 6c9edffe's gpu/vk sources. The replay's writes are the same code on every OS and GPU.
Settings and switches: defaults.
Steps: game-free, from this branch's checkout: python -I -B tests/test_replay_write_order.py --cxx clang++ --headers <Vulkan headers> --source <a tree of main>. In the game: an event signal, a uniform update or a
nvnCommandBufferClearBuffer that is unaligned or crosses two game pools, after a draw that reads those bytes.
Expected: the draw recorded before the write reads the old bytes, and the later write lands last.
Actual: on main the write is made on the CPU at record time, ahead of the GPU: the draw recorded before it reads the
new bytes, and an earlier GPU write can land over it. The test fails 10 of its 18 cases on main.

What changed

The replay's own writes now go on the GPU in command order also when they are unaligned or cross two game pools: a copy
from the recording's stream for unaligned bytes, and one update or fill per import. Only bytes outside every pool are
still written on the CPU, which is exact there.

• Read first: gpu_piece_bind, gpu_put, gpu_write and gpu_fill in vk_replay.cpp, then gpu_pool_piece in
  vk_memory.cpp.
• vk: gpu_write in gpu/vk/vk_replay.cpp (event signals and flush_ub_updates).
  • Each piece an import holds now goes on the GPU, after its barrier as before. Aligned bytes still use
    vkCmdUpdateBuffer. Unaligned bytes go into the recording's stream (stream_copy) and are copied into place with
    vkCmdCopyBuffer, which has no alignment rule (gpu_put).
  • A range no single import holds is written one import at a time (gpu_piece_bind). The pieces end at each game
    pool's end, because pool_lookup imports a range inside one pool only. For a virtual pool's address they also end
    where the host's mapping ends.
  • Bytes outside every pool still go on the CPU. That is exact there: no import holds them, so the GPU neither reads
    nor writes them. A pool whose import fails, or a stream with no room left, also falls back to the CPU, as before.
• vk: NOP_CLEAR_BUF in replay() now calls gpu_fill (vk_replay.cpp).
  • Each piece an import holds is filled with whole words by vkCmdFillBuffer, and stamped as the GPU's as before. The
    up to 3 bytes at either end are copied from the stream, with the value's byte order kept across a pool's end.
  • An aligned clear inside one pool records the same single fill as before.
  • The small clears (RX_VK_CLEAR_CPU_MAX, 256 bytes by default) are unchanged: they are written on the CPU in stream
    order first. Their GPU part now also covers one that is unaligned or crosses two pools, as it already did one inside
    a pool.
  • The RX_VK_WATCH_RANGE clear line now says "NOT all on the GPU" where it said "NOT bound".
• vk: gpu_pool_piece in gpu/vk/vk_memory.cpp (declared in vk_internal.h). It returns how much of a range from an
  address one import can hold: up to the end of the game pool holding it, or, outside the pools, up to the next pool. It
  uses the same "last pool starting at or below" rule as pool_lookup. Only the fallback path calls it.
• tests: tests/test_replay_write_order.py (new).
  • It cuts gpu_write, the NOP_CLEAR_BUF case and their helpers from the source and runs them against a stand-in
    that records the Vulkan commands. The stand-in runs them in order at the submit, after the CPU writes made at record
    time.
  • Guest memory is set up as two adjacent game pools, a gap, a third pool, a pool whose import fails, and a virtual
    pool's mapping across two of them.
  • Each case records a draw's read before the write. Every command is checked against Vulkan's alignment and range
    rules, and every write is checked for an earlier access of the same bytes with no barrier between.
  • It has 18 cases. 10 fail on main and all pass here.
• tests: tests/test_inline_write_order.py now cuts gpu_write with its two helpers, and its stand-in takes
  vkCmdCopyBuffer. An unaligned write now becomes an ordered copy where it was counted as a CPU fallback. Its checks
  that a missing barrier or a missing work count fails are unchanged.
• Design: the CPU path is kept on purpose for bytes outside every pool. No import holds them, so the GPU cannot read them
  in that submission, and a CPU write there is exact.
• Left out: the read side of the same gap. That is a uniform buffer copied from guest memory as submitted, a B2B copy
  across two pools run on the CPU, and a skipped B2T or T2B copy. Each needs reads gathered from several imports, which
  is a separate pull request.

Measurements

Verdict: fixed in the game-free test: it fails 10 of its 18 cases on main and passes all 18 here. The full game-free
suite changes only by the new test. An ordering fix: nothing timed (no GPU run).

Commits: main 6c9edffe5d (2026-11-14); this PR fix/replay-write-order @
fa9b6338a3.
Hardware: Apple M4 Max (16 CPUs), macOS 26.7.1, Apple clang 21. Game-free: no GPU or game involved
Scene: none (an ordering fix; its proof is the test, as CONTRIBUTING's fix row allows)
Method: tests/test_replay_write_order.py against main's gpu/vk sources (6c9edffe) and against this PR;
the full game-free suite on both
How to reproduce: python -I -B tests/test_replay_write_order.py --cxx clang++ --headers <Vulkan headers> (add
--source <a tree of main> for main's side)

	Before: main @ 6c9edffe	After: this PR @ fa9b6338
tests/test_replay_write_order.py	FAIL: 10 of 18 cases	PASS: 18 of 18
full game-free suite on macOS (tests/run_all.py --jobs 6)	531 passed, 32 failed, 17 skipped	532 passed, 32 failed, 17 skipped

main @ 6c9edffe:
FAIL: an unaligned write in one pool, after a draw's read: the draw recorded before it read the new bytes (written on the CPU at record time, ahead of the GPU); written on the CPU
FAIL: an aligned write across two game pools, after a draw's read: the draw recorded before it read the new bytes (written on the CPU at record time, ahead of the GPU); written on the CPU; not one update per pool
FAIL: an unaligned write across two game pools, after a draw's read: the draw recorded before it read the new bytes (written on the CPU at record time, ahead of the GPU); written on the CPU
FAIL: a write from outside every pool into one: the draw recorded before it read the new bytes (written on the CPU at record time, ahead of the GPU); not exactly the bytes outside the pools on the CPU
PASS: a write outside every pool: on the CPU
PASS: a write into a pool whose import fails: on the CPU
PASS: no room in the stream: an unaligned write on the CPU, an aligned one an update
FAIL: an unaligned write over an earlier aligned one: the later bytes stay: the earlier write landed over the later one (the later on the CPU at record time)
FAIL: a virtual pool's write across the end of the host's mapping: the draw recorded before it read the new bytes (written on the CPU at record time, ahead of the GPU); written on the CPU
FAIL: a clear across two game pools, after a draw's read: the draw recorded before it read the new bytes (written on the CPU at record time, ahead of the GPU); written on the CPU; not stamped as the GPU's
FAIL: an unaligned clear, after a draw's read: the draw recorded before it read the new bytes (written on the CPU at record time, ahead of the GPU); written on the CPU; not stamped as the GPU's
PASS: a clear across two game pools from an unaligned address keeps the value's byte order
PASS: an aligned clear in one pool: one fill, stamped
PASS: a small clear in one pool: on the CPU in stream order and on the GPU
FAIL: a small clear across two game pools: on the CPU in stream order and on the GPU per pool: not on the GPU, a pool at a time
FAIL: a parallel replay's small clear across two game pools: the dry pass's on the CPU, the worker's on the GPU: not on the GPU, a pool at a time
PASS: a clear outside every pool: on the CPU
PASS: an unaligned clear with no room in the stream: on the CPU
FAIL: 10 case(s): the replay's own writes do not all keep command order

this PR @ fa9b6338:
PASS: (each of the 18 cases above)
PASS: the replay's own writes keep command order, unaligned and across game pools; only bytes outside every pool go on the CPU

Screenshots: none; nothing on screen in the game-free tests.

Flags and settings

None.

Testing

Tested:

• On an Apple M4 Max with macOS 26.7.1, game-free only.
• tests/test_replay_write_order.py fails on main (10 of 18 cases) and passes here.
• tests/test_inline_write_order.py passes here, and its two negative controls (no barrier, no work count) still fail
  as they should. tests/test_wb_diff_cpu_writes.py passes.
• tests/test_threaded_rendering.py (main's #588, a render thread for the replay) passes on this head, in the suite run.
• The full suite on both sides, compared by test name.
  • By test name, against main 6c9edffe's run on the same Mac: only one status changed: test_replay_write_order,
    absent on main, passes here.
  • These 32 checks fail here and on main on this Mac, by name (not triaged here: some need the macOS port's other pull
    requests, some are checks main added or changed since 5e072fc8): test_appimage_package_retry,
    test_appimage_player_build, test_collect_wide, test_control, test_cpm_discovery, test_crash_dialog,
    test_crash_record, test_env_on, test_game_layout, test_lift_cleanup_error, test_linux_preflight,
    test_longshot_ctl, test_low_memory_notice, test_make_game_cache, test_mem_gate, test_mem_gate_children,
    test_mirror_trim, test_oot_state_tap, test_pcache_header_read, test_pipeline_cache_file,
    test_pipeline_cache_save, test_pipeline_play_cache, test_pipeline_tour, test_prof_symbols,
    test_program_cache_cleanup, test_program_cache_one_variant, test_readback_late, test_report_run,
    test_split_giants, test_vk_pgo, test_vram_budget_row, test_walk_key.
  • The same 17 checks skipped as on main.
  • Crash reports during the run: none (main's run: fault-arm64-2026-11-14-180704.ips,
    recorder_sync_test.exe-2026-11-14-180744.ips).
• Object compiles of gpu/vk/vk_replay.cpp and gpu/vk/vk_memory.cpp with the renderer's flags (nx2/build.py),
  Apple clang 21, arm64 and x86_64.
  • All 8 objects build: 4 on main and 4 here.
  • These two files build on macOS without the compile-only prelude that main's vk_shader.cpp and
    pipeline_cache_file.h need (environ, posix_fallocate). Before the rebase they also built with that prelude.
  • Before the rebase, a -Wall -Wextra syntax-only compile showed no warning on the changed lines of either file.
• After the rebase onto main aa95d35f, at 3156b569 (fa9b6338 before the rebases onto e090cc48 and aa95d35f):
  Sweep3's checks of this head (the C/C++ it changes compiled, and its own tests run, on arm64 and x86_64 where they
  apply) gave the same results as before; python -I -B tests/run_all.py on wave S1's merge 11a5b4e7, which carries
  this head: 568/22/17 (passed/failed/skipped) against main aa95d35f's 543/33/17 on this Mac, no new failure
  (reports/2026-11-15-rebase-sweep-3.md).

Not tested:

• No GPU or game run. The new commands are a vkCmdCopyBuffer from the recording's stream (made with
  TRANSFER_SRC) into a pool buffer or its VRAM mirror (already a transfer destination for updates, fills and B2B
  copies), and more than one fill or update per range. Nobody has run them on a driver.
• Whether the game ever writes unaligned or across two pools is not known, so no in-game change is expected or
  measured.
• A build on Windows (llvm-mingw) or Linux. The change uses only calls the file already makes (stream_copy,
  vkCmdCopyBuffer, vkCmdFillBuffer, gpu_buffer_bind) and the standard library.

Must be tested:

• Windows or Linux, an NVIDIA, AMD or Intel GPU, any driver: build this PR (head 3156b569) and main aa95d35f the same
  way and play the same few minutes on each with RX_VK_VALIDATION=1 (the Khronos validation layer), expecting no
  validation error and no log line on this PR that main does not show, and the same frame rate as main (by hand: test
  request T-2J27).

Risks

What could break:

• Low. Aligned writes inside one pool record the same commands as before: one update, or one fill.
• Only the cases that went on the CPU change. They now record a copy from the stream, or one update or fill per pool.
  Each writes the same bytes the CPU wrote, but in command order. With VRAM mirrors those bytes now go to the mirror,
  as an aligned write's do.
• An unaligned write takes up to 64 KB of the recording's stream per piece. When the stream is full, the write goes on
  the CPU as before.
• Landing order: rebased on main 6c9edffe (2026-11-14) without a conflict. Main changed vk_replay.cpp in six commits
  since this branch's base, among them #588 (a render thread for the replay, behind Threaded rendering): the writes this
  PR moves are recorded where they were (gpu_write, and NOP_CLEAR_BUF in replay), into the recording's command
  buffer and stream, and tests/test_threaded_rendering.py passes on this head (Testing). No conflict with the other
  macOS-port PRs rebased on 6c9edffe (git merge-tree of the new heads, 19:49 CST).

Turning it off: revert the commit. There is no switch.

Compatibility: Nothing changes: vk_replay.cpp, vk_memory.cpp and vk_internal.h are not in the translator stamp
(nx2/translator_files.py), and saves, the options file, the shader and pipeline caches and the vk ABI are unchanged.

Checklist

Everyone

• One topic.
• The title is a Conventional Commit (type(scope): description) of the status line's type.
• Why gives the reason plainly and links the issues that fit (the tracker searched).
• Measurements has a verdict, or one line saying why there is nothing to measure.
• Testing names what was tested, what wasn't, and what must still be tested, where and how.
• Rebased on current main: on aa95d35f (#761, 2026-11-15), head 3156b569, without a conflict; this PR's own
  commits, old → new: fa9b6338 → 3156b569; the commit names elsewhere in this description are from before the
  rebase.
• python -I -B tests/run_all.py passes (any failure that also fails on main is named). On macOS: 532 passed, 17
  skipped, and 32 failures that also fail on main 6c9edffe on this Mac, named under Testing. After the rebase onto main
  aa95d35f (head 3156b569): wave S1's merge 11a5b4e7, 568/22/17 against main aa95d35f's 543/33/17, no new failure.
• No game files, dumps, build output or generated files.
• third_party/ untouched (a new folder with a PROVENANCE.md instead).
• The license is OK: GPL-3.0-or-later, nothing incompatible and nothing decompiled.

When it applies

• AI assistance recorded with Assisted-by: / Co-Authored-By: trailers.
• A game-free test added for the fix, failing on main and passing here.
• A check under tests/ that reads changed source updated in this PR, not weakened.

**Type:** fix · **Risk:** low · **Tested on:** macOS 26.7.1, Apple M4 Max (game-free checks, arm64 and x86_64 object compiles; no game or GPU run) · **Needs:** Windows or Linux, an NVIDIA, AMD or Intel GPU ## Why The replay writes guest memory itself in three places: an event signal, the stream's uniform updates, and nvnCommandBufferClearBuffer. Those writes go on the GPU in command order only when the bytes are 4-byte aligned and one import holds them. Otherwise they were written on the CPU when the replay recorded them, which is before the GPU ran the draws recorded ahead of them. Those draws could then read the new value, and an earlier GPU write of the same bytes could land over it. That happens to an unaligned write and to a range that crosses from one game memory pool into the next. It affects every GPU. Nobody has measured how often the game does this. ## How to reproduce **Seen on:** found by reading the code; not seen in a game. The game-free test shows it on macOS 26.7.1, Apple M4 Max, against main 6c9edffe's `gpu/vk` sources. The replay's writes are the same code on every OS and GPU. **Settings and switches:** defaults. **Steps:** game-free, from this branch's checkout: `python -I -B tests/test_replay_write_order.py --cxx clang++ --headers <Vulkan headers> --source <a tree of main>`. In the game: an event signal, a uniform update or a nvnCommandBufferClearBuffer that is unaligned or crosses two game pools, after a draw that reads those bytes. **Expected:** the draw recorded before the write reads the old bytes, and the later write lands last. **Actual:** on main the write is made on the CPU at record time, ahead of the GPU: the draw recorded before it reads the new bytes, and an earlier GPU write can land over it. The test fails 10 of its 18 cases on main. ## What changed The replay's own writes now go on the GPU in command order also when they are unaligned or cross two game pools: a copy from the recording's stream for unaligned bytes, and one update or fill per import. Only bytes outside every pool are still written on the CPU, which is exact there. - Read first: `gpu_piece_bind`, `gpu_put`, `gpu_write` and `gpu_fill` in `vk_replay.cpp`, then `gpu_pool_piece` in `vk_memory.cpp`. - vk: `gpu_write` in `gpu/vk/vk_replay.cpp` (event signals and `flush_ub_updates`). - Each piece an import holds now goes on the GPU, after its barrier as before. Aligned bytes still use `vkCmdUpdateBuffer`. Unaligned bytes go into the recording's stream (`stream_copy`) and are copied into place with `vkCmdCopyBuffer`, which has no alignment rule (`gpu_put`). - A range no single import holds is written one import at a time (`gpu_piece_bind`). The pieces end at each game pool's end, because `pool_lookup` imports a range inside one pool only. For a virtual pool's address they also end where the host's mapping ends. - Bytes outside every pool still go on the CPU. That is exact there: no import holds them, so the GPU neither reads nor writes them. A pool whose import fails, or a stream with no room left, also falls back to the CPU, as before. - vk: `NOP_CLEAR_BUF` in `replay()` now calls `gpu_fill` (`vk_replay.cpp`). - Each piece an import holds is filled with whole words by `vkCmdFillBuffer`, and stamped as the GPU's as before. The up to 3 bytes at either end are copied from the stream, with the value's byte order kept across a pool's end. - An aligned clear inside one pool records the same single fill as before. - The small clears (`RX_VK_CLEAR_CPU_MAX`, 256 bytes by default) are unchanged: they are written on the CPU in stream order first. Their GPU part now also covers one that is unaligned or crosses two pools, as it already did one inside a pool. - The `RX_VK_WATCH_RANGE` clear line now says "NOT all on the GPU" where it said "NOT bound". - vk: `gpu_pool_piece` in `gpu/vk/vk_memory.cpp` (declared in `vk_internal.h`). It returns how much of a range from an address one import can hold: up to the end of the game pool holding it, or, outside the pools, up to the next pool. It uses the same "last pool starting at or below" rule as `pool_lookup`. Only the fallback path calls it. - tests: `tests/test_replay_write_order.py` (new). - It cuts `gpu_write`, the `NOP_CLEAR_BUF` case and their helpers from the source and runs them against a stand-in that records the Vulkan commands. The stand-in runs them in order at the submit, after the CPU writes made at record time. - Guest memory is set up as two adjacent game pools, a gap, a third pool, a pool whose import fails, and a virtual pool's mapping across two of them. - Each case records a draw's read before the write. Every command is checked against Vulkan's alignment and range rules, and every write is checked for an earlier access of the same bytes with no barrier between. - It has 18 cases. 10 fail on main and all pass here. - tests: `tests/test_inline_write_order.py` now cuts `gpu_write` with its two helpers, and its stand-in takes `vkCmdCopyBuffer`. An unaligned write now becomes an ordered copy where it was counted as a CPU fallback. Its checks that a missing barrier or a missing work count fails are unchanged. - Design: the CPU path is kept on purpose for bytes outside every pool. No import holds them, so the GPU cannot read them in that submission, and a CPU write there is exact. - Left out: the read side of the same gap. That is a uniform buffer copied from guest memory as submitted, a B2B copy across two pools run on the CPU, and a skipped B2T or T2B copy. Each needs reads gathered from several imports, which is a separate pull request. ## Measurements **Verdict:** fixed in the game-free test: it fails 10 of its 18 cases on main and passes all 18 here. The full game-free suite changes only by the new test. An ordering fix: nothing timed (no GPU run). Commits: main 6c9edffe5da1f5b5383977aeab3f544c0de25d4c (2026-11-14); this PR fix/replay-write-order @ fa9b6338a339c802735dcba776ac0576a8510a80. Hardware: Apple M4 Max (16 CPUs), macOS 26.7.1, Apple clang 21. Game-free: no GPU or game involved Scene: none (an ordering fix; its proof is the test, as CONTRIBUTING's `fix` row allows) Method: `tests/test_replay_write_order.py` against main's `gpu/vk` sources (6c9edffe) and against this PR; the full game-free suite on both How to reproduce: `python -I -B tests/test_replay_write_order.py --cxx clang++ --headers <Vulkan headers>` (add `--source <a tree of main>` for main's side) | | Before: main @ 6c9edffe | After: this PR @ fa9b6338 | |---|---|---| | `tests/test_replay_write_order.py` | FAIL: 10 of 18 cases | PASS: 18 of 18 | | full game-free suite on macOS (`tests/run_all.py --jobs 6`) | 531 passed, 32 failed, 17 skipped | 532 passed, 32 failed, 17 skipped | ``` main @ 6c9edffe: FAIL: an unaligned write in one pool, after a draw's read: the draw recorded before it read the new bytes (written on the CPU at record time, ahead of the GPU); written on the CPU FAIL: an aligned write across two game pools, after a draw's read: the draw recorded before it read the new bytes (written on the CPU at record time, ahead of the GPU); written on the CPU; not one update per pool FAIL: an unaligned write across two game pools, after a draw's read: the draw recorded before it read the new bytes (written on the CPU at record time, ahead of the GPU); written on the CPU FAIL: a write from outside every pool into one: the draw recorded before it read the new bytes (written on the CPU at record time, ahead of the GPU); not exactly the bytes outside the pools on the CPU PASS: a write outside every pool: on the CPU PASS: a write into a pool whose import fails: on the CPU PASS: no room in the stream: an unaligned write on the CPU, an aligned one an update FAIL: an unaligned write over an earlier aligned one: the later bytes stay: the earlier write landed over the later one (the later on the CPU at record time) FAIL: a virtual pool's write across the end of the host's mapping: the draw recorded before it read the new bytes (written on the CPU at record time, ahead of the GPU); written on the CPU FAIL: a clear across two game pools, after a draw's read: the draw recorded before it read the new bytes (written on the CPU at record time, ahead of the GPU); written on the CPU; not stamped as the GPU's FAIL: an unaligned clear, after a draw's read: the draw recorded before it read the new bytes (written on the CPU at record time, ahead of the GPU); written on the CPU; not stamped as the GPU's PASS: a clear across two game pools from an unaligned address keeps the value's byte order PASS: an aligned clear in one pool: one fill, stamped PASS: a small clear in one pool: on the CPU in stream order and on the GPU FAIL: a small clear across two game pools: on the CPU in stream order and on the GPU per pool: not on the GPU, a pool at a time FAIL: a parallel replay's small clear across two game pools: the dry pass's on the CPU, the worker's on the GPU: not on the GPU, a pool at a time PASS: a clear outside every pool: on the CPU PASS: an unaligned clear with no room in the stream: on the CPU FAIL: 10 case(s): the replay's own writes do not all keep command order this PR @ fa9b6338: PASS: (each of the 18 cases above) PASS: the replay's own writes keep command order, unaligned and across game pools; only bytes outside every pool go on the CPU ``` Screenshots: none; nothing on screen in the game-free tests. ### Flags and settings None. ## Testing **Tested:** - On an Apple M4 Max with macOS 26.7.1, game-free only. - `tests/test_replay_write_order.py` fails on main (10 of 18 cases) and passes here. - `tests/test_inline_write_order.py` passes here, and its two negative controls (no barrier, no work count) still fail as they should. `tests/test_wb_diff_cpu_writes.py` passes. - `tests/test_threaded_rendering.py` (main's #588, a render thread for the replay) passes on this head, in the suite run. - The full suite on both sides, compared by test name. - By test name, against main 6c9edffe's run on the same Mac: only one status changed: `test_replay_write_order`, absent on main, passes here. - These 32 checks fail here and on main on this Mac, by name (not triaged here: some need the macOS port's other pull requests, some are checks main added or changed since 5e072fc8): `test_appimage_package_retry`, `test_appimage_player_build`, `test_collect_wide`, `test_control`, `test_cpm_discovery`, `test_crash_dialog`, `test_crash_record`, `test_env_on`, `test_game_layout`, `test_lift_cleanup_error`, `test_linux_preflight`, `test_longshot_ctl`, `test_low_memory_notice`, `test_make_game_cache`, `test_mem_gate`, `test_mem_gate_children`, `test_mirror_trim`, `test_oot_state_tap`, `test_pcache_header_read`, `test_pipeline_cache_file`, `test_pipeline_cache_save`, `test_pipeline_play_cache`, `test_pipeline_tour`, `test_prof_symbols`, `test_program_cache_cleanup`, `test_program_cache_one_variant`, `test_readback_late`, `test_report_run`, `test_split_giants`, `test_vk_pgo`, `test_vram_budget_row`, `test_walk_key`. - The same 17 checks skipped as on main. - Crash reports during the run: none (main's run: fault-arm64-2026-11-14-180704.ips, recorder_sync_test.exe-2026-11-14-180744.ips). - Object compiles of `gpu/vk/vk_replay.cpp` and `gpu/vk/vk_memory.cpp` with the renderer's flags (`nx2/build.py`), Apple clang 21, arm64 and x86_64. - All 8 objects build: 4 on main and 4 here. - These two files build on macOS without the compile-only prelude that main's `vk_shader.cpp` and `pipeline_cache_file.h` need (`environ`, `posix_fallocate`). Before the rebase they also built with that prelude. - Before the rebase, a `-Wall -Wextra` syntax-only compile showed no warning on the changed lines of either file. - **After the rebase onto main aa95d35f**, at `3156b569` (`fa9b6338` before the rebases onto e090cc48 and aa95d35f): Sweep3's checks of this head (the C/C++ it changes compiled, and its own tests run, on arm64 and x86_64 where they apply) gave the same results as before; `python -I -B tests/run_all.py` on wave S1's merge `11a5b4e7`, which carries this head: 568/22/17 (passed/failed/skipped) against main aa95d35f's 543/33/17 on this Mac, no new failure (`reports/2026-11-15-rebase-sweep-3.md`). **Not tested:** - No GPU or game run. The new commands are a `vkCmdCopyBuffer` from the recording's stream (made with `TRANSFER_SRC`) into a pool buffer or its VRAM mirror (already a transfer destination for updates, fills and B2B copies), and more than one fill or update per range. Nobody has run them on a driver. - Whether the game ever writes unaligned or across two pools is not known, so no in-game change is expected or measured. - A build on Windows (llvm-mingw) or Linux. The change uses only calls the file already makes (`stream_copy`, `vkCmdCopyBuffer`, `vkCmdFillBuffer`, `gpu_buffer_bind`) and the standard library. **Must be tested:** - Windows or Linux, an NVIDIA, AMD or Intel GPU, any driver: build this PR (head 3156b569) and main aa95d35f the same way and play the same few minutes on each with `RX_VK_VALIDATION=1` (the Khronos validation layer), expecting no validation error and no log line on this PR that main does not show, and the same frame rate as main (by hand: test request T-2J27). ## Risks **What could break:** - Low. Aligned writes inside one pool record the same commands as before: one update, or one fill. - Only the cases that went on the CPU change. They now record a copy from the stream, or one update or fill per pool. Each writes the same bytes the CPU wrote, but in command order. With VRAM mirrors those bytes now go to the mirror, as an aligned write's do. - An unaligned write takes up to 64 KB of the recording's stream per piece. When the stream is full, the write goes on the CPU as before. - Landing order: rebased on `main` 6c9edffe (2026-11-14) without a conflict. Main changed `vk_replay.cpp` in six commits since this branch's base, among them #588 (a render thread for the replay, behind Threaded rendering): the writes this PR moves are recorded where they were (`gpu_write`, and `NOP_CLEAR_BUF` in `replay`), into the recording's command buffer and stream, and `tests/test_threaded_rendering.py` passes on this head (Testing). No conflict with the other macOS-port PRs rebased on 6c9edffe (`git merge-tree` of the new heads, 19:49 CST). **Turning it off:** revert the commit. There is no switch. **Compatibility:** Nothing changes: `vk_replay.cpp`, `vk_memory.cpp` and `vk_internal.h` are not in the translator stamp (`nx2/translator_files.py`), and saves, the options file, the shader and pipeline caches and the vk ABI are unchanged. ## Checklist ### Everyone - [x] One topic. - [x] The title is a Conventional Commit (`type(scope): description`) of the status line's type. - [x] Why gives the reason plainly and links the issues that fit (the tracker searched). - [x] Measurements has a verdict, or one line saying why there is nothing to measure. - [x] Testing names what was tested, what wasn't, and what must still be tested, where and how. - [x] Rebased on current `main`: on aa95d35f (#761, 2026-11-15), head `3156b569`, without a conflict; this PR's own commits, old → new: `fa9b6338` → `3156b569`; the commit names elsewhere in this description are from before the rebase. - [x] `python -I -B tests/run_all.py` passes (any failure that also fails on `main` is named). On macOS: 532 passed, 17 skipped, and 32 failures that also fail on main 6c9edffe on this Mac, named under Testing. After the rebase onto main aa95d35f (head `3156b569`): wave S1's merge `11a5b4e7`, 568/22/17 against main aa95d35f's 543/33/17, no new failure. - [x] No game files, dumps, build output or generated files. - [x] `third_party/` untouched (a new folder with a `PROVENANCE.md` instead). - [x] The license is OK: GPL-3.0-or-later, nothing incompatible and nothing decompiled. ### When it applies - [x] AI assistance recorded with `Assisted-by:` / `Co-Authored-By:` trailers. - [x] A game-free test added for the fix, failing on `main` and passing here. - [x] A check under `tests/` that reads changed source updated in this PR, not weakened.

mooki added 1 commit 2026-11-14 14:01:18 +00:00

fix(vk): the replay's own buffer writes stay in command order when unaligned or split ... 229bc0e62f

gpu_write (event signals, the stream's uniform updates) and NOP_CLEAR_BUF wrote guest memory on the CPU at record time
whenever vkCmdUpdateBuffer or vkCmdFillBuffer could not take the bytes (not 4-byte aligned) or no single import held the
range (a range across two game pools: pool_lookup takes one pool's). The draws recorded before such a write then read its
bytes, and earlier GPU writes of the same bytes landed on top of it.

- gpu_put: an unaligned part goes into the recording's stream and is copied into place (a copy has no alignment rule);
  aligned parts stay updates.
- gpu_piece_bind and vk_memory.cpp's gpu_pool_piece: a range no import holds is written an import at a time, up to each
  game pool's end (and a virtual pool's mapping). Bytes outside every pool stay on the CPU, which is exact there.
- gpu_fill: NOP_CLEAR_BUF's GPU part, per import, as a fill of whole words with the up to 3 bytes on either side copied
  from the stream. An aligned clear in one pool records the same single fill as before. A small clear is still written on
  the CPU in stream order first; its GPU part now also covers one across two pools or unaligned, as it did one in a pool.
- tests/test_replay_write_order.py (new) runs gpu_write and the NOP_CLEAR_BUF case against a recorded-command stand-in;
  10 of its 18 cases fail on main. tests/test_inline_write_order.py follows gpu_write into its helpers.

Assisted-by: Claude Opus 5.5
Co-Authored-By: Claude Opus 5.5 <[email protected]>

mooki added the

kind

bug

priority

high

labels 2026-11-14 14:18:19 +00:00

mooki added the

area/renderer

label 2026-11-14 16:28:00 +00:00

mooki added the

platform/linux

platform/windows

labels 2026-11-14 18:23:59 +00:00

mooki force-pushed fix/replay-write-order from 229bc0e62f to fa9b6338a3 2026-11-15 01:51:54 +00:00 Compare

mooki changed title from WIP: fix(vk): the replay's own buffer writes stay in command order when unaligned or split to fix(vk): the replay's own buffer writes stay in command order when unaligned or split 2026-11-15 03:33:54 +00:00

mooki force-pushed fix/replay-write-order from fa9b6338a3 to c307a843b0 2026-11-15 07:01:33 +00:00 Compare

mooki force-pushed fix/replay-write-order from c307a843b0 to 3156b56989 2026-11-15 13:02:47 +00:00 Compare

Some checks failed

Hide checks

CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Has been skipped

Details

CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Has been skipped

Details

CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Has been skipped

Details

Third-party parts / The same as upstream (pull_request) Has been skipped

Details

Website / Website on Cloudflare (pull_request) Has been skipped

Details

CI / Conventional Commit title (pull_request) Successful in 3s

Details

CI / Workflow lint (actionlint) (pull_request) Successful in 6s

Details

Guide screenshots / Guide images and local UI source review (pull_request) Failing after 6s

Details

Third-party parts / As their notes say (pull_request) Successful in 9s

Details

PR description / Description follows the template (pull_request_target) Successful in 5s

Details

This pull request doesn't have enough approvals yet. 0 of 1 approvals granted.

This branch is out-of-date with the base branch

You are not authorized to merge this pull request.

View command line instructions

Checkout

From your project repository, check out a new branch and test the changes.

git fetch -u 'fix/replay-write-order:mooki-fix/replay-write-order'

git switch 'mooki-fix/replay-write-order'

Sign in to join this conversation.

Reviewers

No reviewers

Labels

Clear labels   

area/accessibility

Barriers for people with disabilities

  

area/audio

Sound and music

  

area/input

Controllers, keyboard, mouse

  

area/install

Installing, updating, building from your game

  

area/menu

In-game menu and settings

  

area/renderer

Vulkan renderer, shaders, effects

  

area/saves

Save files

  

area/upscaling

DLSS, FSR, frame generation

  

beanstalk: approved @ 1697603

Set by Beanstalk for show; the farm's database is what counts

  

beanstalk: approved @ 2c5a62e

Set by Beanstalk for show; the farm's database is what counts

  

beanstalk: approved @ 4fd82b7

Set by Beanstalk for show; the farm's database is what counts

  

beanstalk: approved @ 64d6d09

Set by Beanstalk for show; the farm's database is what counts

  

beanstalk: approved @ 6f91fae

Set by Beanstalk for show; the farm's database is what counts

  

beanstalk: approved @ 70a78f9

Set by Beanstalk for show; the farm's database is what counts

  

beanstalk: approved @ 8d361dc

Set by Beanstalk for show; the farm's database is what counts

  

beanstalk: approved @ 9d8356e

Set by Beanstalk for show; the farm's database is what counts

  

beanstalk: approved @ abf2227

Set by Beanstalk for show; the farm's database is what counts

  

beanstalk: approved @ d82f202

Set by Beanstalk for show; the farm's database is what counts

  

beanstalk: approved @ e5f1e40

Set by Beanstalk for show; the farm's database is what counts

  

beanstalk: approved @ ee8ab5f

Set by Beanstalk for show; the farm's database is what counts

  

ci: approved

A maintainer approved CircleCI for this fork PR (sticky; removed after 14 quiet days)

  

good first issue

A good start for a new contributor

  

gpu/amd

AMD GPUs

  

gpu/igpu

Integrated GPUs

  

gpu/intel

Intel GPUs

  

gpu/nvidia

NVIDIA GPUs

  

help wanted

Extra hands or knowledge wanted

  

kind

bug

Something doesn't work as it should

  

kind

chore

Build, CI, dependencies, tidying

  

kind

docs

Documentation

  

kind

feature

A new feature or an improvement

  

kind

performance

Frame rate, frame time, loading, memory

  

kind

question

A question rather than a change

  

needs measurements

Waiting for before/after measurements

  

platform/linux

Affects Linux (a change for every system carries both Windows and Linux)

  

platform/macos

macOS (Apple silicon or Intel Macs)

  

platform/steam-deck

Steam Deck

  

platform/windows

Affects Windows (a change for every system carries both Windows and Linux)

  

preview

Maintainers: deploy a preview of this PR's website (read its changes first). Removed after the run.

  

priority

critical

Crashes, data loss, or blocks most players

  

priority

high

Important: next in line

  

priority

low

Nice to have

  

regression

Worked in an earlier build, broken in a newer one: highest priority, above other bugs

  

status

blocked

Waits on something else first

  

status

confirmed

Reproduced or agreed on

  

status

duplicate

Already reported or proposed

  

status

invalid

Not a problem in this project

  

status

needs-info

Waiting for more information from the reporter

  

status

wontfix

Will not be worked on

  

testing wanted

Wanted: a test on other hardware (posts a test request in #test-requests)

  

ci: approved

No labels

area/accessibility

area/audio

area/input

area/install

area/menu

area/renderer

area/saves

area/upscaling

beanstalk: approved @ 1697603

beanstalk: approved @ 2c5a62e

beanstalk: approved @ 4fd82b7

beanstalk: approved @ 64d6d09

beanstalk: approved @ 6f91fae

beanstalk: approved @ 70a78f9

beanstalk: approved @ 8d361dc

beanstalk: approved @ 9d8356e

beanstalk: approved @ abf2227

beanstalk: approved @ d82f202

beanstalk: approved @ e5f1e40

beanstalk: approved @ ee8ab5f

ci: approved

good first issue

gpu/amd

gpu/igpu

gpu/intel

gpu/nvidia

help wanted

kind

bug

kind

chore

kind

docs

kind

feature

kind

performance

kind

question

needs measurements

platform/linux

platform/macos

platform/steam-deck

platform/windows

preview

priority

critical

priority

high

priority

low

regression

status

blocked

status

confirmed

status

duplicate

status

invalid

status

needs-info

status

wontfix

testing wanted

ci: approved

Milestone

Clear milestone

No items

No milestone

Projects

Clear projects

No items

No project

Assignees

Clear assignees

No assignees

1 participant

Notifications

Subscribe

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference

MagicBeanDev/Longshot!440

No description provided.