fix(vk): make the GPU's writes visible to the host's reads at each submission's end #1031
No reviewers
Labels
No labels
area/accessibility
area/audio
area/input
area/install
area/menu
area/renderer
area/saves
area/tooling
area/upscaling
beanstalk: approved
beanstalk: failed
beanstalk: needs approval
beanstalk: running
beanstalk: tested
ci: approved
good first issue
gpu/amd
gpu/igpu
gpu/intel
gpu/nvidia
help wanted
kind
bug
kind
chore
kind
docs
kind
feature
kind
performance
kind
question
needs measurements
platform/linux
platform/macos
platform/steam-deck
platform/windows
preview
priority
critical
priority
high
priority
low
regression
risk
high
risk
low
risk
medium
status
blocked
status
confirmed
status
duplicate
status
invalid
status
needs-info
status
wontfix
ci: approved
No milestone
No project
No assignees
2 participants
Notifications
Due date
No due date set.
Dependencies
No dependencies set
Reference
MagicBeanDev/Longshot!1031
Loading…
Reference in a new issue
No description provided.
Delete branch "jcabarca/Longshot:fix/vk-host-read-barrier"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Type: fix · Risk: low · Tested on: Windows, AMD Radeon RX 9070 XT · Needs: none
Why
The game reads what the GPU wrote into its memory (copies to buffers, queries, readbacks) once the timeline semaphore reaches the submission, but a host wait only makes device writes visible to the device: no command buffer had a memory dependency into
HOST_READ, so on a GPU whose caches are not flushed by chance the game could read stale bytes (synchronization.adoc, Fences and Semaphore Signaling: a fence's or a semaphore signal's access scope covers device access only).How to reproduce
Seen on: not seen in the game: found by reading
submit_recording()and the parallel segment jobs ingpu/vk/vk_replay.cpp; on AMD and NVIDIA desktop drivers the end-of-submission cache flush usually hides itSettings and switches: defaults
Steps: game-free:
python -I -B tests/test_host_read_barrier.py --source <main checkout>(the test from this PR); it readsvk_replay.cppExpected: every replay command buffer ends with a barrier from
ALL_COMMANDS/MEMORY_WRITEtoHOST/HOST_READActual: on main there is no such barrier; the test fails with "no memory dependency into the host's reads at a recording's end"
What changed
Each replay command buffer now ends with one global memory barrier into the host's reads, so what the GPU wrote is visible to the game once its wait on the timeline returns.
host_read_barrier()ingpu/vk/vk_replay.cpp: oneVkMemoryBarrier2fromVK_PIPELINE_STAGE_2_ALL_COMMANDS_BIT/VK_ACCESS_2_MEMORY_WRITE_BITtoVK_PIPELINE_STAGE_2_HOST_BIT/VK_ACCESS_2_HOST_READ_BIT, recorded withvkCmdPipelineBarrier2.submit_recording()calls it onr.cmdbeforevkEndCommandBuffer.replay_handle()calls it on its command buffer beforevkEndCommandBuffer.tests/test_host_read_barrier.py(new).The bug, step by step:
submit_recording()signals the timeline semaphore when the command buffer completes; the game's side waits on that value from the host and then reads the memory.HOST_READ, so the read in step 2 was not ordered against them by any Vulkan rule.The Vulkan rules for the new call, from
validusage.json:dstStageMaskisHOST.create_devicealready requiressynchronization2(vk_main.cpp, as on main) and the replay already recordsvkCmdPipelineBarrier2.vkEndCommandBuffer.end_pass) or a full barrier, outside any render pass instance.Before (
submit_recording()invk_replay.cpp):After:
The parallel segment job, after:
Measurements
Verdict: it fixes the bug:
tests/test_host_read_barrier.pyfails on main and passes with this PR, in both interleaved runs. No speed change is measured: one global barrier per command buffer, at its end.Hardware: AMD Radeon RX 9070 XT, Windows; the test itself is game-free (it reads source, it does not use the GPU); Python 3.12
Scene: none, game-free
Method:
tests/test_host_read_barrier.pyfrom this PR with--sourceset to each side's checkout, 2 runs each, interleaved (main, PR, main, PR); the result is pass/fail, so the median is the result both runs gaveHow to reproduce:
python -I -B tests/test_host_read_barrier.py --source <checkout>for each sidecefce0d17e71e972tests/test_host_read_barrier.py, run 1tests/test_host_read_barrier.py, run 2What the test checks:
host_read_barrier()exists invk_replay.cppand recordsALL_COMMANDS/MEMORY_WRITEtoHOST/HOST_READwithvkCmdPipelineBarrier2.submit_recording()callshost_read_barrier(r.cmd)beforevkEndCommandBuffer(r.cmd).host_read_barrier(c)betweenrecord_segment(c, j->g)andvkEndCommandBuffer(j->r->cmd).Flags and settings
None.
Testing
Tested: on this branch, Windows, Python 3.12:
test_host_read_barrierpasses (fails on main), andtest_capture_readback_visibility, the other check that reads the readback path, passes in all 16 of its cases.Built and played: this commit builds on Windows (llvm-mingw back end) and on Linux (
longshot_vk.so, without and with AMD FSR 4). Played on Windows, AMD Radeon RX 9070 XT: the game runs normally.Not tested: a game scene where stale reads showed: none is known (AMD and NVIDIA desktop drivers flush at the end of a submission). Linux was built, not run.
Must be tested: None.
Risks
What could break: one more barrier per command buffer; it waits for all earlier work at the very end, where the semaphore signal already does, so no frame-time change is expected.
Turning it off: revert the commit.
Compatibility: Nothing changes.
Checklist
Everyone
type(scope): description) of the status line's type.main.python -I -B tests/run_all.pypasses (any failure that also fails onmainis named). (the checks that read the changed code, on this PR, Windows:test_host_read_barrier,test_capture_readback_visibilitypass)third_party/untouched (a new folder with aPROVENANCE.mdinstead).When it applies
Assisted-by:/Co-Authored-By:trailers.mainand passing here. (tests/test_host_read_barrier.py)tests/that reads changed source updated in this PR, not weakened. (tests/test_host_read_barrier.pyadded; no check removed)longshot_vk.sowithout and with AMD FSR 4; no OS-specific code or paths)Tested in game: Windows 11 IoT Enterprise LTSC 24H2 (build 26100.9711), AMD Radeon RX 9070 XT, AMD Software: Adrenalin Edition 26.10.1: played with this change; the game runs normally, no stale readbacks, rendering errors or crashes seen.
WIP: fix(vk): make the GPU's writes visible to the host's reads at each submission's endto fix(vk): make the GPU's writes visible to the host's reads at each submission's endCode-only review of
7e71e972d7; informational findings, with final approval left to another reviewer.No introduced correctness defect found in the barrier placement or scopes. The serial path records the dependency after its last GPU commands; the parallel path does so after
record_segmentends rendering. Upload/init command buffers precede the replay command buffer in submission order, so they do not each need their own duplicate barrier. The recorder wrapper copies the barrier structures before the helper's stack frame ends. The device-write → host-read dependency is consistent with the Vulkan synchronization rules.Testing limitation:
tests/test_host_read_barrier.pychecks source strings, not the dependency actually passed to Vulkan. For example, changingd.memoryBarrierCount = 1to0would leave every checked string present while removing the memory dependency. A stubbed call-capture test of the helper should check the count, masks and pointed-to barrier, and reject that mutation. Treat the existing before/after pass as a structural regression check, not a reproduced stale-read failure. The author does separately report Windows RX 9070 XT gameplay and Windows/Linux builds; Linux gameplay and frame-time cost are not demonstrated.Merge coordination: the early-read work and capture visibility work alter related completion/readback paths; these are complementary, not duplicates of this dependency. Preserve the barrier after the final GPU write if recording-end ownership moves.
Read the full diff, submission/parallel recording paths, recorder packing and host-memory choices, prior feedback and related diffs. No tests, builds or game runs performed during this review.
View command line instructions
Checkout
From your project repository, check out a new branch and test the changes.