WIP: fix(vk): seal capture repairs and prove exact CPU visibility #759

Draft

siantm wants to merge 1 commit from siantm/Longshot:fix/capture-repair-publication into main

pull from: siantm/Longshot:fix/capture-repair-publication

merge into: MagicBeanDev:main

MagicBeanDev:main

Conversation 0 Commits 1 Files changed 14 +1489 -183

siantm commented 2026-11-15 10:03:07 +00:00

First-time contributor

• Copy link
• Copy Markdown

Type: fix · Risk: high · Tested on: Windows 11 Pro 26300, compiled fake-Vulkan consumers · Needs: full native renderer build, full suite and real Vulkan validation

Why

CPU buffer and texture consumers must see the completed immutable capture, not stale host-cache bytes or newer guest memory. Repair tables must be sealed before GPU submission becomes visible, and a later successful signal must not conceal a failed earlier recording.

This is a follow-on to #526, not a replacement for its same-part fallback/writeback-shadow fixes. It does not claim the reported player-visible flashes are fixed.

How to reproduce

Seen on: authored game-free Windows fixtures; a player-visible artifact has not been reproduced on this isolated branch.
Settings and switches: fake deferred/failed Vulkan submissions, noncoherent mappings and capture-generation changes.
Steps: run the focused drivers listed under Testing; they exercise production helpers with controlled publication races, held/direct failures and CPU reads.
Expected: exact completion and publication proof before repaired bytes become CPU-visible; tables and capture allocations stay valid through waits.
Actual before this proposal: the missing ordering/lifetime contract is established by source analysis and earlier local synthetic work, not a fresh same-fixture baseline run. Baseline consumer proof and real GPU validation are still required before readiness.

What changed

Seal repair metadata before direct/held queue publication, prove completion of the exact recording part, and pin captures while releasing memory locks for waits. Invalidate only the repaired allocation/page range and restart whole multipage reads if their capture view changes.

• vk_replay.cpp / declarations: exact completion, failed-submission poisoning and publication ordering.
• vk_memory.cpp, vk_vma.cpp: sealed tables, retained storage and checked noncoherent visibility.
• Optional parallel paths emit repairs in an ordered prefix before consumer workers; parallel rendering remains off by default.
• Consumer fixtures and relevant source-cut fixtures migrate with the contract. Unrelated local optimizations and PR 526's separate changes are excluded.

Measurements

Verdict: eight focused compiled-consumer drivers pass. This is correctness work; no FPS gain or flashes-fixed claim.
Hardware: Windows 11 Pro 26300, Intel Core i7-9700, 31.8 GiB usable RAM; Python 3.14.7, LLVM-MinGW Clang 23.1.5, Vulkan-Headers v1.4.345. No GPU used.
Method: authored bytes, deferred/failed work and fake noncoherent visibility; production helper consumers compiled from this PR @ 2de66d862d on main 9e6f231d.

Observed: PASS: capture prefix repair, immutable exact-part CPU/texture reads, cache visibility, fallback and lifetime semantics; direct/held failed work stays unproven after a higher signal; concurrent waiters cannot pass prequeue sealing/publication. No fresh same-fixture baseline pass/fail comparison, complete renderer build or real GPU run is claimed. Historical integrated-local serial/parallel FPS observations are not measurements of this isolated branch.

Flags and settings

No new switch or default. Parallel rendering stays off; threaded-rendering defaults stay unchanged. Existing RX_VK_LATE_FIX=0 retains the wait/copy fallback. Explicitly enabled parallel repair uses the ordered prefix rather than publishing mutable tables.

Testing

Tested: test_late_fix_build, test_readback_part, test_parallel_init_order, test_draw_stream_visibility, test_fsr_image_retirement, test_flush_nothing_held, test_value_rec_last_part and test_recorded_wait_flush: all eight drivers pass. Catalogue renderer accepts the measured-run fragment.
Not tested: full native renderer build, full suite, Linux, real GPU/runtime behavior, or exhaustive transient-artifact detection. Catalogue unit driver has a Windows CRLF-vs-LF fixture failure.
Must be tested: author/Test Farm completes native Windows/Linux builds, full suite and real Vulkan serial/opt-in parallel runs including held batching, resumed submissions, CPU buffer/texture uploads, sparse binding and noncoherent memory. Capture both expected visibility and absence of device loss/deadlocks; coordinate the overlap with PR 526.

Risks

What could break: deadlocks, capture retention, failed-submission handling, noncoherent visibility or retirement ordering. Synthetic success is not exhaustive runtime proof.
Turning it off: disable late fixes for the existing fallback, disable optional parallel replay, or revert the patch.
Compatibility: public renderer ABI, saves/options and disk-cache formats are unchanged; no game-derived artifacts are included.

Checklist

Everyone

• One correctness topic.
• Conventional Commit title: fix(vk): seal capture repairs and prove exact CPU visibility.
• Complete issue matching/owner coordination; related PR 526 reviewed and linked.
• Measurements state the observed checks and do not claim FPS/artifact fixes.
• Testing explicitly distinguishes completed, untested and required coverage.
• Recheck current main before publication; prepared on 9e6f231d.
• Complete game-free suite recorded; not run.
• No game files, dumps, generated game files or build output.
• Third-party sources untouched.
• Existing Longshot source and authored fixtures; GPL-3.0-or-later.

When it applies

• Source commit records Assisted-by: OpenAI Codex.
• Focused game-free run recorded under docs/tested/.
• Baseline behavioral consumer proof completed; current isolated branch checks pass.
• Relevant source-cut fixtures updated without widening obsolete source-window pins.
• Windows/Linux production builds and real Vulkan coverage completed.
• Development documentation updated.

**Type:** fix · **Risk:** high · **Tested on:** Windows 11 Pro 26300, compiled fake-Vulkan consumers · **Needs:** full native renderer build, full suite and real Vulkan validation ## Why CPU buffer and texture consumers must see the completed immutable capture, not stale host-cache bytes or newer guest memory. Repair tables must be sealed before GPU submission becomes visible, and a later successful signal must not conceal a failed earlier recording. This is a follow-on to https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/526, not a replacement for its same-part fallback/writeback-shadow fixes. It does not claim the reported player-visible flashes are fixed. ## How to reproduce **Seen on:** authored game-free Windows fixtures; a player-visible artifact has not been reproduced on this isolated branch. **Settings and switches:** fake deferred/failed Vulkan submissions, noncoherent mappings and capture-generation changes. **Steps:** run the focused drivers listed under Testing; they exercise production helpers with controlled publication races, held/direct failures and CPU reads. **Expected:** exact completion and publication proof before repaired bytes become CPU-visible; tables and capture allocations stay valid through waits. **Actual before this proposal:** the missing ordering/lifetime contract is established by source analysis and earlier local synthetic work, not a fresh same-fixture baseline run. Baseline consumer proof and real GPU validation are still required before readiness. ## What changed Seal repair metadata before direct/held queue publication, prove completion of the exact recording part, and pin captures while releasing memory locks for waits. Invalidate only the repaired allocation/page range and restart whole multipage reads if their capture view changes. - `vk_replay.cpp` / declarations: exact completion, failed-submission poisoning and publication ordering. - `vk_memory.cpp`, `vk_vma.cpp`: sealed tables, retained storage and checked noncoherent visibility. - Optional parallel paths emit repairs in an ordered prefix before consumer workers; parallel rendering remains off by default. - Consumer fixtures and relevant source-cut fixtures migrate with the contract. Unrelated local optimizations and PR 526's separate changes are excluded. ## Measurements **Verdict:** eight focused compiled-consumer drivers pass. This is correctness work; no FPS gain or flashes-fixed claim. Hardware: Windows 11 Pro 26300, Intel Core i7-9700, 31.8 GiB usable RAM; Python 3.14.7, LLVM-MinGW Clang 23.1.5, Vulkan-Headers v1.4.345. No GPU used. Method: authored bytes, deferred/failed work and fake noncoherent visibility; production helper consumers compiled from this PR @ 2de66d862d on main 9e6f231d. Observed: `PASS: capture prefix repair, immutable exact-part CPU/texture reads, cache visibility, fallback and lifetime semantics`; direct/held failed work stays unproven after a higher signal; concurrent waiters cannot pass prequeue sealing/publication. No fresh same-fixture baseline pass/fail comparison, complete renderer build or real GPU run is claimed. Historical integrated-local serial/parallel FPS observations are not measurements of this isolated branch. ### Flags and settings No new switch or default. Parallel rendering stays off; threaded-rendering defaults stay unchanged. Existing `RX_VK_LATE_FIX=0` retains the wait/copy fallback. Explicitly enabled parallel repair uses the ordered prefix rather than publishing mutable tables. ## Testing **Tested:** `test_late_fix_build`, `test_readback_part`, `test_parallel_init_order`, `test_draw_stream_visibility`, `test_fsr_image_retirement`, `test_flush_nothing_held`, `test_value_rec_last_part` and `test_recorded_wait_flush`: all eight drivers pass. Catalogue renderer accepts the measured-run fragment. **Not tested:** full native renderer build, full suite, Linux, real GPU/runtime behavior, or exhaustive transient-artifact detection. Catalogue unit driver has a Windows CRLF-vs-LF fixture failure. **Must be tested:** author/Test Farm completes native Windows/Linux builds, full suite and real Vulkan serial/opt-in parallel runs including held batching, resumed submissions, CPU buffer/texture uploads, sparse binding and noncoherent memory. Capture both expected visibility and absence of device loss/deadlocks; coordinate the overlap with PR 526. ## Risks **What could break:** deadlocks, capture retention, failed-submission handling, noncoherent visibility or retirement ordering. Synthetic success is not exhaustive runtime proof. **Turning it off:** disable late fixes for the existing fallback, disable optional parallel replay, or revert the patch. **Compatibility:** public renderer ABI, saves/options and disk-cache formats are unchanged; no game-derived artifacts are included. ## Checklist ### Everyone - [x] One correctness topic. - [x] Conventional Commit title: `fix(vk): seal capture repairs and prove exact CPU visibility`. - [ ] Complete issue matching/owner coordination; related PR 526 reviewed and linked. - [x] Measurements state the observed checks and do not claim FPS/artifact fixes. - [x] Testing explicitly distinguishes completed, untested and required coverage. - [ ] Recheck current main before publication; prepared on 9e6f231d. - [ ] Complete game-free suite recorded; not run. - [x] No game files, dumps, generated game files or build output. - [x] Third-party sources untouched. - [x] Existing Longshot source and authored fixtures; GPL-3.0-or-later. ### When it applies - [x] Source commit records `Assisted-by: OpenAI Codex`. - [x] Focused game-free run recorded under docs/tested/. - [ ] Baseline behavioral consumer proof completed; current isolated branch checks pass. - [x] Relevant source-cut fixtures updated without widening obsolete source-window pins. - [ ] Windows/Linux production builds and real Vulkan coverage completed. - [x] Development documentation updated.

siantm added 1 commit 2026-11-15 10:03:08 +00:00

fix(vk): seal capture repairs and prove exact CPU visibility ... 2de66d862d

Focused correctness extraction from local 7f96a733, 91de8bc7, 51cc9d17, 85efa229, 7d116221 and documentation 3b286932. No unrelated optimizations or game-derived files.

Assisted-by: OpenAI Codex

GummiNaomi referenced this pull request 2026-11-15 11:00:52 +00:00

fix(vk): with the render thread off, a frame is no longer drawn before the data the game writes after its submit (Link dark for a frame) #769

mooki added the

area/renderer

kind

bug

platform/linux

platform/windows

priority

high

labels 2026-11-15 13:59:29 +00:00

HumanBean referenced this pull request from a commit 2026-11-15 18:29:41 +00:00

fix(vk): with the render thread off, a frame is no longer drawn before the data the game writes after its submit (Link dark for a frame) (#769)

Some checks are pending

Hide checks

CI / Conventional Commit title (pull_request) Blocked by required conditions

Details

CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Blocked by required conditions

Details

CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Blocked by required conditions

Details

CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Blocked by required conditions

Details

CI / Workflow lint (actionlint) (pull_request) Blocked by required conditions

Details

Guide screenshots / Guide images and local UI source review (pull_request) Blocked by required conditions

Details

Third-party parts / As their notes say (pull_request) Blocked by required conditions

Details

Third-party parts / The same as upstream (pull_request) Blocked by required conditions

Details

Website / Website on Cloudflare (pull_request) Blocked by required conditions

Details

PR description / Description follows the template (pull_request_target) Successful in 17s

Details

Website / Website on Cloudflare (pull_request_target) Has been skipped

Details

This pull request has changes conflicting with the target branch.

• docs/development.md
• gpu/vk/vk_memory.cpp
• gpu/vk/vk_replay.cpp
• tests/test_late_fix_build.py

Some workflows are waiting to be reviewed.

View command line instructions

Manual merge helper

Use this merge commit message when completing the merge manually.

Merge commit title WIP: fix(vk): seal capture repairs and prove exact CPU visibility (#759)

Merge commit body Reviewed-on: https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/759

WIP: fix(vk): seal capture repairs and prove exact CPU visibility (#759) Reviewed-on: https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/759 Copy merge commit message

Checkout

From your project repository, check out a new branch and test the changes.

git fetch -u 'fix/capture-repair-publication:siantm-fix/capture-repair-publication'

git switch 'siantm-fix/capture-repair-publication'

Sign in to join this conversation.

Reviewers

No reviewers

Labels

Clear labels   

area/accessibility

Barriers for people with disabilities

  

area/audio

Sound and music

  

area/input

Controllers, keyboard, mouse

  

area/install

Installing, updating, building from your game

  

area/menu

In-game menu and settings

  

area/renderer

Vulkan renderer, shaders, effects

  

area/saves

Save files

  

area/upscaling

DLSS, FSR, frame generation

  

beanstalk: approved @ 1697603

Set by Beanstalk for show; the farm's database is what counts

  

beanstalk: approved @ 2c5a62e

Set by Beanstalk for show; the farm's database is what counts

  

beanstalk: approved @ 4fd82b7

Set by Beanstalk for show; the farm's database is what counts

  

beanstalk: approved @ 64d6d09

Set by Beanstalk for show; the farm's database is what counts

  

beanstalk: approved @ 6f91fae

Set by Beanstalk for show; the farm's database is what counts

  

beanstalk: approved @ 70a78f9

Set by Beanstalk for show; the farm's database is what counts

  

beanstalk: approved @ 8d361dc

Set by Beanstalk for show; the farm's database is what counts

  

beanstalk: approved @ 9d8356e

Set by Beanstalk for show; the farm's database is what counts

  

beanstalk: approved @ abf2227

Set by Beanstalk for show; the farm's database is what counts

  

beanstalk: approved @ d82f202

Set by Beanstalk for show; the farm's database is what counts

  

beanstalk: approved @ e5f1e40

Set by Beanstalk for show; the farm's database is what counts

  

beanstalk: approved @ ee8ab5f

Set by Beanstalk for show; the farm's database is what counts

  

ci: approved

A maintainer approved CircleCI for this fork PR (sticky; removed after 14 quiet days)

  

good first issue

A good start for a new contributor

  

gpu/amd

AMD GPUs

  

gpu/igpu

Integrated GPUs

  

gpu/intel

Intel GPUs

  

gpu/nvidia

NVIDIA GPUs

  

help wanted

Extra hands or knowledge wanted

  

kind

bug

Something doesn't work as it should

  

kind

chore

Build, CI, dependencies, tidying

  

kind

docs

Documentation

  

kind

feature

A new feature or an improvement

  

kind

performance

Frame rate, frame time, loading, memory

  

kind

question

A question rather than a change

  

needs measurements

Waiting for before/after measurements

  

platform/linux

Affects Linux (a change for every system carries both Windows and Linux)

  

platform/macos

macOS (Apple silicon or Intel Macs)

  

platform/steam-deck

Steam Deck

  

platform/windows

Affects Windows (a change for every system carries both Windows and Linux)

  

preview

Maintainers: deploy a preview of this PR's website (read its changes first). Removed after the run.

  

priority

critical

Crashes, data loss, or blocks most players

  

priority

high

Important: next in line

  

priority

low

Nice to have

  

regression

Worked in an earlier build, broken in a newer one: highest priority, above other bugs

  

status

blocked

Waits on something else first

  

status

confirmed

Reproduced or agreed on

  

status

duplicate

Already reported or proposed

  

status

invalid

Not a problem in this project

  

status

needs-info

Waiting for more information from the reporter

  

status

wontfix

Will not be worked on

  

testing wanted

Wanted: a test on other hardware (posts a test request in #test-requests)

  

ci: approved

No labels

area/accessibility

area/audio

area/input

area/install

area/menu

area/renderer

area/saves

area/upscaling

beanstalk: approved @ 1697603

beanstalk: approved @ 2c5a62e

beanstalk: approved @ 4fd82b7

beanstalk: approved @ 64d6d09

beanstalk: approved @ 6f91fae

beanstalk: approved @ 70a78f9

beanstalk: approved @ 8d361dc

beanstalk: approved @ 9d8356e

beanstalk: approved @ abf2227

beanstalk: approved @ d82f202

beanstalk: approved @ e5f1e40

beanstalk: approved @ ee8ab5f

ci: approved

good first issue

gpu/amd

gpu/igpu

gpu/intel

gpu/nvidia

help wanted

kind

bug

kind

chore

kind

docs

kind

feature

kind

performance

kind

question

needs measurements

platform/linux

platform/macos

platform/steam-deck

platform/windows

preview

priority

critical

priority

high

priority

low

regression

status

blocked

status

confirmed

status

duplicate

status

invalid

status

needs-info

status

wontfix

testing wanted

ci: approved

Milestone

Clear milestone

No items

No milestone

Projects

Clear projects

No items

No project

Assignees

Clear assignees

No assignees

1 participant

Notifications

Subscribe

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference

MagicBeanDev/Longshot!759

No description provided.