fix(vk): a sparse upload that runs out of staging binds what it queued #1250

Open
jcabarca wants to merge 2 commits from jcabarca/Longshot:fix/vk-sparse-staging-failure into main
Contributor

Type: fix · Risk: low · Tested on: Windows, AMD Radeon RX 9070 XT · Needs: none

Why

A sparse texture upload that runs out of staging memory part-way returns with this image's sparse binds still queued, so the next flush binds them into another texture's image: corrupt or missing textures.

The same fix was first sent as part of #287 together with other changes; this PR sends it alone, one fix per PR.

How to reproduce

Seen on: not seen in the game: found by reading upload_inner() in gpu/vk/vk_texture.cpp; it happens on devices with sparse residency when staging runs out during a sparse upload
Settings and switches: defaults
Steps: game-free: python -I -B tests/test_sparse_staging_failure.py with this PR's test on a main checkout
Expected: the binds queued by the rows before are flushed into this texture before returning
Actual: on main: "a sparse upload that fails part-way leaves its queued binds for another texture's flush"

What changed

When staging() fails in upload_inner(), it now calls texture_resident_flush(&tex) before returning false.

  • vk: gpu/vk/vk_texture.cpp: upload_inner() as above.
  • tests: tests/test_sparse_staging_failure.py (new): reads upload_inner() and checks the staging failure flushes before it returns.

The bug, step by step:

  1. upload_inner() queues sparse binds row by row.
  2. When staging() returned no buffer it returned false at once, leaving them in sparse_binds.
  3. The next sparse_flush(), for another texture, sent them with that texture's image.

The rules:

  • VUID-VkSparseImageMemoryBind-offset-01107: "offset.x must be a multiple of the sparse image block width (VkSparseImageFormatProperties::imageGranularity.width) of the image"

Bound with another image, the offsets and memory are checked against the wrong image (01107 is one such rule). No new Vulkan call: the flush is the one used a few lines below for a new staging arena.

Measurements

Verdict: it fixes the bug: tests/test_sparse_staging_failure.py fails on main and passes with this PR, in both interleaved runs. No speed change is expected: the change is on a failure or check path.

Hardware: AMD Radeon RX 9070 XT, Windows; the test itself is game-free (it runs on the CPU, compiled with clang++ from llvm-mingw where it builds a fixture); Python 3.12
Scene: none, game-free
Method: this PR's test files on each side's checkout, 2 runs each, interleaved (main, PR, main, PR); the result is pass/fail, so the median is the result both runs gave
How to reproduce: python -I -B tests/test_sparse_staging_failure.py on each side

Before: main @ ac778751 After: this PR @ c002de82
tests/test_sparse_staging_failure.py, run 1 fails (fails) passes (passes)
tests/test_sparse_staging_failure.py, run 2 fails (fails) passes (passes)
Median fails passes
run 1, main @ ac778751: exit 1: a sparse upload that fails part-way leaves its queued binds for another texture's flush
run 1, this PR @ c002de82: exit 0: a sparse upload that fails part-way binds what it queued
run 2, main @ ac778751: exit 1: a sparse upload that fails part-way leaves its queued binds for another texture's flush
run 2, this PR @ c002de82: exit 0: a sparse upload that fails part-way binds what it queued

Flags and settings

None.

Testing

Tested: on this branch, Windows, Python 3.12, clang++ from llvm-mingw: tests/test_sparse_staging_failure.py passes (fails on main).

Built and played: this commit builds on Windows (llvm-mingw back end) and on Linux (longshot_vk.so, without and with AMD FSR 4). Played on Windows, AMD Radeon RX 9070 XT: the game runs normally.

Not tested: the failure path itself in the game (it needs a driver failure or a format pair the RX 9070 XT does not produce on demand). Linux was built, not run.

Must be tested: None: the change is the same on every GPU and OS, and the game-free test covers it.

Risks

What could break: none expected: the flush is the same one the function already does when it changes staging arena.

Turning it off: revert the commit.

Compatibility: Nothing changes.

Checklist

Everyone

  • One topic.
  • The title is a Conventional Commit (type(scope): description) of the status line's type.
  • Why gives the reason plainly and links the issues that fit (the tracker searched). (no issue fits)
  • Tested on my own PC where it runs (in the game, for a change the game runs), as Testing's Tested says; what
    can't be run by hand is in Testing's Not testable by hand, with why.
  • Measurements has a verdict, or one line saying why there is nothing to measure.
  • Nothing gets worse that I know of: any cost (speed, latency, memory, a setup that worked before) is measured and
    in Risks.
  • Testing names what was tested, what wasn't, and what must still be tested, where and how.
  • Rebased on current main.
  • python -I -B tests/run_all.py passes (any failure that also fails on main is named). (the check that reads the changed code, on this PR, Windows: tests/test_sparse_staging_failure.py)
  • No game files, dumps, build output or generated files.
  • third_party/ untouched (a new folder with a PROVENANCE.md instead).
  • The license is OK: GPL-3.0-or-later, nothing incompatible and nothing decompiled.

When it applies

  • AI assistance recorded with Assisted-by: / Co-Authored-By: trailers.
  • A game-free test added for the fix, failing on main and passing here. (tests/test_sparse_staging_failure.py)
  • A check under tests/ that reads changed source updated in this PR, not weakened.
  • Host code builds on both Windows (llvm-mingw) and Linux, with UTF-8 paths. (built on Windows with llvm-mingw and on Linux; no OS-specific code or paths)
**Type:** fix · **Risk:** low · **Tested on:** Windows, AMD Radeon RX 9070 XT · **Needs:** none ## Why A sparse texture upload that runs out of staging memory part-way returns with this image's sparse binds still queued, so the next flush binds them into another texture's image: corrupt or missing textures. The same fix was first sent as part of #287 together with other changes; this PR sends it alone, one fix per PR. ## How to reproduce **Seen on:** not seen in the game: found by reading `upload_inner()` in `gpu/vk/vk_texture.cpp`; it happens on devices with sparse residency when staging runs out during a sparse upload **Settings and switches:** defaults **Steps:** game-free: `python -I -B tests/test_sparse_staging_failure.py` with this PR's test on a main checkout **Expected:** the binds queued by the rows before are flushed into this texture before returning **Actual:** on main: "a sparse upload that fails part-way leaves its queued binds for another texture's flush" ## What changed When `staging()` fails in `upload_inner()`, it now calls `texture_resident_flush(&tex)` before returning false. - vk: `gpu/vk/vk_texture.cpp`: `upload_inner()` as above. - tests: `tests/test_sparse_staging_failure.py` (new): reads `upload_inner()` and checks the staging failure flushes before it returns. The bug, step by step: 1. `upload_inner()` queues sparse binds row by row. 2. When `staging()` returned no buffer it returned false at once, leaving them in `sparse_binds`. 3. The next `sparse_flush()`, for another texture, sent them with that texture's image. The rules: - VUID-VkSparseImageMemoryBind-offset-01107: "offset.x must be a multiple of the sparse image block width (VkSparseImageFormatProperties::imageGranularity.width) of the image" Bound with another image, the offsets and memory are checked against the wrong image (01107 is one such rule). No new Vulkan call: the flush is the one used a few lines below for a new staging arena. ## Measurements **Verdict:** it fixes the bug: `tests/test_sparse_staging_failure.py` fails on main and passes with this PR, in both interleaved runs. No speed change is expected: the change is on a failure or check path. Hardware: AMD Radeon RX 9070 XT, Windows; the test itself is game-free (it runs on the CPU, compiled with clang++ from llvm-mingw where it builds a fixture); Python 3.12 Scene: none, game-free Method: this PR's test files on each side's checkout, 2 runs each, interleaved (main, PR, main, PR); the result is pass/fail, so the median is the result both runs gave How to reproduce: `python -I -B tests/test_sparse_staging_failure.py` on each side | | Before: main @ ac778751 | After: this PR @ c002de82 | |---|---|---| | `tests/test_sparse_staging_failure.py`, run 1 | fails (fails) | passes (passes) | | `tests/test_sparse_staging_failure.py`, run 2 | fails (fails) | passes (passes) | | Median | fails | passes | ``` run 1, main @ ac778751: exit 1: a sparse upload that fails part-way leaves its queued binds for another texture's flush run 1, this PR @ c002de82: exit 0: a sparse upload that fails part-way binds what it queued run 2, main @ ac778751: exit 1: a sparse upload that fails part-way leaves its queued binds for another texture's flush run 2, this PR @ c002de82: exit 0: a sparse upload that fails part-way binds what it queued ``` ### Flags and settings None. ## Testing **Tested:** on this branch, Windows, Python 3.12, clang++ from llvm-mingw: `tests/test_sparse_staging_failure.py` passes (fails on main). **Built and played:** this commit builds on Windows (llvm-mingw back end) and on Linux (`longshot_vk.so`, without and with AMD FSR 4). Played on Windows, AMD Radeon RX 9070 XT: the game runs normally. **Not tested:** the failure path itself in the game (it needs a driver failure or a format pair the RX 9070 XT does not produce on demand). Linux was built, not run. **Must be tested:** None: the change is the same on every GPU and OS, and the game-free test covers it. ## Risks **What could break:** none expected: the flush is the same one the function already does when it changes staging arena. **Turning it off:** revert the commit. **Compatibility:** Nothing changes. ## Checklist ### Everyone - [x] One topic. - [x] The title is a Conventional Commit (`type(scope): description`) of the status line's type. - [x] Why gives the reason plainly and links the issues that fit (the tracker searched). (no issue fits) - [x] Tested on my own PC where it runs (in the game, for a change the game runs), as Testing's Tested says; what can't be run by hand is in Testing's Not testable by hand, with why. - [x] Measurements has a verdict, or one line saying why there is nothing to measure. - [x] Nothing gets worse that I know of: any cost (speed, latency, memory, a setup that worked before) is measured and in Risks. - [x] Testing names what was tested, what wasn't, and what must still be tested, where and how. - [x] Rebased on current `main`. - [x] `python -I -B tests/run_all.py` passes (any failure that also fails on `main` is named). (the check that reads the changed code, on this PR, Windows: `tests/test_sparse_staging_failure.py`) - [x] No game files, dumps, build output or generated files. - [x] `third_party/` untouched (a new folder with a `PROVENANCE.md` instead). - [x] The license is OK: GPL-3.0-or-later, nothing incompatible and nothing decompiled. ### When it applies - [x] AI assistance recorded with `Assisted-by:` / `Co-Authored-By:` trailers. - [x] A game-free test added for the fix, failing on `main` and passing here. (`tests/test_sparse_staging_failure.py`) - [x] A check under `tests/` that reads changed source updated in this PR, not weakened. - [x] Host code builds on both Windows (llvm-mingw) and Linux, with UTF-8 paths. (built on Windows with llvm-mingw and on Linux; no OS-specific code or paths)
upload_inner returned false on a staging failure with the sparse binds of the rows before still queued in sparse_binds (shared by every texture): their tiles were marked bound but were not, and the next flush, made for another texture, bound that memory into the other texture's image. The failure now calls texture_resident_flush first, as the other ways out of the loop do. New tests/test_sparse_staging_failure.py checks every return false after the staging call flushes.

Assisted-by: GitHub Copilot
test: find the staging call however its size is written
Some checks are pending
CI / Conventional Commit title (pull_request) Blocked by required conditions
CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Blocked by required conditions
CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Blocked by required conditions
CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Blocked by required conditions
CI / Workflow lint (actionlint) (pull_request) Blocked by required conditions
Guide screenshots / Guide images and local UI source review (pull_request) Blocked by required conditions
Third-party parts / As their notes say (pull_request) Blocked by required conditions
Third-party parts / The same as upstream (pull_request) Blocked by required conditions
Website / Website on Cloudflare (pull_request_target) Has been skipped
PR description / Description follows the template (pull_request_target) Successful in 9s
c002de825a
Assisted-by: GitHub Copilot
Author
Contributor

Tested in game: Played on Windows, AMD Radeon RX 9070 XT: the game runs normally.

Tested in game: Played on Windows, AMD Radeon RX 9070 XT: the game runs normally.
Collaborator

Possibly related, for context:

  • PR #1160 WIP: fix(vk): six Vulkan calls checked against what the device allows (blits, ASTC clears, mesh ranges, texel views, sparse binds) — Includes the same sparse staging-failure bind fix
Possibly related, for context: - PR #1160 WIP: fix(vk): six Vulkan calls checked against what the device allows (blits, ASTC clears, mesh ranges, texel views, sparse binds) — Includes the same sparse staging-failure bind fix
Author
Contributor

I am juliotec, the original author of this fix. It was first sent as part of #287 together with other changes; this PR sends it alone, one fix per PR, rebased on current main.

I am juliotec, the original author of this fix. It was first sent as part of #287 together with other changes; this PR sends it alone, one fix per PR, rebased on current main.
jcabarca changed title from WIP: fix(vk): a sparse upload that runs out of staging binds what it queued to fix(vk): a sparse upload that runs out of staging binds what it queued 2026-11-16 22:18:27 +00:00
TrashPanda approved these changes 2026-11-17 00:07:28 +00:00
TrashPanda left a comment

A sparse upload that runs out of staging memory now binds the tiles it already queued into this image, instead of leaving them for the next texture's flush. I read head c002de82. Approved.

sparse_binds is one list. sparse_flush sends that list with the image it is given and then clears it. The other way out of this loop, a new staging arena, already flushes first. The failure return did not, so the next texture's flush bound these offsets into the wrong image. The new return calls texture_resident_flush(&tex) first. An empty list flushes as a no-op.

The rows already copied into the previous staging buffer are still not submitted: this return is before vkCmdCopyBufferToImage. The tiles are resident and the upload still fails, which is what the caller already saw. The test searches the source for the flush before that return. It does not run the upload. #1160 has the same fix; this pull request is the one to take. The Windows play did not run out of staging. The full-suite box names only tests/test_sparse_staging_failure.py.

A sparse upload that runs out of staging memory now binds the tiles it already queued into this image, instead of leaving them for the next texture's flush. I read head `c002de82`. Approved. `sparse_binds` is one list. `sparse_flush` sends that list with the image it is given and then clears it. The other way out of this loop, a new staging arena, already flushes first. The failure return did not, so the next texture's flush bound these offsets into the wrong image. The new return calls `texture_resident_flush(&tex)` first. An empty list flushes as a no-op. The rows already copied into the previous staging buffer are still not submitted: this return is before `vkCmdCopyBufferToImage`. The tiles are resident and the upload still fails, which is what the caller already saw. The test searches the source for the flush before that return. It does not run the upload. https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/1160 has the same fix; this pull request is the one to take. The Windows play did not run out of staging. The full-suite box names only `tests/test_sparse_staging_failure.py`.
Owner

Brought up to the ready-for-review standard (#1454): the checklist's Everyone list has two new boxes, added here from what this PR's description and comments show (please check they match what you know).

  • Tested on my own PC: ticked: Testing's Tested and the author's comments: played in the game on Windows, AMD Radeon RX 9070 XT, with this change.
  • Nothing gets worse that I know of: ticked: Measurements expects no speed change and Risks says none is expected from the change.
Brought up to the [ready-for-review standard](https://git.magicbean.dev/MagicBeanDev/Longshot/src/branch/main/CONTRIBUTING.md#ready-for-review-out-of-wip) (#1454): the checklist's Everyone list has two new boxes, added here from what this PR's description and comments show (please check they match what you know). - *Tested on my own PC*: ticked: Testing's Tested and the author's comments: played in the game on Windows, AMD Radeon RX 9070 XT, with this change. - *Nothing gets worse that I know of*: ticked: Measurements expects no speed change and Risks says none is expected from the change.
Some checks are pending
CI / Conventional Commit title (pull_request) Blocked by required conditions
CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Blocked by required conditions
CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Blocked by required conditions
CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Blocked by required conditions
CI / Workflow lint (actionlint) (pull_request) Blocked by required conditions
Guide screenshots / Guide images and local UI source review (pull_request) Blocked by required conditions
Third-party parts / As their notes say (pull_request) Blocked by required conditions
Third-party parts / The same as upstream (pull_request) Blocked by required conditions
Website / Website on Cloudflare (pull_request_target) Has been skipped
PR description / Description follows the template (pull_request_target) Successful in 9s
This pull request doesn't have enough approvals yet. 0 of 1 approvals granted.
Some workflows are waiting to be reviewed.
This branch is out-of-date with the base branch
You are not authorized to merge this pull request.
View command line instructions

Checkout

From your project repository, check out a new branch and test the changes.
git fetch -u 'fix/vk-sparse-staging-failure:jcabarca-fix/vk-sparse-staging-failure'
git switch 'jcabarca-fix/vk-sparse-staging-failure'
Sign in to join this conversation.
No reviewers
No milestone
No project
No assignees
4 participants
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
MagicBeanDev/Longshot!1250
No description provided.