fix(vk): cooperative matrix probe checks map and fence, keeps in-flight resources #1247

Open
jcabarca wants to merge 2 commits from jcabarca/Longshot:fix/vk-coopmat-probe into main
Contributor

Type: fix · Risk: low · Tested on: Windows, AMD Radeon RX 9070 XT · Needs: none

Why

At start-up the cooperative matrix probe writes through an unchecked mapping, waits on a fence it may not have, and frees the pipeline, pool, fence and buffers while its dispatch may still run after a timed-out wait (VUID-vkDestroyPipeline-pipeline-00765 and others): a crash or a lost device at start-up on a slow or failing driver.

The same fix was first sent as part of #287 together with other changes; this PR sends it alone, one fix per PR.

How to reproduce

Seen on: not seen in the game: found by reading run_shader() in gpu/vk/vk_coopmat.cpp; it happens when vkMapMemory or vkCreateFence fails, or when the probe's dispatch does not finish within its 5 s wait
Settings and switches: defaults
Steps: game-free: python -I -B tests/test_coopmat_probe_failures.py --cxx clang++ --headers <Vulkan-Headers include> with this PR's test on a main checkout
Expected: a failed map or fence returns cleanly; a dispatch that did not finish keeps what it uses
Actual: on main the map-fail case fails (exit 2): the probe writes through a null mapping

What changed

run_shader() checks vkMapMemory and vkCreateFence, submits only with a fence, waits only after a successful submit, and when a submitted dispatch did not finish it returns without destroying what the dispatch uses (only the shader module, which may be destroyed once the pipeline exists).

  • vk: gpu/vk/vk_coopmat.cpp: run_shader() as above.
  • tests: tests/test_coopmat_probe_failures.py (new): four cases (map failure, fence failure, submit failure, wait timeout) against a cut-out of run_shader(); the fixture stands in for the Intel compiler gate so every case reaches the probe.

The bug, step by step:

  1. vkMapMemory's result was ignored and memcpy wrote through words.
  2. vkCreateFence's result was ignored; vkQueueSubmit2 and vkWaitForFences then used a null fence.
  3. After vkWaitForFences timed out or failed, the code destroyed the fence, command pool, pipeline and buffers while the dispatch could still be pending.

The rules:

  • VUID-vkDestroyPipeline-pipeline-00765: "All submitted commands that refer to pipeline must have completed execution"
  • VUID-vkDestroyCommandPool-commandPool-00041: "All VkCommandBuffer objects allocated from commandPool must not be in the pending state"
  • VUID-vkDestroyFence-fence-01120: "All queue submission commands that refer to fence must have completed execution"
  • VUID-vkFreeMemory-memory-00677: "All submitted commands that refer to memory (via images or buffers) must have completed execution"
  • VUID-vkDestroyBuffer-buffer-00922: "All submitted commands that refer to buffer, either directly or via a VkBufferView, must have completed execution"
  • VUID-vkWaitForFences-pFences-parameter: "pFences must be a valid pointer to an array of fenceCount valid VkFence handles"

When the dispatch did not finish the branch leaks the fence, pool, pipeline and buffers on purpose: Vulkan has no way to free them safely until the work completes, and the probe runs once at start-up.

Measurements

Verdict: it fixes the bug: tests/test_coopmat_probe_failures.py fails on main and passes with this PR, in both interleaved runs. No speed change is expected: the change is on a failure or check path.

Hardware: AMD Radeon RX 9070 XT, Windows; the test itself is game-free (it runs on the CPU, compiled with clang++ from llvm-mingw where it builds a fixture); Python 3.12
Scene: none, game-free
Method: this PR's test files on each side's checkout, 2 runs each, interleaved (main, PR, main, PR); the result is pass/fail, so the median is the result both runs gave
How to reproduce: python -I -B tests/test_coopmat_probe_failures.py --cxx clang++ --headers <Vulkan-Headers include> on each side

Before: main @ ac778751 After: this PR @ b59cbcdf
tests/test_coopmat_probe_failures.py, run 1 fails (fails at map-fail) passes (4 of 4 pass)
tests/test_coopmat_probe_failures.py, run 2 fails (fails at map-fail) passes (4 of 4 pass)
Median fails passes
run 1, main @ ac778751: exit 1: subprocess.CalledProcessError: Command '['C:\\Users\\t3040794\\AppData\\Local\\Temp\\coopmat-probe-17z6ydbl\\fixture.exe', 'map-fail']' returned non-zero exit status 2.
run 1, this PR @ b59cbcdf: exit 0: 4 cooperative-matrix probe failure cases passed
run 2, main @ ac778751: exit 1: subprocess.CalledProcessError: Command '['C:\\Users\\t3040794\\AppData\\Local\\Temp\\coopmat-probe-y4_n06hb\\fixture.exe', 'map-fail']' returned non-zero exit status 2.
run 2, this PR @ b59cbcdf: exit 0: 4 cooperative-matrix probe failure cases passed

Flags and settings

None.

Testing

Tested: on this branch, Windows, Python 3.12, clang++ from llvm-mingw: tests/test_coopmat_probe_failures.py passes (fails on main).

Built and played: this commit builds on Windows (llvm-mingw back end) and on Linux (longshot_vk.so, without and with AMD FSR 4). Played on Windows, AMD Radeon RX 9070 XT: the game runs normally. (The build that was played had this change before a whitespace-only fix: indentation, no code change.)

Not tested: the failure path itself in the game (it needs a driver failure or a format pair the RX 9070 XT does not produce on demand). Linux was built, not run.

Must be tested: None: the change is the same on every GPU and OS, and the game-free test covers it.

Risks

What could break: a probe whose dispatch did not finish leaks one small buffer, a pipeline, a pool and a fence, once, at start-up.

Turning it off: revert the commit.

Compatibility: Nothing changes.

Checklist

Everyone

  • One topic.
  • The title is a Conventional Commit (type(scope): description) of the status line's type.
  • Why gives the reason plainly and links the issues that fit (the tracker searched). (no issue fits)
  • Tested on my own PC where it runs (in the game, for a change the game runs), as Testing's Tested says; what
    can't be run by hand is in Testing's Not testable by hand, with why.
  • Measurements has a verdict, or one line saying why there is nothing to measure.
  • Nothing gets worse that I know of: any cost (speed, latency, memory, a setup that worked before) is measured and
    in Risks.
  • Testing names what was tested, what wasn't, and what must still be tested, where and how.
  • Rebased on current main.
  • python -I -B tests/run_all.py passes (any failure that also fails on main is named). (the check that reads the changed code, on this PR, Windows: tests/test_coopmat_probe_failures.py)
  • No game files, dumps, build output or generated files.
  • third_party/ untouched (a new folder with a PROVENANCE.md instead).
  • The license is OK: GPL-3.0-or-later, nothing incompatible and nothing decompiled.

When it applies

  • AI assistance recorded with Assisted-by: / Co-Authored-By: trailers.
  • A game-free test added for the fix, failing on main and passing here. (tests/test_coopmat_probe_failures.py)
  • A check under tests/ that reads changed source updated in this PR, not weakened.
  • Host code builds on both Windows (llvm-mingw) and Linux, with UTF-8 paths. (built on Windows with llvm-mingw and on Linux; no OS-specific code or paths)
**Type:** fix · **Risk:** low · **Tested on:** Windows, AMD Radeon RX 9070 XT · **Needs:** none ## Why At start-up the cooperative matrix probe writes through an unchecked mapping, waits on a fence it may not have, and frees the pipeline, pool, fence and buffers while its dispatch may still run after a timed-out wait (VUID-vkDestroyPipeline-pipeline-00765 and others): a crash or a lost device at start-up on a slow or failing driver. The same fix was first sent as part of #287 together with other changes; this PR sends it alone, one fix per PR. ## How to reproduce **Seen on:** not seen in the game: found by reading `run_shader()` in `gpu/vk/vk_coopmat.cpp`; it happens when `vkMapMemory` or `vkCreateFence` fails, or when the probe's dispatch does not finish within its 5 s wait **Settings and switches:** defaults **Steps:** game-free: `python -I -B tests/test_coopmat_probe_failures.py --cxx clang++ --headers <Vulkan-Headers include>` with this PR's test on a main checkout **Expected:** a failed map or fence returns cleanly; a dispatch that did not finish keeps what it uses **Actual:** on main the `map-fail` case fails (exit 2): the probe writes through a null mapping ## What changed `run_shader()` checks `vkMapMemory` and `vkCreateFence`, submits only with a fence, waits only after a successful submit, and when a submitted dispatch did not finish it returns without destroying what the dispatch uses (only the shader module, which may be destroyed once the pipeline exists). - vk: `gpu/vk/vk_coopmat.cpp`: `run_shader()` as above. - tests: `tests/test_coopmat_probe_failures.py` (new): four cases (map failure, fence failure, submit failure, wait timeout) against a cut-out of `run_shader()`; the fixture stands in for the Intel compiler gate so every case reaches the probe. The bug, step by step: 1. `vkMapMemory`'s result was ignored and `memcpy` wrote through `words`. 2. `vkCreateFence`'s result was ignored; `vkQueueSubmit2` and `vkWaitForFences` then used a null fence. 3. After `vkWaitForFences` timed out or failed, the code destroyed the fence, command pool, pipeline and buffers while the dispatch could still be pending. The rules: - VUID-vkDestroyPipeline-pipeline-00765: "All submitted commands that refer to pipeline must have completed execution" - VUID-vkDestroyCommandPool-commandPool-00041: "All VkCommandBuffer objects allocated from commandPool must not be in the pending state" - VUID-vkDestroyFence-fence-01120: "All queue submission commands that refer to fence must have completed execution" - VUID-vkFreeMemory-memory-00677: "All submitted commands that refer to memory (via images or buffers) must have completed execution" - VUID-vkDestroyBuffer-buffer-00922: "All submitted commands that refer to buffer, either directly or via a VkBufferView, must have completed execution" - VUID-vkWaitForFences-pFences-parameter: "pFences must be a valid pointer to an array of fenceCount valid VkFence handles" When the dispatch did not finish the branch leaks the fence, pool, pipeline and buffers on purpose: Vulkan has no way to free them safely until the work completes, and the probe runs once at start-up. ## Measurements **Verdict:** it fixes the bug: `tests/test_coopmat_probe_failures.py` fails on main and passes with this PR, in both interleaved runs. No speed change is expected: the change is on a failure or check path. Hardware: AMD Radeon RX 9070 XT, Windows; the test itself is game-free (it runs on the CPU, compiled with clang++ from llvm-mingw where it builds a fixture); Python 3.12 Scene: none, game-free Method: this PR's test files on each side's checkout, 2 runs each, interleaved (main, PR, main, PR); the result is pass/fail, so the median is the result both runs gave How to reproduce: `python -I -B tests/test_coopmat_probe_failures.py --cxx clang++ --headers <Vulkan-Headers include>` on each side | | Before: main @ ac778751 | After: this PR @ b59cbcdf | |---|---|---| | `tests/test_coopmat_probe_failures.py`, run 1 | fails (fails at `map-fail`) | passes (4 of 4 pass) | | `tests/test_coopmat_probe_failures.py`, run 2 | fails (fails at `map-fail`) | passes (4 of 4 pass) | | Median | fails | passes | ``` run 1, main @ ac778751: exit 1: subprocess.CalledProcessError: Command '['C:\\Users\\t3040794\\AppData\\Local\\Temp\\coopmat-probe-17z6ydbl\\fixture.exe', 'map-fail']' returned non-zero exit status 2. run 1, this PR @ b59cbcdf: exit 0: 4 cooperative-matrix probe failure cases passed run 2, main @ ac778751: exit 1: subprocess.CalledProcessError: Command '['C:\\Users\\t3040794\\AppData\\Local\\Temp\\coopmat-probe-y4_n06hb\\fixture.exe', 'map-fail']' returned non-zero exit status 2. run 2, this PR @ b59cbcdf: exit 0: 4 cooperative-matrix probe failure cases passed ``` ### Flags and settings None. ## Testing **Tested:** on this branch, Windows, Python 3.12, clang++ from llvm-mingw: `tests/test_coopmat_probe_failures.py` passes (fails on main). **Built and played:** this commit builds on Windows (llvm-mingw back end) and on Linux (`longshot_vk.so`, without and with AMD FSR 4). Played on Windows, AMD Radeon RX 9070 XT: the game runs normally. (The build that was played had this change before a whitespace-only fix: indentation, no code change.) **Not tested:** the failure path itself in the game (it needs a driver failure or a format pair the RX 9070 XT does not produce on demand). Linux was built, not run. **Must be tested:** None: the change is the same on every GPU and OS, and the game-free test covers it. ## Risks **What could break:** a probe whose dispatch did not finish leaks one small buffer, a pipeline, a pool and a fence, once, at start-up. **Turning it off:** revert the commit. **Compatibility:** Nothing changes. ## Checklist ### Everyone - [x] One topic. - [x] The title is a Conventional Commit (`type(scope): description`) of the status line's type. - [x] Why gives the reason plainly and links the issues that fit (the tracker searched). (no issue fits) - [x] Tested on my own PC where it runs (in the game, for a change the game runs), as Testing's Tested says; what can't be run by hand is in Testing's Not testable by hand, with why. - [x] Measurements has a verdict, or one line saying why there is nothing to measure. - [x] Nothing gets worse that I know of: any cost (speed, latency, memory, a setup that worked before) is measured and in Risks. - [x] Testing names what was tested, what wasn't, and what must still be tested, where and how. - [x] Rebased on current `main`. - [x] `python -I -B tests/run_all.py` passes (any failure that also fails on `main` is named). (the check that reads the changed code, on this PR, Windows: `tests/test_coopmat_probe_failures.py`) - [x] No game files, dumps, build output or generated files. - [x] `third_party/` untouched (a new folder with a `PROVENANCE.md` instead). - [x] The license is OK: GPL-3.0-or-later, nothing incompatible and nothing decompiled. ### When it applies - [x] AI assistance recorded with `Assisted-by:` / `Co-Authored-By:` trailers. - [x] A game-free test added for the fix, failing on `main` and passing here. (`tests/test_coopmat_probe_failures.py`) - [x] A check under `tests/` that reads changed source updated in this PR, not weakened. - [x] Host code builds on both Windows (llvm-mingw) and Linux, with UTF-8 paths. (built on Windows with llvm-mingw and on Linux; no OS-specific code or paths)
run_shader now fails cleanly when vkMapMemory fails (no memcpy through a null pointer), does not submit without a fence when vkCreateFence fails, and when the wait gives up it leaves what the dispatch may still use alive instead of destroying it under the GPU. New tests/test_coopmat_probe_failures.py runs run_shader against a stand-in driver for healthy, map-fail, fence-fail and wait-fail.

Assisted-by: GitHub Copilot
test(vk): stand in for the Intel compiler gate in the coopmat probe fixture
Some checks are pending
CI / Conventional Commit title (pull_request) Blocked by required conditions
CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Blocked by required conditions
CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Blocked by required conditions
CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Blocked by required conditions
CI / Workflow lint (actionlint) (pull_request) Blocked by required conditions
Guide screenshots / Guide images and local UI source review (pull_request) Blocked by required conditions
Third-party parts / As their notes say (pull_request) Blocked by required conditions
Third-party parts / The same as upstream (pull_request) Blocked by required conditions
Website / Website on Cloudflare (pull_request_target) Has been skipped
PR description / Description follows the template (pull_request_target) Successful in 9s
b59cbcdf84
main (#252) makes run_shader create its module and pipeline through IntelDriverGate, extern_guarded_shader_module and extern_guarded_compute.

Assisted-by: GitHub Copilot
Author
Contributor

Tested in game: Played on Windows, AMD Radeon RX 9070 XT: the game runs normally.

Tested in game: Played on Windows, AMD Radeon RX 9070 XT: the game runs normally.
Collaborator

Possibly related, for context:

  • PR #912 WIP: fix(vk): the cooperative matrix test waits for its dispatch before destroying what it uses — Overlapping cooperative-matrix probe lifetime fix
Possibly related, for context: - PR #912 WIP: fix(vk): the cooperative matrix test waits for its dispatch before destroying what it uses — Overlapping cooperative-matrix probe lifetime fix
Author
Contributor

I am juliotec, the original author of this fix. It was first sent as part of #287 together with other changes; this PR sends it alone, one fix per PR, rebased on current main.

I am juliotec, the original author of this fix. It was first sent as part of #287 together with other changes; this PR sends it alone, one fix per PR, rebased on current main.
jcabarca changed title from WIP: fix(vk): cooperative matrix probe checks map and fence, keeps in-flight resources to fix(vk): cooperative matrix probe checks map and fence, keeps in-flight resources 2026-11-16 22:10:26 +00:00
TrashPanda approved these changes 2026-11-17 00:12:58 +00:00
TrashPanda left a comment

The cooperative-matrix probe no longer writes through a failed mapping, submits without a fence, or destroys a dispatch that has not finished. I read head b59cbcdf. Approved.

run_shader checks vkMapMemory and frees the buffer and its memory when the map fails. A fence that does not create is left null, and the submit is skipped, so vkWaitForFences is not called on a null fence. When a submit did happen and the 5 s wait did not succeed, the function destroys the shader module and returns. The fence, command pool, descriptor pool, pipeline, pipeline layout, descriptor-set layout, buffer and memory stay. A pending dispatch still refers to all of those except the shader module, which Vulkan allows to be destroyed once the pipeline exists. A failed submit takes the old path and frees them, because nothing is pending.

The fixture compiles that function and runs it. Map failure, fence failure and a timed-out wait are real calls. A failed vkQueueSubmit2 is not one of the cases, though the description names it; that path frees everything because submitted is false. The Windows play on an RX 9070 XT did not hit a failed map, fence or wait. #912 is the other fix for the same wait: it waits without a limit and then destroys. It is still work in progress. Take this pull request. Do not land both.

The cooperative-matrix probe no longer writes through a failed mapping, submits without a fence, or destroys a dispatch that has not finished. I read head `b59cbcdf`. Approved. `run_shader` checks `vkMapMemory` and frees the buffer and its memory when the map fails. A fence that does not create is left null, and the submit is skipped, so `vkWaitForFences` is not called on a null fence. When a submit did happen and the 5 s wait did not succeed, the function destroys the shader module and returns. The fence, command pool, descriptor pool, pipeline, pipeline layout, descriptor-set layout, buffer and memory stay. A pending dispatch still refers to all of those except the shader module, which Vulkan allows to be destroyed once the pipeline exists. A failed submit takes the old path and frees them, because nothing is pending. The fixture compiles that function and runs it. Map failure, fence failure and a timed-out wait are real calls. A failed `vkQueueSubmit2` is not one of the cases, though the description names it; that path frees everything because `submitted` is false. The Windows play on an RX 9070 XT did not hit a failed map, fence or wait. https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/912 is the other fix for the same wait: it waits without a limit and then destroys. It is still work in progress. Take this pull request. Do not land both.
Owner

Brought up to the ready-for-review standard (#1454): the checklist's Everyone list has two new boxes, added here from what this PR's description and comments show (please check they match what you know).

  • Tested on my own PC: ticked: Testing's Tested and the author's comments: played in the game on Windows, AMD Radeon RX 9070 XT, with this change.
  • Nothing gets worse that I know of: ticked: Measurements expects no speed change and Risks names only a small start-up leak on the failure path.
Brought up to the [ready-for-review standard](https://git.magicbean.dev/MagicBeanDev/Longshot/src/branch/main/CONTRIBUTING.md#ready-for-review-out-of-wip) (#1454): the checklist's Everyone list has two new boxes, added here from what this PR's description and comments show (please check they match what you know). - *Tested on my own PC*: ticked: Testing's Tested and the author's comments: played in the game on Windows, AMD Radeon RX 9070 XT, with this change. - *Nothing gets worse that I know of*: ticked: Measurements expects no speed change and Risks names only a small start-up leak on the failure path.
Some checks are pending
CI / Conventional Commit title (pull_request) Blocked by required conditions
CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Blocked by required conditions
CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Blocked by required conditions
CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Blocked by required conditions
CI / Workflow lint (actionlint) (pull_request) Blocked by required conditions
Guide screenshots / Guide images and local UI source review (pull_request) Blocked by required conditions
Third-party parts / As their notes say (pull_request) Blocked by required conditions
Third-party parts / The same as upstream (pull_request) Blocked by required conditions
Website / Website on Cloudflare (pull_request_target) Has been skipped
PR description / Description follows the template (pull_request_target) Successful in 9s
This pull request doesn't have enough approvals yet. 0 of 1 approvals granted.
Some workflows are waiting to be reviewed.
This branch is out-of-date with the base branch
You are not authorized to merge this pull request.
View command line instructions

Checkout

From your project repository, check out a new branch and test the changes.
git fetch -u 'fix/vk-coopmat-probe:jcabarca-fix/vk-coopmat-probe'
git switch 'jcabarca-fix/vk-coopmat-probe'
Sign in to join this conversation.
No reviewers
No milestone
No project
No assignees
4 participants
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
MagicBeanDev/Longshot!1247
No description provided.