WIP: fix(tests): three renderer checks build and run on main again #2037

Draft
siantm wants to merge 3 commits from siantm/Longshot:fix/tests-renderer-fixtures-build into main
Contributor

Summary

Three game-free renderer checks fail on main before they run a single case: each cuts renderer functions out of the
source by their text, and the source moved on under them. #759 and #1995 change these checks (#1995 adds cases to the
uniform one), and their changes cannot run either. This brings each back to running its cases against the code as it
is on main @ fd7dc40d, one commit per check; nothing outside tests/ changes.

  • tests/test_fsr_image_retirement.py (FSR images kept until the GPU is past their last use): submit_recording ends
    each recording with host_read_barrier(r.cmd) since 905e36a9 (#1031), which the check did not cut, so its fixture
    stopped at "use of undeclared identifier 'host_read_barrier'". The check now cuts host_read_barrier with the rest of
    the submission code, and the fixture gives it vkCmdPipelineBarrier2 (no GPU). No case or assertion changes.
  • tests/test_fsr_uniform_ownership.py (FSR constants stay owned by their recording): the recording code it cuts gained
    calls its fixture did not provide: recording_wait's FpsRtWait (c64e864f, #928), begin_recording's
    texture_complete_writes (7daafa7a, #1553), draw_begin_recording's pool_shift_first, make_pool(shift) and
    pools_grown (382a3d67, #1158) and mesh_conv_begin_recording (9027e737, #1846): ten compile errors. The check now
    cuts pool_shift_first too; the fixture's make_pool takes the shift, pools_grown is counted, and the FPS counter's
    timer, the texture writes' completion and the mesh draws' conversion stand empty (none of them here). No case or
    assertion changes.
  • tests/test_draw_stream_visibility.py (the draw streams' bytes are flushed before the GPU reads them): it cut the
    parallel replay's segment submit out of vk_replay.cpp (from if (j->inits < 0) init_lost = true; to the device-lost
    check), and a3565ac5 (#1763) deleted the segment workers and that loop with them: "substring not found", before
    anything compiled. Dropped: the cases segments and failed-segment, which ran that loop: the code they tested is
    gone in a3565ac5 (no segment worker records a slot, no segment submit orders them, no init command buffer whose failure
    stops the submits after it). Every recording is now submitted by submit_recording, which the seven other cases run;
    the fixture's harness for the loop (Job, submit_segments, the init command buffer, the submitted-slot bookkeeping)
    goes with the two cases. The rest builds against main: host_read_barrier and pool_shift_first are cut from the
    source, make_pool, pools_grown and mesh_conv_begin_recording as in the uniform check. The other cases and their
    assertions are unchanged.

How to see it on main: with the Vulkan headers found (RX_CPM pointing at the CPM cache, or
LONGSHOT_VULKAN_HEADERS), python -I -B tests/run_all.py --strict --exact tests/test_fsr_image_retirement.py (and the
same for the other two) fails with the errors above. Without the headers, --strict fails each with "not found:
vulkan_headers" and nothing is compiled.

Read first: the three tests/test_*.py drivers and their fixtures (tests/fsr_image_retirement_test.cpp,
tests/fsr_uniform_ownership_test.cpp, tests/draw_stream_visibility_test.cpp).

Testing

Tested: Windows 11 (10.0.26300), llvm-mingw clang 23.1.5, the Vulkan headers 1.4.345 from the CPM cache (RX_CPM);
game-free checks only: nothing here runs in the game, so there was nothing to run in it. Each check ran alone
(python -I -B tests/run_all.py --strict --exact tests/<check>.py), in a hidden console of its own, in a build job of
this PC's job queue (fb3-fix-3), on main @ e6196f0f and on this branch @ 7354e49d:

  • On main each fails as the Summary says: the uniform check with its ten compile errors, the retirement check with
    host_read_barrier undeclared, the draw check with "substring not found" before compiling.
  • Here each passes: the retirement check's 14 cases for both backends, the uniform check's 11 cases with a coherent and
    a non-coherent cache for both backends, the draw check's 7 cases with both caches.
  • Each fails again when the code it checks is broken, in a scratch copy of this branch with one function it cuts changed
    (not committed):
    • vk_fsr3.cpp free_dead freeing a retired image one timeline value early (now + 1 >= ...): the retirement check
      fails at resumed, "view destroyed while GPU use is pending".
    • vk_draw.cpp uniform_upload handing back the stream's start instead of its slice (cur->mapped for
      cur->mapped + at): the uniform check fails at queued, "GPU must see this evaluation's authored constants".
    • vk_draw.cpp draw_flush_recording flushing only the stream in use, not the ones the recording outgrew: the draw
      check fails at grow with the non-coherent cache, "GPU draw input must see the CPU-authored bytes".
  • python -I -B tests/run_all.py --strict --changed origin/main (the three checks read the changed files, and the
    byte-compile of the tracked Python): 4 passed.

The same results came earlier on main @ 40bfceb0 (fb3-fix-1 on main; fb3-fix-2 on these commits there, @ 30589ebe,
with the same scratch breaks). Then rebased onto main @ fd7dc40d without a conflict, the three commits unchanged
(git range-diff), now 98f1ad99; the checks have not run again on that base: its three new commits (#2007, #2017,
#1657) change none of the text these checks cut, and the C++ each check writes is byte-for-byte the same as at 7354e49d
(written out without compiling, and compared). Not run: the whole mise run test.

Still to test:

  • Linux: python3 -I -B tests/run_all.py --strict --exact with each of the three checks, expecting each to pass, as on
    Windows (by hand, or CI's run)

Measurements

Nothing to measure: only three game-free checks under tests/ change; nothing the game runs.

Risks

Nothing a player notices: only tests/ changes, and nothing the game runs. The three checks now run their cases where
CI runs them; if the renderer reshapes the functions they cut again, they stop with tests/srccut.py's message naming
the marker, or a compile error. A revert gives back main's three failing checks.

Checklist

Ready for review

  • One topic, on current main; the title is a Conventional Commit.
  • Tested on my own PC, in the game for a change the game runs, as Testing says.
  • mise run test passes (any failure that also fails on main is named).
  • Measured against main, or Measurements says why there is nothing to measure.
  • No game files, build output or hand-edited generated files; third_party/ untouched; nothing GPL-incompatible.

When it applies

  • A fix: a game-free test that fails on main and passes here.
  • AI assistance recorded with Assisted-by: / Co-Authored-By: trailers.

Ready to merge

  • Everything in Still to test has run, its results in Tested.
  • Tier 1 hardware it can affect is tested (docs/dev/standards.md, "Hardware tiers").
  • Approved in review, every comment answered.
  • CI passes on the head; on current main, measured again or a comment says why the numbers still hold.
## Summary Three game-free renderer checks fail on `main` before they run a single case: each cuts renderer functions out of the source by their text, and the source moved on under them. #759 and #1995 change these checks (#1995 adds cases to the uniform one), and their changes cannot run either. This brings each back to running its cases against the code as it is on `main` @ fd7dc40d, one commit per check; nothing outside `tests/` changes. - `tests/test_fsr_image_retirement.py` (FSR images kept until the GPU is past their last use): `submit_recording` ends each recording with `host_read_barrier(r.cmd)` since 905e36a9 (#1031), which the check did not cut, so its fixture stopped at "use of undeclared identifier 'host_read_barrier'". The check now cuts `host_read_barrier` with the rest of the submission code, and the fixture gives it `vkCmdPipelineBarrier2` (no GPU). No case or assertion changes. - `tests/test_fsr_uniform_ownership.py` (FSR constants stay owned by their recording): the recording code it cuts gained calls its fixture did not provide: `recording_wait`'s `FpsRtWait` (c64e864f, #928), `begin_recording`'s `texture_complete_writes` (7daafa7a, #1553), `draw_begin_recording`'s `pool_shift_first`, `make_pool(shift)` and `pools_grown` (382a3d67, #1158) and `mesh_conv_begin_recording` (9027e737, #1846): ten compile errors. The check now cuts `pool_shift_first` too; the fixture's `make_pool` takes the shift, `pools_grown` is counted, and the FPS counter's timer, the texture writes' completion and the mesh draws' conversion stand empty (none of them here). No case or assertion changes. - `tests/test_draw_stream_visibility.py` (the draw streams' bytes are flushed before the GPU reads them): it cut the parallel replay's segment submit out of `vk_replay.cpp` (from `if (j->inits < 0) init_lost = true;` to the device-lost check), and a3565ac5 (#1763) deleted the segment workers and that loop with them: "substring not found", before anything compiled. **Dropped: the cases `segments` and `failed-segment`**, which ran that loop: the code they tested is gone in a3565ac5 (no segment worker records a slot, no segment submit orders them, no init command buffer whose failure stops the submits after it). Every recording is now submitted by `submit_recording`, which the seven other cases run; the fixture's harness for the loop (`Job`, `submit_segments`, the init command buffer, the submitted-slot bookkeeping) goes with the two cases. The rest builds against `main`: `host_read_barrier` and `pool_shift_first` are cut from the source, `make_pool`, `pools_grown` and `mesh_conv_begin_recording` as in the uniform check. The other cases and their assertions are unchanged. How to see it on `main`: with the Vulkan headers found (`RX_CPM` pointing at the CPM cache, or `LONGSHOT_VULKAN_HEADERS`), `python -I -B tests/run_all.py --strict --exact tests/test_fsr_image_retirement.py` (and the same for the other two) fails with the errors above. Without the headers, `--strict` fails each with "not found: vulkan_headers" and nothing is compiled. Read first: the three `tests/test_*.py` drivers and their fixtures (`tests/fsr_image_retirement_test.cpp`, `tests/fsr_uniform_ownership_test.cpp`, `tests/draw_stream_visibility_test.cpp`). ## Testing **Tested:** Windows 11 (10.0.26300), llvm-mingw clang 23.1.5, the Vulkan headers 1.4.345 from the CPM cache (`RX_CPM`); game-free checks only: nothing here runs in the game, so there was nothing to run in it. Each check ran alone (`python -I -B tests/run_all.py --strict --exact tests/<check>.py`), in a hidden console of its own, in a build job of this PC's job queue (fb3-fix-3), on `main` @ e6196f0f and on this branch @ 7354e49d: - On `main` each fails as the Summary says: the uniform check with its ten compile errors, the retirement check with `host_read_barrier` undeclared, the draw check with "substring not found" before compiling. - Here each passes: the retirement check's 14 cases for both backends, the uniform check's 11 cases with a coherent and a non-coherent cache for both backends, the draw check's 7 cases with both caches. - Each fails again when the code it checks is broken, in a scratch copy of this branch with one function it cuts changed (not committed): - `vk_fsr3.cpp` `free_dead` freeing a retired image one timeline value early (`now + 1 >= ...`): the retirement check fails at `resumed`, "view destroyed while GPU use is pending". - `vk_draw.cpp` `uniform_upload` handing back the stream's start instead of its slice (`cur->mapped` for `cur->mapped + at`): the uniform check fails at `queued`, "GPU must see this evaluation's authored constants". - `vk_draw.cpp` `draw_flush_recording` flushing only the stream in use, not the ones the recording outgrew: the draw check fails at `grow` with the non-coherent cache, "GPU draw input must see the CPU-authored bytes". - `python -I -B tests/run_all.py --strict --changed origin/main` (the three checks read the changed files, and the byte-compile of the tracked Python): 4 passed. The same results came earlier on `main` @ 40bfceb0 (fb3-fix-1 on `main`; fb3-fix-2 on these commits there, @ 30589ebe, with the same scratch breaks). Then rebased onto `main` @ fd7dc40d without a conflict, the three commits unchanged (`git range-diff`), now 98f1ad99; the checks have not run again on that base: its three new commits (#2007, #2017, #1657) change none of the text these checks cut, and the C++ each check writes is byte-for-byte the same as at 7354e49d (written out without compiling, and compared). Not run: the whole `mise run test`. **Still to test:** - Linux: `python3 -I -B tests/run_all.py --strict --exact` with each of the three checks, expecting each to pass, as on Windows (by hand, or CI's run) ## Measurements Nothing to measure: only three game-free checks under `tests/` change; nothing the game runs. ## Risks Nothing a player notices: only `tests/` changes, and nothing the game runs. The three checks now run their cases where CI runs them; if the renderer reshapes the functions they cut again, they stop with `tests/srccut.py`'s message naming the marker, or a compile error. A revert gives back `main`'s three failing checks. ## Checklist ### Ready for review - [x] One topic, on current `main`; the title is a Conventional Commit. - [x] Tested on my own PC, in the game for a change the game runs, as Testing says. - [ ] `mise run test` passes (any failure that also fails on `main` is named). - [x] Measured against `main`, or Measurements says why there is nothing to measure. - [x] No game files, build output or hand-edited generated files; `third_party/` untouched; nothing GPL-incompatible. ### When it applies - [x] A fix: a game-free test that fails on `main` and passes here. - [x] AI assistance recorded with `Assisted-by:` / `Co-Authored-By:` trailers. ### Ready to merge - [ ] Everything in Still to test has run, its results in Tested. - [ ] Tier 1 hardware it can affect is tested (docs/dev/standards.md, "Hardware tiers"). - [ ] Approved in review, every comment answered. - [ ] CI passes on the head; on current `main`, measured again or a comment says why the numbers still hold.
tests/test_fsr_image_retirement.py has not built since 905e36a9 (#1031): submit_recording ends each recording with
host_read_barrier(r.cmd), which the check did not cut, so its fixture stopped at "use of undeclared identifier
'host_read_barrier'" and none of its 14 cases ran for either backend. The check now cuts host_read_barrier out of
vk_replay.cpp with the rest of the submission code, and the fixture gives it vkCmdPipelineBarrier2 (no GPU here). No
case or assertion changes.

Assisted-by: Claude Code
tests/test_fsr_uniform_ownership.py cuts draw_begin_recording, recording_wait and begin_recording out of the renderer,
and its fixture stopped compiling as they gained calls it did not provide: recording_wait's FpsRtWait (c64e864f,
#928), begin_recording's texture_complete_writes (7daafa7a, #1553), draw_begin_recording's pool_shift_first,
make_pool(shift) and pools_grown (382a3d67, #1158) and mesh_conv_begin_recording (9027e737, #1846). None of its 11
cases ran (each with a coherent and a non-coherent cache, for both backends). The check now cuts pool_shift_first out
of vk_draw.cpp too; the fixture's make_pool takes the shift, pools_grown is counted with the other stream counters,
and the FPS counter's render-lane timer, the texture writes' completion and the mesh draws' conversion stand empty (no
FPS counter, textures or mesh draws here). No case or assertion changes.

Assisted-by: Claude Code
fix(tests): the draw stream visibility check runs again; its two segment cases are dropped with the segment workers
Some checks are pending
CI / Conventional Commit title (pull_request) Blocked by required conditions
CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Blocked by required conditions
CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Blocked by required conditions
CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Blocked by required conditions
CI / Workflow lint (actionlint) (pull_request) Blocked by required conditions
CI / Website build and checks (pull_request) Blocked by required conditions
Guide screenshots / Guide images and local UI source review (pull_request) Blocked by required conditions
Third-party parts / As their notes say (pull_request) Blocked by required conditions
Third-party parts / The same as upstream (pull_request) Blocked by required conditions
PR description / Description follows the template (pull_request_target) Successful in 18s
Website / Website on Cloudflare (pull_request_target) Has been skipped
98f1ad9944
tests/test_draw_stream_visibility.py stopped before compiling anything: it cut the parallel replay's segment submit
out of vk_replay.cpp by its text (from "if (j->inits < 0) init_lost = true;" to the device-lost check), and
a3565ac5 (#1763) deleted the segment workers and that loop with them ("substring not found").

Dropped: the cases segments and failed-segment, which ran that loop. The code they tested is gone in a3565ac5: there
is no segment worker left to record a slot, no segment submit to order, and no init command buffer whose failure stops
the submits after it. Every recording is now submitted by submit_recording, which the seven other cases run. The
fixture's harness for that loop (Job, submit_segments, the init command buffer, the submitted-slot bookkeeping) goes
with them.

The rest builds against main again: submit_recording's host_read_barrier (905e36a9, #1031) and draw_begin_recording's
pool_shift_first (382a3d67, #1158) are cut from the source too; the fixture's make_pool takes the shift, pools_grown
is counted, vkCmdPipelineBarrier2 and mesh_conv_begin_recording (9027e737, #1846) stand empty (no GPU or mesh draws
here). The other cases and their assertions are unchanged.

Assisted-by: Claude Code
Member

Possibly related, for context:

  • PR #2031 WIP: perf(vk): a program load no draw asked for is published when the game frees its programs (RX_VK_LOAD_RETIRE=0 to disable) — Calls out the same three test fixtures as failing on main; no code overlap.
  • PR #2030 WIP: perf(vk): the stopped queues' watch sleeps until a queue stops or the CPU signals (RX_VK_SYNC_WATCH_EVENT=0 to disable) — Calls out the same three test fixtures as failing on main; no code overlap.
Possibly related, for context: - PR #2031 WIP: perf(vk): a program load no draw asked for is published when the game frees its programs (RX_VK_LOAD_RETIRE=0 to disable) — Calls out the same three test fixtures as failing on main; no code overlap. - PR #2030 WIP: perf(vk): the stopped queues' watch sleeps until a queue stops or the CPU signals (RX_VK_SYNC_WATCH_EVENT=0 to disable) — Calls out the same three test fixtures as failing on main; no code overlap.
Some checks are pending
CI / Conventional Commit title (pull_request) Blocked by required conditions
CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Blocked by required conditions
CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Blocked by required conditions
CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Blocked by required conditions
CI / Workflow lint (actionlint) (pull_request) Blocked by required conditions
CI / Website build and checks (pull_request) Blocked by required conditions
Guide screenshots / Guide images and local UI source review (pull_request) Blocked by required conditions
Third-party parts / As their notes say (pull_request) Blocked by required conditions
Third-party parts / The same as upstream (pull_request) Blocked by required conditions
PR description / Description follows the template (pull_request_target) Successful in 18s
Website / Website on Cloudflare (pull_request_target) Has been skipped
This pull request is marked as a work in progress.
Some workflows are waiting to be reviewed.
This branch is out-of-date with the base branch
View command line instructions

Checkout

From your project repository, check out a new branch and test the changes.
git fetch -u 'fix/tests-renderer-fixtures-build:siantm-fix/tests-renderer-fixtures-build'
git switch 'siantm-fix/tests-renderer-fixtures-build'
Sign in to join this conversation.
No reviewers
No milestone
No project
No assignees
2 participants
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
MagicBeanDev/Longshot!2037
No description provided.