WIP: perf(vk): the submit's collection leaves the capture's ranges to the captures (RX_VK_COLLECT_ONCE=0 to disable) #2027

Draft
siantm wants to merge 1 commit from siantm/Longshot:perf/vk-collect-once into main
Contributor

Summary

A submit's capture reads and clears the write marks of its ranges (the mirrored pools, the texture storage, the
pools up to 64 MB), and the collection of written pages that follows read every GPU-visible range again, those
included, only to put back the marks it found there for the next capture.

With this, memory_collect_written asks the host for the marks outside those ranges (collect_written_in with a count
below 0, host_collect_outside in host/rx_backend.c; the smallest int asks whether the host knows that form), and the
marks inside stay as they are, which is what putting them back came to. The pools with a write-back shadow are still
scanned (collect_skip_ranges, rebuilt when a shadow is made or freed), so the shadows are told of the same pages as
before. The full scan stays before the first capture, with RX_VK_CAPTURE=0 and with a host without the outside form.
A log line every 600 collections says what was found and how much was left to the captures. The result expected: one
scan of the capture's ranges a submit instead of two.

The switch: RX_VK_COLLECT_ONCE (new, a kill switch, bool, its row in gpu/vk/switches.def), on by default as the
branch shipped it, read once; =0 gives back main's code: every range scanned and its marks put back.

From d1e33769 on exp/x86-asm-sync (there it was off with the render thread; here it is not: the ranges left out are
the ones the full scan's put-back reads, under the same collect_mu, whichever thread collects).

Read first: memory_collect_written and collect_skip_ranges in gpu/vk/vk_memory.cpp, host_collect_outside in
host/rx_backend.c, then tests/test_collect_outside.py.

Testing

Tested: Windows 11, i7-9700, RTX 4060, the builder's llvm-mingw clang 23; head 48e5a97e on main 75317e1f. Build jobs of this PC's job queue (batch CapCut3-b3), the checks in a hidden console: tests/run_all.py --strict --exact on tests/test_collect_outside.py and every check --changed picks for the files this changes: 231 checks, 198 passed, 33 failed, 0 skipped; its own check passes, the switch's read included (fresh processes); the failing ones fail on their base too, run from a base worktree in the same batch: tests/test_capture_read_once.py (fails on main @ 75317e1f too), tests/test_capture_readback_visibility.py (fails on main @ 75317e1f too), tests/test_crash_dialog.py (fails on main @ 75317e1f too), tests/test_crash_record.py (fails on main @ 75317e1f too), tests/test_dev_ab.py (fails on main @ 75317e1f too), tests/test_dev_build.py (fails on main @ 75317e1f too), tests/test_dev_packs.py (fails on main @ 75317e1f too), tests/test_device_lost_exit.py (fails on main @ 75317e1f too), tests/test_draw_stream_visibility.py (fails on main @ 75317e1f too), tests/test_flush_nothing_held.py (fails on main @ 75317e1f too), tests/test_frame_overlap.py (fails on main @ 75317e1f too), tests/test_fsr_image_retirement.py (fails on main @ 75317e1f too), tests/test_fsr_uniform_ownership.py (fails on main @ 75317e1f too), tests/test_gpu_wait_stats.py (fails on main @ 75317e1f too), tests/test_guest_imported.py (fails on main @ 75317e1f too), tests/test_host_guest_writes.py (fails on main @ 75317e1f too), tests/test_linux_pressure.py (fails on main @ 75317e1f too), tests/test_mesh_conv.py (fails on main @ 75317e1f too), tests/test_pending_guest_import.py (fails on main @ 75317e1f too), tests/test_perf_report.py (fails on main @ 75317e1f too), tests/test_present_queue_default.py (fails on main @ 75317e1f too), tests/test_present_source_crop.py (fails on main @ 75317e1f too), tests/test_profiling_mode.py (fails on main @ 75317e1f too), tests/test_recorder_own_idle.py (fails on main @ 75317e1f too), tests/test_renderer_new_handler.py (fails on main @ 75317e1f too), tests/test_replay_write_order.py (fails on main @ 75317e1f too), tests/test_slow_frames.py (fails on main @ 75317e1f too), tests/test_sparse_mirror.py (fails on main @ 75317e1f too), tests/test_target_reuse_memory.py (fails on main @ 75317e1f too), tests/test_texel_buffer_ownership.py (fails on main @ 75317e1f too), tests/test_ub_update_batch.py (fails on main @ 75317e1f too), tests/test_upscaler_rows.py (fails on main @ 75317e1f too), tests/test_vk_occlusion_queries.py (fails on main @ 75317e1f too).

It has NOT run in the game on this PC yet.

Still to test:

  • Windows 11, i7-9700, RTX 4060 (this PC): Kokiri Forest, play three minutes turning the camera and walking into Link's house and out: main @ 75317e1f, this branch with RX_VK_COLLECT_ONCE=0 and this branch with it unset (on), each warmed up a minute in the scene first, interleaved (ABBA), at least two runs each, expecting the same picture (no stale or flickering texture, no missing draw), no FATAL, and a median frame time unset lower than with RX_VK_COLLECT_ONCE=0 and than main, and RX_VK_COLLECT_ONCE=0 the same as main (by hand)
  • Linux NVIDIA and AMD, Windows AMD and Intel, Steam Deck: Kokiri Forest, play three minutes: main @ 75317e1f, this branch with RX_VK_COLLECT_ONCE=0 and this branch with it unset (on), warmed up, ABBA, expecting the same picture, no stale texture and no FATAL, and a median frame time unset lower than with RX_VK_COLLECT_ONCE=0 and than main, and RX_VK_COLLECT_ONCE=0 the same as main (@Beanstalk linux+nvidia, linux+amd, windows+amd, windows+intel, steamdeck: Kokiri Forest three minutes: main @ 75317e1f, this branch with RX_VK_COLLECT_ONCE=0 and this branch with it unset (on), warmed up, ABBA, the log, frame times and a screenshot)
  • Linux: build the host and the renderer and run python3 -I -B tests/test_collect_outside.py, expecting PASS (by hand)

Measurements

Verdict: not measured: no micro-benchmark of this port and no run in the game on this PC yet; the in-game A/B is
in Still to test. RX_VK_COLLECT_ONCE ships on, as the branch shipped it; no measurement chose that default yet, the in-game A/B in Still to test does.

Risks

What could break: a page written by the CPU and left out of both the capture and the collection would reach the GPU a
submit late: a stale or flickering texture, a wrong value read back. The test cuts host_collect_in and
host_collect_outside from host/rx_backend.c and checks against a plain walk that only the whole pages of the given
ranges are left out; the write-back shadows' checks run collect_skip_ranges and check the shadowed pools stay
scanned. Turn it off with RX_VK_COLLECT_ONCE=0. Host and renderer code: the runtime is rebuilt; nothing changes for
players' saves, options or caches.

Checklist

Ready for review

  • One topic, on current main; the title is a Conventional Commit.
  • Tested on my own PC, in the game for a change the game runs, as Testing says.
  • mise run test passes (any failure that also fails on main is named).
  • Measured against main, or Measurements says why there is nothing to measure.
  • No game files, build output or hand-edited generated files; third_party/ untouched; nothing GPL-incompatible.

When it applies

  • A switch added or changed: it ships at the default the measurements chose, with its row in gpu/vk/switches.def.
  • A check under tests/ that reads changed source is updated, not weakened.
  • Host code builds on Windows (llvm-mingw) and Linux, with UTF-8 paths.
  • AI assistance recorded with Assisted-by: / Co-Authored-By: trailers.

Ready to merge

  • Everything in Still to test has run, its results in Tested.
  • Tier 1 hardware it can affect is tested (docs/dev/standards.md, "Hardware tiers").
  • Approved in review, every comment answered.
  • CI passes on the head; on current main, measured again or a comment says why the numbers still hold.
## Summary A submit's capture reads and clears the write marks of its ranges (the mirrored pools, the texture storage, the pools up to 64 MB), and the collection of written pages that follows read every GPU-visible range again, those included, only to put back the marks it found there for the next capture. With this, `memory_collect_written` asks the host for the marks outside those ranges (`collect_written_in` with a count below 0, `host_collect_outside` in `host/rx_backend.c`; the smallest int asks whether the host knows that form), and the marks inside stay as they are, which is what putting them back came to. The pools with a write-back shadow are still scanned (`collect_skip_ranges`, rebuilt when a shadow is made or freed), so the shadows are told of the same pages as before. The full scan stays before the first capture, with `RX_VK_CAPTURE=0` and with a host without the outside form. A log line every 600 collections says what was found and how much was left to the captures. The result expected: one scan of the capture's ranges a submit instead of two. The switch: `RX_VK_COLLECT_ONCE` (new, a kill switch, bool, its row in `gpu/vk/switches.def`), on by default as the branch shipped it, read once; `=0` gives back main's code: every range scanned and its marks put back. From d1e33769 on `exp/x86-asm-sync` (there it was off with the render thread; here it is not: the ranges left out are the ones the full scan's put-back reads, under the same `collect_mu`, whichever thread collects). Read first: `memory_collect_written` and `collect_skip_ranges` in `gpu/vk/vk_memory.cpp`, `host_collect_outside` in `host/rx_backend.c`, then `tests/test_collect_outside.py`. ## Testing **Tested:** Windows 11, i7-9700, RTX 4060, the builder's llvm-mingw clang 23; head `48e5a97e` on main `75317e1f`. Build jobs of this PC's job queue (batch `CapCut3-b3`), the checks in a hidden console: `tests/run_all.py --strict --exact` on `tests/test_collect_outside.py` and every check `--changed` picks for the files this changes: 231 checks, 198 passed, 33 failed, 0 skipped; its own check passes, the switch's read included (fresh processes); the failing ones fail on their base too, run from a base worktree in the same batch: `tests/test_capture_read_once.py` (fails on main @ 75317e1f too), `tests/test_capture_readback_visibility.py` (fails on main @ 75317e1f too), `tests/test_crash_dialog.py` (fails on main @ 75317e1f too), `tests/test_crash_record.py` (fails on main @ 75317e1f too), `tests/test_dev_ab.py` (fails on main @ 75317e1f too), `tests/test_dev_build.py` (fails on main @ 75317e1f too), `tests/test_dev_packs.py` (fails on main @ 75317e1f too), `tests/test_device_lost_exit.py` (fails on main @ 75317e1f too), `tests/test_draw_stream_visibility.py` (fails on main @ 75317e1f too), `tests/test_flush_nothing_held.py` (fails on main @ 75317e1f too), `tests/test_frame_overlap.py` (fails on main @ 75317e1f too), `tests/test_fsr_image_retirement.py` (fails on main @ 75317e1f too), `tests/test_fsr_uniform_ownership.py` (fails on main @ 75317e1f too), `tests/test_gpu_wait_stats.py` (fails on main @ 75317e1f too), `tests/test_guest_imported.py` (fails on main @ 75317e1f too), `tests/test_host_guest_writes.py` (fails on main @ 75317e1f too), `tests/test_linux_pressure.py` (fails on main @ 75317e1f too), `tests/test_mesh_conv.py` (fails on main @ 75317e1f too), `tests/test_pending_guest_import.py` (fails on main @ 75317e1f too), `tests/test_perf_report.py` (fails on main @ 75317e1f too), `tests/test_present_queue_default.py` (fails on main @ 75317e1f too), `tests/test_present_source_crop.py` (fails on main @ 75317e1f too), `tests/test_profiling_mode.py` (fails on main @ 75317e1f too), `tests/test_recorder_own_idle.py` (fails on main @ 75317e1f too), `tests/test_renderer_new_handler.py` (fails on main @ 75317e1f too), `tests/test_replay_write_order.py` (fails on main @ 75317e1f too), `tests/test_slow_frames.py` (fails on main @ 75317e1f too), `tests/test_sparse_mirror.py` (fails on main @ 75317e1f too), `tests/test_target_reuse_memory.py` (fails on main @ 75317e1f too), `tests/test_texel_buffer_ownership.py` (fails on main @ 75317e1f too), `tests/test_ub_update_batch.py` (fails on main @ 75317e1f too), `tests/test_upscaler_rows.py` (fails on main @ 75317e1f too), `tests/test_vk_occlusion_queries.py` (fails on main @ 75317e1f too). **It has NOT run in the game on this PC yet.** **Still to test:** - Windows 11, i7-9700, RTX 4060 (this PC): Kokiri Forest, play three minutes turning the camera and walking into Link's house and out: main @ 75317e1f, this branch with `RX_VK_COLLECT_ONCE=0` and this branch with it unset (on), each warmed up a minute in the scene first, interleaved (ABBA), at least two runs each, expecting the same picture (no stale or flickering texture, no missing draw), no FATAL, and a median frame time unset lower than with `RX_VK_COLLECT_ONCE=0` and than main, and `RX_VK_COLLECT_ONCE=0` the same as main (by hand) - Linux NVIDIA and AMD, Windows AMD and Intel, Steam Deck: Kokiri Forest, play three minutes: main @ 75317e1f, this branch with `RX_VK_COLLECT_ONCE=0` and this branch with it unset (on), warmed up, ABBA, expecting the same picture, no stale texture and no FATAL, and a median frame time unset lower than with `RX_VK_COLLECT_ONCE=0` and than main, and `RX_VK_COLLECT_ONCE=0` the same as main (@Beanstalk linux+nvidia, linux+amd, windows+amd, windows+intel, steamdeck: Kokiri Forest three minutes: main @ 75317e1f, this branch with RX_VK_COLLECT_ONCE=0 and this branch with it unset (on), warmed up, ABBA, the log, frame times and a screenshot) - Linux: build the host and the renderer and run `python3 -I -B tests/test_collect_outside.py`, expecting PASS (by hand) ## Measurements **Verdict:** not measured: no micro-benchmark of this port and no run in the game on this PC yet; the in-game A/B is in Still to test. `RX_VK_COLLECT_ONCE` ships on, as the branch shipped it; no measurement chose that default yet, the in-game A/B in Still to test does. ## Risks What could break: a page written by the CPU and left out of both the capture and the collection would reach the GPU a submit late: a stale or flickering texture, a wrong value read back. The test cuts `host_collect_in` and `host_collect_outside` from `host/rx_backend.c` and checks against a plain walk that only the whole pages of the given ranges are left out; the write-back shadows' checks run `collect_skip_ranges` and check the shadowed pools stay scanned. Turn it off with `RX_VK_COLLECT_ONCE=0`. Host and renderer code: the runtime is rebuilt; nothing changes for players' saves, options or caches. ## Checklist ### Ready for review - [x] One topic, on current `main`; the title is a Conventional Commit. - [ ] Tested on my own PC, in the game for a change the game runs, as Testing says. - [ ] `mise run test` passes (any failure that also fails on `main` is named). - [ ] Measured against `main`, or Measurements says why there is nothing to measure. - [x] No game files, build output or hand-edited generated files; `third_party/` untouched; nothing GPL-incompatible. ### When it applies - [ ] A switch added or changed: it ships at the default the measurements chose, with its row in `gpu/vk/switches.def`. - [x] A check under `tests/` that reads changed source is updated, not weakened. - [ ] Host code builds on Windows (llvm-mingw) and Linux, with UTF-8 paths. - [x] AI assistance recorded with `Assisted-by:` / `Co-Authored-By:` trailers. ### Ready to merge - [ ] Everything in Still to test has run, its results in Tested. - [ ] Tier 1 hardware it can affect is tested (docs/dev/standards.md, "Hardware tiers"). - [ ] Approved in review, every comment answered. - [ ] CI passes on the head; on current `main`, measured again or a comment says why the numbers still hold.
perf(vk): the submit's collection leaves the capture's ranges to the captures
Some checks are pending
CI / Conventional Commit title (pull_request) Blocked by required conditions
CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Blocked by required conditions
CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Blocked by required conditions
CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Blocked by required conditions
CI / Workflow lint (actionlint) (pull_request) Blocked by required conditions
CI / Website build and checks (pull_request) Blocked by required conditions
Guide screenshots / Guide images and local UI source review (pull_request) Blocked by required conditions
Third-party parts / As their notes say (pull_request) Blocked by required conditions
Third-party parts / The same as upstream (pull_request) Blocked by required conditions
PR description / Description follows the template (pull_request_target) Successful in 14s
Website / Website on Cloudflare (pull_request_target) Has been skipped
48e5a97e97
- A submit's capture reads and clears the write marks of its ranges (the mirrored pools, the texture storage, the pools
  up to 64 MB), and the collection that follows read every GPU-visible range again, those included, only to put back
  the marks it found there for the next capture. memory_collect_written now asks the host for the marks outside those
  ranges (collect_written_in with a count below 0, host/rx_backend.c host_collect_outside; the smallest int asks
  whether the host knows the form), and their marks stay as they are, which is what putting them back came to. The
  pools with a write-back shadow are still scanned (collect_skip_ranges, rebuilt when a shadow is made or freed:
  shadow_ver), so the shadows are told of the same pages as before. The full scan stays before the first capture,
  with RX_VK_CAPTURE=0 and with a host without the outside form.
- Switch RX_VK_COLLECT_ONCE (kill, on); =0 scans every range as before. A log line every 600 collections says what
  was found and how much was left to the captures.
- Unlike the branch, it is not turned off with the render thread: the branch read RX_VK_ASYNC alone (so a run with the
  Threaded rendering row's render thread had it on there too), and the ranges left out are the ones the full scan's
  put-back reads, capture_ranges_merged under the same collect_mu, whichever thread collects.
- Branch's numbers (d1e33769, profile, i7-9700, Kokiri Forest): collect_in was 5.1% of a core on the two threads the
  frame ran on, the capture's ranges 2.9 GB in ~1,465 ranges; at most the capture's own scan again, ~0.11 ms a submit,
  is saved. Not measured on the branch beyond that, and not measured in play on main.
- Tests: new tests/test_collect_outside.py (host_collect_in and host_collect_outside cut from rx_backend.c against a
  plain walk: the whole pages of the given ranges left out and nothing else, the question answered, the renderer's
  numbers the host's); test_wb_diff_stale_shadow runs collect_skip_ranges; test_wb_diff_cpu_writes checks the
  shadowed pools stay scanned; test_vk_collect_window runs the capture's window with the outside form too;
  test_wb_shadow_remake gets shadow_ver.

Switch reads: the test also runs RX_VK_COLLECT_ONCE as the source reads it, in fresh processes (unset, 0, 1, 2, off, each twice, the variable flipped after the first read: read once). The host's collection outside ranges falls back to the whole collection when its range list cannot be allocated.

Assisted-by: Claude Code
Member

Possibly related, for context:

  • issue #750 Game: Hyrule Town spin costs ~30% fps (Deck LCD) - GPU-written 691x432 texture re-uploaded per present + page-capture storm — Related capture/write-mark overhead and GPU-written texture reuploads on Deck
Possibly related, for context: - issue #750 Game: Hyrule Town spin costs ~30% fps (Deck LCD) - GPU-written 691x432 texture re-uploaded per present + page-capture storm — Related capture/write-mark overhead and GPU-written texture reuploads on Deck
BeanBot left a comment

Recommendation: Request changes. The shortened scan can miss a dirty page when a pool leaves the capture list during collection. It also makes the early-readback diagnostic count partial scans as full coverage, and the performance gain on this head is unproven.

The change aims to reduce CPU work per Vulkan submit by leaving capture-owned write marks for the next capture while still scanning pools with write-back shadows. It adds a host callback protocol, cached range subtraction, a pool lock acquisition, and scratch storage. Players have no demonstrated frame-time gain yet.

@fix will do (13)

  1. L1 gpu/vk/vk_memory.cpp:938: The skip list is built before the host scan, while the put-back decision uses a fresh range snapshot afterward. (more on the line)
  2. L2 gpu/vk/vk_memory.cpp:969: RX_VK_LOG_EARLY_CHECK still advances its full-scan generation for an outside scan. Its classifier closes an interval after the second full collection begun after a snapshot. With two submits already queued before a CPU store to an excluded unshadowed page, replay can finish both partial scans and close the interval before any later capture reports the store. The diagnostic can then attribute the changed word to the GPU. Keep the full scan while that diagnostic is enabled, or make interval closure depend on actual coverage.
  3. L3 host/rx_backend.c:325: The low-memory fallback calls host_collect, which allocates another 64 KiB range buffer and passes it to hos_gpu_guest_ranges unchecked. (more on the line)
  4. L4 host/rx_backend.c:313: This adds a new contract to MkwVkHost while vk_api.h still documents only an inside scan. (more on the line)
  5. L5 (minor) host/rx_backend.c:327: The advertised negative-count protocol accepts counts above the renderer's local guard. (more on the line)
  6. L6 (minor) gpu/vk/vk_memory.cpp:946: The C callback indexes an array of uint64_t[2], but the caller passes std::pair objects through a cast. (more on the line)
  7. L7 (minor) gpu/vk/vk_memory.cpp:985: The 600-submit log combines averaged page counts with the mode, range count and bytes from only the last submit. (more on the line)
  8. L8 tests/test_collect_outside.py:223: The new test ignores CC and uses one driver for both languages. (more on the line)
  9. L9 (minor) tests/test_wb_diff_stale_shadow.py:23: The newly added shadow-exclusion case sits behind a literal g++ runner argument. A Windows llvm-mingw setup with clang++ and no g++ cannot run that coverage even with CXX configured. Use the runner's {cxx} tool and honor CXX for direct invocation. The direct defaults in the other changed C++ fixtures should use the same selection.
  10. L10 (minor) tests/test_vk_collect_window.py:203: The new enabled run checks final marks and stamps, which also pass if the renderer silently uses the full scan and puts page 9 back. (more on the line)
  11. L11 (minor) gpu/vk/switches.def:299: The switch row omits the full-scan fallback before capture coverage exists and when RX_VK_CAPTURE=0. (more on the line)
  12. L12 (its part an edit can do): Cache the excluded byte total with the skip list, avoid the extra lock on unchanged state where safe, and report actual scan coverage alongside existing collection and capture timings.
  13. L13 (minor): The checklist marks the branch as current with main, but the supplied checkout's main is e6196f0f, eight commits beyond 75317e1f. Untick the box until the branch and its measurements are refreshed against current main.

For the author (1) (a run, a measurement or a decision)

  • L12 (minor): The PR opened without an author game run, and the plan measures gameplay only by median frame time in Kokiri Forest. Run the changed code in game on the author's PC and the affected Tier 1 platforms, including pool churn during area changes. Report repeated, warmed, interleaved current-main and head runs with the switch on and off, average FPS, 1% lows, median and p99 frame time, collection time, capture lock waits and RAM. Include a heavy scene and one with little useful exclusion, where the range walk and pool lock can cost more than they save. Put the setup, results and chosen defaults in Measurements.

Checks: Still running: CI / Conventional Commit title (pull_request), CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request), CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request), CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request), CI / Workflow lint (actionlint) (pull_request), CI / Website build and checks (pull_request), Guide screenshots / Guide images and local UI source review (pull_request), PR description / Description follows the template (pull_request_target), Third-party parts / As their notes say (pull_request), Third-party parts / The same as upstream (pull_request).

More: does it help, other systems, switches, what was read

Does it help: Unproven. The commit cites a profile from experimental commit d1e33769 on an i7-9700 in Kokiri Forest: collection used 5.1% of one core, the capture covered about 2.9 GB across 1,465 ranges, and the estimated upper bound was 0.11 ms saved per submit. There are no game timings for 48e5a97e against 75317e1f, no same-head switch A/B, and no frame-time or 1% low results.

Other systems: The Vulkan collection path is shared by Windows and Linux across NVIDIA, AMD, Intel and Steam Deck, including mesh fallback. Threaded replay can encounter the mapping change described below. Shadowed pools stay in the scan for a stable range snapshot, and capture-disabled runs retain the full scan. Early-readback diagnostics depend on full coverage. The shader stamp already excludes vk_api.h. The diff contains no save, audio, input, shader, or build-script changes.

Switches: RX_VK_COLLECT_ONCE is a developer kill switch. It is read once and defaults on with a capable host and eligible capture ranges. Empty, 0, off, false and no disable it. The row is present and sorted. Merge defaults still need measurements on supported hardware, with the correctness fixes in place.

Reviewed: 48e5a97e97 against 75317e1f31: Read AGENTS.md, CONTRIBUTING.md, docs/dev/standards.md, the Forgejo PR template, the commit and the full eight-file diff. Read all five changed test files and the changed production sections with their surrounding code. Traced collection and capture callers, pool mapping and shadow lifetimes, early-readback accounting, host API declarations, allocation helpers and shader stamp inputs. Skimmed the prior vector-walk run and related existing tests. No local review-checklist file, PR comments, or prior reviews were present. Unrelated portions of the large production files, other recorded runs, translations and generated files were outside the review.; nothing run (a code review).

Checked and fine:

  • The commit stays on one topic, uses a Conventional Commit title after the WIP prefix, and carries the Assisted-by trailer.
  • The diff contains source and authored fixtures. No game-derived fixture data, generated shader headers, build outputs, or third-party changes were found.
  • Both production callers serialize memory_collect_written with replay_mu. collect_mu serializes mark consumption against capture scans, and shadow version updates use pools_mu.
  • For sorted, disjoint guest ranges and valid endpoints, the host subtraction emits sorted parts and preserves marks on whole excluded pages. The randomized fixture checks page order and remaining marks.
  • The new code adds no runtime file I/O, external commands, or CPU feature requirement. The test changes compile and run local fixtures.

Testing: Read-only review. The PR reports 198 passes and 33 failures also seen on base out of 231 selected Windows checks, including a pass for the new fixture. Linux builds and game runs are pending. The supplied CI snapshot has no completed head-only failures and still lists checks in progress.

Overlaps and merge order: No overlapping open PR was identified from the supplied review data and local refs. The local main has a separate switch-table edit from merged #1977 with no overlapping hunk. Forgejo reports mergeable false and supplies no conflict details.


Automated code review by BeanBot (gpt-6.2-sol), on the pull request's opening. Nothing was run; a maintainer decides.

**Recommendation: Request changes.** The shortened scan can miss a dirty page when a pool leaves the capture list during collection. It also makes the early-readback diagnostic count partial scans as full coverage, and the performance gain on this head is unproven. The change aims to reduce CPU work per Vulkan submit by leaving capture-owned write marks for the next capture while still scanning pools with write-back shadows. It adds a host callback protocol, cached range subtraction, a pool lock acquisition, and scratch storage. Players have no demonstrated frame-time gain yet. **`@fix` will do (13)** 1. **L1** `gpu/vk/vk_memory.cpp:938`: The skip list is built before the host scan, while the put-back decision uses a fresh range snapshot afterward. (more on the line) 2. **L2** `gpu/vk/vk_memory.cpp:969`: RX_VK_LOG_EARLY_CHECK still advances its full-scan generation for an outside scan. Its classifier closes an interval after the second full collection begun after a snapshot. With two submits already queued before a CPU store to an excluded unshadowed page, replay can finish both partial scans and close the interval before any later capture reports the store. The diagnostic can then attribute the changed word to the GPU. Keep the full scan while that diagnostic is enabled, or make interval closure depend on actual coverage. 3. **L3** `host/rx_backend.c:325`: The low-memory fallback calls host_collect, which allocates another 64 KiB range buffer and passes it to hos_gpu_guest_ranges unchecked. (more on the line) 4. **L4** `host/rx_backend.c:313`: This adds a new contract to MkwVkHost while vk_api.h still documents only an inside scan. (more on the line) 5. **L5** (minor) `host/rx_backend.c:327`: The advertised negative-count protocol accepts counts above the renderer's local guard. (more on the line) 6. **L6** (minor) `gpu/vk/vk_memory.cpp:946`: The C callback indexes an array of uint64_t[2], but the caller passes std::pair objects through a cast. (more on the line) 7. **L7** (minor) `gpu/vk/vk_memory.cpp:985`: The 600-submit log combines averaged page counts with the mode, range count and bytes from only the last submit. (more on the line) 8. **L8** `tests/test_collect_outside.py:223`: The new test ignores CC and uses one driver for both languages. (more on the line) 9. **L9** (minor) `tests/test_wb_diff_stale_shadow.py:23`: The newly added shadow-exclusion case sits behind a literal g++ runner argument. A Windows llvm-mingw setup with clang++ and no g++ cannot run that coverage even with CXX configured. Use the runner's {cxx} tool and honor CXX for direct invocation. The direct defaults in the other changed C++ fixtures should use the same selection. 10. **L10** (minor) `tests/test_vk_collect_window.py:203`: The new enabled run checks final marks and stamps, which also pass if the renderer silently uses the full scan and puts page 9 back. (more on the line) 11. **L11** (minor) `gpu/vk/switches.def:299`: The switch row omits the full-scan fallback before capture coverage exists and when RX_VK_CAPTURE=0. (more on the line) 12. **L12** (its part an edit can do): Cache the excluded byte total with the skip list, avoid the extra lock on unchanged state where safe, and report actual scan coverage alongside existing collection and capture timings. 13. **L13** (minor): The checklist marks the branch as current with main, but the supplied checkout's main is e6196f0f, eight commits beyond 75317e1f. Untick the box until the branch and its measurements are refreshed against current main. **For the author (1)** (a run, a measurement or a decision) - **L12** (minor): The PR opened without an author game run, and the plan measures gameplay only by median frame time in Kokiri Forest. Run the changed code in game on the author's PC and the affected Tier 1 platforms, including pool churn during area changes. Report repeated, warmed, interleaved current-main and head runs with the switch on and off, average FPS, 1% lows, median and p99 frame time, collection time, capture lock waits and RAM. Include a heavy scene and one with little useful exclusion, where the range walk and pool lock can cost more than they save. Put the setup, results and chosen defaults in Measurements. **Checks:** Still running: `CI / Conventional Commit title (pull_request)`, `CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request)`, `CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request)`, `CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request)`, `CI / Workflow lint (actionlint) (pull_request)`, `CI / Website build and checks (pull_request)`, `Guide screenshots / Guide images and local UI source review (pull_request)`, `PR description / Description follows the template (pull_request_target)`, `Third-party parts / As their notes say (pull_request)`, `Third-party parts / The same as upstream (pull_request)`. <details> <summary>More: does it help, other systems, switches, what was read</summary> **Does it help:** Unproven. The commit cites a profile from experimental commit d1e33769 on an i7-9700 in Kokiri Forest: collection used 5.1% of one core, the capture covered about 2.9 GB across 1,465 ranges, and the estimated upper bound was 0.11 ms saved per submit. There are no game timings for 48e5a97e against 75317e1f, no same-head switch A/B, and no frame-time or 1% low results. **Other systems:** The Vulkan collection path is shared by Windows and Linux across NVIDIA, AMD, Intel and Steam Deck, including mesh fallback. Threaded replay can encounter the mapping change described below. Shadowed pools stay in the scan for a stable range snapshot, and capture-disabled runs retain the full scan. Early-readback diagnostics depend on full coverage. The shader stamp already excludes vk_api.h. The diff contains no save, audio, input, shader, or build-script changes. **Switches:** RX_VK_COLLECT_ONCE is a developer kill switch. It is read once and defaults on with a capable host and eligible capture ranges. Empty, 0, off, false and no disable it. The row is present and sorted. Merge defaults still need measurements on supported hardware, with the correctness fixes in place. **Reviewed:** 48e5a97e97e5 against 75317e1f31ff: Read AGENTS.md, CONTRIBUTING.md, docs/dev/standards.md, the Forgejo PR template, the commit and the full eight-file diff. Read all five changed test files and the changed production sections with their surrounding code. Traced collection and capture callers, pool mapping and shadow lifetimes, early-readback accounting, host API declarations, allocation helpers and shader stamp inputs. Skimmed the prior vector-walk run and related existing tests. No local review-checklist file, PR comments, or prior reviews were present. Unrelated portions of the large production files, other recorded runs, translations and generated files were outside the review.; nothing run (a code review). **Checked and fine:** - The commit stays on one topic, uses a Conventional Commit title after the WIP prefix, and carries the Assisted-by trailer. - The diff contains source and authored fixtures. No game-derived fixture data, generated shader headers, build outputs, or third-party changes were found. - Both production callers serialize memory_collect_written with replay_mu. collect_mu serializes mark consumption against capture scans, and shadow version updates use pools_mu. - For sorted, disjoint guest ranges and valid endpoints, the host subtraction emits sorted parts and preserves marks on whole excluded pages. The randomized fixture checks page order and remaining marks. - The new code adds no runtime file I/O, external commands, or CPU feature requirement. The test changes compile and run local fixtures. **Testing:** Read-only review. The PR reports 198 passes and 33 failures also seen on base out of 231 selected Windows checks, including a pass for the new fixture. Linux builds and game runs are pending. The supplied CI snapshot has no completed head-only failures and still lists checks in progress. **Overlaps and merge order:** No overlapping open PR was identified from the supplied review data and local refs. The local main has a separate switch-table edit from merged #1977 with no overlapping hunk. Forgejo reports mergeable false and supplies no conflict details. </details> --- <sub>Automated code review by BeanBot (gpt-6.2-sol), on the pull request's opening. Nothing was run; a maintainer decides.</sub>
@ -296,6 +296,7 @@ SW(RX_VK_CASCADE_BIAS, tune, string, "0,0: nothing added", "Adds a depth bias <u
SW(RX_VK_CHECKPOINTS, kill, enum, "on: NVIDIA diagnostic checkpoints before each draw where the driver has the extension", "0 turns off draw checkpoints; ext enables the extension only; 1 also uses AMD buffer markers where NV's are missing.")
SW(RX_VK_CHECK_BANKS, diag, bool, "off: no check", "Compares each constant-bank fill against the per-word path and logs up to 20 mismatches.")
SW(RX_VK_CLEAR_CPU_MAX, tune, int, "256 bytes", "Largest buffer clear also applied on the CPU in stream order; 0 leaves every clear to the GPU.")
SW(RX_VK_COLLECT_ONCE, kill, bool, "on (unset): the submit's collection leaves the capture's ranges to the captures and scans the rest and the pools with a write-back shadow (not with a host without the collection outside ranges)", "0, off, false or no: the submit's collection scans every GPU-visible range, the capture's ranges included, as before. Read once, at the first collection (the log says which).")
Member

L11 (minor): The switch row omits the full-scan fallback before capture coverage exists and when RX_VK_CAPTURE=0. It also omits the empty value from the off values. Document both so an operator can tell when an unset switch will actually shorten a scan.

**L11** (minor): The switch row omits the full-scan fallback before capture coverage exists and when RX_VK_CAPTURE=0. It also omits the empty value from the off values. Document both so an operator can tell when an unset switch will actually shorten a scan.
@ -861,0 +935,4 @@
once = collect_once_on();
if (once) { // (only what the captures do not own: see above)
uint64_t owned_ver = 0;
const auto& ranges = capture_ranges_merged(&owned_ver);
Member

L1: The skip list is built before the host scan, while the put-back decision uses a fresh range snapshot afterward. The game can replace a small capture-eligible pool with a larger GPU-visible pool at the same address between those snapshots. A dirty page in that span is skipped, and the next capture also excludes it. Main's full scan can stamp that page for the current submit, so a texture or buffer can stay stale until a later collection. Revalidate capture ownership before completing collection and scan any ranges that lost ownership, or synchronize the host scan and classification against a stable mapping snapshot. Cover removal during the scan in the window fixture.

**L1**: The skip list is built before the host scan, while the put-back decision uses a fresh range snapshot afterward. The game can replace a small capture-eligible pool with a larger GPU-visible pool at the same address between those snapshots. A dirty page in that span is skipped, and the next capture also excludes it. Main's full scan can stamp that page for the current submit, so a texture or buffer can stay stale until a later collection. Revalidate capture ownership before completing collection and scan any ranges that lost ownership, or synchronize the host scan and classification against a stable mapping snapshot. Cover removal during the scan in the window fixture.
@ -861,0 +943,4 @@
if (once) {
left_ranges = skip.size();
for (const auto& r : skip) left_bytes += r.second - r.first;
n = host->collect_written_in(reinterpret_cast<const uint64_t(*)[2]>(skip.data()), -(int)skip.size() - 1, &pages);
Member

L6 (minor): The C callback indexes an array of uint64_t[2], but the caller passes std::pair objects through a cast. C++ gives no array layout or aliasing guarantee for those pairs, so a build with a different pair layout could feed the host incorrect endpoints. Store the exclusion list in the actual wire type and share that type through the API.

**L6** (minor): The C callback indexes an array of uint64_t[2], but the caller passes std::pair objects through a cast. C++ gives no array layout or aliasing guarantee for those pairs, so a build with a different pair layout could feed the host incorrect endpoints. Store the exclusion list in the actual wire type and share that type through the API.
@ -887,0 +982,4 @@
calls++, found += n > 0 ? n : 0, back += put_back;
if (calls % 600 == 0) {
logf("vk: writes collected (600 submits): %.1f pages a submit, %.1f of them in the captures' ranges (marks put back for "
"the next capture), %s", found / 600.0, back / 600.0, once ? "the captures' ranges not scanned again:" : "every range scanned");
Member

L7 (minor): The 600-submit log combines averaged page counts with the mode, range count and bytes from only the last submit. It also claims ranges were skipped when the host fell back to a full scan after realloc failed. Report actual completed full and outside scans for the window, with actual excluded bytes or an explicit fallback count. The current byte tally also walks every exclusion on every submit and can be computed when the list is rebuilt.

**L7** (minor): The 600-submit log combines averaged page counts with the mode, range count and bytes from only the last submit. It also claims ranges were skipped when the host fell back to a full scan after realloc failed. Report actual completed full and outside scans for the window, with actual excluded bytes or an explicit fallback count. The current byte tally also walks every exclusion on every submit and can be computed when the list is rebuilt.
@ -307,0 +310,4 @@
rest; the renderer names the ones it leaves to its captures, whose marks stay as they are. skip is sorted and its
ranges are apart, as the renderer's list is; only the whole pages inside one are left out, and a list that is not in
order leaves less out, never more: a page is left out only where it lies in one of the ranges given.
It is reached through collect_written_in's count, with no callback of its own (MkwVkHost is in gpu/vk/vk_api.h, which
Member

L4: This adds a new contract to MkwVkHost while vk_api.h still documents only an inside scan. The probe assumes any accepted old callback safely handles an out-of-contract negative count and null ranges. The stated reason for avoiding a versioned callback is stale because nx2/translator_files.py explicitly excludes vk_api.h from the shader stamp. Declare the outside operation in the shared API, bump MKWVK_API_VERSION per the architecture rules, and gate the callback on host version and availability.

**L4**: This adds a new contract to MkwVkHost while vk_api.h still documents only an inside scan. The probe assumes any accepted old callback safely handles an out-of-contract negative count and null ranges. The stated reason for avoiding a versioned callback is stale because nx2/translator_files.py explicitly excludes vk_api.h from the shader stamp. Declare the outside operation in the shared API, bump MKWVK_API_VERSION per the architecture rules, and gate the callback on host version and availability.
@ -307,0 +322,4 @@
static uint64_t (*ranges)[2], (*parts)[2];
static int parts_cap;
if (!ranges) ranges = malloc(sizeof *ranges * 4096);
if (!ranges) return host_collect(pages); /* (no memory for the list: the whole collection, as before) */
Member

L3: The low-memory fallback calls host_collect, which allocates another 64 KiB range buffer and passes it to hos_gpu_guest_ranges unchecked. If the first collection starts with captures enabled and both allocations fail at the commit limit, the host copies the mapping list through a null pointer. Use one shared mapping buffer for both paths or the checked host allocator. Exercise both mapping-buffer failure and parts-growth failure with a fixture whose full-scan fallback actually collects the marks.

**L3**: The low-memory fallback calls host_collect, which allocates another 64 KiB range buffer and passes it to hos_gpu_guest_ranges unchecked. If the first collection starts with captures enabled and both allocations fail at the commit limit, the host copies the mapping list through a null pointer. Use one shared mapping buffer for both paths or the checked host allocator. Exercise both mapping-buffer failure and parts-growth failure with a fixture whose full-scan fallback actually collects the marks.
@ -307,0 +324,4 @@
if (!ranges) ranges = malloc(sizeof *ranges * 4096);
if (!ranges) return host_collect(pages); /* (no memory for the list: the whole collection, as before) */
const int n = hos_gpu_guest_ranges(ranges, 4096);
if (parts_cap < n + nskip) { /* (a range's rest, and at most one part before each range left out) */
Member

L5 (minor): The advertised negative-count protocol accepts counts above the renderer's local guard. For example, INT_MIN + 1 decodes to INT_MAX - 1, so adding even two visible ranges overflows int before the allocation's size_t casts. Validate counts at the callback boundary and check the capacity, slack, and multiplication in size_t before allocating, or put a shared enforced limit in the new API.

**L5** (minor): The advertised negative-count protocol accepts counts above the renderer's local guard. For example, INT_MIN + 1 decodes to INT_MAX - 1, so adding even two visible ranges overflows int before the allocation's size_t casts. Validate counts at the callback boundary and check the capacity, slack, and multiplication in size_t before allocating, or put a shared enforced limit in the new API.
@ -0,0 +220,4 @@
p = argparse.ArgumentParser(description=__doc__)
p.add_argument("--cc", default=None)
a = p.parse_args()
a.cc = a.cc or default_cxx()
Member

L8: The new test ignores CC and uses one driver for both languages. The documented --cc gcc invocation uses gcc to link the C++ switch fixture, which needs the C++ runtime for its function-local static initialization. Give the C and C++ fixtures separate compiler selection, honor CC and CXX, and pass the corresponding tools from the runner.

**L8**: The new test ignores CC and uses one driver for both languages. The documented --cc gcc invocation uses gcc to link the C++ switch fixture, which needs the C++ runtime for its function-local static initialization. Give the C and C++ fixtures separate compiler selection, honor CC and CXX, and pass the corresponding tools from the runner.
@ -178,6 +201,8 @@ void run(Enter e, const char* when) {
int main() {
run(kAfterTake, "just after the collection took the marks");
run(kAtRemark, "as the collection marked the captured page again");
Member

L10 (minor): The new enabled run checks final marks and stamps, which also pass if the renderer silently uses the full scan and puts page 9 back. The switch fixture always supplies a capable host and checks compatibility selection through source text. Count calls to each callback and assert that the enabled path leaves the captured mark untouched throughout. Run the real selection with an unsupported or absent callback and with empty capture ranges to verify the fallback.

**L10** (minor): The new enabled run checks final marks and stamps, which also pass if the renderer silently uses the full scan and puts page 9 back. The switch fixture always supplies a capable host and checks compatibility selection through source text. Count calls to each callback and assert that the enabled path leaves the captured mark untouched throughout. Run the real selection with an unsupported or absent callback and with empty capture ranges to verify the fallback.
Some checks are pending
CI / Conventional Commit title (pull_request) Blocked by required conditions
CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Blocked by required conditions
CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Blocked by required conditions
CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Blocked by required conditions
CI / Workflow lint (actionlint) (pull_request) Blocked by required conditions
CI / Website build and checks (pull_request) Blocked by required conditions
Guide screenshots / Guide images and local UI source review (pull_request) Blocked by required conditions
Third-party parts / As their notes say (pull_request) Blocked by required conditions
Third-party parts / The same as upstream (pull_request) Blocked by required conditions
PR description / Description follows the template (pull_request_target) Successful in 14s
Website / Website on Cloudflare (pull_request_target) Has been skipped
This pull request is marked as a work in progress.
Some workflows are waiting to be reviewed.
This branch is out-of-date with the base branch
View command line instructions

Checkout

From your project repository, check out a new branch and test the changes.
git fetch -u 'perf/vk-collect-once:siantm-perf/vk-collect-once'
git switch 'siantm-perf/vk-collect-once'
Sign in to join this conversation.
No reviewers
No milestone
No project
No assignees
2 participants
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
MagicBeanDev/Longshot!2027
No description provided.