WIP: perf(vk): a draw's write masks from its program's word, not its stages walked for each target #2049

Draft
siantm wants to merge 3 commits from siantm/Longshot:perf/vk-write-mask-word into main
Contributor

Summary

target_write_mask (gpu/vk/vk_draw.cpp) walked the program's stages for its fragment stage at every target of every draw that worked out its dynamic state, and at every target in draw_hazards. program_wm_sig already keeps all that function reads of a program as one word (6 bits a target), read by dyn_pack; write_mask_of makes each mask from that word and the channel mask, and dynamic_state reads the word once for dyn_pack and the masks. The masks are the same; the result expected is less work a draw.

Depends on #2048: it needs only RX_VK_BIND_CHANGED's switch from #2047, and goes after #2048 so that the next pull request, which needs both, has one base. Their commits show in this diff until they merge.

Switch: RX_VK_BIND_CHANGED (kill, on). =0: the stages walked for each target, main's code.

From bd3ea307 on exp/x86-asm-sync.

Read first: write_mask_of and its use in dynamic_state and draw_hazards in gpu/vk/vk_draw.cpp.

Testing

Tested: Windows 11, i7-9700, RTX 4060, the builder's llvm-mingw clang 23; head 3562fe6b on #2048's head 3432e729 (on main @ 75317e1f). Every check ran through tests/run_all.py --strict --exact in build jobs of the PC's job queue, in a hidden console (log t-dc3-vk-write-mask-word):

  • Its own: tests/test_write_mask_word.py (new: write_mask_of of the word is target_write_mask for every target, stored or not, components and channel masks; the switch in dynamic_state and draw_hazards); tests/test_same_dyn.py (updated: reads the word as dynamic_state now takes it): pass.
  • The 178 checks that read a file this change touches (--changed 3432e729; its own are among them): 157 passed, 21 failed. Failed: tests/test_sampler_admission.py, tests/test_fsr_uniform_ownership.py, tests/test_pipeline_play_cache.py, tests/test_draw_stream_visibility.py, tests/test_device_lost_exit.py, tests/test_dev_packs.py, tests/test_dev_ab.py, tests/test_draw_binds.py, tests/test_crash_dialog.py, tests/test_frame_overlap.py, tests/test_linux_pressure.py, tests/test_mesh_conv.py, tests/test_prepare_cache_save.py, tests/test_present_queue_default.py, tests/test_recorder_own_idle.py, tests/test_target_reuse_memory.py, tests/test_upscaler_rows.py, tests/test_vk_occlusion_queries.py: each fails on main @ 75317e1f too (the same checks on main in the same queue batch, log t-dc3-base). tests/test_fallback_buffer_cleanup.py, tests/test_mesh_args_cleanup.py and tests/test_pipeline_cache_save.py also failed in that run, each stopped at its 60-second limit, and passed when run again on this head (log t-dc4-vk-write-mask-word).

It has NOT run in the game on this PC yet.

Still to test:

  • Windows 11, i7-9700, RTX 4060 (this PC): start in Kokiri Forest from the scene's save, play a minute (walking, turning the camera), #2048's head, this PR with RX_VK_BIND_CHANGED=0 and this PR with it unset (on), each warmed a minute in the scene first, then interleaved (base, =0, unset, unset, =0, base), two runs each, expecting the same picture all three ways (no missing colour channel or transparency) and unset the replay's time a frame the same as or lower than the base's (by hand)
  • Linux NVIDIA and AMD, Windows AMD and Intel, Steam Deck: start in Kokiri Forest from the scene's save, play a minute (walking, turning the camera), its base PR alone against this PR with RX_VK_BIND_CHANGED=0 and unset, warmed, in ABBA order, expecting the same picture every way and no crash or validation error (@Beanstalk linux+nvidia, linux+amd, windows+amd, windows+intel, steamdeck: Kokiri Forest a minute, its base PR alone against this PR with RX_VK_BIND_CHANGED=0 and unset, ABBA, the same picture)
  • Linux (any GPU): build the renderer and run tests/test_write_mask_word.py, tests/test_same_dyn.py with --cxx g++ and --cxx clang++, expecting PASS (by hand)

Measurements

Verdict: not measured: no micro-benchmark of this port and no run in the game on this PC yet; the in-game A/B is in Still to test. RX_VK_BIND_CHANGED ships on (kind kill), the branch's default; it is not measured on main.

Risks

A mask that differs from target_write_mask's would write the wrong colour channels (a wrong picture) or miss a pass hazard (a flicker); the test compares them for every case. Turned off by RX_VK_BIND_CHANGED=0. The renderer is compiled again after updating.

Checklist

Ready for review

  • One topic, on current main; the title is a Conventional Commit.
  • Tested on my own PC, in the game for a change the game runs, as Testing says. Not yet: it has not run in the game.
  • mise run test passes (any failure that also fails on main is named). Not yet: only the checks in Testing ran.
  • Measured against main, or Measurements says why there is nothing to measure. Not yet (Measurements).
  • No game files, build output or hand-edited generated files; third_party/ untouched; nothing GPL-incompatible.

When it applies

  • A switch added or changed: it ships at the default the measurements chose, with its row in gpu/vk/switches.def. Its row is in; not measured yet.
  • A check under tests/ that reads changed source is updated, not weakened.
  • Host code builds on Windows (llvm-mingw) and Linux, with UTF-8 paths. Not yet: only the checks' cuts compiled (Testing).
  • AI assistance recorded with Assisted-by: / Co-Authored-By: trailers.
## Summary `target_write_mask` (`gpu/vk/vk_draw.cpp`) walked the program's stages for its fragment stage at every target of every draw that worked out its dynamic state, and at every target in `draw_hazards`. `program_wm_sig` already keeps all that function reads of a program as one word (6 bits a target), read by `dyn_pack`; `write_mask_of` makes each mask from that word and the channel mask, and `dynamic_state` reads the word once for `dyn_pack` and the masks. The masks are the same; the result expected is less work a draw. Depends on #2048: it needs only `RX_VK_BIND_CHANGED`'s switch from #2047, and goes after #2048 so that the next pull request, which needs both, has one base. Their commits show in this diff until they merge. Switch: `RX_VK_BIND_CHANGED` (kill, on). `=0`: the stages walked for each target, main's code. From bd3ea307 on `exp/x86-asm-sync`. Read first: `write_mask_of` and its use in `dynamic_state` and `draw_hazards` in `gpu/vk/vk_draw.cpp`. ## Testing **Tested:** Windows 11, i7-9700, RTX 4060, the builder's llvm-mingw clang 23; head 3562fe6b on #2048's head 3432e729 (on main @ 75317e1f). Every check ran through `tests/run_all.py --strict --exact` in build jobs of the PC's job queue, in a hidden console (log `t-dc3-vk-write-mask-word`): - Its own: `tests/test_write_mask_word.py` (new: `write_mask_of` of the word is `target_write_mask` for every target, stored or not, components and channel masks; the switch in `dynamic_state` and `draw_hazards`); `tests/test_same_dyn.py` (updated: reads the word as `dynamic_state` now takes it): pass. - The 178 checks that read a file this change touches (`--changed 3432e729`; its own are among them): 157 passed, 21 failed. Failed: `tests/test_sampler_admission.py`, `tests/test_fsr_uniform_ownership.py`, `tests/test_pipeline_play_cache.py`, `tests/test_draw_stream_visibility.py`, `tests/test_device_lost_exit.py`, `tests/test_dev_packs.py`, `tests/test_dev_ab.py`, `tests/test_draw_binds.py`, `tests/test_crash_dialog.py`, `tests/test_frame_overlap.py`, `tests/test_linux_pressure.py`, `tests/test_mesh_conv.py`, `tests/test_prepare_cache_save.py`, `tests/test_present_queue_default.py`, `tests/test_recorder_own_idle.py`, `tests/test_target_reuse_memory.py`, `tests/test_upscaler_rows.py`, `tests/test_vk_occlusion_queries.py`: each fails on main @ 75317e1f too (the same checks on main in the same queue batch, log `t-dc3-base`). `tests/test_fallback_buffer_cleanup.py`, `tests/test_mesh_args_cleanup.py` and `tests/test_pipeline_cache_save.py` also failed in that run, each stopped at its 60-second limit, and passed when run again on this head (log `t-dc4-vk-write-mask-word`). It has NOT run in the game on this PC yet. **Still to test:** - Windows 11, i7-9700, RTX 4060 (this PC): start in Kokiri Forest from the scene's save, play a minute (walking, turning the camera), #2048's head, this PR with `RX_VK_BIND_CHANGED=0` and this PR with it unset (on), each warmed a minute in the scene first, then interleaved (base, =0, unset, unset, =0, base), two runs each, expecting the same picture all three ways (no missing colour channel or transparency) and unset the replay's time a frame the same as or lower than the base's (by hand) - Linux NVIDIA and AMD, Windows AMD and Intel, Steam Deck: start in Kokiri Forest from the scene's save, play a minute (walking, turning the camera), its base PR alone against this PR with `RX_VK_BIND_CHANGED=0` and unset, warmed, in ABBA order, expecting the same picture every way and no crash or validation error (@Beanstalk linux+nvidia, linux+amd, windows+amd, windows+intel, steamdeck: Kokiri Forest a minute, its base PR alone against this PR with RX_VK_BIND_CHANGED=0 and unset, ABBA, the same picture) - Linux (any GPU): build the renderer and run `tests/test_write_mask_word.py`, `tests/test_same_dyn.py` with `--cxx g++` and `--cxx clang++`, expecting PASS (by hand) ## Measurements **Verdict:** not measured: no micro-benchmark of this port and no run in the game on this PC yet; the in-game A/B is in Still to test. `RX_VK_BIND_CHANGED` ships on (kind kill), the branch's default; it is not measured on main. ## Risks A mask that differs from `target_write_mask`'s would write the wrong colour channels (a wrong picture) or miss a pass hazard (a flicker); the test compares them for every case. Turned off by `RX_VK_BIND_CHANGED=0`. The renderer is compiled again after updating. ## Checklist ### Ready for review - [x] One topic, on current `main`; the title is a Conventional Commit. - [ ] Tested on my own PC, in the game for a change the game runs, as Testing says. Not yet: it has not run in the game. - [ ] `mise run test` passes (any failure that also fails on `main` is named). Not yet: only the checks in Testing ran. - [ ] Measured against `main`, or Measurements says why there is nothing to measure. Not yet (Measurements). - [x] No game files, build output or hand-edited generated files; `third_party/` untouched; nothing GPL-incompatible. ### When it applies - [ ] A switch added or changed: it ships at the default the measurements chose, with its row in `gpu/vk/switches.def`. Its row is in; not measured yet. - [x] A check under `tests/` that reads changed source is updated, not weakened. - [ ] Host code builds on Windows (llvm-mingw) and Linux, with UTF-8 paths. Not yet: only the checks' cuts compiled (Testing). - [x] AI assistance recorded with `Assisted-by:` / `Co-Authored-By:` trailers.
perf(vk): a draw leaves out a block of push constants its command buffer already has (RX_VK_BIND_CHANGED, on)
Some checks are pending
CI / Conventional Commit title (pull_request) Blocked by required conditions
CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Blocked by required conditions
CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Blocked by required conditions
CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Blocked by required conditions
CI / Workflow lint (actionlint) (pull_request) Blocked by required conditions
CI / Website build and checks (pull_request) Blocked by required conditions
Guide screenshots / Guide images and local UI source review (pull_request) Blocked by required conditions
Third-party parts / As their notes say (pull_request) Blocked by required conditions
Third-party parts / The same as upstream (pull_request) Blocked by required conditions
PR description / Description follows the template (pull_request_target) Successful in 5s
Website / Website on Cloudflare (pull_request_target) Has been skipped
beanstalk/approval waiting for a BigBean's 👍 on the request
aebcf74638
push_constants pushed the program's whole block (page table, task first, render area, fallback addresses) at every
draw whose program reads it, though the draws of a pass push the same block one after another. It now leaves the
push out when the last block this thread recorded was the same bytes, for the same stages, into the same command
buffer (push_standing). vk_work.h counts every vkCmdPushConstants on the thread (push_epoch), so a push of anyone's
in between makes the next draw push again. The block is forgotten where the command buffer's may be something else:
at each recording's begin (binds_begin_recording), a masked clear, the temporal upscalers' stand-ins, each
dispatch's end (its stand-ins call NGX / FidelityFX, which push past vk_work.h), and (new here, beside the branch's
sites) in descriptors_disturbed, which every upscaler and library pass on main calls after recording its own
pipelines. A direct mesh draw no longer pushes entry 0's first a second time: it is in the block just sent or left
standing.

The first of RX_VK_BIND_CHANGED's parts on the branch (bd3ea307, "draw-binds"); its other parts follow as their own
commits. Its index-buffer part (bind at the pool buffer's start, draw from the first index) is main's
RX_VK_INDEX_REBASE already and is not ported.

Switch: RX_VK_BIND_CHANGED, kill, on by default (the branch's default); =0 (or false, off, no) pushes at every draw as
before. Read once, at its first use. Row in gpu/vk/switches.def. Not measured on its own on the branch ("the saving is
not measured yet"); not measured in play.

Test: tests/test_push_changed.py runs push_standing (cut from the source) and reads the wiring: vk_work.h's count, the
check before the push, the block kept after it, every place it is forgotten, the mesh first; and the switch's reader
runs in fresh processes (unset, 0, off, false, 1, on and another value, twice each, set again after its first read).
The three fixtures that compile draw_begin_recording stand in binds_begin_recording.

Assisted-by: Claude Code
perf(vk): a draw takes its targets by reference and their colour types as worked out at their resolution
Some checks are pending
CI / Conventional Commit title (pull_request) Blocked by required conditions
CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Blocked by required conditions
CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Blocked by required conditions
CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Blocked by required conditions
CI / Workflow lint (actionlint) (pull_request) Blocked by required conditions
CI / Website build and checks (pull_request) Blocked by required conditions
Guide screenshots / Guide images and local UI source review (pull_request) Blocked by required conditions
Third-party parts / As their notes say (pull_request) Blocked by required conditions
Third-party parts / The same as upstream (pull_request) Blocked by required conditions
PR description / Description follows the template (pull_request_target) Successful in 5s
Website / Website on Cloudflare (pull_request_target) Has been skipped
beanstalk/approval waiting for a BigBean's 👍 on the request
3432e729e7
draw() copied the thread's resolved targets twice a draw (targets(), then bare_targets()) and worked out their
colour types (color_types_of) at every draw. With RX_VK_BIND_CHANGED (on) it takes them by reference
(draw_targets: the thread's resolved ones, or the viewport's area in a scratch for a draw with no target) and the
colour types from a memo made again whenever targets() resolves anew (targets_gen). =0: copied and worked out at
every draw, as before.

Part of RX_VK_BIND_CHANGED on the branch (bd3ea307). Needs "perf(vk): a draw leaves out a block of push constants its
command buffer already has (RX_VK_BIND_CHANGED, on)" first (the switch). Not ported from the same part: the branch
also kept whether the targets can be the scene (colour 0 in B10G11R11) and asked once a recording whether an upscaler
wants the scene notes, then skipped temporal_note_scene_draw, fsr3_inputs::note_scene_draw and skin_check for other
draws. On main temporal_scene_format also takes R16G16B16A16_SFLOAT targets (PsychoV35 with the Upscaler Off or a
network that takes it), so that test would leave out draws main notes: those calls stay as main has them.

Not measured on its own on the branch; not measured in play.

Test: tests/test_targets_kept.py cuts draw_targets, draw_targets_copied, TargetsMemo and targets_memo_of and runs them
with stand-ins over 200,000 random target changes, viewports and new submissions against the copies and
color_types_of, and reads targets() and draw() for the generation and the switch.

RX_VK_BIND_CHANGED's row in gpu/vk/switches.def names this part too (what the switch keeps, what =0 gives back).

Assisted-by: Claude Code
perf(vk): a draw's write masks from its program's word, not its stages walked for each target
Some checks are pending
CI / Conventional Commit title (pull_request) Blocked by required conditions
CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Blocked by required conditions
CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Blocked by required conditions
CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Blocked by required conditions
CI / Workflow lint (actionlint) (pull_request) Blocked by required conditions
CI / Website build and checks (pull_request) Blocked by required conditions
Guide screenshots / Guide images and local UI source review (pull_request) Blocked by required conditions
Third-party parts / As their notes say (pull_request) Blocked by required conditions
Third-party parts / The same as upstream (pull_request) Blocked by required conditions
PR description / Description follows the template (pull_request_target) Successful in 5s
Website / Website on Cloudflare (pull_request_target) Has been skipped
beanstalk/approval waiting for a BigBean's 👍 on the request
3562fe6b59
target_write_mask walked the program's stages for its fragment stage at every target of every draw that worked out
its dynamic state, and at every target in draw_hazards. program_wm_sig already keeps all that function reads of a
program as one word (6 bits a target), read by dyn_pack; write_mask_of makes each mask from that word and the
channel mask. dynamic_state reads the word once for dyn_pack and the masks. =0 (RX_VK_BIND_CHANGED): the stages
walked, as before.

Part of RX_VK_BIND_CHANGED on the branch (bd3ea307). Needs "perf(vk): a draw leaves out a block of push constants its
command buffer already has (RX_VK_BIND_CHANGED, on)" first (the switch); it goes after "perf(vk): a draw takes its
targets by reference ..." only so that the next commit, which needs both, has one base. Not measured on its own on the branch; not
measured in play.

Test: tests/test_write_mask_word.py cuts target_write_mask, program_wm_sig and write_mask_of and checks write_mask_of
of the word is target_write_mask for every target, stored or not, components and channel masks; and reads
dynamic_state and draw_hazards for the switch. test_same_dyn.py reads the word as dynamic_state now takes it.

RX_VK_BIND_CHANGED's row in gpu/vk/switches.def names this part too (what the switch keeps, what =0 gives back).

Assisted-by: Claude Code
Author
Contributor

@Beanstalk This changes the renderer so that every draw's dynamic state and pass hazards are tracked with one write-mask word (gpu/vk), behind RX_VK_BIND_CHANGED, a kill switch on by default; RX_VK_BIND_CHANGED=0 is main's way. It is on #2048 and #2047, whose commits are in its diff. It has not run in the game on these PCs yet. Please play Kokiri Forest and Hyrule Field, two minutes each, the PR against main on the same PC, on the PR's side with the switch unset and with RX_VK_BIND_CHANGED=0, with a picture in each compared with main's, on Linux with NVIDIA, Linux with AMD (RADV) and the Steam Deck. The same picture, no crash, device lost, validation error or FATAL line; unset, frame time p99 equal to main's or better and to the switch off.

@Beanstalk This changes the renderer so that every draw's dynamic state and pass hazards are tracked with one write-mask word (gpu/vk), behind RX_VK_BIND_CHANGED, a kill switch on by default; RX_VK_BIND_CHANGED=0 is main's way. It is on #2048 and #2047, whose commits are in its diff. It has not run in the game on these PCs yet. Please play Kokiri Forest and Hyrule Field, two minutes each, the PR against main on the same PC, on the PR's side with the switch unset and with RX_VK_BIND_CHANGED=0, with a picture in each compared with main's, on Linux with NVIDIA, Linux with AMD (RADV) and the Steam Deck. The same picture, no crash, device lost, validation error or FATAL line; unset, frame time p99 equal to main's or better and to the switch off.
Member

@siantm only members of BigBeans can start the farm. One of them can approve this request by reacting 👍 to it or to this comment.

@siantm only members of `BigBeans` can start the farm. One of them can approve this request by reacting 👍 to it or to this comment.
Some checks are pending
CI / Conventional Commit title (pull_request) Blocked by required conditions
CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Blocked by required conditions
CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Blocked by required conditions
CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Blocked by required conditions
CI / Workflow lint (actionlint) (pull_request) Blocked by required conditions
CI / Website build and checks (pull_request) Blocked by required conditions
Guide screenshots / Guide images and local UI source review (pull_request) Blocked by required conditions
Third-party parts / As their notes say (pull_request) Blocked by required conditions
Third-party parts / The same as upstream (pull_request) Blocked by required conditions
PR description / Description follows the template (pull_request_target) Successful in 5s
Website / Website on Cloudflare (pull_request_target) Has been skipped
beanstalk/approval waiting for a BigBean's 👍 on the request
This pull request is marked as a work in progress.
Some workflows are waiting to be reviewed.
This branch is out-of-date with the base branch
View command line instructions

Checkout

From your project repository, check out a new branch and test the changes.
git fetch -u 'perf/vk-write-mask-word:siantm-perf/vk-write-mask-word'
git switch 'siantm-perf/vk-write-mask-word'
Sign in to join this conversation.
No reviewers
No milestone
No project
No assignees
2 participants
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
MagicBeanDev/Longshot!2049
No description provided.