fix(vk): Keep memory-writing draws while pipelines compile #1573
No reviewers
Labels
No labels
accessibility
autorelease: pending
autorelease: tagged
bug
dependencies
documentation
duplicate
enhancement
github_actions
good first issue
help wanted
invalid
question
testing wanted
wontfix
No milestone
No assignees
1 participant
Due date
No due date set.
Dependencies
No dependencies set
Reference
MagicBeanDev/Hookshot!1573
Loading…
Reference in a new issue
No description provided.
Delete branch "codex/preserve-graphics-storage-writes"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
What this fixes
A draw producing data for later draws can be skipped while its shader or pipeline compiles. An older framebuffer cannot preserve separate buffer or image outputs.
Why merge
RX_VK_PIPE_SKIP=0. Mode2restores the old untranslated-draw skip.Change
gpu/vk/vk_draw.cpp: preserve memory-writing draws during compilation.tests/test_draw_side_effects.py: execute the production decision in four modes.docs/development.md: document modes0,1and2.docs/tested-configurations.md: record the three Hyrule checks.Tested
python -I -B tests/run_all.py --jobs 1 --strict draw_side_effects shader_stamp_cache run_all_declarations: 23 cases per mode, shader stamp and runner checks.Not covered
2can omit unknown writes.Thanks, the pipeline-pending part looks right to me. Once translation is published,
stores_global_memoryplusis_writtenon storage buffers, image buffers and images covers global stores, SSBO writes and image stores/atomics. Those are the same flags the binding code already uses for write-back (memory_note_gpu_write,texture_written). Ready pipelines never reach the scan, so the hot path is unchanged. Waiting there goes through the samejobs_waitthe!may_skippath already used, so I don't see any new lock or deadlock risk. The test pulls the real block out ofvk_draw.cpp, checks for early metadata reads, and is registered inrun_all.pywith{cxx}. CI is green.The part I'd like settled before merge is the translation-pending change (inline). It's a bigger behaviour change than the title suggests. The old code skipped every not-yet-translated draw with persistent targets. The comment this PR deletes says that was hookshot's deliberate default, because waiting "froze Ocarina 0.2-3 s at each area's first new programs, the most reported stutter". After this PR every untranslated draw blocks the render thread, color-only ones too, not only the memory writers. You list this under "Not covered", but the routes you tested don't say whether the program cache was cold. Could you post
st_waits/st_wait_ms(or just felt stutter) for a cold-cache first visit to a couple of areas, main vs. this branch? If it's noticeably worse, an opt-out likeRX_VK_PIPE_SKIP=2(old translation skip, new pipeline rule) would keep the fix while giving people a way back.Nits:
This now waits for every program still being translated, including color-only draws that used to be skipped when their targets were persistent. That's the 0.2-3 s first-visit freeze the old comment described as the most reported stutter. Waiting here is the only way to know whether a stage writes memory, so I get why it's needed. But it would be good to see cold-cache wait numbers vs. main, and maybe an env opt-out for the old translation skip.
Heads-up: this conflicts with main now (a lot landed today). Could you merge main into the branch (or rebase) and resolve the conflicts? Thanks!
Addressed the source changes in the refreshed branch:
RUN_ALL_ARGS, matching the new runner.RX_VK_PIPE_SKIP=2. It restores the old skip only for unknown translations with persistent targets; published memory-writing pipelines still wait. Defaults and=0retain the correctness fix.=0/nonpersistent pipeline waits. Three focused checks pass.Your cache question is valid: the earlier routes tested first-use graphics pipelines with an already prepared program bank. They do not measure cold translation. The description now says that explicitly. I am keeping this draft until the cold-program main/candidate comparison is done; current main also needs its API13 host for the new DLLs, so the earlier API12 game runs are not being presented as a new-head hardware check.
Cold-program tests are now run: one 96-second launch per arm in Hyrule and the Kokiri house, RX 9070 XT / Ryzen 7 5700X3D, 32:9, API13, serial replay. Each started from the same visually checked title/save-menu program bank; area programs were absent. Play-generation cache files were reset too. Validation was off; original saves were unchanged.
Wait totals are approximate sums of rounded 120-frame summaries, including compute-program waits. Intervals include menus/loading and screenshot costs; they are application-present times, not display times.
This does not establish a latency improvement: driver/OS caches were not cleared, worker compute compiles reached 18.7–22.0 seconds, and the first Hyrule control did not capture gameplay. Candidate shots show Hyrule; both Kokiri arms show intact house geometry. No unexpected translation errors appeared; the reported network omissions are the card policy.
I am keeping the PR draft while isolating those common compute costs. The documented
RX_VK_PIPE_SKIP=2fallback and first-visit history comment are present, and the refreshed Windows/Linux CI is green.The cold-program Hyrule check now includes complete gameplay in both arms. Three 96-second launches used matched API13 binaries on RX 9070 XT / Ryzen 7 5700X3D, 32:9, serial replay. Each restored identical saved inputs: a menu-only program bank and warmed application pipeline caches, including play generations. Area CPU programs were absent. Validation and GPU profiling were off; sparse screenshots stayed enabled.
Repeating unchanged control removed its long freeze. This is not a PR speedup: implicit driver/OS caches and successful initial driver-cache admission were uncontrolled. Compared with the later control, the PR accumulated 2.20 seconds more readiness waiting. These totals include compute and graphics; they do not isolate translation CPU time. The first control's empty 40–60-second measurement window was rejected.
All three reached Hyrule Field without a fatal error; checked screenshots show intact terrain, grass and castle. Original saves were unchanged. This supplies first-visit cost evidence, not native memory-writer output proof.
The documented
RX_VK_PIPE_SKIP=2fallback and first-visit history comment remain. The refreshed readiness block is identical to the hardware-tested block. Three focused checks pass. Full suite: 321 passed, 23 skipped; all four failures reproduce on unchanged current main (symlink privilege, builder UI fixture, FPX DLL loading, MinGWNOMINMAX). Both independent source reviews found no defects.Linux source checks, Linux AppImage and Windows Builder pass on the refreshed head. Windows source checks finished with 325 passed, 1 failed and 22 skipped: only
test_bake_skeletontimed out at 600 seconds. The draw-side-effects check passes on both platforms.The same skeleton timeout occurs in the base main run, which also failed the log-batch timing assertion. The PR's log-batch check passes. No claim that the entire CI suite is green.
Thanks for the follow-up, this settles what I asked for.
RX_VK_PIPE_SKIP=2brings back the old skip for untranslated draws with persistent targets. Since the pipeline check runs after translation, published memory-writers still wait in that mode, which is the right split. Defaults,1and0keep the fix.I re-read the new block.
p.stagesis only read aftercp->readywas acquired (or after a wait on it). Mode 2 returns before it gets there, so there are no early reads. The fixture covers all four modes.CI: the only red check is Windows source checks, where
test_bake_skeletontimed out at 600 s. That's unrelated, and main's latest source-check runs fail too. No conflicts with main.One nit inline (a mis-encoded dash in the docs). It's fine to merge as is.
Nit:
40–60is a double-encoded en dash (UTF-8 read as cp1252). A plain40-60would do.Thanks, that answers it. The mode 2 opt-out and the restored comment cover the concern, and the waiting it adds against the warm control (~2.2 s over 96 s, about the same longest interval) is acceptable. Resolved from my side.
Fixed the range text to
40-60inb0c03b4. The refreshed table also keeps the AMD and NVIDIA “as above” groups separate (6041ea0).Linux run with NVIDIA's driver (the part you list as untested): no regression, and no visible gain on this route. RTX 4090, NVIDIA 615.78.08, i9-13900K, Arch Linux (kernel 7.2.15). Head
6041ea0merged onto main364d199(10 Nov; it still merges cleanly onto today's main), against main364d199itself. Headless (offscreen 1920x1080, the game's 2048x1152), defaults.To make pipelines build in play, both builds started with the Vulkan pipeline cache and the driver's shader cache deleted and
RX_VK_PRECOMPILE=0; the translated programs were kept (the log saystranslation 0 ms in allon every one of its 32 to 37 such lines in each run, so your first hunk, the wait for translation, never ran). Then a save in Kokiri Forest and 45 s of camera turns by script. Runs in turn in one hold of the card, load average 10-37 (other jobs on the machine; no frame-rate claim).(The totals are the log's 120-present lines
a frame: N draws left out (not ready), M waits for shaders or pipelines (T ms), each times 120 and summed.)Not tested: a folder without the program cache, where draws wait for translation (your first hunk);
RX_VK_PIPE_SKIP=2; AMD, Intel, Windows; a window.Re-checked at
6041ea0. Since my last review the branch took main (65 commits, no conflicts) and added two small commits of its own:vk_draw.cppchange is the same as the one I approved. The fixture got apipeline_topologystub for main's topology normalization (#2046), which is what that change needed.40–60nit is fixed, and the cold-cache rows intested-configurations.mdnow sit together.@mooki000's Linux NVIDIA run (RTX 4090, cold pipeline and driver caches) covers part of what you listed as untested: wait counts and times about the same as main's, slightly fewer draws left out, no fault. The white wash on a cold load shows on main too, so it doesn't come from the draws this PR keeps. That run had translated programs ready, so the translation wait wasn't exercised there. Your AMD cold-program runs cover that part.
Overlap with #1946: it changes the same skip decision. It adds an "Asynchronous" mode that never waits, and makes
targets_persistentfalse for targets cleared whole this frame. Whichever PR lands second has to merge the two rules. I'd expect the memory-writer check from this PR to still hold in #1946's default mode. Its author already said they'll rebase if this one goes first.CI is all green on this head, Windows source checks included. Good to merge.
github-sync referenced this pull request2026-11-12 06:22:54 +00:00