WIP: perf(macos): a pass's leading whole clears as its attachments' LOAD_OP_CLEAR, off by default (RX_VK_LOAD_CLEAR=1) #780
No reviewers
Labels
No labels
area/accessibility
area/audio
area/input
area/install
area/menu
area/renderer
area/saves
area/upscaling
beanstalk: approved @ 1697603
beanstalk: approved @ 2c5a62e
beanstalk: approved @ 4fd82b7
beanstalk: approved @ 64d6d09
beanstalk: approved @ 6f91fae
beanstalk: approved @ 70a78f9
beanstalk: approved @ 8015c9a
beanstalk: approved @ 8d361dc
beanstalk: approved @ 9d8356e
beanstalk: approved @ abf2227
beanstalk: approved @ d82f202
beanstalk: approved @ e5f1e40
beanstalk: approved @ ee8ab5f
ci: approved
good first issue
gpu/amd
gpu/igpu
gpu/intel
gpu/nvidia
help wanted
kind
bug
kind
chore
kind
docs
kind
feature
kind
performance
kind
question
needs measurements
platform/linux
platform/macos
platform/steam-deck
platform/windows
preview
priority
critical
priority
high
priority
low
regression
status
blocked
status
confirmed
status
duplicate
status
invalid
status
needs-info
status
wontfix
testing wanted
ci: approved
No milestone
No project
No assignees
1 participant
Notifications
Due date
No due date set.
Dependencies
No dependencies set
Reference
MagicBeanDev/Longshot!780
Loading…
Reference in a new issue
No description provided.
Delete branch "mooki/Longshot:mac/load-clear"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Type: perf · Risk: low · Tested on: macOS 26.7.1, Apple M4 Max (MoltenVK 1.4.2 and MoltenVK main 5189f8de + argbuf-slots), arm64 and x86_64 under Rosetta · Needs: macOS, Apple silicon with less memory bandwidth than an M4 Max (a hold with the switch on); Windows and Linux, any GPU (build and start)
Why
Every render pass LS records begins by loading each attachment (
LOAD_OP_LOAD). On Apple's tile GPUs a load copies the attachmentfrom memory into tile memory, and when the pass's first commands clear the attachments whole, that copy is thrown away at once; the
clear is then one more MoltenVK quad, with the state it disturbs. In run #31's field-dawn hold 37.8 passes a frame began with whole
clears: 46.8 attachments, 76 MB of the 650 MB loaded a frame (11.7%), loaded only to be overwritten. This PR can make those clears the
attachments'
LOAD_OP_CLEAR, Metal's clear load action, which only fills the tiles, wherever that draws exactly what the clears draw.On an M4 Max it measured under the run-to-run noise, so it ships off, behind
RX_VK_LOAD_CLEAR=1, for GPUs with less memorybandwidth to try. It is part of the macOS port's performance work (MacPerf §4 #7). See #420.
What changed
With
RX_VK_LOAD_CLEAR=1a pass whose first operations are whole clears begins with them as its load ops: itsvkCmdBeginRenderingis held while only those clears and changes of state come, and recorded with
LOAD_OP_CLEARfor the cleared attachments beforeanything else goes into the command buffer. Everything else is recorded as before; a clear that cannot ride on a load op stays a
vkCmdClearAttachments. Off by default on every driver (Integration 40, from the E-series): unset or0, every pass is recorded asbefore.
gpu/vk/vk_draw.cppPassgainsload_clear::Held lead(the tenVkRenderingAttachmentInfoLS givesvkCmdBeginRendering, the render area,held) andlead_cmd;begin_pass(cmd, at, lead)fillsleadinstead of stack structsand, for a clear that may fold, holds the begin;
pass_settlerecords a held begin, called bybegin_passfor anything that is nota foldable clear on the same targets, by
end_passright beforevkCmdEndRendering, and by the replay (draw_pass_settle).gpu/vk/vk_replay.cppreplay()settles a held pass before any operation outsidekeeps_lead(the clears and theoperations that record nothing on the GPU, a set of bits), before it records or admits anything. Nothing reaches the command buffer
between a pass's begin and its settle, so the command buffer is the one recorded before, less the folded clears, with those
attachments' load ops changed.
gpu/vk/load_clear.h, new;clear_color_target,clear_depth_stencil,lead_why): everyattachment the render area's size and single-sampled, no BC slice or raw view of BC (
Targets::whole_extentintargets():MoltenVK's clear load action clears each attachment whole, past a smaller render area); the clear of the whole render area and of
every channel or the aspects it names (the depth's and the stencil's load ops each their own); a value MoltenVK's two ways write
alike (its clear load action and its
vkCmdClearAttachmentsquad: no conversion that rounds, per format); the pass's begin not yetrecorded.
gpu/vk/vk_vendor.hvendor fixload_clear, off on every driver (x.load_clear = false),RX_VK_LOAD_CLEAR=1turnsit on; the device line
vk: a pass's leading whole clears as its attachments' load ops (LOAD_OP_CLEAR): off (RX_VK_LOAD_CLEAR)onMoltenVK,
onwherever it is on (gpu/vk/vk_main.cpp), and the first pass that folds says so once. The bench's B1 arm:RX_VK_BENCH_AB=load_clear(the load ops off for the run, on in the bench's B block: A is the clears in the pass, B the load ops,on the same frame).
7c6f0e9c, Integration 40's2b4b7b8efor these files (vk_vendor.h, the docs rows,test_load_clear).L2line (gpu/vk/perf_lines.h,gpu/vk/vk_perf.cpp) also counts what each leading whole clear became:a load op, switched off, other attachment sizes or samples, its value or format, the pass begun already (a frame, and in MB).
docs/development.md(theRX_VK_LOAD_CLEARrow, the bench arm,L2),gpu/vk/VULKAN.md. Tests:tests/test_load_clear.py(new; it pins the default off on every driver and
1on);tests/test_perf_lines.pyreadsbegin_pass's new signature.Measurements
Verdict: doesn't help measurably on an M4 Max: in the field-dawn hold all on but LoadClear (E4c) and all on (E2) both ran
33.7 fps and 29.67 ms a frame, GPU 19.2 against 19.1 ms, under the run-to-run noise of 0.17 ms of frame, 0.0 ms of GPU and 0.2 fps
(E7), though every one of the 46.8 leading whole clears a frame folded (76 MB of loads skipped). So it is off by default. It is exact
on the GPU where probed: LC-01 drew every folded clear's attachments texel for texel as the clears did, on both MoltenVK builds.
Baseline: main does not start on macOS. The E-series runs are the integration build I39
7a7726fe(this PR's code with#782's and #783's, and the other integrated macOS topics; LoadClear on by default there), arm64, one run each: before
is E4c, all on but LoadClear (
RX_VK_LOAD_CLEAR=0), after is E2, all three on. This head's LoadClear functions are I39's text; onlythe default differs (off here, as Integration 40 made it). LC-01 records, in a standalone MoltenVK process per build, each pass as LS
records it with the switch off (A:
LOAD_OP_LOAD, thenvkCmdClearAttachments, then a draw over the cleared values) and on (B:LOAD_OP_CLEAR, the same draw), with the PR's ownload_clear.hdeciding what folds, and compares every texel of every layer,level, slice and aspect read back.
Hardware: macOS 26.7.1 (25G241), Apple M4 Max (40-core GPU), MoltenVK main 5189f8de + argbuf-slots (
c170a745; LC-01 also MoltenVK1.4.2), 16-core CPU, 64 GB
Scene: the E-series: the field-dawn save, slot 00, entered by START and A×5, then 600 s held in play, windowless; the instrument set
on in every run (
RX_VK_PERF_LINES=1,RX_VK_GPU_TIMELINE=1, the present trace); quiet required before each start, the other lanespaused. LC-01: 373 cases (every colour and depth-stencil format LS can fold, each foldable value; an array layer, a mip level with
and without depth, a 3D slice, the depth alone, the stencil alone, two and three colours; three cases of other attachment sizes) and
64 sweeps of the values LS refuses
Method: the E-series: the medians of the hold's 120-present periods from +20 s (E4c 162 periods, E2 141; the bench's and the chain's
periods left out of E2); one run each, the noise from E7 (a repeat of E2); runs
mac-arm64-7a7726fe-06(E4c),-02(E2),-08(E7); reports
reports/2026-11-15-mac-perf-E4c.md,-E2.md,-E7.md, the seriesreports/2026-11-15-mac-perf-before-after.md.LC-01:
reports/LoadClear-files/lc01/runs-lc01-arm64/result.json(787dc0cf…), released once on arm64 by the parent; reviewf00090ed…, build00ac1f0a…, plana651ba74…How to reproduce: the series'
ls-run.py prepare/runcommands and common switches (reports/2026-11-15-mac-perf-runs-i39.md§2-§3), with
RX_VK_LOAD_CLEAR=0for the before run; LC-01's release command (reports/2026-11-14-load-clear.md§7)L2leading whole clears a frameswitched offX.smaller_depth)whole_extent)Run #31's prediction (about 0.2-0.5 ms of the GPU frame if most clears fold) did not hold on the M4 Max: all folded, 0.1 ms of GPU.
A Mac with less memory bandwidth may gain more [inference, not measured].
Flags and settings
One new switch, and one new arm of #704's bench switch; off by default.
RX_VK_LOAD_CLEAR=0or1LOAD_OP_LOADandvkCmdClearAttachments0: off;1on, on any driver1to try where memory bandwidth is shortRX_VK_BENCH_AB=load_clearRX_VK_BENCHTesting
Tested: on macOS 26.7.1, Apple M4 Max:
7c6f0e9c, on #70446eabf24, maine090cc48):test_load_clearpasses on arm64 and x86_64 (2,000 sequencesthrough the bench's arms, 779 clears made load ops in B; the default off,
1on), and fails on #704 with this head's test, botharches (no
keeps_lead);test_perf_lines(178 checks),test_vendor_fixes,test_bank_gen,test_parallel_dry_marks,test_same_bind,test_bc_raw_targets,test_pass_hazardpass;test_env_onfails as on maine090cc48(main's ownRX_VK_PRESENT_AFTER_SIGNAL, #588);vk_draw.cpp,vk_main.cpp,vk_perf.cpp,vk_replay.cppcompiled whole for both arches,0 warnings (
reports/PubWave-files/PubLate5/PerfTrio/proof/,compile/load-clear-7c6f0e9c/).macos/load-clear0d15403f, on the integration branch I383f872b0c, whose changed lines are thishead's line for line but the default):
tests/test_load_clear.py(LS's own code cut from the source against a model of MoltenVK'spasses: 25 fixed sequences command by command, 6,000 random sequences switch off against on, 2,000 through the bench's arms, the
value rules over 40 formats,
whole_extentover 16 target sets): pass; 19 of 19 mutants killed; full suite 586 passed, 1 failed(
test_menu_i18n, as on I38), 16 skipped (reports/2026-11-14-load-clear.md§8).7a7726fe):test_load_clearwith the widening and themesh batches in its fixture, both arches, among 103 of 103 checks; full suite 588 passed, 1 failed (
test_menu_i18n), 16 skipped(
reports/2026-11-14-integration-2.md§Integration 39).unifdef -U__APPLE__, #704 against this head): every changed file differs, the change being gated by the switch(
reports/PubWave-files/PubLate5/PerfTrio/offapple-load-clear.txt).Not tested: a game run at this head (the E-series ran I39, LoadClear on by default there); a Mac with less memory bandwidth than
an M4 Max; the fold on any driver but MoltenVK; not built on Windows or Linux; its test request covers it.
Must be tested:
RX_VK_LOAD_CLEAR=1andRX_VK_PERF_LINES=1against the default, expectingL2's fates to show folds and the GPU time per frame to fall by more than thenoise, if it is to be turned on there (by hand: the owner's hardware).
7c6f0e9c, it carries #704 and what it carries) and #704's head the same way,start each to the title, expecting no
LOAD_OP_CLEARdevice line, every other line as #704's, and the same picture (by hand: testrequest T-XXXX).
Risks
What could break: with
RX_VK_LOAD_CLEAR=1only: a pass'svkCmdBeginRenderingmoves after its leading clears, which become itsload ops. A command recorded between a held begin and its settle would land outside the pass; the replay settles before every
operation that records, and the test checks it on every sequence. Folds are limited to values and extents LC-01 showed exact. Off
(the default), a pass's rendering info is built in the pass (
lead) and its begin recorded at once, as before.Turning it off: it is off by default (unset
RX_VK_LOAD_CLEAR, or0), or revert the PR.Compatibility: Nothing changes for players outside macOS. No file in the shader stamp changed; no
MKWVK_API_VERSIONchange.Reviewer notes
46eabf24..7c6f0e9c(3 commits):cherry-pick -xofmacos/load-cleare0d3af8fand0d15403fonto #704, then7c6f0e9c(own commit: off by default, Integration 40's2b4b7b8eline for line for these files, but for theRX_VK_BENCH_ABrow:the same change to its
load_clearsentence, on the row without thewiden8andmesh_batcharms the stacked PRs add). Thepicks' conflicts were adjacency only, and only this topic's lines were kept:
vk_vendor.h(the field, default and switch beside#652's
compiler_serviceand #678'sindex8_size),vk_main.cpp(beside #678'svk: 8-bit indicesline),vk_draw.cpp'sincludes (#680's, #701's, #703's, #705's, #707's, #702's), the
docs/development.mdrow (beside #678's; main e090cc48's newerRX_VK_NATIVE_FADDrow kept). Every changed line of the picks is the topic's, line for line.perf_lines.h'sL2counters,perf_lines::bench_b, the perf hooks this PR extends) and what it carries (#679,#677, #663 …).
11fe233a: onepass_keepsfor this PR'spass_onand Widen8's;the widening recorded right before a held begin), #783 its fix B (
7a7726fe: this PR's test with the mesh batches).in #782's merge
ffb5dfba).LOAD_OP_CLEARis exact on every driver by Vulkan's rules; turning it on by default anywhere (a Macwith less bandwidth that measures faster, other tile GPUs) is a separate PR.
e090cc48; main has moved since, and a delta sweep follows.Issues
See #420
Checklist
Everyone
type(scope): description) of the status line's type.main. (On maine090cc48through #704; main has moved since: a delta sweep follows.)python -I -B tests/run_all.pypasses (any failure that also fails onmainis named). (On wave P = #783's head94fca509, which carries this PR, #782 and their bases: 585 passed, 31 failed, 18 skipped against main e090cc48's 544/32/17 on this Mac; the 31 failures are main's own and nothing that passes on main fails;test_collect_widenow passes; 40 new tests pass;reports/PubWave-files/PubLate5/wave/full-P-94fca509/.)third_party/untouched (a new folder with aPROVENANCE.mdinstead).When it applies
Assisted-by:/Co-Authored-By:trailers.mainand passing here.tests/that reads changed source updated in this PR, not weakened.Every scan down vk_memory.cpp's import table by start (pool_lookup's covering import and merge plan, bg_drain, prefetch_chunks, import_in_background, memory_note_gpu_write and memory_guest_imported) stopped once an import ended more than 1 GiB below its address, on the belief that no import is larger than that. pool_lookup's own merge escape leaves an import above 1 GiB over the smaller ones inside it ("left overlapping"), and the device allows imports up to max_import_size: a scan stopped at a small import inside a large one, missed the large one, and the same bytes were imported again, two memories over the same bytes. import_reach.h counts the live imports by span class; its bound is above every live span, within twice the largest. imported_add and imported_take, now the table's only ways in and out, keep the count. A scan stops only where no import further down can reach its address, so it also stops sooner than before when the imports are small. The covering-import checks in pool_lookup and import_in_background look past the last import at or below the range for the same reason. Tests: test_import_reach.py runs the actual scans with a 4 GiB import over a 16 MiB chunk, left by the merge escape (imports are recorded ranges: no payload, no Vulkan, no game). Its lookup, query, note, drain, background and prefetch cases fail on main5e072fc8and pass here, and a source check keeps every change to the table in the two helpers. test_import_windows runs pool_lookup with the helpers; test_mirror_gpu_only names the new publication point. Assisted-by: Claude Opus 5.5 Co-Authored-By: Claude Opus 5.5 <[email protected]>A guest span larger than one device allocation (the cap on MoltenVK/M4 Max is 41,747,087,360 bytes) is bound as pieces: one descriptor per allocation segment, at the descriptor's own binding (reports/2026-11-14-full-range-design.md sections 4.1-4.4). This is the shader half; the draw side (descriptors, layout, gathers) comes later. - runtime_info.h: SegEntry {Storage|Table|TexBuf|ImgBuf, desc, pieces} and RuntimeInfo::segmented. - Back end: a segmented storage buffer is an array of pieces at its binding (a table count * pieces, entry k's piece j at k * pieces + j); its loads, stores and atomics call DontInline helpers made after main, which find the piece by the pieces' OpArrayLength in order, with constant indices. The global memory fallbacks sum the pieces' lengths for a segmented descriptor's bound end and branch to its piece (single, native candidate, table written out and in the loop). Texel and image buffers are arrays too, their fetches, reads, writes and atomics through helpers that pick the piece by OpImageQuerySize, and a texel buffer's size query sums the pieces. ImageQuery was declared for every module already. With RuntimeInfo::segmented empty nothing new is made: the SPIR-V is byte-identical. - vk_shader: SegSig (entries, hash) on Program, seg_sig and seg_variant; program_segmented finds or translates the variant of a plain program with plain's texture questions and answers and checks its descriptors are plain's; program_pick, mesh_program_locked, program_find_plain, loaded_variants and portable_variant keep the plain and segmented variants apart; every stage's RuntimeInfo::segmented comes from Program::seg. - vk_progcache: a segmented variant is never written to the program cache nor listed among the loaded programs. - tests/test_seg_shader.py + tests/seg_shader_ir.cpp: synthetic IR programs through the real back end, plain and segmented; structure checks, spirv-val (Vulkan 1.3, uniform buffer standard layout), and with --base-tree a byte comparison of the plain modules against an older tree. Five source-reading checks follow the new Program field and the atomics' new branch. The translator stamp changes with this (eden/, vk_shader.cpp, sizeof(Program)): every program cache is rebuilt once. Land it in the same release as the vk_api.h change that already does so. Assisted-by: Claude Opus 5.5 Co-Authored-By: Claude Opus 5.5 <[email protected]> (cherry picked from commit b7928876dab846485f667c1c934d34d25ecef035)044635ea added vma_startup_rollback (vk_vma.cpp, run after a failed replay_start), which calls vmaDestroyAllocator. vk_vma.cpp includes vk_work.h last so that RX_VK_RECORDER=defer/thread drains before its destructions, and every such boundary call in a vk_work.h file must be in RX_REC_BOUNDARIES with a drain-first macro. This one was not, and test_recorder_sync caught it ("vk_vma.cpp calls ['vmaDestroyAllocator'], which vk_work.h does not drain before"): it passes at 58c5c5aa (044635ea's parent) and fails at 044635ea. Add it to the list and to vk_work.h, next to vmaDestroyBuffer and vmaDestroyImage. With the recorder off (the default) boundary() is a relaxed load and a branch. Rollback runs before any image or buffer exists, so nothing should be held there today, but the allocator is no longer destroyed while a recorder could still hold packets. The test is unchanged (sources: 42 boundaries drained, was 41). Assisted-by: Claude Opus 5.5 Co-Authored-By: Claude Opus 5.5 <[email protected]>The interfaces the four segmented-consumer lanes (SegBind, SegXfer, TexelDesc, NullShader; reports/2026-11-14-seg-consumer-plan.md section 3) code against, frozen before they fork. Inert: no caller of anything new, no SPIR-V, layout, push range or stream change, sizeof(Program) unchanged. - vk_internal.h / vk_memory.cpp, SegMem's consumer interface: gpu_span_any() (a relaxed count of the live domains, stored in seg_commit; domains are kept for the run), gpu_span (gpu_buffer's flavour of gpu_span_bind: import buffers, no written marks, no mirror), SpanLayout::bind_stamp (gpu_bind_stamp(): a segment can be imported again with its cuts kept, so cut_gen alone never keys piece buffers), span_unused_piece's semantics ({null_storage, 0, 1}), and the SegArena API (SegArenaRef, seg_arena_take, seg_gather, seg_scatter; stub bodies after stream_alloc that give no room and log, SegBind's to fill). - Review fixes in the same interface (renderer-core review P2-3, P3.2): every piece of gpu_span_bind carries its buffer's device address (the import's or the mirror's; 0 only with bufferDeviceAddress off), and its one-piece path stores the translated guest address, not a virtual one. - vk_texture.h/.cpp: texel_view_pieces / texel_piece_views, stubs that return 1 and false (TexelDesc's to fill). - vk_shader.h: Program::presence_words, in what was padding. - Back end: PRESENCE_PUSH_OFFSET (88), Profile::emulate_null_descriptors, Bindings::presence / presence_read with the bit numbering rule. - tests: test_resolver_coverage lists gpu_span_bind / gpu_span sites too (scoped to gpu/vk: gpu/render has an rx::gpu_span of its own), checks that gpu_span follows its own gpu_buffer, and splits SITES into SITES_BASE plus one block per lane. test_segmented_domains executes the contract (the count 0, 1, 2 -> 1 on a merge, 0; gpu_span's pieces; addresses with the fixture's gpu_buffer_bind returning none as the real one does; the bind stamp; a virtual address's guest). test_seg_shader accepts a base tree of this back end's generation and compares every module. - docs/segmented-binding.md: the gate (every new behaviour Apple-only, other platforms byte-identical) and one heading per lane. Conflict on this branch (not with main): tests/test_seg_shader.py's docstring, against the array_pointer_calls paragraph #P20's msl-arrayptr port added; both kept, this paragraph first. Assisted-by: Claude Opus 5.5 Co-Authored-By: Claude Opus 5.5 <[email protected]> (cherry picked from commit 95fa06d4e6526601bd2e1bc88c2f8f5720855350)The segmented single storage array's case (plan 4.3 item 5) had no negative: the check always cut its own tree. With --source <B2 archive> it fails as required ("moltenvk.segmented_single want=set got=push"). Conflict with #P28 (read_write textures; not with main): the docstring keeps #P28's paragraph with this usage line, and main() cuts from --source's vk_draw.cpp with #P28's read_write count (as in the macOS integration, Integration 17). Assisted-by: Claude Opus 5.5 Co-Authored-By: Claude Opus 5.5 <[email protected]> (cherry picked from commit 5824f6828e21cf810dcceaa6be425a9224b15994)plain7a498e5865plain1e1a3b3e0dKosmicKrisp has no fillModeNonSolid and MoltenVK no POINT polygons, and Metal restarts an indexed strip or fan at the all-ones index whatever Vulkan's restart enable says. Before anything is fixed, count whether the game needs it (mode_census.h, Apple only; nothing built or drawn changes): - the graphics pipelines built, where st_pipelines counts them, and the draws recorded, where st_draws counts them, by NVN's polygon mode, with the first 8 LINE and POINT pipelines (program, pipeline key, topology) and programs named; - the indexed strip and fan draws with restart off; a direct one's indices are read whole, a page at a time, through memory_read_as_submitted for the index type's all-ones value (0xFF, 0xFFFF, 0xFFFFFFFF). An indirect one's range is the GPU's and a range the CPU cannot read is not guessed: both are counted as not scanned. Any other draw costs restart_off_strip's compares and its polygon mode's count on the thread's own counters. Printed with the frame statistics every 120 presents ("at present N") and at close ("at close"). The two tests that compile build_graphics and draw()'s binds from the source get a stand-in for the counts. Assisted-by: Claude Opus 5.5 Co-Authored-By: Claude Opus 5.5 <[email protected]> (cherry picked from commit 1bd30dceb348909ca03c4cf12709dccbb0900f4a)PerfInstr's full_barrier asks perf_lines::bench_b("dedupe_off") (RX_VK_BENCH_AB's B arm; false unless the bench runs it). main's test_inline_write_order (#329) compiles full_barrier cut out of vk_texture.cpp, and did not compile at this head ("use of undeclared identifier 'perf_lines'"; found by the wave's full suite: the topic's base predates the test). It now includes perf_lines.h, as the topic did for test_dump_readback's fixture; its three variants (the current code and two negative controls) run as before, both compilers. Assisted-by: Claude Opus 5.5 Co-Authored-By: Claude Opus 5.5 <[email protected]>7c6f0e9cee12fe15f62dFolded into #852 (perf(macos): Mac performance on MoltenVK)
Pull request closed