WIP: feat(vk): a packed record layout for the mesh fallback's export and reader (#1477 BC-1a) #1561
No reviewers
Labels
No labels
area/accessibility
area/audio
area/input
area/install
area/menu
area/renderer
area/saves
area/tooling
area/upscaling
beanstalk: approved
beanstalk: failed
beanstalk: needs approval
beanstalk: running
beanstalk: tested
ci: approved
good first issue
gpu/amd
gpu/igpu
gpu/intel
gpu/nvidia
help wanted
kind
bug
kind
chore
kind
docs
kind
feature
kind
performance
kind
question
needs measurements
platform/linux
platform/macos
platform/steam-deck
platform/windows
preview
priority
critical
priority
high
priority
low
regression
risk
high
risk
low
risk
medium
status
blocked
status
confirmed
status
duplicate
status
invalid
status
needs-info
status
wontfix
ci: approved
No milestone
No project
No assignees
2 participants
Notifications
Due date
No due date set.
Dependencies
No dependencies set
Reference
MagicBeanDev/Longshot!1561
Loading…
Reference in a new issue
No description provided.
Delete branch "omgitsgela/Longshot:perf/1477-bc1-packed-export"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Type: feat · Risk: low · Tested on: Steam Deck LCD (VanGogh, RADV), desktop session, headless test profile, in game · Needs: the release Windows job to compile the touched files (see Must be tested)
TL;DR
BC-0 (#1531) measured that ~1.5 ms of the compute mesh fallback's 6.8 ms is our record-export traffic. BC-1a makes that traffic wide: the fallback's epilogue now writes a record's vertex lines four at a time (one 16-byte store instead of four strided scalar words) and the generated vertex stage reads them back the same way. On the Deck LCD, A/B/A/B on one build: +0.3 fps steady against a 0.2 spread, and each log's whole-frame
GPU timeline says −0.8 to −1.1 ms a frame — matching BC-0's bound, so the wide access is real on VANGOGH. New valueRX_VK_MESH_FB_LAYOUT=packed4, off by default; both packed4 arms' pictures are visually clean.Breakdown
packed4, for programs whose records are only their vertex lines: lines grouped in fours, 16-byte stores and loads, no validity word. Everything else — header, bump allocator, indirect draw, counts — is untouched. A program whose records carry per-primitive lines cannot take it and is laid out soa, its own module exactly (one shared decision helper,LaidOutLayout, asked by both the epilogue and the vertex body of the same program, so a record's write and read cannot disagree whatever the switch said).soaoraos: the emitter takes the same paths as before (the fork adds Packed4 only); native mesh is untouched; saves, ini, ABI unchanged. The switch's name is under the program cache stamp'sRX_VK_MESHsweep, so a packed4 program is never read back by a run that did not ask for it, and mesh + vertex stages of a program are translated in one call and saved together.vk: mesh fallback records:line; both packed4 end-view.png frames inspected for addressing garbage (none); a delegated read-only review of the whole diff returned SHIP with zero findings, re-running the harness independently and checking 27,948 address expressions (cites in #1477 comment 48148).fb mesh prog <key>span A/B (does the fallback's own span shrink) needs #1460's span machinery, which is not in main — recorded in the docs row; it can run cheaply once #1460 lands.Why
#1477 BC-1: restructure the fallback's output after BC-0 bounded our export traffic at ~1.5 ms of the fallback's 6.8 ms (comment 47400). BC-2a came back disproven (the hot programs are rank-identity programs, comment 47796), and indexed/dedup turned out measured-and-dead from PR #906 (comment 48043 and 47993): so the remaining honest prize in this pool is the access width of the export and reader. Packed4 is the smallest shippable increment of that: no index buffer, no new passes, opt-in.
What changed
The layout enum gains
Packed4 = 2, selected bymesh_rule::fb_layouton the wordpacked4. The emitter decides the layout once per program throughLaidOutLayout(runtime_info, mesh)— Packed4 only where the program has no per-primitive lines, soa otherwise — and both the epilogue and the generated vertex body ask the same helper. Packed4 addressing: groupgof recordratH + g*4*fit + r*4(H the 8 header words, fit from the area's words over the vertex-line count — a few more records than soa holds, since its records carry no validity word), remainder lines in soa-shaped columns atH + l*fit + r; the writer uses the allocator'sbase + kas the record number, the reader'sgl_Vertexindexes the same space because the draw's count is the records taken and every taken record wrote its lines.gpu/vk/eden/shader_recompiler/backend/spirv/emit_spirv_mesh.cpp—LaidOutLayout,WritePackedRecords, vec4BufferVecAt/LoadVec/StoreVec, the epilogue fork, the vertex body's packed reader.gpu/vk/eden/shader_recompiler/backend/spirv/spirv_emit_context.{h,cpp}—MeshFallbackVec4: the PhysicalStorageBuffer block of four words the vec4 loads/stores address.gpu/vk/eden/shader_recompiler/backend/spirv/emit_spirv.h— the enum value andMeshFallbackFitPacked4(the host asks capacity through it).gpu/vk/mesh_mode.h—packed4as the wordfb_layoutaccepts (the first cut documented it without parsing it: an arm would have run soa silently; caught before any arm ran, fixed and covered by the model test).gpu/vk/vk_shader.cpp— thevk: mesh fallback records:log says which layout a start chose, and names the soa-fallback rule.tests/mesh_fallback_layout_model.cpp/_ir.cpp/test_mesh_fallback_layout.py— the model draws packed4's records exactly as aos/soa's where capacity allows; packed4-cannot-apply fixtures byte-identical to soa; the switch-word parsing pinned (including that unknown words still fall to the default); 76 fixture modules across the harness, 48 validated under spv1.6 + vulkan1.3.docs/development.md— the layout row gainspacked4with its rule and its measured provenance.Measurements
Verdict: packed4 is real but small, exactly as BC-0's split bounds it: +0.3 fps steady against a 0.2 spread, +0.5/+1.3 over whole-drive windows, −0.8 to −1.1 ms a frame of whole-frame GPU time. The fallback pool's cheap half is now collected; 30-while-moving is not in this pool.
Hardware: Steam Deck LCD, AMD Custom GPU 0405 (RADV VANGOGH), Mesa 26.2.0-devel (git-035ae2f854), auto DPM (sclk median 1300 MHz), 16 GB shared.
Scene: Hyrule Field save (copied, slot 05), 1280x800 output, resolution=4 (640x400 internal), compute fallback,
RX_VK_PASS_TIMING=2, no PROF_FINE, no fps cap; scripted route (warm, walks, pans, full-stick turns, circle) bymesh2245-arm3.sh.Method: A/B/A/B on ONE overlay (
rt-rc6-bc1, this branch at195cd15d,so=df64452c, caches unlinked, h1 pays the cold translation, h2-h4 warm): h1/h3RX_VK_MESH_FB_LAYOUT=soa, h2/h4=packed4. Every arm's layout pinned from its own log line, not assumed;RX_VK_GM_LOOPat its default (off on RADV) in all four — dukenheimer's variant table (#1477) had not posted at arm time.Baseline: soa arms vs packed4 arms on the same binary — main is not the comparison because both arms must share one build.
195cd15d, RX_VK_MESH_FB_LAYOUT=soa (h1/h3)195cd15d(h2/h4)GPU timemedianGPU time(the log's own unprofiled line, 12-13 play-window reports per arm): 25.4/25.1 vs 24.3/24.3 ms — matches BC-0's ~1.5 ms export bound, which is the mechanism evidence.~/longshot-investigation/mesh2245-arm3.sh LABEL ~/longshot-nvme/rt-rc6-bc1 compute 4 RX_VK_MESH_FB_LAYOUT=packed4(and=soaagainst it). Rows:docs/tested/2026-11-17-packed4-abba-omgitsgela-run19.md; raw run foldersmesh2245-bc1a-h1..h4on the Deck.Flags and settings
This PR adds one value to an existing switch:
packed4forRX_VK_MESH_FB_LAYOUT, shipped unset (soa stays the default).RX_VK_MESH_FB_LAYOUT=packed4aos/soaonlypacked4Testing
Tested: Steam Deck LCD desktop mode, in game, this branch at
195cd15d(four full scripted arms, A/B/A/B; no crash, no amdgpu kernel lines, layout pin correct in all four logs, both packed4 end views visually clean — checked for sheared grass, exploded quads, holes, wrong-colored patches). The branch head0c32b62d(docs only past the arms' tree) plus merged main1333302d: fullpython3 -I tests/run_all.py610 passed / 36 failed / 128 skipped — and clean main atd1a307cbrun in a separate worktree fails 39 of the same tests; the branch's 36 failures are a strict subset of main's (main additionally fails bug_report, crash_groups, hardware_report), so none is caused by this PR.tests/test_mesh_fallback_layout.py62 cases 0 failed, 48 modules spirv-val clean (VM: g++, NDK spirv-val).tests/test_tested_configurations.pyOK (it caught my first attempt at the docs rows and the rows were rewritten in the fragment format).tests/test_mesh_fallback_failed_once.py, stamp tests pass. A delegated read-only review of the diff (independent worktree, cited findings): SHIP, no CRITICAL/MAJOR/MINOR findings across writer/reader address agreement, validity-word removal (the allocator's gate isk < takenwith the host barrier ordering stores before the indirect draw; no other production reader of the validity word), host↔shader capacity (both askMeshFallbackFitPacked4; no separate host cap), vec4 alignment legality, cache stamping and paired-stage translation, soa byte-identity of packed4-cannot-apply programs, and count-vs-written records; it re-ran the harness clean and checked 27,948 address expressions (#1477 comment 48148).Not tested: the keyed
fb-span A/B on this lineage (needs #1460's span labels, not in main — the GPU-time line carries the mechanism instead); other GPUs (soa remains the measured-and-shipped default; packed4 stays opt-in); other titles; AMD's other architectures; the Game-Mode path.Must be tested: Windows, llvm-mingw cross-compile (I have no Windows toolchain on this host): the
build-windows.ymljob on this PR (by hand: read the check on the PR page), expecting green for the touchedemit_spirv_mesh.cpp,spirv_emit_context.{h,cpp},vk_shader.cpp(all compiled on the Deck's clang and the VM's gcc here). Nothing else beyond the switch's own per-GPU measurement, which the flag table marks untested.Risks
What could break: a packed4 addressing bug would show as displaced geometry in packed4 runs only — guarded by the model's draw-equality (packed4 draws the same primitives as aos/soa through capacity), 48 validated fixture modules, byte-identity where packed4 cannot apply, and both Deck arms' clean pictures. A capacity bug would overflow the area — guarded by the reviewer-checked
base <= fit && taken <= fit-baseallocator gate and the model's per-layout caps. Nothing for a normal run: the word is unset by default and the fork adds only the Packed4 path.Turning it off: unset the variable (soa remains the default). The layout is part of the program cache stamp's
RX_VK_MESHsweep, so switching back re-keys caches exactly like anyRX_VK_MESH*change — one cold translation.Compatibility: no save, ini, ABI or tool change; native-mesh GPUs never see a fallback program. The vec4 accesses declare
Aligned 4as every fallback access does — legal, with the intra-buffer offsets multiples of 16 bytes; whether a driver issues 16-byte transactions is driver-side, and the Deck's −1 ms GPU-time delta is the evidence for VANGOGH.Issues
See #1477 (BC-1a of the campaign plan; the arms' table and the review outcome are comment 48148). Builds on BC-0's measured split (#1531) but does not depend on it: this branch is main + this change.
Checklist
Everyone
feat(vk): ...) of the status line's type.python -I -B tests/run_all.pypasses (any failure that also fails onmainis named). On the merged head (maind1a307cbmerged as1333302d): 610 passed / 36 failed / 128 skipped, and clean main atd1a307cbin its own worktree fails 39 of the same tests — the branch's 36 are a strict subset of main's (main also fails bug_report, crash_groups, hardware_report); none is caused by this PR.main(merged1333302d, main atd1a307cb).third_party/untouched (a new folder with aPROVENANCE.mdinstead).When it applies
docs/tested/for each new measured run, fragment format (the configurations checker passes).tests/that read changed source updated, not weakened: the layout model grew packed4's cases and the switch-word pins; nothing weakened (the harness's existing aos-as-main equality still passes).The first BC-1a commit did not build: spirv_emit_context.cpp used mesh_fallback_vec4_ptr before the header declared it; the epilogue's and reader's Const(4) were ambiguous with the Const(u32,u32,..) composites (need 4U); OpCompositeExtract wants a literal index, not the Id Const(c) gave it; the record line's attribute u32 needs IR::Attribute{} at the StoreFallbackOutput call; and the loop body left a bare block where the If(...) closer belonged. The mesh translation unit now passes a syntax check on the harness's own include set.Tested in game: Steam Deck LCD desktop mode, four scripted A/B/A/B arms at
195cd15d(rt-rc6-bc1, so=df64452c): layout pin correct in all four logs, both packed4 end views visually clean, no crash or kernel linesRead ahead of a Steam Deck OLED run of this (head
864078296; built as main @9af4808fplus exactly this PR's own commits, which apply there unchanged with lines identical to the PR's diff; onlylongshot_vk.sodiffers from that main). The layout model gives 62 cases, 0 failed and 48 modulesspirv-valclean here too, and the 7 failures of the changed set all fail on that main. The OLED arms are queued (the Field at 50% and at 100% standing, Kokiri at 50%, unset againstpacked4, the log'sGPU timeline per window); numbers will follow here. From the read:Found sound
MeshFallbackFitPacked4(V); groups and tail columns cannot overlap (r < fit), and everything ends inside the host's area, whose size does not depend on the layout. Both stages askLaidOutLayout, so a program's write and read cannot disagree.base, and only when the block fits does it add to the draw count and publishbase; the workgroup then writes all it took with no early exit. Since the reservations are monotonic, every workgroup before the first that does not fit does fit and every one after hasbase > fit, so the draw's count is exactly the contiguous run of written records. Past capacity, zero-output meshlets and out-of-range slots add, write and draw nothing; a reservation with no write loop cannot happen.Worth a line in the description
packed4is a second stamp, so a tester needs two preparations on a Deck (an explicit=soawould be a third stamp text for the same code: arms that pin soa by leaving the switch unset run what a player gets).Small things
Aligned 4: legal, and the buffer's base is in practice far more aligned, though nothing asserts it;Aligned 16would say what is relied on.LaidOutLayout.Tester's first numbers on a Steam Deck OLED, the setting where its GPU binds: packed4 takes 0.4 ms of GPU time a frame off the Field standing at 100% (every packed4 run below every default run), and nothing measurable while turning. The Field at 50%, Kokiri, and both again with
RX_VK_READBACK_LATE=newest(the wait out of the way, as in #1477) are running now.This PR's code on plain main
9af4808f, one binary for both arms; A =RX_VK_MESH_FB_LAYOUTunset (soa), B =packed4, each on its own prepared cache, order A B A B B A. Hyrule Field, Lost Woods tunnel mouth at night, 1280x800 native (Render 100%), the game's own upscaler, vsync off,HOS_FPS_MAX=1000, 45 s standing then two full-stick pans. SteamOS 3.9.5, Mesa 26.2.0-devel, GPU at manual 1600 MHz before and after every run.line after line (soa) (the default)/packed four lines (#1477 BC-1a; …)), the running game's environment, onelongshot_vk.sohash in all six runs, and each run keeping its own arm's cache (47,798 program keys kept, no preparation). Draws and dispatches a frame are equal in A and B (1,091-1,095 and 145 standing).Two more measured rows on this branch since the description was written, both strengthening the case:
docs/tested/2026-11-18-probed-layout-ab-omgitsgela-run27.md: with the host-side read-back wait probed out (RX_VK_READBACK_LATE=newest), Kokiri arms lift to ~33 fps and the layout stays fps-null whileGPU timekeeps its ~1 ms (16.0/15.0 against 14.5/14.4). The ms are robust to the regime.docs/tested/2026-11-18-field100-packed4-abba-omgitsgela-run28.md: at Render 100% in Hyrule Field the ms convert to frames — standing soa 25.1/25.3 against packed4 25.6/25.5, window groups barely overlapping,GPU time31.9/31.9 against 31.0/31.0 ms. At 100% the Field frame is GPU-bound (below the read-back plateau), which is where users play when they complain about detail loss at 50%.So the fps claim in the TL;DR should read: invisible at 50% render (wait-limited), visible at 100% render (GPU-limited), −0.9 to −1.1 ms whole-frame GPU time in both. Full thread: #1477 comments 48564, 48600, 48643.
feat(vk): a packed record layout for the mesh fallback's export and reader (#1477 BC-1a)to WIP: feat(vk): a packed record layout for the mesh fallback's export and reader (#1477 BC-1a)Heads-up: the branch now carries one more code commit past the reviewed packed4 tree
195cd15d:5e64d691(#1477 CS-1) enforces a 16-byte-aligned records buffer base (create_buffergains an alignment request;fallback_buffers_ofdemands and checks it on new and reused buffers) and raises the packed4 vector loads/stores' SPIR-V alignment declaration from 4 to 16, following a delegated arithmetic audit (offsets land at base+16k; the base was unpromised). Docs rows run27-run34 also sit on the branch. I am putting the PR back to WIP: the CS-1 change is not yet Deck-measured, and the earlier review covered only packed4 at195cd15d. Deck arms (cs1 overlay build running now) + a fresh review of the new diff will restore ready; measurement decides whether CS-1 ships with this PR or as its own follow-up.3ecd17a0be0c7b08e9e3View command line instructions
Checkout
From your project repository, check out a new branch and test the changes.