feat(vk): add a segment planner for guest spans larger than one device allocation #647
No reviewers
Labels
No labels
area/accessibility
area/audio
area/input
area/install
area/menu
area/renderer
area/saves
area/upscaling
beanstalk: approved @ 1697603
beanstalk: approved @ 2c5a62e
beanstalk: approved @ 4fd82b7
beanstalk: approved @ 64d6d09
beanstalk: approved @ 6f91fae
beanstalk: approved @ 70a78f9
beanstalk: approved @ 8d361dc
beanstalk: approved @ 9d8356e
beanstalk: approved @ abf2227
beanstalk: approved @ d82f202
beanstalk: approved @ e5f1e40
beanstalk: approved @ ee8ab5f
ci: approved
good first issue
gpu/amd
gpu/igpu
gpu/intel
gpu/nvidia
help wanted
kind
bug
kind
chore
kind
docs
kind
feature
kind
performance
kind
question
needs measurements
platform/linux
platform/macos
platform/steam-deck
platform/windows
preview
priority
critical
priority
high
priority
low
regression
status
blocked
status
confirmed
status
duplicate
status
invalid
status
needs-info
status
wontfix
testing wanted
ci: approved
No milestone
No project
No assignees
1 participant
Notifications
Due date
No due date set.
Dependencies
No dependencies set
Reference
MagicBeanDev/Longshot!647
Loading…
Reference in a new issue
No description provided.
Delete branch "mooki/Longshot:mac/seg-plan"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Type: feat · Risk: low · Tested on: macOS 26.7.1, Apple M4 Max (arm64, and x86_64 under Rosetta; no GPU work) · Needs: Windows, Linux (the test only)
Why
The macOS port (#420). On a Mac the renderer reaches guest memory by importing it into Vulkan, and one import cannot be
larger than one device allocation (41,747,087,360 bytes, about 38.9 GiB, on this Mac), while overlapping views of the
game's memory can together span more than that. A
span of guest memory larger than one allocation has to be imported as several segments, and where the cuts fall decides which of
the game's buffers end up across a cut, so that the renderer has to bind them piece by piece or gather them. This PR adds
the planner that decides where to cut: it puts the cuts where they split the fewest and cheapest buffers, keeps earlier
cuts where it can, and adds a segment only when that is cheaper than gathering a large buffer. Nothing calls it yet; the
renderer's segmented imports (#658) do. See #420.
What changed
A header-only planner,
gpu/vk/span_segments.h(namespacemkwvk::span_segments, no Vulkan, no platform calls), and itstest. No renderer file changes.
plan(L, H, cap, g, demands, previous, weights)returns the cut list of least cost for the domain[L, H):every segment at most
cap, cuts on multiples of the granularityg. Each demand ({lo, hi, class, may_write}, classGather, Uniform, Texel or Storage, heaviest first) that a cut lands inside costs its class weight once; a gather costs
its bytes (twice when written back); moving an earlier cut costs
move; each segment beyond the fewest costssegment. Ties go to fewer segments, then the lowest cuts. When the earlier cuts still fit and split nothing, they comeback unchanged (the common case at each admission). The search is a DP over candidate cut positions (each demand's end,
the earlier cuts, and the cap-sized chains down from them); its range of segment counts is derived in the header's
contract comment.
windows()gives each cut's exact feasible range;pieces()andtexel_pieces()map a view onto the segments(texel views in whole elements, split at cuts and at
max_elements, with one-element seams where a cut falls inside anelement; a written view with a seam is refused).
check_cuts()checks a list for the caller's debug invariants.demand, more than 4096 segments (Vulkan's least
maxMemoryAllocationCount), bad cuts, outside the domain, a badelement, a written seam. Window arithmetic is exact up to
UINT64_MAX; costs are a portable 128-bit type.tests/span_segments_test.cppandtests/test_span_segments.py(run_all.pyruns it with--cxx). It comparesevery plan with two independent oracles: a branch-and-bound enumeration of every cut list over the derived range of
segment counts, and a plain DP over every multiple of
gand every count. It sweeps every single demand in 4 KiB stepsat two caps, runs randomized cases at four caps (each again in shuffled order and as a second admission), checks the
real 48 GiB and 512 GiB cases, the top of the
u64range and the 4096 limit, and 20,000 random view mappings.Measurements
Verdict: nothing to measure: the planner has no caller in this PR, and its test (under Testing) passes on both CPUs.
Flags and settings
None.
Testing
Tested: on macOS 26.7.1, Apple M4 Max, Apple clang 21.0.0, at this PR's head before the rebases onto
e090cc48andaa95d35f(6fd02124), on main6c9edffe:tests/test_span_segments.py --cxx /usr/bin/clang++ --arch arm64 --arch x86_64: passes on arm64 and on x86_64 underRosetta, with the same counts, every plan equal to both oracles:
tests/run_all.py span_segments: 1 passed.History, at the topic's own commits (
macos/seg-plan189d0636, the same header, test and driver blobs): the sametest under ASan and UBSan on both CPUs, and 18 planner mutations, each caught; with only the oracle sections, every
search mutation is caught by the oracles themselves.
No file of the game's build changes: of the 3 changed C/C++ files, all new, the header
gpu/vk/span_segments.hisincluded by no source in this PR, and
tests/span_segments_test.cppis compiled only by its test. So no host or rendererbuild changes on any platform, and the host-build box does not apply.
After the rebase onto main
aa95d35f, at52def683(6fd02124before the rebases ontoe090cc48andaa95d35f):Sweep3's checks of this head (the C/C++ it changes compiled, and its own tests run, on arm64 and x86_64 where they
apply) gave the same results as before;
python -I -B tests/run_all.pyon wave B's mergedb0e2e16, which carriesthis head: 569/32/18 (passed/failed/skipped) against main aa95d35f's 543/33/17 on this Mac, no new failure
(
reports/2026-11-15-rebase-sweep-3.md).Not tested: Windows and Linux (no compiler for them here; the header uses no platform call and the test prints its
Rosetta line only under
__APPLE__). No game run: nothing calls the planner in this PR.Must be tested:
52def683) runpython -I -B tests/run_all.py span_segments, expecting1 passedand thespan_segments: ok (683899 plans …)line; the header is in no game buildyet, so nothing else changes against main
aa95d35f(by hand: test request T-EKVM)Risks
What could break: nothing at run time: no renderer file includes the header in this PR. A compiler that rejects the
header or its test would fail
run_all.pyonly.Turning it off: revert the PR.
Compatibility: Nothing changes: no shader stamp, cache format or
MKWVK_API_VERSIONchange.Reviewer notes
411e2885..6fd02124(two commits): the planner, then the priced extra segments (the second commit replaces thefirst one's fixed-count window DP). Start with the contract comment at the top of
span_segments.h.6c9edffe; no dependency. The two topic commits were cherry-picked with-xfrommacos/seg-planwith noconflict (three new files).
Weights' defaults, one place tochange.
git merge-tree).Issues
See #420
Checklist
Everyone
type(scope): description) of the status line's type.main: onaa95d35f(#761, 2026-11-15), head52def683, without a conflict; this PR's owncommits, old → new:
411e2885→601c00cc,6fd02124→52def683; the commit names elsewhere in this descriptionare from before the rebase.
python -I -B tests/run_all.pypasses (any failure that also fails onmainis named). (On the wave's merge of every macOS PR built with this one,453eb678: 553 passed, 32 failed, 18 skipped against main 6c9edffe's 531/32/17 on this Mac; the failures are main's own excepttest_split_ub_flush, which #658's tree at the time broke and its head fixes;test_collect_widenow passes; the new tests pass,test_span_segmentsamong them, andtest_darwin_audio_service, Darwin-only, is skipped by the runner;reports/PubWave-files/waveA/. After the rebase onto mainaa95d35f(head52def683): wave B's mergedb0e2e16, 569/32/18 against main aa95d35f's 543/33/17, no new failure.)third_party/untouched (a new folder with aPROVENANCE.mdinstead).When it applies
Assisted-by:/Co-Authored-By:trailers.6fd02124bcaae09e20f7aae09e20f752def6832cWIP: feat(vk): add a segment planner for guest spans larger than one device allocationto feat(vk): add a segment planner for guest spans larger than one device allocationFolded into #848 (feat(vk): the SPIR-V back end on MoltenVK and guest spans past one device allocation)
Pull request closed