WIP: perf(vk): Stage 0 of the memory-sync redesign: the ceiling report, a content oracle for the write marks, and four unmarked writes fixed #1034
No reviewers
Labels
No labels
area/accessibility
area/audio
area/input
area/install
area/menu
area/renderer
area/saves
area/tooling
area/upscaling
beanstalk: approved
beanstalk: failed
beanstalk: needs approval
beanstalk: running
beanstalk: tested
ci: approved
good first issue
gpu/amd
gpu/igpu
gpu/intel
gpu/nvidia
help wanted
kind
bug
kind
chore
kind
docs
kind
feature
kind
performance
kind
question
needs measurements
platform/linux
platform/macos
platform/steam-deck
platform/windows
preview
priority
critical
priority
high
priority
low
regression
risk
high
risk
low
risk
medium
status
blocked
status
confirmed
status
duplicate
status
invalid
status
needs-info
status
wontfix
ci: approved
No milestone
No project
No assignees
1 participant
Notifications
Due date
No due date set.
Dependencies
No dependencies set
Reference
MagicBeanDev/Longshot!1034
Loading…
Reference in a new issue
No description provided.
Delete branch "TrashPanda/Longshot:perf/stage0-ceiling-oracle"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Type: perf · Risk: low · Tested on: Linux, RTX 4070 Ti · Needs: none (measurement only; the host fixes are guest-memory marks)
Why
To make the renderer faster we first need to know how far it is from what the game itself allows on a PC, and a way to catch a renderer change that loses a CPU write. Neither existed:
scripts/chain.pyshowed the frame's critical path but couldn't tell the game's own work from the renderer's, and nothing checked that every CPU write to guest memory the GPU reads is marked. This is Stage 0 of a staged renderer memory-sync redesign. Each later stage is judged by these numbers.What changed
Three measurements, all off unless asked for, plus four guest writes the new check found unmarked.
scripts/chain.py --log): the renderer brackets its work in spans (MkwVkHost.chain_span,MKWVK_SPAN_*ingpu/vk/vk_api.h: the replay, a submit's memory work, the present, and its waits for the GPU) recorded byHOS_CHAIN. The report splits each thread's running time on the frame's path into the game's own and the renderer's, takes the GPU's time from the same run'svk: GPU timelines, and prints the frame time over the ceiling (the larger of the GPU's time and the game's own path; 1.0 means nothing of the port's is above it). One load per span when nothing records.RX_VK_MARK_AUDIT=N,gpu/vk/vk_memory.cpp): every Nth submit it copies a window of the synced pages (mirrored pools, texture storage, the capture ranges) and, a submit later, compares them with guest memory. A page that changed while its mark is still clear is a missed write, logged with its address.scripts/memtour.pysessions can be compared with it.host/nvn_native.c,host/hos_mem.c,host/hos_svc.c):nvnCommandBufferEndRecording's copy mark in control memory,nvnQueueSubmitCommands' submit counter, and the zeroing inFreeMemoryPages/discard_pagesandUnmapPhysicalMemory. Each lay in a mirrored pool, so a capture could keep the page's old bytes and a write-back could put them back.tests/test_collect_concurrent.py(collect_infinds every mark set before it starts and loses none set while it runs, with threads marking at the same time),tests/test_chain.py(the spans and the report).MKWVK_API_VERSION25 to 26 (two members appended after #1102'smic_test);tests/test_audio_output_devices.pynow checks that the playback and microphone callbacks followoptions_path, rather than that they're the struct's last seven members, and it andtests/test_microphone_test_gain.pyaccept any version from 25 on.Measurements
Verdict: On this PC both scenes run at 1.26 times the ceiling, which is the GPU here. The renderer adds 1.5-1.7 ms to the frame's path, about 1.1-1.3 ms of it waiting for the GPU on the main thread. Recording costs nothing measurable: 120.1 fps recording against 120.1 not (Kokiri Forest), 114.5 against 112.9 (Hyrule Field). The oracle finds no missed write after the fixes, and the renderer's memory is bounded over a 25-minute tour except for retired pool records (Stage 1).
Hardware: Ubuntu 26.04.1 LTS (kernel 7.0.0-43), RTX 4070 Ti 12 GB, NVIDIA 595.99.02, Ryzen 7 9800X3D, 32 GB
Scene: Kokiri Forest (teleport 0) and Hyrule Field (teleport 1) from the repo save
kokiri-start, the camera turning; 1920x1080, no upscaler,HOS_FPS_MAX=240, headless (HOS_NO_SDL=1)Method: a 20 s window with
HOS_CHAINand a 20 s window without in the same run, fps fromHOS_PRESENT_TRACE;python -I scripts/chain.py <chain> --log <log>for the reportHow to reproduce:
HOS_CHAIN=<file>and touch<file>.gofor the window, then the command aboveBaseline: main has neither the spans nor the oracle, so there is no main column. The first table is this PR's build with
HOS_CHAINoff (Before) and on (After): what recording costs. The build is37085a07, this branch's first commit before its rebases, with uncommitted Stage 0 changes (-dirtyin thedocs/tested/rows); the ceiling table below is from the same runs.The ceiling report, the same runs:
The content oracle (
RX_VK_MARK_AUDIT), the same scenes:+0x1c8, the queue's submit counter, freed texture storage zeroed)The renderer's memory over a session:
scripts/memtour.pythrough Zora's Domain, Hyrule Field, Kakariko, the Market, Death Mountain Trail and Goron City, 1 lap and 2 areas (an event at Goron City then refused the teleports), 1,321 memory lines ond66b4ac6(this branch before its rebase). Captures 70-276 MB, spares 2-32 MB, mirrors up to 1,223 MB of a 2,048 MB budget, live imports flat at 100-137 (about 1.8-2.1 GB) after the first areas, shadows 124-384 MB, stamps 0-7 MB, VRAM 5.4-5.6 GB of 5.65. Only the retired pool records grow: 0 to 159, about 10 per area change, though all of their imports are given back. The process's private memory rose about 1 GB during the first lap, as each new area loaded, and was flat on the second.Flags and settings
One new switch and its window size, off by default (testing only);
HOS_CHAINgains the spans.RX_VK_MARK_AUDIT=NRX_VK_MARK_AUDIT_MB=NRX_VK_MARK_AUDITTesting
Tested:
python -I -B tests/run_all.py --jobs 6on Linux at the head of the stacked #1037 (this PR plus four commits): 649 passed, 24 failed, 21 skipped. All 24 also fail onmainac778751on this PC (25 there:test_fsr_uniform_ownershipfails on main and passes here, with this PR's change to it): the import checkstest_guest_imported,test_pending_guest_importandtest_import_reach;test_replay_write_order,test_walk_key,test_run_all_changed,test_make_game_cacheandtest_bug_report_defaults; and tests this PC can't run (thelz4andnumpyPython modules missing, Windows-only fixtures, a Vulkan 1.3 offscreen device). The renderer (RX_VK_ONLY=1 nx2/build.py --standalone, llvm-mingw 20261104) cross-builds and links for Windows, and the four changed host files (host/rx_backend.c,host/hos_mem.c,host/hos_svc.c,host/nvn_native.c) compile with llvm-mingw for Windows and with clang for Linux (-fsyntax-onlywith the build's flags and a local lift's headers). No path handling changed. In the game: the scenes above on Linux with an RTX 4070 Ti, with and withoutHOS_CHAIN, withRX_VK_MARK_AUDIT=1, and the 25-minute memory tour, all with no error in the logs.Not tested: Windows in the game, and a full Windows link of the game (the host files were compiled for Windows, not linked into the game). That code is portable C, and the host fixes are one
HOS_WRITTENeach.Must be tested: None. The new switch is off by default, and the host fixes only add marks the renderer already handles.
Risks
What could break: the four new marks make the renderer copy those pages again at the next submit: a few bytes a submit, and a freed or unmapped range once.
Turning it off:
RX_VK_MARK_AUDITunset (the default); revert for the marks.Compatibility:
MKWVK_API_VERSION26, so the game and its renderer must come from the same build (as always). Nothing else changes.Checklist
Everyone
type(scope): description) of the status line's type.main.python -I -B tests/run_all.pypasses (any failure that also fails onmainis named).third_party/untouched (a new folder with aPROVENANCE.mdinstead).When it applies
docs/tested/for each new measured run (format indocs/tested-configurations.md).MKWVK_API_VERSIONbumped (gpu/vk/vk_api.h'sMkwVkHostorMkwVkRendererchanged).tests/that reads changed source updated in this PR, not weakened.Tested in game: Linux (Ubuntu 26.04.1, kernel 7.0.0-43), RTX 4070 Ti, NVIDIA 595.99.02, Ryzen 7 9800X3D: Kokiri Forest and Hyrule Field with and without HOS_CHAIN (120.1/120.1 and 114.5/112.9 fps, frame over ceiling 1.26 in both), RX_VK_MARK_AUDIT=1 in both (0 missed writes after the host fixes, of 16.5 M and 30.2 M pages compared), and a 25-minute memtour.py session through six areas; no errors in the logs.
5b156bfdda34c15dcc6f34c15dcc6f52a3547745Rebased onto main @
71b1edd8(head52a35477). Main's #1102 tookMKWVK_API_VERSION25 formic_test, so this PR's twochain_span/chain_onmembers now followmic_testas version 26.tests/test_audio_output_devices.pychecks that the playback and microphone callbacks followoptions_pathwherever later versions append, and it andtests/test_microphone_test_gain.pyaccept any version from 25 on, so the next bump doesn't break them. The description says 26 now.python -I -B tests/run_all.pyon52a35477(Linux): 635 passed, 22 failed, 21 skipped. All 22 fail the same way on main @71b1edd8on this PC (lift prerequisites and Windows-only fixtures), none from this branch.52a35477457046f6da36Rebased onto main @
ac778751(head7046f6da; was52a35477).Conflict: one hunk in
gpu/vk/vk_memory.cpp'stake_capture. Main's #977 (1b2e7555, early reads) replaced the late pages' wait forc->late_valuewithif (wait) { vk_timeline_wait(wait); ... }, wherelate_defer_splitleaves some late pages to their reads. This PR had wrapped the old wait inChainSpan span(MKWVK_SPAN_LATE_WAIT). Resolved by keeping main's block whole and wrapping itsvk_timeline_wait(wait)in the same span. The new wait #977 adds inlate_defer_cpu_read(a read reached frommemory_read_as_submitted) is left unspanned, like thelate_fixwaits on that path were already.MKWVK_API_VERSIONstays 26: main is still at 25. The other three commits applied as they were.Tests: the touched checks (
test_collect_concurrent,test_chain,test_audio_output_devices,test_microphone_test_gain,test_retired_give_backand every check that readsvk_memory.cpp) pass, except the import and write-order checks that also fail on main.python -I -B tests/run_all.py --jobs 6at the stacked #1037's headc328a0c4: 649 passed, 24 failed, 21 skipped. All 24 also fail on mainac778751on this PC (25 there;test_fsr_uniform_ownershippasses only with this PR). The Windows renderer cross-builds and links with llvm-mingw 20261104. The four changed host files compile for Windows (llvm-mingw) and Linux (clang).Description: the Measurements table now names its commit: recording off against on, both on build 37085a07, with a Baseline line explaining why there's no main column. Also added
RX_VK_MARK_AUDIT_MBto Flags and the host-code box, and refreshed the test results to this run.View command line instructions
Checkout
From your project repository, check out a new branch and test the changes.