WIP: perf(vk): the host's question how far the replay is answered from the timeline as last read (RX_VK_REPLAYED_CACHE=0 to disable) #1994
No reviewers
Labels
No labels
area/accessibility
area/audio
area/input
area/install
area/menu
area/renderer
area/saves
area/tooling
area/upscaling
beanstalk: approved
beanstalk: failed
beanstalk: needs approval
beanstalk: running
beanstalk: tested
ci: approved
good first issue
gpu/amd
gpu/igpu
gpu/intel
gpu/nvidia
help wanted
kind
bug
kind
chore
kind
docs
kind
feature
kind
performance
kind
question
needs measurements
platform/linux
platform/macos
platform/steam-deck
platform/windows
preview
priority
critical
priority
high
priority
low
regression
risk
high
risk
low
risk
medium
status
blocked
status
confirmed
status
duplicate
status
invalid
status
needs-info
status
wontfix
ci: approved
No milestone
No project
No assignees
3 participants
Notifications
Due date
No due date set.
Dependencies
No dependencies set
Reference
MagicBeanDev/Longshot!1994
Loading…
Reference in a new issue
No description provided.
Delete branch "siantm/Longshot:perf/vk-replayed-cache"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Summary
The host asks how far the replay is (
get_replayed) under its recordings' lock, held exclusively, at everynvnCommandBufferAddControlMemory(free_dead_copiesinhost/nvn_native.c), to let go of the copies the GPU is past,and every answer was a
vkGetSemaphoreCounterValue.With this,
completed()keeps the highest value it has read successfully (it only goes up), andget_replayedanswersfrom it, reading the driver at the first call and every 64th call after, so that it moves when nothing replays. While
the bench holds the answer (
replayed_hold) the driver is read at every call, and those calls are not counted. A readthat fails keeps nothing and answers 0 (nothing taken as done); one that comes back
VK_ERROR_DEVICE_LOSTgoes todevice_lost(), as the GPU hang watch's read does. The result expected: far fewer driver calls under the host's lock.The cost: an answer lower than the GPU keeps a copy the GPU is past until a later answer passes it, up to 63 calls when
nothing else reads the timeline, however long those calls take to come; the copies kept meanwhile go together, in one
pass under the lock. So the host's report every 50,000 recordings (
nvn:in the log) now also says the bytes of thecopies let go and waiting for the replay, their peak, and the passes that let them go with their time under the copies'
lock, since the last report: what a run needs to tell a shorter lock from larger, rarer passes.
The switch:
RX_VK_REPLAYED_CACHE(new, a kill switch, bool, its row ingpu/vk/switches.def), on by default as thebranch shipped it, read and logged once by
replay_start, before the timeline is made and the renderer is given to thehost; empty,
0,off,falseornoread the driver at every call and keep no last-read value, as main does.From bd3ea307 on
exp/x86-asm-sync(its replay-parts work).Read first:
replayed_cache_on,completedandget_replayedingpu/vk/vk_replay.cpp,free_dead_copiesand thereport in
host/nvn_native.c, thentests/test_replayed_cache.py.Testing
Tested: Windows 11, i7-9700, RTX 4060, the builder's llvm-mingw clang 23; head
d8a583a6on mainf1f2dc5b. Buildjobs of this PC's job queue (
fb-1994-1,fb-1995-1,fb-1995-2), the checks in a hidden console:tests/test_replayed_cache.pyfails at the test commit
4b3e94e7(the failed read kept, the lost device not reported, the off switch keeping thevalue) and passes at the head;
tests/run_all.py --strict --changed origin/mainat the head (the CPM cache's Vulkan headers): 169 checks, 135 passed, 29 failed, 5 skipped (Linux only or a device); the 29 fail on main @f1f2dc5btoo, run from a main worktree in the same job:tests/test_capture_readback_visibility.py,tests/test_crash_dialog.py,tests/test_crash_record.py,tests/test_dev_ab.py,tests/test_dev_packs.py,tests/test_device_lost_exit.py,tests/test_draw_stream_visibility.py,tests/test_flush_nothing_held.py,tests/test_frame_overlap.py,tests/test_fsr_image_retirement.py,tests/test_fsr_uniform_ownership.py,tests/test_gpu_wait_stats.py,tests/test_host_guest_writes.py,tests/test_linux_pressure.py,tests/test_mesh_conv.py,tests/test_perf_report.py,tests/test_present_queue_default.py,tests/test_present_source_crop.py,tests/test_profiling_mode.py,tests/test_recorder_own_idle.py,tests/test_renderer_new_handler.py,tests/test_replay_write_order.py,tests/test_slow_frames.py,tests/test_sparse_mirror.py,tests/test_target_reuse_memory.py,tests/test_ub_update_batch.py,tests/test_upscaler_rows.py,tests/test_vk_occlusion_queries.py,tests/test_walk_key.py;gpu/vk/vk_replay.cppandhost/nvn_native.ccompiled (syntax) with the gamebuild's own commands.
It has NOT run in the game on this PC yet.
Still to test:
scene change that recycles recordings (into and out of a shrine): main @
f1f2dc5b, this branch withRX_VK_REPLAYED_CACHE=0and this branch with it unset (on), each warmed up a minute in the scene first, interleaved(ABBA), at least two runs each, expecting the same picture, no FATAL, device lost or replay error; report average FPS,
1% lows, median and p99 frame times, the
nvn:report's copies waiting (MB, peak) and let-go passes (count, time,longest) and the process's peak RAM; a gain is median and p99 frame time unset lower than with
RX_VK_REPLAYED_CACHE=0and than main, with no rise in the peak of copies waiting or in the longest pass,and
RX_VK_REPLAYED_CACHE=0the same as main (by hand)f1f2dc5b, this branch withRX_VK_REPLAYED_CACHE=0and with it unset (on), warmed up, ABBA, expecting the same pictureand no FATAL, device lost or replay error, and the same figures as above (by hand or on the farm, not run yet)
python3 -I -B tests/test_replayed_cache.py, expecting PASS (by hand)Measurements
Verdict: not measured: no micro-benchmark of this port and no run in the game on this PC yet; the baseline is main
@
f1f2dc5b, and the in-game A/B is in Still to test.RX_VK_REPLAYED_CACHEships on, as the branch shipped it; nomeasurement chose that default yet, the in-game A/B in Still to test does.
Risks
What could break: an answer above what the GPU reached would free a copy it still reads (a wrong draw or a device
lost); the answer is never above a value the driver returned from a read that succeeded. At worst it is lower: a copy
the GPU is past stays up to 63 calls longer when nothing else reads the timeline (longer in wall time when the calls are
sparse), so more memory is held between passes and a pass under the copies' lock lets more go at once. The test runs
the real
completed(),get_replayed()and the host'sfree_dead_copies()against a stand-in driver. Turn it off withRX_VK_REPLAYED_CACHE=0. Renderer and host code: the runtime is rebuilt; nothing changes for players' saves, options orcaches.
Checklist
Ready for review
main; the title is a Conventional Commit.mise run testpasses (any failure that also fails onmainis named).main, or Measurements says why there is nothing to measure.third_party/untouched; nothing GPL-incompatible.When it applies
gpu/vk/switches.def.tests/that reads changed source is updated, not weakened.Assisted-by:/Co-Authored-By:trailers.Ready to merge
main, measured again or a comment says why the numbers still hold.Possibly related, for context:
Recommendation: Request changes. Publish only successful timeline queries and make the kill switch disable cache updates as well as cache reads. The fixture shows fewer driver calls, but the player benefit and the cost of delayed reclamation are still unmeasured.
The host usually reuses a recent GPU completion value while holding its recording-copy lock. That can shorten the lock hold, but it can also delay returning dead copies to the pool and concentrate reclamation into fewer passes. The new cache also adds atomic work to completion queries elsewhere in the renderer.
@fixwill do (11)gpu/vk/vk_replay.cpp:284: The new cache publishes the output without checking VkResult. (more on the line)gpu/vk/vk_replay.cpp:285: RX_VK_REPLAYED_CACHE=0 restores the driver's query frequency but leaves cache maintenance active in every completed() caller. (more on the line)gpu/vk/vk_replay.cpp:3053: The first reclamation query can log while the host holds copy_lock exclusively. (more on the line)tests/test_replayed_cache.py:27: The test's compiler fallback can be bypassed in both directions. (more on the line)tests/test_replayed_cache.py:128: The read-once check ignores failure from _putenv_s and setenv. (more on the line)tests/test_replayed_cache.py:138: ATTEMPT_OFF and the stubs parameter are unused, and the test compiles separate binaries to drive one switch through a stub and through the real helper. (more on the line)gpu/vk/switches.def:640: The row omits the empty value, which disables the cache through env_on_or, and should describe the cadence as the first eligible call and every 64 eligible calls thereafter. (more on the line)gpu/vk/vk_replay.cpp:3070: Record peak pending dead-copy bytes and reclamation time under copy_lock so the run can distinguish faster polling from larger reclamation bursts.For the author (2) (a run, a measurement or a decision)
gpu/vk/vk_replay.cpp:3070: The planned median and live-copy count cannot establish the gain or show its cost. (more on the line)Checks: Still running:
PR description / Description follows the template (pull_request_target),CI / Conventional Commit title (pull_request),CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request),CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request),CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request),CI / Workflow lint (actionlint) (pull_request),CI / Website build and checks (pull_request),Guide screenshots / Guide images and local UI source review (pull_request),Third-party parts / As their notes say (pull_request),Third-party parts / The same as upstream (pull_request).More: does it help, other systems, switches, what was read
Does it help: Unproven. The author reports that the Windows fixture passed. It checks that 63 of 64 eligible calls skip the driver when no other path refreshes the cache. The commit cites 117 calls per frame in a Hyrule Field profile from the development branch. There are no matched in-game results for main, head with the switch off, and head with it on.
Other systems: The cache applies across Vulkan drivers and recorder modes. Batch flush polling still runs on every host query, including on RADV and Steam Deck, and a sampled benchmark hold still caps the answer. Sync-wait debug logs can now print stale progress. Waits, readbacks, screenshots, and renderer resource retirement still query the driver directly, but their queries can update the shared atomic. Delayed host-copy reuse can raise RAM use and bunch reclamation work under the lock. The diff touches no save, audio, input, shader-cache format, or renderer ABI code.
Switches: RX_VK_REPLAYED_CACHE is a new developer kill switch with a sorted table row. It defaults on when unset and is read once. Empty, 0, off, false, and no disable cached answers. Cache updates currently remain active with it off. The on default awaits matched results on Tier 1 hardware.
Reviewed:
ab3a2544a1against75317e1f31: Read tests/test_replayed_cache.py in full, the complete diff and commit, and every changed section of gpu/vk/vk_replay.cpp and gpu/vk/switches.def. Traced the host callers, copy pool, recorder dispatch, batch flush, benchmark hold, logging, timeline lifetime, and test runner. Read AGENTS.md, CONTRIBUTING.md, the PR template, the standards, and the relevant architecture and switch rules. Skimmed unrelated recorded runs and source-check references. The rest of the replay file and switch table were outside the changed sections. There is no local review-checklist file or sibling Tooling checkout. No translations or generated files changed.; nothing run (a code review).Checked and fine:
Testing: Read-only review. No builds or tests were run. The author reports 131 of 158 Windows checks passed, including the new fixture, and all 27 failures reproduced on the base. Linux renderer and fixture results, in-game results, and measurements are pending. The supplied CI checks were still running and listed no head-only failures.
Overlaps and merge order: No overlapping pull request was identified from the supplied record or local history. The local main change for #1948 touches none of this PR's three files.
Automated code review by BeanBot (gpt-6.2-sol), on the pull request's opening. Nothing was run; a maintainer decides.
@ -637,6 +637,7 @@ SW(RX_VK_REFLEX, player, enum, "the menu's NVIDIA Reflex row", "Sets NVIDIA ReflSW(RX_VK_REFLEX_LOG, diag, bool, "off", "Logs Reflex's latency timings every 300 frames.")SW(RX_VK_REJECT_KEYS, diag, string, "empty: no program rejected", "Comma list of program key prefixes taken as untranslatable, for bisecting a bad translation.")SW(RX_VK_REJECT_STAGES, diag, string, "empty: no stage rejected", "Comma list of NVN stage numbers whose programs are taken as untranslatable, for bisecting a bad translation.")SW(RX_VK_REPLAYED_CACHE, kill, bool, "on (unset): the host's question how far the replay is answered from the timeline as last read, read again every 64th call", "0, off, false or no: the timeline read from the driver at every call, as before. Read once, at first use (the log says which).")L8 (minor): The row omits the empty value, which disables the cache through env_on_or, and should describe the cadence as the first eligible call and every 64 eligible calls thereafter. Calls during a benchmark hold always query the driver and do not advance that cadence.
@ -279,3 +281,4 @@std::atomic<uint64_t> completed_seen{0};uint64_t completed() {uint64_t v = 0;vkGetSemaphoreCounterValue(dev.device, timeline, &v);L1: The new cache publishes the output without checking VkResult. If a driver writes to the output and returns an error, that value can become the permanent high-water mark and a later cache hit can let free_dead_copies recycle a recording without confirmed completion. Publish only after VK_SUCCESS, follow the renderer's device-loss handling for VK_ERROR_DEVICE_LOST, and add a fixture case where a failed query writes a larger value.
@ -279,3 +282,4 @@uint64_t completed() {uint64_t v = 0;vkGetSemaphoreCounterValue(dev.device, timeline, &v);uint64_t was = completed_seen.load(std::memory_order_relaxed);L2: RX_VK_REPLAYED_CACHE=0 restores the driver's query frequency but leaves cache maintenance active in every completed() caller. A readback or present on the off build can still write the shared cache line as the GPU advances, so off retains the work the switch is meant to bypass. Initialize the switch before replay starts, gate publication on it, and assert that the off fixture leaves the cache untouched.
@ -3044,0 +3050,4 @@bool replayed_cache_on() {static const bool v = [] {const bool on = env_on_or("RX_VK_REPLAYED_CACHE", 1);logf("vk: the host's question how far the replay is answered from the timeline as last read: %s (RX_VK_REPLAYED_CACHE=0: "L3 (minor): The first reclamation query can log while the host holds copy_lock exclusively. Linux writes to stderr synchronously, as does Windows with HOS_LOG_SYNC=1. A slow log sink can then stall other recording threads. Read and log the switch during renderer startup.
@ -3047,0 +3067,4 @@// the bench holds it. The saving is not measured yet. RX_VK_REPLAYED_CACHE=0: the driver asked at every call, as beforeif (!hold && replayed_cache_on()) {static std::atomic<uint32_t> asked{0};if (asked.fetch_add(1, std::memory_order_relaxed) & 63) return completed_seen.load(std::memory_order_relaxed);L10 (minor): The planned median and live-copy count cannot establish the gain or show its cost. That count excludes dead copies waiting for reuse. Compare main with head off and on in repeated, interleaved runs at matched settings. Report driver calls and time under copy_lock, average FPS, 1% lows, median and p99 frame times, and peak dead-copy bytes and process RAM. Include a heavy scene and the affected Tier 1 Windows, Linux, and Steam Deck configurations before settling the default.
@ -0,0 +24,4 @@from srccut import block # noqa: E402import srccut # noqa: E402RUN_ALL_ARGS = ["--cxx", "{cxx}"]L4: The test's compiler fallback can be bypassed in both directions. On Windows with llvm-mingw configured but off PATH, run_all resolves {cxx} before starting the script and can skip or fail the check before default_cxx runs. On Linux, a direct run with LONGSHOT_MINGW set for Windows cross-builds can pick that compiler and try to execute a Windows binary. Use native-platform compiler selection for both entry points, honoring an explicit --cxx or CXX.
@ -0,0 +125,4 @@int main(int, char** argv) {const long long first = (long long)(@CALL@);#ifdef _WIN32_putenv_s(argv[1], argv[2]);L5 (minor): The read-once check ignores failure from _putenv_s and setenv. If the update fails, an implementation that rereads the environment can pass because it sees the original value twice. Check the update result and verify that getenv sees the new value before the second call.
@ -0,0 +135,4 @@}"""KILL_ON = ((None, 1), ("0", 0), ("1", 1), ("2", 1), ("off", 0)) # (unset: on; 0 or off: off; any other value: on)ATTEMPT_OFF = ((None, 0), ("0", 0), ("1", 1), ("2", 1), ("off", 0)) # (unset: off)L6 (minor): ATTEMPT_OFF and the stubs parameter are unused, and the test compiles separate binaries to drive one switch through a stub and through the real helper. Use the real helper in a single fixture with fresh subprocesses for the on and off cases, and remove the unused scaffolding.
ab3a2544a1d8a583a60cThanks for the review (#issuecomment-62985). Pushed: the branch is rebased onto main
f1f2dc5b(the PR's own commit unchanged,becb9c7b), with a test commit4b3e94e7and a fixd8a583a6.tests/test_replayed_cache.pyfails at4b3e94e7and passes atd8a583a6(this PC's job queue, jobsfb-1994-1,fb-1995-1andfb-1995-2); the renderer'svk_replay.cppand the host'snvn_native.ccompile with the game build's own commands. Not run in the game yet.d8a583a6:completed()keeps only a read that returnedVK_SUCCESS; a failed read keeps nothing and answers 0 (nothing taken as done), andVK_ERROR_DEVICE_LOSTgoes todevice_lost(), as the GPU hang watch's read does. The fixture has a read that fails after writing 1000 (nothing kept, no answer above the last good read, atcompleted()and atget_replayed's own read) and a lost device.d8a583a6: the switch is read byreplay_startinto an atomic before the timeline is made; with it offcompleted()keeps nothing. The off cases assertcompleted_seenstays 0 after 128 calls.d8a583a6: read and logged once at the renderer's start, not under the host's copy lock.4b3e94e7:--cxx, thenCXX, thenLONGSHOT_MINGW's clang++ on Windows only, then clang++, g++ or c++ on PATH;RUN_ALL_ARGSis gone, so run_all no longer resolves a compiler for it.4b3e94e7: the flip's_putenv_s/setenvresult is checked andgetenvmust return the new value.4b3e94e7: one fixture with the realenv_on_or, each case (unset, empty, 0, 1, 2, off, no, twice) in a fresh process; the unused scaffolding is gone.get_replayed(d8a583a6): up to 63 calls when nothing else reads the timeline, longer in wall time when calls are sparse, and the copies kept meanwhile let go together.d8a583a6: the row names empty as off, the cadence (the first call and every 64th after, calls under the bench's hold not counted) and the delayed let-go.f1f2dc5b; Measurements names that baseline.d8a583a6: the host's report every 50,000 recordings also logs the bytes of copies waiting for the replay, their peak, and the let-go passes with their time under the copies' lock (the fixture checks the counters). The in-game A/B with those figures, FPS, 1% lows, median/p99 frame times and RAM on Tier 1 hardware is pending, in Still to test.4b3e94e7(the host's realfree_dead_copies: a copy the GPU is past stays on a stale answer and goes at the next read). The own-PC run with the cache on and off through a scene change is pending, in Still to test.@Beanstalk This adds RX_VK_REPLAYED_CACHE, a kill switch, on by default: the host's question how far the replay is (asked under the recordings' lock at every control memory add) is answered from the timeline value last read, reading the driver at every 64th call (gpu/vk, host/nvn_native.c); RX_VK_REPLAYED_CACHE=0 reads the driver at every call, as main does. It has not run in the game on these PCs yet. Please play Hyrule Field for three minutes, running and turning the camera, after a minute's warm-up, the PR against main on the same PC, on the PR's side with the switch unset and with RX_VK_REPLAYED_CACHE=0, on Linux with NVIDIA, Linux with AMD (RADV), Windows with Intel and the Steam Deck. The same picture, no crash, no device lost and no FATAL line; unset, frame time p99 equal to main's or better and to the switch off; with it off, as main. Please report the peak copies waiting in the log's nvn: report line on each side.
@siantm only members of
BigBeanscan start the farm. One of them can approve this request by reacting 👍 to it or to this comment.View command line instructions
Checkout
From your project repository, check out a new branch and test the changes.