vk: VRAM, system RAM, GPU and replay CPU time: streams and staging out of VRAM, memory given back when idle, redundant barriers left out, draw setup cached #585
Closed
BlAnK182C-bak wants to merge 8 commits from
vram-streams-staging into main
pull from: vram-streams-staging
merge into: MagicBeanDev:main
MagicBeanDev:main
MagicBeanDev:github/pr/2893
MagicBeanDev:github/pr/2890
MagicBeanDev:github/pr/2887
MagicBeanDev:github/pr/2878
MagicBeanDev:github/pr/2875
MagicBeanDev:github/pr/2874
MagicBeanDev:github/pr/2873
MagicBeanDev:github/pr/2871
MagicBeanDev:github/pr/2756
MagicBeanDev:github/pr/2897
MagicBeanDev:github/pr/2896
MagicBeanDev:github/pr/2718
MagicBeanDev:github/pr/2842
MagicBeanDev:github/pr/2691
MagicBeanDev:github/pr/2895
MagicBeanDev:github/pr/2743
MagicBeanDev:github/pr/2765
MagicBeanDev:github/pr/2708
MagicBeanDev:github/pr/2732
MagicBeanDev:github/pr/2755
MagicBeanDev:github/pr/2817
MagicBeanDev:github/pr/2876
MagicBeanDev:release-please--branches--main
MagicBeanDev:release-please--branches--main--release-notes
MagicBeanDev:github/pr/2778
MagicBeanDev:github/pr/2888
MagicBeanDev:github/pr/2866
MagicBeanDev:github/pr/2877
MagicBeanDev:github/pr/2800
MagicBeanDev:github/pr/2829
MagicBeanDev:github/pr/2850
MagicBeanDev:github/pr/2889
MagicBeanDev:github/pr/2882
MagicBeanDev:github/pr/2860
MagicBeanDev:github/pr/2849
MagicBeanDev:github/pr/2886
MagicBeanDev:github/pr/2794
MagicBeanDev:github/pr/2840
MagicBeanDev:github/pr/2830
MagicBeanDev:github/pr/2834
MagicBeanDev:github/pr/2719
MagicBeanDev:github/pr/2742
MagicBeanDev:github/pr/2741
MagicBeanDev:github/pr/2820
MagicBeanDev:github/pr/2771
MagicBeanDev:github/pr/2859
MagicBeanDev:github/pr/2858
MagicBeanDev:github/pr/2823
MagicBeanDev:github/pr/2757
MagicBeanDev:github/pr/2769
MagicBeanDev:github/pr/2844
MagicBeanDev:github/pr/2843
MagicBeanDev:github/pr/2673
MagicBeanDev:github/pr/2855
MagicBeanDev:github/pr/2739
MagicBeanDev:github/pr/2807
MagicBeanDev:github/pr/2851
MagicBeanDev:github/pr/2789
MagicBeanDev:github/pr/2814
MagicBeanDev:github/pr/2818
MagicBeanDev:github/pr/2774
MagicBeanDev:github/pr/2772
MagicBeanDev:github/pr/2750
MagicBeanDev:github/pr/2786
MagicBeanDev:github/pr/2785
MagicBeanDev:github/pr/2782
MagicBeanDev:github/pr/2779
MagicBeanDev:github/pr/2775
MagicBeanDev:github/pr/2706
MagicBeanDev:github/pr/2766
MagicBeanDev:github/pr/2747
MagicBeanDev:github/pr/2759
MagicBeanDev:github/pr/2745
MagicBeanDev:github/pr/2740
MagicBeanDev:github/pr/2733
MagicBeanDev:github/pr/2725
MagicBeanDev:github/pr/2727
MagicBeanDev:github/pr/2728
MagicBeanDev:github/pr/2729
MagicBeanDev:github/pr/2721
MagicBeanDev:github/pr/2722
MagicBeanDev:github/pr/2715
MagicBeanDev:github/pr/2714
MagicBeanDev:github/pr/2707
MagicBeanDev:github/pr/2703
MagicBeanDev:github/pr/2705
MagicBeanDev:github/pr/2697
MagicBeanDev:github/pr/2679
MagicBeanDev:github/pr/2676
MagicBeanDev:github/pr/2669
MagicBeanDev:docs/what-hookshot-connects-to
MagicBeanDev:fix/hdr-encode-test
MagicBeanDev:fix/main-tests
MagicBeanDev:fix/play-cache-files
MagicBeanDev:perf/game-barrier-order
MagicBeanDev:feat/heap-lazy
MagicBeanDev:feat/texture-detail
MagicBeanDev:perf/astc-opaque-bc1
MagicBeanDev:fix/fs-dir-lock
MagicBeanDev:fix/framegen-native-kept
MagicBeanDev:fix/update-no-downgrade
MagicBeanDev:port/k10-gcn-f16-mirrors
MagicBeanDev:fix/framegen-mult-cap
MagicBeanDev:chore/release-as-2.1.1
MagicBeanDev:perf/pcache-load-switch
MagicBeanDev:fix/prepare-progress-log
MagicBeanDev:fix/update-closes-stale-game
MagicBeanDev:fix/hdr-scaled-sampled
MagicBeanDev:perf/first-start-preset
MagicBeanDev:perf/zero-upload
MagicBeanDev:fix/vcore-escape-2s
MagicBeanDev:test/fake-gh-atomic
MagicBeanDev:perf/batch-idle-flush
MagicBeanDev:test/finish-stub-a
MagicBeanDev:chore/record-2.1.0
MagicBeanDev:fix/main-red-tests
MagicBeanDev:fix/test-display-startup-driver-cache
MagicBeanDev:feat/mesh-shader-mode
MagicBeanDev:fix/menu-probes-at-startup
MagicBeanDev:fix/fps-counter-classic
MagicBeanDev:fix/test-log-batch-flake
MagicBeanDev:fix/narrow-screen-hud
MagicBeanDev:ci/prereleases
MagicBeanDev:release/v2.0.15
MagicBeanDev:hotfix/v2.0.15
MagicBeanDev:fix/present-ahead-replay
MagicBeanDev:release/v2.0.13
MagicBeanDev:fix/bracket-paths-2013
MagicBeanDev:fix/hdr-overlays-2013
MagicBeanDev:fix/fps-counter-scale-2013
MagicBeanDev:fix/stick-seen-at-rest
MagicBeanDev:hotfix/menu-fits-screen
MagicBeanDev:fix/f1-always-opens-menu
MagicBeanDev:fix/ini-keep-lines
MagicBeanDev:claude/cranky-pike-7cf247
MagicBeanDev:perf/texture-answer-flat
MagicBeanDev:fix/stamp-test-modules
MagicBeanDev:fix/appimage-fonts
MagicBeanDev:fix/display-awake
MagicBeanDev:release/v2.0.12
MagicBeanDev:fix/discord-windows
MagicBeanDev:rc/v2.0.12-tests
MagicBeanDev:fix/dlss-keep-feature
MagicBeanDev:rc/v2.0.12
MagicBeanDev:button-icons
No reviewers
Labels
Clear labels
accessibility
Barrier affecting people with disabilities
autorelease: pending
autorelease: tagged
bug
Something isn't working
dependencies
Pull requests that update a dependency file
documentation
Improvements or additions to documentation
duplicate
This issue or pull request already exists
enhancement
New feature or request
github_actions
Pull requests that update GitHub Actions code
good first issue
Good for newcomers
help wanted
Extra attention is needed
invalid
This doesn't seem right
question
Further information is requested
testing wanted
wontfix
This will not be worked on
No labels
accessibility
autorelease: pending
autorelease: tagged
bug
dependencies
documentation
duplicate
enhancement
github_actions
good first issue
help wanted
invalid
question
testing wanted
wontfix
Milestone
Clear milestone
No items
No milestone
Assignees
Clear assignees
No assignees
1 participant
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".
No due date set.
Dependencies
No dependencies set
Reference
MagicBeanDev/Hookshot!585
Loading…
Reference in a new issue
No description provided.
Delete branch "vram-streams-staging"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
vk: VRAM, system RAM, GPU and replay CPU time: streams and staging out of VRAM, memory given back when idle, redundant barriers left out, draw setup cached
Refs #519 (4 GB RTX 3050 Laptop:
textures 432 MB (budget 0 MB)with961 MB of it free in the blocks); likely related: #254, #394.This PR started with the first four commits (VRAM held by recording streams and staging; part 1 below). It now also carries a second set (part 2): system memory given back when idle, redundant full barriers left out (the GPU-time win), a GPU pass-timing profiler, and an anisotropy override; and a third (part 3): less CPU per draw on the replay thread. Every behaviour change has an env switch back to the old behaviour.
Part 1: ~1 GB of VRAM back on resizable-BAR cards (commits 1-4)
What was using the VRAM
On an RTX 4050 Laptop (6 GB, the driver allows ~5.7 GB) the renderer sat at the driver's limit (
VRAM: 5668 MB used of 5708 MB allowed), and the texture budget was squeezed to ~2.0-2.2 GB, while textures (2.2 GB) and mirrors (752 MB) explained only ~3 GB of the ~4.8 GB VMA had allocated. The rest:draw_begin_recording->create_buffer(host_visible)staging()memory_guest_pagesWith resizable BAR, NVIDIA exposes a DEVICE_LOCAL | HOST_VISIBLE type inside the VRAM heap, and VMA (
VMA_MEMORY_USAGE_AUTO+HOST_ACCESS_SEQUENTIAL_WRITE, usage beyond TRANSFER) prefers it, so these CPU-written buffers land in VRAM. The streams' measured peak in Kokiri is ~760 KB of their 24 MB.Changes
RX_VK_RECORDINGS_48=1: 48). The 48 came with RX_VK_BATCH for RADV; 24 was the non-batched count before. Stays 48 with batching or RX_VK_PARALLEL > 1.RX_VK_BLOCK_BUFFERS=1: shares of 64 MB blocks), so a 24 MB stream no longer leaves 16 MB of a block empty.RX_VK_STREAM_MB=<n>sets its size. Log:vk: the recording streams: N of M MB, the most one held X KB. (Part 2 makes the streams growable, see below.)RX_VK_STAGING_KEEP_MB=256: as before).Results (part 1)
Kokiri Forest (Link on his house's balcony), 20 s settle, then 30 s standing and 15 s turning the camera. "Before" and "after" differ only by commits 1-4, measured on a local branch carrying other local changes, identical between the two runs.
nvidia-smi)Part 2: memory given back when idle, redundant barriers left out (commits 5-7)
GPU time: redundant full barriers left out (on by default,
RX_VK_BARRIER_DEDUPE_OFF=1: as before)gpu/vk/vk_work.hcounts GPU work recorded on a command buffer (every work-doingvkCmd*goes through its macros; all files that record work include it). A full barrier that directly follows another on the same command buffer with nothing recorded between them is left out;transfer_barriergoes through the same check. Off while parallel replay is on. The work count is afetch_add(two threads can record at once).a frame: N full barriers recorded, M left out (nothing recorded since the one before).System RAM and VRAM given back when idle (each on by default, with its switch)
RX_VK_STREAM_GROW_OFF=1(24 MB, skips),RX_VK_STREAM_KEEP=1RX_VK_CAP_MIN_MB=16,RX_VK_CAP_BLOCKS=1,RX_VK_CAP_SPARE_MB=0RX_VK_BC_IDLE_S=0,RX_VK_BC_POOL=1RX_VK_COUNT_BY_ESTIMATE=1RX_VK_TAAU_HIST_LEAK=1flush_texturereads back into cached host memory, not through the sequential-write bufferRX_VK_FLUSH_UNCACHED=1RX_VK_STAGING_IDLE_S=0RX_VK_SHADOW_SWEEP_OFF=1malloc_trim(0)when glibc's arenas hold 192 MB free; avk: malloc:census line every 1800 presentsRX_VK_MALLOC_TRIM_MB=0RX_VK_DESC_POOL_DIV=<n>,RX_VK_IMAGE_BLOCK_MB=<n>,HOS_NO_THP=1(Linux: transparent huge pages off for the process)tests/test_mem_trim.pychecks the code keeps what these trims must: what is kept as an offset into a stream names its buffer or reserves its room first (a stream may move mid-recording), every user of the BC3 scratch asks for it first, and capture buffers are sized by class and counted out as well as in.Diagnostics (off by default)
RX_VK_PASS_TIMING=1|2(aliasRX_VK_GPU_PROFILE): GPU timestamps around passes, dispatches, barriers (by origin), copies, mirror syncs and write-backs, reported every 600 presents, with the 25 costliest spans.=2adds a timestamp per program run inside passes.RX_VK_ANISO=<n>: samplers with linear min and mip filters get at least n-times anisotropy (default 0: the game's own). The game asks for 0-2x. Avk: sampler anisotropy: the game asks for ...line is logged once per distinct value.Results (part 2)
Same machine and method as part 1, save in Kokiri on Link's balcony, Firefox and Discord closed. Before = a local branch with part 1; after = the same branch plus these commits. Caveat: that local "after" branch also carried three other open PRs' commits (#479, #423, #435), which are not in this PR; with #435 the replay's descriptor time rose ~1 ms (3.35 -> 4.32 ms standing; its bookkeeping found almost nothing to reuse in this game), so the CPU replay figures below understate this PR. The exact branch pushed here has been compiled for Linux x86_64 (all
gpu/vk/vk_*.cpp),tests/test_mem_trim.pypasses, and a run of this exact branch is still to come; I'll post it as a comment.nvidia-smi)smaps_rollup, 10 s into standingnvidia-smi)Where the GPU frame goes after this PR (
RX_VK_PASS_TIMING=1, ~18 ms profiled): render passes 52%, compute dispatches 32%, write-back to guest memory 7% (13 MB a frame), barriers 6%, uploads 8%.Tried and left out
vk: view of texture ... not made) and the lighting goes wrong (dark, posterized, a black title movie); it was also slower (36 vs 43 fps).Part 3: less CPU per draw on the replay thread (commit 8)
The replay thread is on the critical path (about 15 ms of a 22 ms frame here, ~2,400 draws). Four places did per draw what the draw before had already done; now they keep it, keyed by exactly what it depends on. Same descriptors, state, banks and programs come out.
RX_VK_PUSH_STATE_OLD=1RX_VK_DYN_PROG_KEY=1RX_VK_BANK_GEN_OFF=1RX_VK_PICK_BIND_GEN=1RX_VK_PROF_FINE=1(off by default): the draw's time split into finer sections, and what each draw doestests/test_push_state.py,test_bank_gen.py,test_pick_tex_gen.pyandtest_same_dyn.pycheck the invariants.Program::wm_sigis atomic (replay threads may ask for the same program at once).Measured so far only as part of a local build that carries these changes with others: the replay thread's time a frame went from ~16.1 ms to ~12.7 ms standing (and fps 43.4 -> 47.8, with the 3D scene at a higher resolution), but that comparison is not isolated (other changes and a different save). An isolated A/B of this exact change (same build, the four switches off vs on) is next; I'll post it as a comment. This branch's commit compiles for Linux x86_64 (all
gpu/vk/vk_*.cpp) and its tests pass; it has not been run as this exact branch yet.Tested on
__NV_PRIME_RENDER_OFFLOAD=1 __VK_LAYER_NV_optimus=NVIDIA_only)HOS_INPUT_LIVE), the game's 120-frame stat lines,nvidia-smievery 2 s,/proc/<pid>/smaps_rollup, andRX_VK_FRAMELOG+scripts/framelog.pyfor frame times and flashes.nx2/make_game.py, Linux). This branch is onmain7e9bf73; every changedgpu/vkfile compiles for x86_64 Linux.malloc_trimandHOS_NO_THP(both under#ifdef __linux__/glibc), but I have not built or run it elsewhere.🤖 Generated with Claude Code
Fix conflicts please
@shayderrr rebased and resolved conflicts
Hi, @NotRexed was just curious as to why this PR was closed