perf(vk): VRAM mirrors on GCN too (RX 400/500, Vega): an RX 580 runs 28 to 54% faster with them #1187
No reviewers
Labels
No labels
area/accessibility
area/audio
area/input
area/install
area/menu
area/renderer
area/saves
area/tooling
area/upscaling
beanstalk: approved
beanstalk: failed
beanstalk: needs approval
beanstalk: running
beanstalk: tested
ci: approved
good first issue
gpu/amd
gpu/igpu
gpu/intel
gpu/nvidia
help wanted
kind
bug
kind
chore
kind
docs
kind
feature
kind
performance
kind
question
needs measurements
platform/linux
platform/macos
platform/steam-deck
platform/windows
preview
priority
critical
priority
high
priority
low
regression
risk
high
risk
low
risk
medium
status
blocked
status
confirmed
status
duplicate
status
invalid
status
needs-info
status
wontfix
ci: approved
No milestone
No project
No assignees
1 participant
Notifications
Due date
No due date set.
Dependencies
No dependencies set
Reference
MagicBeanDev/Longshot!1187
Loading…
Reference in a new issue
No description provided.
Delete branch "HumanBean/Longshot:perf/vram-mirrors-gcn"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Type: perf · Risk: medium · Tested on: Linux, RX 580 8 GB (RADV) · Needs: more GCN cards (a 4 GB RX 470/570, Vega), and a GCN PC with 12+ threads
Why
The VRAM mirrors are off by default on GCN cards (RX 400/500, Vega). In k10 (
1f8c02f9), an RX 580 ran Kokiri Forest at 20 fps with them and 24.5 without. On today's renderer that has reversed. On the same kind of card, an RX 580 8 GB, main withRX_VK_MIRROR=1runs 28 to 54% faster than main's default in every area. The rule now costs GCN players most of that.What changed
mirror_on()ingpu/vk/vk_memory.cppgives every discrete card its mirrors. The GCN test (sg32_stages == 0 && subgroup_size > 32) and its(GCN)log note are gone.RX_VK_MIRROR=0still turns them off.docs/development.md's Threaded rendering row no longer lists GCN among the PCs whose pools have no mirrors.Threaded rendering's Auto turns the render thread on only where the pools have mirrors (
pools_mirrored(),vk_menu.cpp). So on a GCN PC with 12 or more logical processors, Auto now picks On. An RX 580 PC with 8 threads keeps Off. The measurements below include that mode forced on the RX 580.Measurements
Verdict: faster on the RX 580 in every phase. fps is +28 to +54% (Kokiri 20.9 → 29.6, Hyrule Field 27.5 → 42.3 standing, the Market 24.2 → 31.6), and GPU time is 2 to 8 ms lower a frame. This is from one session of main with
RX_VK_MIRROR=1, which behaves exactly like this change, against one of main's default. Interleaved runs of this PR's own build are in progress and will be added before this leaves WIP.Hardware: Linux, AMD Radeon RX 580 8 GB (Polaris10, RADV), 8 threads, KDE
Scene: Kokiri Forest, Hyrule Field, the Castle Town Market (debug teleport), Hyrule Field again; each standing then turning the camera
Method: one game session per arm, the pristine Kokiri save; fps from the frame log's median frame time; GPU time the renderer's per-frame median
How to reproduce: run the lab bench's route with
RX_VK_MIRROR=1against unset on main, or this PR againstRX_VK_MIRROR=0Baseline: main
c167ec67(the lab's base), its own build, withRX_VK_MIRRORunset (mirrors off on GCN) and=1c167ec67c167ec67withRX_VK_MIRROR=1(this PR's default)The same lab round had main with both
RX_VK_MIRROR=1andRX_VK_WB_MAX_KB=1024, and it agreed: Kokiri 28.8, Hyrule Field 45.1, the Market 29.4 fps.Flags and settings
RX_VK_MIRROR=0for an A/BTesting
Tested: Linux, RX 580 8 GB, RADV, in the game: Kokiri Forest, Hyrule Field and the Market, main with
RX_VK_MIRROR=1against main's default, one session each, no device loss and no amdgpu faults. In progress: this PR's own build on the RX 580, interleaved with main, withRX_VK_MIRROR=0, and with the threaded mode forced (RX_VK_ASYNC=1 RX_VK_RECORDER=thread).Not tested:
python -I -B tests/run_all.py: not run on this PC (no test suites run on it); left to CI. No check cutsmirror_on()'s body (tests/test_import_chunk_default.pyreads onlyimport_chunk()'smirror_on() ? 4 : 16; others stubmirror_on).mirror_budget.h), Vega, and a GCN PC with 12+ threads where Auto now turns on Threaded rendering.Must be tested:
RX_VK_MIRROR=0, expecting fps at least=0's (Test Farm)Risks
What could break: GCN cards with little VRAM, where mirrors take part of it; Threaded rendering now on by Auto on GCN PCs with 12+ threads; the k10 slowdown, if something specific to that build or driver comes back.
Turning it off:
RX_VK_MIRROR=0, or revert the commit.Compatibility: no cache, save or options file changes; the renderer's interface is unchanged.
Issues
Prior work: none found (searched "mirror GCN", "GCN", "RX_VK_MIRROR", "mirror_on", "Polaris", "RX 580"). Related: #1141 (which pools get the mirrors on 4 to 6 GB cards), #906 and #1053 (other RX 580 gains measured in the same round).
Checklist
Everyone
type(scope): description) of the status line's type.main.python -I -B tests/run_all.pypasses (any failure that also fails onmainis named). Not run on this PC (no test suites run on it); left to CI.third_party/untouched (a new folder with aPROVENANCE.mdinstead).When it applies
Assisted-by:/Co-Authored-By:trailers.docs/tested/for each new measured run (format indocs/tested-configurations.md). To add with the interleaved runs.🤖 Generated with Claude Code
Tested in game: RX 580 8 GB, RADV, Linux: Kokiri Forest, Hyrule Field and the Market with the mirrors on (RX_VK_MIRROR=1 on main, the behaviour this PR makes the default) against main's default: fps +28 to +54% in every phase, no device loss, no amdgpu faults. This PR's own build is being measured on the same card.
WIP: perf(vk): VRAM mirrors on GCN too (RX 400/500, Vega): an RX 580 runs 28 to 54% faster with themto perf(vk): VRAM mirrors on GCN too (RX 400/500, Vega): an RX 580 runs 28 to 54% faster with themFor the record, since this merged before I could add it to the description:
Where the rule came from:
1f8c02f9ported it from deadlynch/hookshot-k10 (commit e09059c1). It turned mirrors and batched submits off together: "an RX 580 ran Kokiri Forest at 20 fps with either on and 24.5 with both off". That was tested on the fork's renderer by the fork's author, and "not yet run here". This PR turns only the mirrors back on. Batched submits stay off on GCN (gpu/vk/vk_replay.cpp). Main has also changed a good deal since, including #1091's fix to how AMD's VRAM allowance is read.Effect on #977: with the mirrors on, GCN no longer takes #977's early-read path, which acts on results written in place. On the RX 580 the mirrors gave more (+28 to +54%) than early reads did (+10 to +24%), so #977's value moves to integrated GPUs, which keep no mirrors by design.
Still to come: the lab is running this PR's own build on the RX 580, interleaved with main, with
RX_VK_MIRROR=0, and with the threaded mode forced (RX_VK_ASYNC=1 RX_VK_RECORDER=thread). That last one is what Auto now picks on a GCN PC with 12 or more threads. Results follow in a comment.🤖 Generated with Claude Code
This PR's own build on the RX 580 8 GB (RADV, 8 threads, Linux), as promised.
Build: this change applied to
mainc167ec67, the lab's base, as a pack. Main's own folder is the before.Route: Kokiri Forest, Hyrule Field, the Market, Hyrule Field again; standing then turning. One session per run, interleaved M P P M P0 T T after a warm-up.
RX_VK_MIRROR=0RX_VK_MIRROR=0: gives main's numbers back, so the switch still restores the old behaviour.VRAM mirror of buffer pools on, with 0 amdgpu fault, timeout or reset lines in any session, and no device loss.RX_VK_ASYNC=1 RX_VK_RECORDER=thread, what Auto now picks on a GCN PC with 12+ threads): still well ahead of main everywhere, but 5 to 20% behind this PR alone in the Market (noisy: 40.3 and 27.6 turning). This 8-thread PC is a poor stand-in for a 12+ thread one, so that case still wants a run on such a PC. If it shows the same loss, Auto could keep the render thread off on GCN.🤖 Generated with Claude Code