perf(vk): VRAM mirrors on GCN too (RX 400/500, Vega): an RX 580 runs 28 to 54% faster with them #1187

Merged
HumanBean merged 1 commit from HumanBean/Longshot:perf/vram-mirrors-gcn into main 2026-11-16 17:07:29 +00:00
Owner

Type: perf · Risk: medium · Tested on: Linux, RX 580 8 GB (RADV) · Needs: more GCN cards (a 4 GB RX 470/570, Vega), and a GCN PC with 12+ threads

Why

The VRAM mirrors are off by default on GCN cards (RX 400/500, Vega). In k10 (1f8c02f9), an RX 580 ran Kokiri Forest at 20 fps with them and 24.5 without. On today's renderer that has reversed. On the same kind of card, an RX 580 8 GB, main with RX_VK_MIRROR=1 runs 28 to 54% faster than main's default in every area. The rule now costs GCN players most of that.

What changed

  • vk/memory: mirror_on() in gpu/vk/vk_memory.cpp gives every discrete card its mirrors. The GCN test (sg32_stages == 0 && subgroup_size > 32) and its (GCN) log note are gone. RX_VK_MIRROR=0 still turns them off.
  • docs: docs/development.md's Threaded rendering row no longer lists GCN among the PCs whose pools have no mirrors.

Threaded rendering's Auto turns the render thread on only where the pools have mirrors (pools_mirrored(), vk_menu.cpp). So on a GCN PC with 12 or more logical processors, Auto now picks On. An RX 580 PC with 8 threads keeps Off. The measurements below include that mode forced on the RX 580.

Measurements

Verdict: faster on the RX 580 in every phase. fps is +28 to +54% (Kokiri 20.9 → 29.6, Hyrule Field 27.5 → 42.3 standing, the Market 24.2 → 31.6), and GPU time is 2 to 8 ms lower a frame. This is from one session of main with RX_VK_MIRROR=1, which behaves exactly like this change, against one of main's default. Interleaved runs of this PR's own build are in progress and will be added before this leaves WIP.

Hardware: Linux, AMD Radeon RX 580 8 GB (Polaris10, RADV), 8 threads, KDE
Scene: Kokiri Forest, Hyrule Field, the Castle Town Market (debug teleport), Hyrule Field again; each standing then turning the camera
Method: one game session per arm, the pristine Kokiri save; fps from the frame log's median frame time; GPU time the renderer's per-frame median
How to reproduce: run the lab bench's route with RX_VK_MIRROR=1 against unset on main, or this PR against RX_VK_MIRROR=0
Baseline: main c167ec67 (the lab's base), its own build, with RX_VK_MIRROR unset (mirrors off on GCN) and =1

fps (GPU ms) Before: main @ c167ec67 After: main @ c167ec67 with RX_VK_MIRROR=1 (this PR's default)
Kokiri Forest, standing 20.9 (35.60) 29.6 (27.35)
Kokiri Forest, turning 22.4 (31.70) 31.7 (24.90)
Hyrule Field, standing 27.5 (24.30) 42.3 (19.60)
Hyrule Field, turning 33.6 (20.25) 48.1 (17.60)
the Market, standing 24.2 (28.90) 31.6 (22.30)
the Market, turning 36.5 (17.50) 46.6 (15.30)
Hyrule Field again, standing 27.3 (24.60) 38.0 (21.80)

The same lab round had main with both RX_VK_MIRROR=1 and RX_VK_WB_MAX_KB=1024, and it agreed: Kokiri 28.8, Hyrule Field 45.1, the Market 29.4 fps.

Flags and settings

Flag What it is Before this PR After this PR Effect Helps / doesn't help / untested Recommendation
RX_VK_MIRROR VRAM mirrors of the game's buffer pools on for discrete cards except GCN on for every discrete card +28 to 54% fps on an RX 580 helps on the RX 580; untested on 4 GB GCN and Vega default (on); =0 for an A/B

Testing

Tested: Linux, RX 580 8 GB, RADV, in the game: Kokiri Forest, Hyrule Field and the Market, main with RX_VK_MIRROR=1 against main's default, one session each, no device loss and no amdgpu faults. In progress: this PR's own build on the RX 580, interleaved with main, with RX_VK_MIRROR=0, and with the threaded mode forced (RX_VK_ASYNC=1 RX_VK_RECORDER=thread).

Not tested:

  • python -I -B tests/run_all.py: not run on this PC (no test suites run on it); left to CI. No check cuts mirror_on()'s body (tests/test_import_chunk_default.py reads only import_chunk()'s mirror_on() ? 4 : 16; others stub mirror_on).
  • Windows, a 4 GB GCN card (the mirrors' budget there is at most 256 MB, mirror_budget.h), Vega, and a GCN PC with 12+ threads where Auto now turns on Threaded rendering.

Must be tested:

  • Linux and Windows, a 4 GB RX 470/570 and a Vega: Kokiri Forest and the Market, this PR against RX_VK_MIRROR=0, expecting fps at least =0's (Test Farm)
  • A GCN PC with 12+ threads: the same, with Threaded rendering on Auto, expecting no flashes or device loss

Risks

What could break: GCN cards with little VRAM, where mirrors take part of it; Threaded rendering now on by Auto on GCN PCs with 12+ threads; the k10 slowdown, if something specific to that build or driver comes back.

Turning it off: RX_VK_MIRROR=0, or revert the commit.

Compatibility: no cache, save or options file changes; the renderer's interface is unchanged.

Issues

Prior work: none found (searched "mirror GCN", "GCN", "RX_VK_MIRROR", "mirror_on", "Polaris", "RX 580"). Related: #1141 (which pools get the mirrors on 4 to 6 GB cards), #906 and #1053 (other RX 580 gains measured in the same round).

Checklist

Everyone

  • One topic.
  • The title is a Conventional Commit (type(scope): description) of the status line's type.
  • Why gives the reason plainly and links the issues that fit (the tracker searched).
  • Measurements has a verdict, or one line saying why there is nothing to measure.
  • Testing names what was tested, what wasn't, and what must still be tested, where and how.
  • Rebased on current main.
  • python -I -B tests/run_all.py passes (any failure that also fails on main is named). Not run on this PC (no test suites run on it); left to CI.
  • No game files, dumps, build output or generated files.
  • third_party/ untouched (a new folder with a PROVENANCE.md instead).
  • The license is OK: GPL-3.0-or-later, nothing incompatible and nothing decompiled.

When it applies

  • AI assistance recorded with Assisted-by: / Co-Authored-By: trailers.
  • Numbers from the same PC, scene and settings, at least 2 interleaved runs each, the median reported. One session each so far; the interleaved runs are in progress.
  • Switches documented (docs/development.md; the README for a menu row) and shipped at their recommended value.
  • A file in docs/tested/ for each new measured run (format in docs/tested-configurations.md). To add with the interleaved runs.
  • Docs updated.

🤖 Generated with Claude Code

**Type:** perf · **Risk:** medium · **Tested on:** Linux, RX 580 8 GB (RADV) · **Needs:** more GCN cards (a 4 GB RX 470/570, Vega), and a GCN PC with 12+ threads ## Why The VRAM mirrors are off by default on GCN cards (RX 400/500, Vega). In k10 (1f8c02f9), an RX 580 ran Kokiri Forest at 20 fps with them and 24.5 without. On today's renderer that has reversed. On the same kind of card, an RX 580 8 GB, main with `RX_VK_MIRROR=1` runs **28 to 54% faster** than main's default in every area. The rule now costs GCN players most of that. ## What changed - **vk/memory:** `mirror_on()` in `gpu/vk/vk_memory.cpp` gives every discrete card its mirrors. The GCN test (`sg32_stages == 0 && subgroup_size > 32`) and its `(GCN)` log note are gone. `RX_VK_MIRROR=0` still turns them off. - **docs:** `docs/development.md`'s Threaded rendering row no longer lists GCN among the PCs whose pools have no mirrors. Threaded rendering's Auto turns the render thread on only where the pools have mirrors (`pools_mirrored()`, `vk_menu.cpp`). So on a GCN PC with 12 or more logical processors, Auto now picks On. An RX 580 PC with 8 threads keeps Off. The measurements below include that mode forced on the RX 580. ## Measurements **Verdict:** faster on the RX 580 in every phase. fps is +28 to +54% (Kokiri 20.9 → 29.6, Hyrule Field 27.5 → 42.3 standing, the Market 24.2 → 31.6), and GPU time is 2 to 8 ms lower a frame. This is from one session of main with `RX_VK_MIRROR=1`, which behaves exactly like this change, against one of main's default. Interleaved runs of this PR's own build are in progress and will be added before this leaves WIP. Hardware: Linux, AMD Radeon RX 580 8 GB (Polaris10, RADV), 8 threads, KDE Scene: Kokiri Forest, Hyrule Field, the Castle Town Market (debug teleport), Hyrule Field again; each standing then turning the camera Method: one game session per arm, the pristine Kokiri save; fps from the frame log's median frame time; GPU time the renderer's per-frame median How to reproduce: run the lab bench's route with `RX_VK_MIRROR=1` against unset on main, or this PR against `RX_VK_MIRROR=0` Baseline: main c167ec67 (the lab's base), its own build, with `RX_VK_MIRROR` unset (mirrors off on GCN) and `=1` | fps (GPU ms) | Before: main @ c167ec67 | After: main @ c167ec67 with `RX_VK_MIRROR=1` (this PR's default) | |---|---|---| | Kokiri Forest, standing | 20.9 (35.60) | **29.6** (27.35) | | Kokiri Forest, turning | 22.4 (31.70) | **31.7** (24.90) | | Hyrule Field, standing | 27.5 (24.30) | **42.3** (19.60) | | Hyrule Field, turning | 33.6 (20.25) | **48.1** (17.60) | | the Market, standing | 24.2 (28.90) | **31.6** (22.30) | | the Market, turning | 36.5 (17.50) | **46.6** (15.30) | | Hyrule Field again, standing | 27.3 (24.60) | **38.0** (21.80) | The same lab round had main with both `RX_VK_MIRROR=1` and `RX_VK_WB_MAX_KB=1024`, and it agreed: Kokiri 28.8, Hyrule Field 45.1, the Market 29.4 fps. ### Flags and settings | Flag | What it is | Before this PR | After this PR | Effect | Helps / doesn't help / untested | Recommendation | |---|---|---|---|---|---|---| | `RX_VK_MIRROR` | VRAM mirrors of the game's buffer pools | on for discrete cards except GCN | on for every discrete card | +28 to 54% fps on an RX 580 | helps on the RX 580; untested on 4 GB GCN and Vega | default (on); `=0` for an A/B | ## Testing **Tested:** Linux, RX 580 8 GB, RADV, in the game: Kokiri Forest, Hyrule Field and the Market, main with `RX_VK_MIRROR=1` against main's default, one session each, no device loss and no amdgpu faults. In progress: this PR's own build on the RX 580, interleaved with main, with `RX_VK_MIRROR=0`, and with the threaded mode forced (`RX_VK_ASYNC=1 RX_VK_RECORDER=thread`). **Not tested:** - `python -I -B tests/run_all.py`: not run on this PC (no test suites run on it); left to CI. No check cuts `mirror_on()`'s body (`tests/test_import_chunk_default.py` reads only `import_chunk()`'s `mirror_on() ? 4 : 16`; others stub `mirror_on`). - Windows, a 4 GB GCN card (the mirrors' budget there is at most 256 MB, `mirror_budget.h`), Vega, and a GCN PC with 12+ threads where Auto now turns on Threaded rendering. **Must be tested:** - Linux and Windows, a 4 GB RX 470/570 and a Vega: Kokiri Forest and the Market, this PR against `RX_VK_MIRROR=0`, expecting fps at least `=0`'s (Test Farm) - A GCN PC with 12+ threads: the same, with Threaded rendering on Auto, expecting no flashes or device loss ## Risks **What could break:** GCN cards with little VRAM, where mirrors take part of it; Threaded rendering now on by Auto on GCN PCs with 12+ threads; the k10 slowdown, if something specific to that build or driver comes back. **Turning it off:** `RX_VK_MIRROR=0`, or revert the commit. **Compatibility:** no cache, save or options file changes; the renderer's interface is unchanged. ## Issues Prior work: none found (searched "mirror GCN", "GCN", "RX_VK_MIRROR", "mirror_on", "Polaris", "RX 580"). Related: #1141 (which pools get the mirrors on 4 to 6 GB cards), #906 and #1053 (other RX 580 gains measured in the same round). ## Checklist ### Everyone - [x] One topic. - [x] The title is a Conventional Commit (`type(scope): description`) of the status line's type. - [x] Why gives the reason plainly and links the issues that fit (the tracker searched). - [x] Measurements has a verdict, or one line saying why there is nothing to measure. - [x] Testing names what was tested, what wasn't, and what must still be tested, where and how. - [x] Rebased on current `main`. - [ ] `python -I -B tests/run_all.py` passes (any failure that also fails on `main` is named). Not run on this PC (no test suites run on it); left to CI. - [x] No game files, dumps, build output or generated files. - [x] `third_party/` untouched (a new folder with a `PROVENANCE.md` instead). - [x] The license is OK: GPL-3.0-or-later, nothing incompatible and nothing decompiled. ### When it applies - [x] AI assistance recorded with `Assisted-by:` / `Co-Authored-By:` trailers. - [ ] Numbers from the same PC, scene and settings, at least 2 interleaved runs each, the median reported. One session each so far; the interleaved runs are in progress. - [x] Switches documented (docs/development.md; the README for a menu row) and shipped at their recommended value. - [ ] A file in `docs/tested/` for each new measured run (format in `docs/tested-configurations.md`). To add with the interleaved runs. - [x] Docs updated. 🤖 Generated with [Claude Code](https://claude.com/claude-code)
perf(vk): VRAM mirrors on GCN too (RX 400/500, Vega): an RX 580 runs 28 to 54% faster with them
Some checks failed
CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Has been skipped
CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Has been skipped
CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Has been skipped
Third-party parts / The same as upstream (pull_request) Has been skipped
Website / Website on Cloudflare (pull_request) Has been skipped
CI / Conventional Commit title (pull_request) Successful in 8s
Guide screenshots / Guide images and local UI source review (pull_request) Failing after 10s
CI / Workflow lint (actionlint) (pull_request) Successful in 12s
Website / Website on Cloudflare (pull_request_target) Has been skipped
Third-party parts / As their notes say (pull_request) Successful in 14s
PR description / Description follows the template (pull_request_target) Failing after 7s
18ef0ac584
k10 (1f8c02f9) turned the mirrors off by default on GCN after an RX 580 ran Kokiri Forest at 20 fps with them and 24.5
without. On today's renderer the same RX 580 runs faster with them in every area (RX_VK_MIRROR=1 against main's default):
so mirror_on() gives every discrete card its mirrors, and RX_VK_MIRROR=0 still turns them off.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Claude-Session: https://claude.ai/code/session_017vw8dzpak9jUZJcRrqmxhM
Author
Owner

Tested in game: RX 580 8 GB, RADV, Linux: Kokiri Forest, Hyrule Field and the Market with the mirrors on (RX_VK_MIRROR=1 on main, the behaviour this PR makes the default) against main's default: fps +28 to +54% in every phase, no device loss, no amdgpu faults. This PR's own build is being measured on the same card.

Tested in game: RX 580 8 GB, RADV, Linux: Kokiri Forest, Hyrule Field and the Market with the mirrors on (RX_VK_MIRROR=1 on main, the behaviour this PR makes the default) against main's default: fps +28 to +54% in every phase, no device loss, no amdgpu faults. This PR's own build is being measured on the same card.
HumanBean changed title from WIP: perf(vk): VRAM mirrors on GCN too (RX 400/500, Vega): an RX 580 runs 28 to 54% faster with them to perf(vk): VRAM mirrors on GCN too (RX 400/500, Vega): an RX 580 runs 28 to 54% faster with them 2026-11-16 17:07:21 +00:00
HumanBean deleted branch perf/vram-mirrors-gcn 2026-11-16 17:07:29 +00:00
Author
Owner

For the record, since this merged before I could add it to the description:

Where the rule came from: 1f8c02f9 ported it from deadlynch/hookshot-k10 (commit e09059c1). It turned mirrors and batched submits off together: "an RX 580 ran Kokiri Forest at 20 fps with either on and 24.5 with both off". That was tested on the fork's renderer by the fork's author, and "not yet run here". This PR turns only the mirrors back on. Batched submits stay off on GCN (gpu/vk/vk_replay.cpp). Main has also changed a good deal since, including #1091's fix to how AMD's VRAM allowance is read.

Effect on #977: with the mirrors on, GCN no longer takes #977's early-read path, which acts on results written in place. On the RX 580 the mirrors gave more (+28 to +54%) than early reads did (+10 to +24%), so #977's value moves to integrated GPUs, which keep no mirrors by design.

Still to come: the lab is running this PR's own build on the RX 580, interleaved with main, with RX_VK_MIRROR=0, and with the threaded mode forced (RX_VK_ASYNC=1 RX_VK_RECORDER=thread). That last one is what Auto now picks on a GCN PC with 12 or more threads. Results follow in a comment.

🤖 Generated with Claude Code

For the record, since this merged before I could add it to the description: **Where the rule came from:** 1f8c02f9 ported it from [deadlynch/hookshot-k10](https://github.com/deadlynch/hookshot-k10) (commit e09059c1). It turned mirrors and batched submits off together: "an RX 580 ran Kokiri Forest at 20 fps with either on and 24.5 with both off". That was tested on the fork's renderer by the fork's author, and "not yet run here". This PR turns only the mirrors back on. Batched submits stay off on GCN (`gpu/vk/vk_replay.cpp`). Main has also changed a good deal since, including #1091's fix to how AMD's VRAM allowance is read. **Effect on #977:** with the mirrors on, GCN no longer takes #977's early-read path, which acts on results written in place. On the RX 580 the mirrors gave more (+28 to +54%) than early reads did (+10 to +24%), so #977's value moves to integrated GPUs, which keep no mirrors by design. **Still to come:** the lab is running this PR's own build on the RX 580, interleaved with main, with `RX_VK_MIRROR=0`, and with the threaded mode forced (`RX_VK_ASYNC=1 RX_VK_RECORDER=thread`). That last one is what Auto now picks on a GCN PC with 12 or more threads. Results follow in a comment. 🤖 Generated with [Claude Code](https://claude.com/claude-code)
Author
Owner

This PR's own build on the RX 580 8 GB (RADV, 8 threads, Linux), as promised.

Build: this change applied to main c167ec67, the lab's base, as a pack. Main's own folder is the before.
Route: Kokiri Forest, Hyrule Field, the Market, Hyrule Field again; standing then turning. One session per run, interleaved M P P M P0 T T after a warm-up.

fps (GPU ms) main (2 runs) this PR (2 runs) this PR, RX_VK_MIRROR=0 this PR, threaded mode forced (2 runs)
Kokiri, standing 21.0, 20.9 (35.6) 29.3, 29.0 (27.8) 20.9 (35.6) 29.4, 26.7 (27.6)
Kokiri, turning 22.4, 22.5 (31.5) 32.4, 32.0 (24.7) 22.4 (31.6) 28.5, 27.5 (25.2)
Hyrule Field, standing 27.8, 27.8 (24.2) 45.1, 44.9 (18.1) 27.8 (24.2) 43.3, 42.8 (18.2)
Hyrule Field, turning 34.2, 34.5 (20.0) 51.6, 51.5 (15.8) 33.8 (20.2) 50.2, 49.9 (16.0)
the Market, standing 23.4, 23.4 (29.2) 32.5, 32.4 (22.4) 24.1 (28.6) 26.5, 29.0 (22.1)
the Market, turning 34.8, 37.8 (18.0) 45.8, 48.0 (14.8) 39.7 (17.3) 40.3, 27.6 (15.9)
Hyrule Field again, standing 27.2, 26.2 (24.5) 43.0, 42.8 (18.6) 27.5 (24.2) 40.7, 37.7 (19.9)
  • fps: +27% (the Market turning) to +62% (Hyrule Field standing), in both runs of every phase. GPU time is 3 to 8 ms lower.
  • RX_VK_MIRROR=0: gives main's numbers back, so the switch still restores the old behaviour.
  • No faults: the log says VRAM mirror of buffer pools on, with 0 amdgpu fault, timeout or reset lines in any session, and no device loss.
  • Threaded mode forced (RX_VK_ASYNC=1 RX_VK_RECORDER=thread, what Auto now picks on a GCN PC with 12+ threads): still well ahead of main everywhere, but 5 to 20% behind this PR alone in the Market (noisy: 40.3 and 27.6 turning). This 8-thread PC is a poor stand-in for a 12+ thread one, so that case still wants a run on such a PC. If it shows the same loss, Auto could keep the render thread off on GCN.

🤖 Generated with Claude Code

This PR's own build on the RX 580 8 GB (RADV, 8 threads, Linux), as promised. **Build:** this change applied to `main` c167ec67, the lab's base, as a pack. Main's own folder is the before. **Route:** Kokiri Forest, Hyrule Field, the Market, Hyrule Field again; standing then turning. One session per run, interleaved M P P M P0 T T after a warm-up. | fps (GPU ms) | main (2 runs) | this PR (2 runs) | this PR, `RX_VK_MIRROR=0` | this PR, threaded mode forced (2 runs) | |---|---|---|---|---| | Kokiri, standing | 21.0, 20.9 (35.6) | **29.3, 29.0** (27.8) | 20.9 (35.6) | 29.4, 26.7 (27.6) | | Kokiri, turning | 22.4, 22.5 (31.5) | **32.4, 32.0** (24.7) | 22.4 (31.6) | 28.5, 27.5 (25.2) | | Hyrule Field, standing | 27.8, 27.8 (24.2) | **45.1, 44.9** (18.1) | 27.8 (24.2) | 43.3, 42.8 (18.2) | | Hyrule Field, turning | 34.2, 34.5 (20.0) | **51.6, 51.5** (15.8) | 33.8 (20.2) | 50.2, 49.9 (16.0) | | the Market, standing | 23.4, 23.4 (29.2) | **32.5, 32.4** (22.4) | 24.1 (28.6) | 26.5, 29.0 (22.1) | | the Market, turning | 34.8, 37.8 (18.0) | **45.8, 48.0** (14.8) | 39.7 (17.3) | 40.3, 27.6 (15.9) | | Hyrule Field again, standing | 27.2, 26.2 (24.5) | **43.0, 42.8** (18.6) | 27.5 (24.2) | 40.7, 37.7 (19.9) | - **fps:** +27% (the Market turning) to +62% (Hyrule Field standing), in both runs of every phase. GPU time is 3 to 8 ms lower. - **`RX_VK_MIRROR=0`:** gives main's numbers back, so the switch still restores the old behaviour. - **No faults:** the log says `VRAM mirror of buffer pools on`, with 0 amdgpu fault, timeout or reset lines in any session, and no device loss. - **Threaded mode forced** (`RX_VK_ASYNC=1 RX_VK_RECORDER=thread`, what Auto now picks on a GCN PC with 12+ threads): still well ahead of main everywhere, but 5 to 20% behind this PR alone in the Market (noisy: 40.3 and 27.6 turning). This 8-thread PC is a poor stand-in for a 12+ thread one, so that case still wants a run on such a PC. If it shows the same loss, Auto could keep the render thread off on GCN. 🤖 Generated with [Claude Code](https://claude.com/claude-code)
Sign in to join this conversation.
No reviewers
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
MagicBeanDev/Longshot!1187
No description provided.