perf(vk): give back retired pools' imports once the GPU is past them #446

Merged

azlekayn merged 1 commit from fix/vk-retired-imports into main

2026-11-08 08:19:57 +00:00

azlekayn commented

2026-11-08 02:45:13 +00:00

(Migrated from github.com)

What this fixes

In Kokiri Forest on Linux with RADV, the frame rate falls to 3-5 fps while the GPU is mostly idle. The cost is in vkQueueSubmit (#312, #413). One reason is that retired memory pools were never given back.

When a pool is remade larger, the old one is retired, and its host-pointer import was kept forever. On amdgpu each import is a userptr buffer object. RADV lists every buffer object in every submission, because of buffer device addresses. So at every DRM_IOCTL_AMDGPU_CS the kernel re-checks every page of every userptr object, and the set it checks only grew with each pool merge. strace shows 15-45 ms per CS ioctl in Kokiri, with about 6 submits a frame.

Change

gpu/vk/vk_memory.cpp:

A retired pool's buffer and memory are now destroyed once all of these hold:
- it has no mirror and no inherit pending;
- the GPU has completed its retired_value and last_value.
Destruction is deferred past everything recorded so far, as dead_mirrors already is.
Its guest page-table entries were cleared at retirement in that same recording, so nothing reaches it afterwards.
The node stays in retired_pools, since pointers to it are kept, and pools_gen is bumped.
memory_guest_imported skips freed retired pools.
A summary is logged every 1200 submissions: vk: imports: N live, N retired still imported, N retired given back.
RX_VK_KEEP_RETIRED=1 keeps them, as before.

Tested

Hardware: RX 7800 XT, RADV (Mesa, Vulkan 1.4), Fedora 44, Linux 7.2.
Kokiri Forest: 3.7 → 5.6 fps on the release build alone, with 31 retired pools (720-848 MB) given back per run.
Stability: no device loss or crash in about 15 runs of 4-8 minutes, from the title through the intro to Kokiri gameplay.
Builds measured: the numbers come from the foliage-linux-updater release with this patch.
Against main: the patch is rebased onto current main, and hookshot_vk builds there. I have not run a full main build.
Not tested: Windows, NVIDIA.

This is the smaller half of the submit cost. #413's batching and the udmabuf PR (#448) address the rest.

Analysis and patch made with Claude (Anthropic); I ran every test on my machine.

🤖 Generated with Claude Code

## What this fixes In Kokiri Forest on Linux with RADV, the frame rate falls to 3-5 fps while the GPU is mostly idle. The cost is in `vkQueueSubmit` (#312, #413). One reason is that retired memory pools were never given back. When a pool is remade larger, the old one is retired, and its host-pointer import was kept forever. On amdgpu each import is a userptr buffer object. RADV lists every buffer object in every submission, because of buffer device addresses. So at every `DRM_IOCTL_AMDGPU_CS` the kernel re-checks every page of every userptr object, and the set it checks only grew with each pool merge. strace shows 15-45 ms per CS ioctl in Kokiri, with about 6 submits a frame. ## Change `gpu/vk/vk_memory.cpp`: - A retired pool's buffer and memory are now destroyed once all of these hold: - it has no mirror and no inherit pending; - the GPU has completed its `retired_value` and `last_value`. - Destruction is deferred past everything recorded so far, as `dead_mirrors` already is. - Its guest page-table entries were cleared at retirement in that same recording, so nothing reaches it afterwards. - The node stays in `retired_pools`, since pointers to it are kept, and `pools_gen` is bumped. - `memory_guest_imported` skips freed retired pools. - A summary is logged every 1200 submissions: `vk: imports: N live, N retired still imported, N retired given back`. - `RX_VK_KEEP_RETIRED=1` keeps them, as before. ## Tested - **Hardware:** RX 7800 XT, RADV (Mesa, Vulkan 1.4), Fedora 44, Linux 7.2. - **Kokiri Forest:** 3.7 → 5.6 fps on the release build alone, with 31 retired pools (720-848 MB) given back per run. - **Stability:** no device loss or crash in about 15 runs of 4-8 minutes, from the title through the intro to Kokiri gameplay. - **Builds measured:** the numbers come from the `foliage-linux-updater` release with this patch. - **Against `main`:** the patch is rebased onto current `main`, and `hookshot_vk` builds there. I have not run a full `main` build. - **Not tested:** Windows, NVIDIA. This is the smaller half of the submit cost. #413's batching and the udmabuf PR (#448) address the rest. Analysis and patch made with Claude (Anthropic); I ran every test on my machine. 🤖 Generated with [Claude Code](https://claude.com/claude-code)

Commenting is not possible because the repository is archived.

No reviewers