fix(vk): retain guest pages during import handoff and retirement #468

Open

zeldaootr wants to merge 3 commits from zeldaootr/Longshot:import-ownership into main

pull from: zeldaootr/Longshot:import-ownership

merge into: MagicBeanDev:main

MagicBeanDev:main

Conversation 4 Commits 3 Files changed 12 +118 -14

zeldaootr commented 2026-11-14 16:03:44 +00:00

Contributor

• Copy link
• Copy Markdown

Type: fix · Risk: low · Tested on: Steam Deck OLED, SteamOS 3.9.5, RADV (ours and a tester's runs); Linux x86-64 (game-free) · Needs: Windows, any GPU

Why

When the game frees a range that Vulkan still owns through an import that is queued, being made, made but not yet published, or retired but waiting for the GPU, the renderer can report the pages as free. The host can then decommit pages that a pending GPU allocation still references.

How to reproduce

Seen on: Linux x86-64, game-free (the real memory_guest_imported() with its import tables filled by hand); not seen as a fault in the game, where the race does not show in the counters (see Measurements)
Settings and switches: defaults
Steps: on main aa95d35f with this PR's test added (cd7e4de1), run python -I -B tests/test_pending_guest_import.py --cxx g++: it asks about a range held by a queued import, one being made, one made but not published and one retired but not yet destroyed
Expected: each is reported as imported
Actual: Assertion 'memory_guest_imported(0x2000, 0x3000)' failed: a range only the import thread holds is reported free

What changed

memory_guest_imported() now also counts the imports the background thread has queued, is making or has made but not yet published, and the retired ones still waiting for the GPU, so the host keeps those pages instead of decommitting them under a live import.

• vk memory: memory_guest_imported() in gpu/vk/vk_memory.cpp also checks the ranges the import thread has queued or is importing (bg_inflight, now [lo, hi) per range, under bg_mu), the ranges imported but not yet published (bg_ready), and the retired imports waiting for the GPU before they are destroyed (dead_imports, which now keeps the guest start, DeadImport::lo). An empty range answers 0 before taking the lock.
• tests: tests/test_pending_guest_import.py (new) cuts memory_guest_imported() out of the source and checks deferred, in-flight and ready ranges plus a disjoint query; tests/texel_buffer_ownership_test.cpp and nx2/test_import_windows.py follow the new DeadImport and bg_inflight shapes, and the latter now also checks the end recorded for each queued range.
• docs/tested: the Deck runs of this branch and of main at the Field transition (the four main records are the same files in #469 and #472: one shared main arm, which merges as one).

Measurements

Verdict: passes: no change in fps, frame times or memory on a Steam Deck at the Field transition (20.60 against 20.53 fps, inside the run-to-run spread; no fault in 4 runs each), as expected for a correctness fix; the game-free test fails on main and passes here. A tester's Deck run with and without this PR also saw no difference.

Hardware: Steam Deck OLED, AMD Custom APU 0932 (RADV VANGOGH), Mesa 26.2.0-devel (git-5c65d4fc31), 16 GB shared, SteamOS 3.9.5 (kernel 7.2.7-valve2)
Scene: Hyrule Field (child), arriving from Kokiri Forest by teleport to the Lost Woods bridge entrance, day; 1280×720 output, render resolution option 4, no frame generation, precompile off, 1536 MiB texture budget, 256 MiB mirror budget, headless
Method: one scripts/dev_packs.py base at main 9c9d69ed (release optimisation, x86-64-v3); main is the base's runtime with no pack, this PR a renderer pack (longshot_vk.so only; the host binary is byte-identical). One warm-up pass of main made the caches, copied fresh into every run. Each run: child Kokiri save settled 20 s, teleport to HyruleField_Child, 20 s warm-up, 45 s stationary sample. Four rounds in one session, interleaved main, #468, #469, #472; medians of the 4 runs per arm. FPS and present intervals from HOS_PRESENT_TRACE in the 45 s window; GPU clock and busy from sysfs every 0.5 s; guest memory backing from the guest memfd's allocated blocks.
Rebased since: the branch is now on main aa95d35f, the fix at d2b39b8f (the same change as 46b467d0, git patch-id). Not measured again on aa95d35f. Since 9c9d69ed main changed gpu/vk/vk_memory.cpp only in how the mirror budget is set (#735), not in memory_guest_imported() or the import tables; everything else it changed both arms would take alike, so these numbers still compare the change. The game-free test was run again on aa95d35f (below).
How to reproduce: build a base at main and a pack of this branch (python -I scripts/dev_packs.py base|pack|rt, docs/development.md "Testing a change against a base"), start each runtime with HOS_CONTROL and a child save at Kokiri spawn, then longshot_ctl.py teleport HyruleField_Child, wait 20 s and sample 45 s.

	Before: main @ 9c9d69ed	After: this PR @ 46b467d0
FPS, Field sample (median of 4)	20.60	20.53
Present interval median (ms)	48.56	48.60
Present interval p95 (ms)	51.79	51.87
GPU clock median (MHz) / busy	1040 / 99%	1040 / 99%
Guest memory backed at sample end (MiB)	5867	5865
Lowest available memory (MiB)	2185	2291
Device faults, run memory guard, crashes	none in 4 runs	none in 4 runs

Per run, FPS: main 20.56, 20.54, 20.69, 20.64; this PR 20.44, 20.71, 20.62, 20.30. The difference is inside the run-to-run spread: no frame-rate change, as expected for a correctness fix. The game's own log shows the same import traffic on both (vk: imports: about 50 live imports, 9-13 retired given back). The live game does not exercise the race this fixes in a way the counters show; the game-free test is the evidence for the fix itself.

main-r2:   [hos host] longshot v3.0.0-rc.3-75-g9c9d69ed (source 6b76ded3)
           vk: 5160 presents, 20.6 fps; a frame: 1176 draws (0 skipped), 145 dispatches, 155 passes, ...
           vk: imports: 48 live (2016 MB), 0 retired still imported (0 MB), 13 retired given back (1056 MB)
pr468-r2:  renderer v3.0.0-rc.3-77-g46b467d0, host as main's
           vk: 5160 presents, 20.6 fps; a frame: 1167 draws (0 skipped), 145 dispatches, 155 passes, ...
           vk: imports: 57 live (2032 MB), 0 retired still imported (0 MB), 9 retired given back (672 MB)

$ python3 -I -B tests/test_pending_guest_import.py --cxx g++      (at cd7e4de1, main aa95d35f plus the test; rounds 1 and 2)
check.exe: check.cpp:40: main()::<lambda()>: Assertion `memory_guest_imported(0x2000, 0x3000)' failed.
$ python3 -I -B tests/test_pending_guest_import.py --cxx g++      (at 2273c8d3; rounds 1 and 2)
(passes, exit 0)

A tester's Deck run, with and without this PR: dukenheimer, 2026-11-14 (#468 (comment)), not a test request (T-HHWD asks for Windows). Steam Deck OLED (AMD Custom APU 0932, RADV VANGOGH, 16 GB; SteamOS 3.9.5 build 20261023.100, kernel 7.2.7-valve2, Mesa 26.2.0-devel git-5c65d4fc31), stock clocks, Game Mode, docked. Build: a Deck user build of main 543846eb with #227, #225, #223, #232 and #303, with #384, this PR (e5889e76), #488 and #496 merged on top; "without" is the same tree with this PR's merge reverted, only longshot_vk.so swapped in one warm test folder. That owner's settings: the handheld profile, output 1280x800, render resolution 100%, the game's own upscaler, frame generation 3x, a 60 fps cap, vsync on, RX_VK_READBACK_EARLY=1 and HOS_VIDEO_LOG=1. A scripted moving route from a Kokiri Forest save, 80 s measured, warm caches; runs in a mirrored order, 2 each way.

dukenheimer's Deck	GPU memory in the window, median (min-max), MB	its slope, MB a minute	real fps	p95 ms
with this PR (runs 1 and 8)	5,826 (5,692-6,091); 5,866 (5,465-6,006)	+108; +296	13.1; 12.8	73.4; 73.6
without (runs 4 and 5)	5,932 (5,640-6,278); 6,014 (5,452-6,078)	+55; +244	13.2; 12.8	95.2; 100.7

No crash, exception, device lost, vk: device fault, the GPU has finished nothing or read-back timeout line in 9 route runs (7 with this PR, among them the build's cold first start) and 4 title-screen movie runs with it. GPU memory is the process's drm-memory-gtt + drm-memory-vram from /proc/<pid>/fdinfo, every 5 s; it grows in every run as textures stream in, and no growth is peculiar to either arm. The tester's reading: no difference in faults, memory or fps; the p95 gap follows the run order (rising toward the middle of the series whatever ran there), not the build. Its limits, as the tester put them: the routes stay in Kokiri Forest (in run 4 the route went into a house), so they may never reach the import handoff this PR guards; the Field transition is not tested there.

Flags and settings

None

Testing

Tested: On the Steam Deck OLED above, 4 runs of this PR interleaved with 4 of main through the Kokiri Forest to Hyrule Field transition: Field reached every time (status map HyruleField_Child, 914-932 presents in each 45 s sample), no device fault, no VK_ERROR, no crash record, the run's low-memory guard never tripped, every run exited 0. dukenheimer's Deck route runs with and without this PR (in Measurements): no fault, no difference. Game-free, on this branch rebased onto main aa95d35f: tests/test_pending_guest_import.py fails at the test-only commit cd7e4de1 and passes at 2273c8d3, twice each, alternating, and under tests/run_all.py with g++ 13.3 and with clang 23.1.5 at both commits; nx2/test_import_windows.py passes at the tip with both. The 43 checks that read gpu/vk/vk_memory.cpp or the changed tests pass at the tip, 43 of 43. With #469 and #472, as rebased, the branch merges cleanly (git merge-tree).

Not tested: A forced background handoff or an induced GPU fault in the live game; the Field transition with this PR in anyone's run but ours; Windows and other GPUs and drivers in the game.

Must be tested:

• Windows 10/11, any GPU: build this PR (with llvm-mingw), load a child save in Kokiri Forest, go to Hyrule Field, stay 1-2 minutes and back, expecting no device lost, no VK_ERROR, no crash record in longshot_log.txt and fps like main at the same spot (by hand, test request T-HHWD: post the fps, the vk: imports: lines and the GPU, driver and OS)

Risks

What could break: this query recognizes logical ranges held at the time it runs; the host releases the renderer lock before performing the discard. Sparse-buffer padding beyond the logical binding is outside the test. Lock order is pools_mu then bg_mu, as in bg_drain and pool_lookup; the import thread never takes pools_mu. A range held by a pending import is now kept backed a little longer than before.

Turning it off: revert the fix commit

Compatibility: Nothing changes

Issues

None filed; found while tracing guest memory on the Steam Deck. Overlaps two other PRs in memory_guest_imported: #469 adds the same empty-range guard, and #431 changes the 1 GiB stop in the same loop. This and #469 merge with each other cleanly; whichever of these lands after #431 needs a small rebase.

Checklist

Everyone

• One topic.
• The title is a Conventional Commit (type(scope): description) of the status line's type.
• Why gives the reason plainly and links the issues that fit (the tracker searched).
• Measurements has a verdict, or one line saying why there is nothing to measure.
• Testing names what was tested, what wasn't, and what must still be tested, where and how.
• Rebased on current main. Onto main aa95d35f, no conflict; the test, fix and docs commits are the same changes as before (git patch-id).
• python -I -B tests/run_all.py passes (any failure that also fails on main is named). The full suite on main aa95d35f (Linux, g++ 13.3, the Vulkan headers set): 528 passed, 15 failed, 50 skipped; on this branch the new test and the 43 checks that read the changed files ran, with no failure that main does not have (see Testing). The 15 that fail on main: test_appimage_player_build, test_bake_portability, test_cpm_discovery, test_crash_dialog, test_env_on, test_lift_cache, test_lift_cleanup_error, test_make_game_cache, test_mods_size_table, test_options_budget_once, test_pipeline_tour, test_readback_late, test_vk_pgo, test_walk_key and nx2/test_texture_init_barrier.py.
• No game files, dumps, build output or generated files.
• third_party/ untouched (a new folder with a PROVENANCE.md instead).
• The license is OK: GPL-3.0-or-later, nothing incompatible and nothing decompiled.

When it applies

• AI assistance recorded with Assisted-by: / Co-Authored-By: trailers.
• Numbers from the same PC, scene and settings, at least 2 interleaved runs each, the median reported.
• A file in docs/tested/ for each new measured run (format in docs/tested-configurations.md).
• A game-free test added for the fix, failing on main and passing here.
• A check under tests/ that reads changed source updated in this PR, not weakened.
• Host code builds on both Windows (llvm-mingw) and Linux, with UTF-8 paths. Before the rebase, built on Linux for both: RX_VK_ONLY=1 RX_RELEASE=1 RX_TARGET=windows with llvm-mingw 20261104 linked longshot_vk.dll, and the same without RX_TARGET linked longshot_vk.so; after the rebase, gpu/vk/vk_memory.cpp at 2273c8d3 checked whole (-fsyntax-only -Wall -Wextra) with clang 23.1.5 for Linux and llvm-mingw 20261104 for Windows: the same diagnostics as main aa95d35f's (54 and 43), none new. The diff has no path handling.

**Type:** fix · **Risk:** low · **Tested on:** Steam Deck OLED, SteamOS 3.9.5, RADV (ours and a tester's runs); Linux x86-64 (game-free) · **Needs:** Windows, any GPU ## Why When the game frees a range that Vulkan still owns through an import that is queued, being made, made but not yet published, or retired but waiting for the GPU, the renderer can report the pages as free. The host can then decommit pages that a pending GPU allocation still references. ## How to reproduce **Seen on:** Linux x86-64, game-free (the real `memory_guest_imported()` with its import tables filled by hand); not seen as a fault in the game, where the race does not show in the counters (see Measurements) **Settings and switches:** defaults **Steps:** on `main` aa95d35f with this PR's test added (cd7e4de1), run `python -I -B tests/test_pending_guest_import.py --cxx g++`: it asks about a range held by a queued import, one being made, one made but not published and one retired but not yet destroyed **Expected:** each is reported as imported **Actual:** `Assertion 'memory_guest_imported(0x2000, 0x3000)' failed`: a range only the import thread holds is reported free ## What changed `memory_guest_imported()` now also counts the imports the background thread has queued, is making or has made but not yet published, and the retired ones still waiting for the GPU, so the host keeps those pages instead of decommitting them under a live import. - vk memory: `memory_guest_imported()` in `gpu/vk/vk_memory.cpp` also checks the ranges the import thread has queued or is importing (`bg_inflight`, now `[lo, hi)` per range, under `bg_mu`), the ranges imported but not yet published (`bg_ready`), and the retired imports waiting for the GPU before they are destroyed (`dead_imports`, which now keeps the guest start, `DeadImport::lo`). An empty range answers 0 before taking the lock. - tests: `tests/test_pending_guest_import.py` (new) cuts `memory_guest_imported()` out of the source and checks deferred, in-flight and ready ranges plus a disjoint query; `tests/texel_buffer_ownership_test.cpp` and `nx2/test_import_windows.py` follow the new `DeadImport` and `bg_inflight` shapes, and the latter now also checks the end recorded for each queued range. - docs/tested: the Deck runs of this branch and of main at the Field transition (the four main records are the same files in https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/469 and https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/472: one shared main arm, which merges as one). ## Measurements **Verdict:** passes: no change in fps, frame times or memory on a Steam Deck at the Field transition (20.60 against 20.53 fps, inside the run-to-run spread; no fault in 4 runs each), as expected for a correctness fix; the game-free test fails on `main` and passes here. A tester's Deck run with and without this PR also saw no difference. Hardware: Steam Deck OLED, AMD Custom APU 0932 (RADV VANGOGH), Mesa 26.2.0-devel (git-5c65d4fc31), 16 GB shared, SteamOS 3.9.5 (kernel 7.2.7-valve2) Scene: Hyrule Field (child), arriving from Kokiri Forest by teleport to the Lost Woods bridge entrance, day; 1280×720 output, render resolution option 4, no frame generation, precompile off, 1536 MiB texture budget, 256 MiB mirror budget, headless Method: one `scripts/dev_packs.py` base at main 9c9d69ed (release optimisation, x86-64-v3); main is the base's runtime with no pack, this PR a renderer pack (`longshot_vk.so` only; the host binary is byte-identical). One warm-up pass of main made the caches, copied fresh into every run. Each run: child Kokiri save settled 20 s, teleport to HyruleField_Child, 20 s warm-up, 45 s stationary sample. Four rounds in one session, interleaved main, #468, #469, #472; medians of the 4 runs per arm. FPS and present intervals from `HOS_PRESENT_TRACE` in the 45 s window; GPU clock and busy from sysfs every 0.5 s; guest memory backing from the guest memfd's allocated blocks. Rebased since: the branch is now on main aa95d35f, the fix at d2b39b8f (the same change as 46b467d0, `git patch-id`). Not measured again on aa95d35f. Since 9c9d69ed `main` changed `gpu/vk/vk_memory.cpp` only in how the mirror budget is set (#735), not in `memory_guest_imported()` or the import tables; everything else it changed both arms would take alike, so these numbers still compare the change. The game-free test was run again on aa95d35f (below). How to reproduce: build a base at main and a pack of this branch (`python -I scripts/dev_packs.py base|pack|rt`, docs/development.md "Testing a change against a base"), start each runtime with `HOS_CONTROL` and a child save at Kokiri spawn, then `longshot_ctl.py teleport HyruleField_Child`, wait 20 s and sample 45 s. | | Before: main @ 9c9d69ed | After: this PR @ 46b467d0 | |---|---|---| | FPS, Field sample (median of 4) | 20.60 | 20.53 | | Present interval median (ms) | 48.56 | 48.60 | | Present interval p95 (ms) | 51.79 | 51.87 | | GPU clock median (MHz) / busy | 1040 / 99% | 1040 / 99% | | Guest memory backed at sample end (MiB) | 5867 | 5865 | | Lowest available memory (MiB) | 2185 | 2291 | | Device faults, run memory guard, crashes | none in 4 runs | none in 4 runs | Per run, FPS: main 20.56, 20.54, 20.69, 20.64; this PR 20.44, 20.71, 20.62, 20.30. The difference is inside the run-to-run spread: no frame-rate change, as expected for a correctness fix. The game's own log shows the same import traffic on both (`vk: imports:` about 50 live imports, 9-13 retired given back). The live game does not exercise the race this fixes in a way the counters show; the game-free test is the evidence for the fix itself. ``` main-r2: [hos host] longshot v3.0.0-rc.3-75-g9c9d69ed (source 6b76ded3) vk: 5160 presents, 20.6 fps; a frame: 1176 draws (0 skipped), 145 dispatches, 155 passes, ... vk: imports: 48 live (2016 MB), 0 retired still imported (0 MB), 13 retired given back (1056 MB) pr468-r2: renderer v3.0.0-rc.3-77-g46b467d0, host as main's vk: 5160 presents, 20.6 fps; a frame: 1167 draws (0 skipped), 145 dispatches, 155 passes, ... vk: imports: 57 live (2032 MB), 0 retired still imported (0 MB), 9 retired given back (672 MB) $ python3 -I -B tests/test_pending_guest_import.py --cxx g++ (at cd7e4de1, main aa95d35f plus the test; rounds 1 and 2) check.exe: check.cpp:40: main()::<lambda()>: Assertion `memory_guest_imported(0x2000, 0x3000)' failed. $ python3 -I -B tests/test_pending_guest_import.py --cxx g++ (at 2273c8d3; rounds 1 and 2) (passes, exit 0) ``` A tester's Deck run, with and without this PR: dukenheimer, 2026-11-14 (https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/468#issuecomment-20355), not a test request (T-HHWD asks for Windows). Steam Deck OLED (AMD Custom APU 0932, RADV VANGOGH, 16 GB; SteamOS 3.9.5 build 20261023.100, kernel 7.2.7-valve2, Mesa 26.2.0-devel git-5c65d4fc31), stock clocks, Game Mode, docked. Build: a Deck user build of main 543846eb with #227, #225, #223, #232 and #303, with https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/384, this PR (e5889e76), https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/488 and https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/496 merged on top; "without" is the same tree with this PR's merge reverted, only `longshot_vk.so` swapped in one warm test folder. That owner's settings: the handheld profile, output 1280x800, render resolution 100%, the game's own upscaler, frame generation 3x, a 60 fps cap, vsync on, `RX_VK_READBACK_EARLY=1` and `HOS_VIDEO_LOG=1`. A scripted moving route from a Kokiri Forest save, 80 s measured, warm caches; runs in a mirrored order, 2 each way. | dukenheimer's Deck | GPU memory in the window, median (min-max), MB | its slope, MB a minute | real fps | p95 ms | |---|---|---|---|---| | with this PR (runs 1 and 8) | 5,826 (5,692-6,091); 5,866 (5,465-6,006) | +108; +296 | 13.1; 12.8 | 73.4; 73.6 | | without (runs 4 and 5) | 5,932 (5,640-6,278); 6,014 (5,452-6,078) | +55; +244 | 13.2; 12.8 | 95.2; 100.7 | No crash, exception, device lost, `vk: device fault`, `the GPU has finished nothing` or read-back timeout line in 9 route runs (7 with this PR, among them the build's cold first start) and 4 title-screen movie runs with it. GPU memory is the process's `drm-memory-gtt` + `drm-memory-vram` from `/proc/<pid>/fdinfo`, every 5 s; it grows in every run as textures stream in, and no growth is peculiar to either arm. The tester's reading: no difference in faults, memory or fps; the p95 gap follows the run order (rising toward the middle of the series whatever ran there), not the build. Its limits, as the tester put them: the routes stay in Kokiri Forest (in run 4 the route went into a house), so they may never reach the import handoff this PR guards; the Field transition is not tested there. ### Flags and settings None ## Testing **Tested:** On the Steam Deck OLED above, 4 runs of this PR interleaved with 4 of main through the Kokiri Forest to Hyrule Field transition: Field reached every time (status map `HyruleField_Child`, 914-932 presents in each 45 s sample), no device fault, no `VK_ERROR`, no crash record, the run's low-memory guard never tripped, every run exited 0. dukenheimer's Deck route runs with and without this PR (in Measurements): no fault, no difference. Game-free, on this branch rebased onto `main` aa95d35f: `tests/test_pending_guest_import.py` fails at the test-only commit cd7e4de1 and passes at 2273c8d3, twice each, alternating, and under `tests/run_all.py` with g++ 13.3 and with clang 23.1.5 at both commits; `nx2/test_import_windows.py` passes at the tip with both. The 43 checks that read `gpu/vk/vk_memory.cpp` or the changed tests pass at the tip, 43 of 43. With https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/469 and https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/472, as rebased, the branch merges cleanly (`git merge-tree`). **Not tested:** A forced background handoff or an induced GPU fault in the live game; the Field transition with this PR in anyone's run but ours; Windows and other GPUs and drivers in the game. **Must be tested:** - Windows 10/11, any GPU: build this PR (with llvm-mingw), load a child save in Kokiri Forest, go to Hyrule Field, stay 1-2 minutes and back, expecting no device lost, no `VK_ERROR`, no crash record in `longshot_log.txt` and fps like main at the same spot (by hand, test request T-HHWD: post the fps, the `vk: imports:` lines and the GPU, driver and OS) ## Risks **What could break:** this query recognizes logical ranges held at the time it runs; the host releases the renderer lock before performing the discard. Sparse-buffer padding beyond the logical binding is outside the test. Lock order is `pools_mu` then `bg_mu`, as in `bg_drain` and `pool_lookup`; the import thread never takes `pools_mu`. A range held by a pending import is now kept backed a little longer than before. **Turning it off:** revert the fix commit **Compatibility:** Nothing changes ## Issues None filed; found while tracing guest memory on the Steam Deck. Overlaps two other PRs in `memory_guest_imported`: https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/469 adds the same empty-range guard, and https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/431 changes the 1 GiB stop in the same loop. This and #469 merge with each other cleanly; whichever of these lands after #431 needs a small rebase. ## Checklist ### Everyone - [x] One topic. - [x] The title is a Conventional Commit (`type(scope): description`) of the status line's type. - [x] Why gives the reason plainly and links the issues that fit (the tracker searched). - [x] Measurements has a verdict, or one line saying why there is nothing to measure. - [x] Testing names what was tested, what wasn't, and what must still be tested, where and how. - [x] Rebased on current `main`. Onto main aa95d35f, no conflict; the test, fix and docs commits are the same changes as before (`git patch-id`). - [x] `python -I -B tests/run_all.py` passes (any failure that also fails on `main` is named). The full suite on main aa95d35f (Linux, g++ 13.3, the Vulkan headers set): 528 passed, 15 failed, 50 skipped; on this branch the new test and the 43 checks that read the changed files ran, with no failure that `main` does not have (see Testing). The 15 that fail on main: `test_appimage_player_build`, `test_bake_portability`, `test_cpm_discovery`, `test_crash_dialog`, `test_env_on`, `test_lift_cache`, `test_lift_cleanup_error`, `test_make_game_cache`, `test_mods_size_table`, `test_options_budget_once`, `test_pipeline_tour`, `test_readback_late`, `test_vk_pgo`, `test_walk_key` and `nx2/test_texture_init_barrier.py`. - [x] No game files, dumps, build output or generated files. - [x] `third_party/` untouched (a new folder with a `PROVENANCE.md` instead). - [x] The license is OK: GPL-3.0-or-later, nothing incompatible and nothing decompiled. ### When it applies - [x] AI assistance recorded with `Assisted-by:` / `Co-Authored-By:` trailers. - [x] Numbers from the same PC, scene and settings, at least 2 interleaved runs each, the median reported. - [x] A file in `docs/tested/` for each new measured run (format in `docs/tested-configurations.md`). - [x] A game-free test added for the fix, failing on `main` and passing here. - [x] A check under `tests/` that reads changed source updated in this PR, not weakened. - [x] Host code builds on both Windows (llvm-mingw) and Linux, with UTF-8 paths. Before the rebase, built on Linux for both: `RX_VK_ONLY=1 RX_RELEASE=1 RX_TARGET=windows` with llvm-mingw 20261104 linked `longshot_vk.dll`, and the same without `RX_TARGET` linked `longshot_vk.so`; after the rebase, `gpu/vk/vk_memory.cpp` at 2273c8d3 checked whole (`-fsyntax-only -Wall -Wextra`) with clang 23.1.5 for Linux and llvm-mingw 20261104 for Windows: the same diagnostics as `main` aa95d35f's (54 and 43), none new. The diff has no path handling.

zeldaootr added 3 commits 2026-11-14 16:03:47 +00:00

reproduce missed imports during handoff and retirement ... e71bda20b0

Assisted-by: OpenAI Codex

keep guest pages while background or deferred imports own them ... 81995b529e

Assisted-by: OpenAI Codex

check the full range of each queued import ... e5889e7605

Assisted-by: OpenAI Codex

zeldaootr referenced this pull request 2026-11-14 16:06:02 +00:00

fix(vk): release pages ending at a GPU import boundary #469

zeldaootr referenced this pull request 2026-11-14 16:06:21 +00:00

docs(tested): measure partially covered guest frees on Deck #471

mooki added the

area/renderer

kind

bug

priority

high

labels 2026-11-14 16:28:07 +00:00

mooki added the

platform/linux

platform/windows

labels 2026-11-14 18:24:12 +00:00

zeldaootr force-pushed import-ownership from e5889e7605 to 8fd6d885d6 2026-11-14 18:57:47 +00:00 Compare

zeldaootr commented 2026-11-14 18:57:50 +00:00

Author

Contributor

• Copy link
• Copy Markdown

Rebased onto main 1be60eae and measured on a Steam Deck OLED against main: four interleaved rounds at the Kokiri Forest to Hyrule Field transition, medians 20.60 fps on main and 20.53 here (within the run-to-run spread), no fault in any run. The run records are in docs/tested. Still WIP: not built for Windows yet; the test request asks for that.

Rebased onto main 1be60eae and measured on a Steam Deck OLED against main: four interleaved rounds at the Kokiri Forest to Hyrule Field transition, medians 20.60 fps on main and 20.53 here (within the run-to-run spread), no fault in any run. The run records are in docs/tested. Still WIP: not built for Windows yet; the test request asks for that.

zeldaootr referenced this pull request 2026-11-14 18:57:58 +00:00

perf(vk): scan pending uniform updates in place #472

zeldaootr commented 2026-11-14 19:24:30 +00:00

Author

Contributor

• Copy link
• Copy Markdown

Built for Windows from Linux with llvm-mingw 20261104 (the renderer alone, RX_VK_ONLY, as the CI job does) and for Linux: both link, and the changed file has no new warning under -Wall -Wextra on either. The Windows box is ticked; the Windows test request stays open for a run on real hardware.

Built for Windows from Linux with llvm-mingw 20261104 (the renderer alone, RX_VK_ONLY, as the CI job does) and for Linux: both link, and the changed file has no new warning under -Wall -Wextra on either. The Windows box is ticked; the Windows test request stays open for a run on real hardware.

zeldaootr changed title from WIP: fix(vk): retain guest pages during import handoff and retirement to fix(vk): retain guest pages during import handoff and retirement 2026-11-14 19:24:31 +00:00

dukenheimer referenced this pull request 2026-11-14 19:44:16 +00:00

perf(vk): the BC3 scratch has the layers a run uses, 20 MB instead of 160 #384

dukenheimer commented 2026-11-14 19:44:16 +00:00

Contributor

• Copy link
• Copy Markdown

Steam Deck OLED test, with and without this PR (dukenheimer's Deck)

• Deck: Steam Deck OLED: AMD Custom APU 0932 (RADV VANGOGH), 16 GB; SteamOS 3.9.5 build 20261023.100, kernel 7.2.7-valve2,
  Mesa 26.2.0-devel git-5c65d4fc31. Stock clocks. Game Mode, docked: the game gets a 1280x720 surface.
• Build: a Deck user build: main 543846eb + #227 0fc2e8c9 (with #225) + #223 52a61f40 + #232 3ccb48a8 + #303 995aab15, with
  #384 a9b0c880, #468 e5889e76, #488 abf2227f and #496 7fb4e2c1 merged on top. All four merged clean. Linux suite: 544 passed,
  main's 5 known failures. Built with x86-64-v3 and the release play profile, LLVM 23.1.5 (glibc 2.35).
• Without this PR: the same tree with this PR's merge reverted (git revert -m 1), built the same way. Only that file is swapped
  in one warm test folder: the host longshot for #496, longshot_vk.so for #384 and #468.
• Settings (the Deck owner's own): handheld profile (window crop 1536x864), output 1280x800, Render resolution 100%, the game's own
  upscaler, frame generation 3x, frame rate cap 60, vsync on. RX_VK_READBACK_EARLY=1 (#303) and HOS_VIDEO_LOG=1.
• Scene: Continue from a Kokiri Forest save, then a scripted moving route. 80 s are measured; caches are warm.
• Order: all four PRs (F), -#496, -#384, -#468, -#468, -#384, -#496, F: 2 runs each, mirrored about the middle.
  In runs 2 (-#496) and 4 (-#468) the scripted route went into a house after its start, so most of those two windows is an interior.

Results for this PR (#468)

Faults. None with this PR and none without. Over 9 route runs (7 with it, among them the build's cold first start; 2 without)
and 4 title-screen movie runs with it, there were none of these:

• crash, exception or device-lost lines;
• vk: device fault lines;
• the GPU has finished nothing lines;
• readback timeouts.

Memory.

	GPU memory in the window: median (min-max), MB	its slope over the 80 s, MB a minute	real fps	p95 ms
with (F, runs 1 and 8)	5,826 (5,692-6,091); 5,866 (5,465-6,006)	+108; +296	13.1; 12.8	73.4; 73.6
without (runs 4 and 5)	5,932 (5,640-6,278); 6,014 (5,452-6,078)	+55; +244	13.2; 12.8	95.2; 100.7

• The other runs that have this PR had slopes of +225 and +191 (without #496) and +140 and +225 MB a minute (without #384).
• GPU memory is the process's drm-memory-gtt + drm-memory-vram from /proc/<pid>/fdinfo, every 5 s. It grows in every run as
  textures stream in along the route.
• RSS falls 0.4 to 3.1 GB in every run as the Deck swaps.
• No growth is peculiar to either arm.

Reading. No difference in faults, memory or fps. The p95 gap follows the run order: the runs without this PR were the two in the
middle, and by position in the series p95 was 73.4, 78.3, 79.4, 95.2, 100.7, 99.7, 79.1, 73.6 ms, rising toward the middle
whatever ran there. These routes stay in Kokiri Forest (the first run without the PR went into a house), so they may never reach
the import handoff this PR guards. Your repro is a Field transition, and nothing here tests that.

**Steam Deck OLED test, with and without this PR** (dukenheimer's Deck) - **Deck:** Steam Deck OLED: AMD Custom APU 0932 (RADV VANGOGH), 16 GB; SteamOS 3.9.5 build 20261023.100, kernel 7.2.7-valve2, Mesa 26.2.0-devel git-5c65d4fc31. Stock clocks. Game Mode, docked: the game gets a 1280x720 surface. - **Build:** a Deck user build: main 543846eb + #227 0fc2e8c9 (with #225) + #223 52a61f40 + #232 3ccb48a8 + #303 995aab15, with #384 a9b0c880, #468 e5889e76, #488 abf2227f and #496 7fb4e2c1 merged on top. All four merged clean. Linux suite: 544 passed, main's 5 known failures. Built with x86-64-v3 and the release play profile, LLVM 23.1.5 (glibc 2.35). - **Without this PR:** the same tree with this PR's merge reverted (`git revert -m 1`), built the same way. Only that file is swapped in one warm test folder: the host `longshot` for #496, `longshot_vk.so` for #384 and #468. - **Settings (the Deck owner's own):** handheld profile (window crop 1536x864), output 1280x800, Render resolution 100%, the game's own upscaler, frame generation 3x, frame rate cap 60, vsync on. `RX_VK_READBACK_EARLY=1` (#303) and `HOS_VIDEO_LOG=1`. - **Scene:** Continue from a Kokiri Forest save, then a scripted moving route. 80 s are measured; caches are warm. - **Order:** all four PRs (F), -#496, -#384, -#468, -#468, -#384, -#496, F: 2 runs each, mirrored about the middle. In runs 2 (-#496) and 4 (-#468) the scripted route went into a house after its start, so most of those two windows is an interior. **Results for this PR (#468)** **Faults.** None with this PR and none without. Over 9 route runs (7 with it, among them the build's cold first start; 2 without) and 4 title-screen movie runs with it, there were none of these: - crash, exception or device-lost lines; - `vk: device fault` lines; - `the GPU has finished nothing` lines; - readback timeouts. **Memory.** | | GPU memory in the window: median (min-max), MB | its slope over the 80 s, MB a minute | real fps | p95 ms | |---|---|---|---|---| | with (F, runs 1 and 8) | 5,826 (5,692-6,091); 5,866 (5,465-6,006) | +108; +296 | 13.1; 12.8 | 73.4; 73.6 | | without (runs 4 and 5) | 5,932 (5,640-6,278); 6,014 (5,452-6,078) | +55; +244 | 13.2; 12.8 | 95.2; 100.7 | - The other runs that have this PR had slopes of +225 and +191 (without #496) and +140 and +225 MB a minute (without #384). - GPU memory is the process's `drm-memory-gtt` + `drm-memory-vram` from `/proc/<pid>/fdinfo`, every 5 s. It grows in every run as textures stream in along the route. - RSS falls 0.4 to 3.1 GB in every run as the Deck swaps. - No growth is peculiar to either arm. **Reading.** No difference in faults, memory or fps. The p95 gap follows the run order: the runs without this PR were the two in the middle, and by position in the series p95 was 73.4, 78.3, 79.4, 95.2, 100.7, 99.7, 79.1, 73.6 ms, rising toward the middle whatever ran there. These routes stay in Kokiri Forest (the first run without the PR went into a house), so they may never reach the import handoff this PR guards. Your repro is a Field transition, and nothing here tests that.

dukenheimer referenced this pull request 2026-11-14 19:44:17 +00:00

fix(vi): pace at 240 while a movie plays, so its player stops losing pictures #488

dukenheimer referenced this pull request 2026-11-14 19:44:18 +00:00

fix(host): a thread lowered right after it is created runs lowered on Linux #496

zeldaootr referenced this pull request 2026-11-15 00:19:25 +00:00

fix(vk): give memory back under Linux memory pressure and after the precompile #223

zeldaootr commented 2026-11-15 16:41:37 +00:00

Author

Contributor

• Copy link
• Copy Markdown

Thank you, dukenheimer, for the Deck runs with and without this PR: they are in the description as you reported them, with your reading (no difference in faults, memory or fps) and your limit (the Kokiri routes may never reach the import handoff). Rebased onto main aa95d35f with no conflict (the same three changes, git patch-id) and the description moved to the current template.
The game-free test was run again on the new base (fails at the test-only commit, passes at the tip, two rounds, g++ and clang; the 43 checks that read the changed files pass); the Field numbers still hold because main changed vk_memory.cpp since 9c9d69ed only in how the mirror budget is set (#735), not in memory_guest_imported() or the import tables. T-HHWD (Windows) still waits for the testing wanted label.

Thank you, dukenheimer, for the Deck runs with and without this PR: they are in the description as you reported them, with your reading (no difference in faults, memory or fps) and your limit (the Kokiri routes may never reach the import handoff). Rebased onto `main` aa95d35f with no conflict (the same three changes, `git patch-id`) and the description moved to the current template. The game-free test was run again on the new base (fails at the test-only commit, passes at the tip, two rounds, g++ and clang; the 43 checks that read the changed files pass); the Field numbers still hold because `main` changed `vk_memory.cpp` since 9c9d69ed only in how the mirror budget is set (#735), not in `memory_guest_imported()` or the import tables. T-HHWD (Windows) still waits for the `testing wanted` label.

zeldaootr force-pushed import-ownership from 8fd6d885d6 to 2273c8d36e 2026-11-15 16:42:08 +00:00 Compare

Some checks are pending

Hide checks

CI / Conventional Commit title (pull_request) Blocked by required conditions

Details

CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Blocked by required conditions

Details

CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Blocked by required conditions

Details

CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Blocked by required conditions

Details

CI / Workflow lint (actionlint) (pull_request) Blocked by required conditions

Details

Guide screenshots / Guide images and local UI source review (pull_request) Blocked by required conditions

Details

Third-party parts / As their notes say (pull_request) Blocked by required conditions

Details

Third-party parts / The same as upstream (pull_request) Blocked by required conditions

Details

Website / Website on Cloudflare (pull_request) Blocked by required conditions

Details

PR description / Description follows the template (pull_request_target) Successful in 4s

Details

This pull request doesn't have enough approvals yet. 0 of 1 approvals granted.

Some workflows are waiting to be reviewed.

This branch is out-of-date with the base branch

You are not authorized to merge this pull request.

View command line instructions

Checkout

From your project repository, check out a new branch and test the changes.

git fetch -u 'import-ownership:zeldaootr-import-ownership'

git switch 'zeldaootr-import-ownership'

Sign in to join this conversation.

Reviewers

No reviewers

Labels

Clear labels   

area/accessibility

Barriers for people with disabilities

  

area/audio

Sound and music

  

area/input

Controllers, keyboard, mouse

  

area/install

Installing, updating, building from your game

  

area/menu

In-game menu and settings

  

area/renderer

Vulkan renderer, shaders, effects

  

area/saves

Save files

  

area/upscaling

DLSS, FSR, frame generation

  

beanstalk: approved @ 1697603

Set by Beanstalk for show; the farm's database is what counts

  

beanstalk: approved @ 2c5a62e

Set by Beanstalk for show; the farm's database is what counts

  

beanstalk: approved @ 4fd82b7

Set by Beanstalk for show; the farm's database is what counts

  

beanstalk: approved @ 64d6d09

Set by Beanstalk for show; the farm's database is what counts

  

beanstalk: approved @ 6f91fae

Set by Beanstalk for show; the farm's database is what counts

  

beanstalk: approved @ 70a78f9

Set by Beanstalk for show; the farm's database is what counts

  

beanstalk: approved @ 8d361dc

Set by Beanstalk for show; the farm's database is what counts

  

beanstalk: approved @ 9d8356e

Set by Beanstalk for show; the farm's database is what counts

  

beanstalk: approved @ abf2227

Set by Beanstalk for show; the farm's database is what counts

  

beanstalk: approved @ d82f202

Set by Beanstalk for show; the farm's database is what counts

  

beanstalk: approved @ e5f1e40

Set by Beanstalk for show; the farm's database is what counts

  

beanstalk: approved @ ee8ab5f

Set by Beanstalk for show; the farm's database is what counts

  

ci: approved

A maintainer approved CircleCI for this fork PR (sticky; removed after 14 quiet days)

  

good first issue

A good start for a new contributor

  

gpu/amd

AMD GPUs

  

gpu/igpu

Integrated GPUs

  

gpu/intel

Intel GPUs

  

gpu/nvidia

NVIDIA GPUs

  

help wanted

Extra hands or knowledge wanted

  

kind

bug

Something doesn't work as it should

  

kind

chore

Build, CI, dependencies, tidying

  

kind

docs

Documentation

  

kind

feature

A new feature or an improvement

  

kind

performance

Frame rate, frame time, loading, memory

  

kind

question

A question rather than a change

  

needs measurements

Waiting for before/after measurements

  

platform/linux

Affects Linux (a change for every system carries both Windows and Linux)

  

platform/macos

macOS (Apple silicon or Intel Macs)

  

platform/steam-deck

Steam Deck

  

platform/windows

Affects Windows (a change for every system carries both Windows and Linux)

  

preview

Maintainers: deploy a preview of this PR's website (read its changes first). Removed after the run.

  

priority

critical

Crashes, data loss, or blocks most players

  

priority

high

Important: next in line

  

priority

low

Nice to have

  

regression

Worked in an earlier build, broken in a newer one: highest priority, above other bugs

  

status

blocked

Waits on something else first

  

status

confirmed

Reproduced or agreed on

  

status

duplicate

Already reported or proposed

  

status

invalid

Not a problem in this project

  

status

needs-info

Waiting for more information from the reporter

  

status

wontfix

Will not be worked on

  

testing wanted

Wanted: a test on other hardware (posts a test request in #test-requests)

  

ci: approved

No labels

area/accessibility

area/audio

area/input

area/install

area/menu

area/renderer

area/saves

area/upscaling

beanstalk: approved @ 1697603

beanstalk: approved @ 2c5a62e

beanstalk: approved @ 4fd82b7

beanstalk: approved @ 64d6d09

beanstalk: approved @ 6f91fae

beanstalk: approved @ 70a78f9

beanstalk: approved @ 8d361dc

beanstalk: approved @ 9d8356e

beanstalk: approved @ abf2227

beanstalk: approved @ d82f202

beanstalk: approved @ e5f1e40

beanstalk: approved @ ee8ab5f

ci: approved

good first issue

gpu/amd

gpu/igpu

gpu/intel

gpu/nvidia

help wanted

kind

bug

kind

chore

kind

docs

kind

feature

kind

performance

kind

question

needs measurements

platform/linux

platform/macos

platform/steam-deck

platform/windows

preview

priority

critical

priority

high

priority

low

regression

status

blocked

status

confirmed

status

duplicate

status

invalid

status

needs-info

status

wontfix

testing wanted

ci: approved

Milestone

Clear milestone

No items

No milestone

Projects

Clear projects

No items

No project

Assignees

Clear assignees

No assignees

2 participants

Notifications

Subscribe

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference

MagicBeanDev/Longshot!468

No description provided.