WIP: fix(linux): RX_VK_NO_DMABUF leaves the guest window anonymous memory, so its pools can be host-pointer imports #2100
No reviewers
Labels
No labels
area/accessibility
area/audio
area/input
area/install
area/menu
area/renderer
area/saves
area/tooling
area/upscaling
beanstalk: approved
beanstalk: failed
beanstalk: needs approval
beanstalk: running
beanstalk: tested
ci: approved
good first issue
gpu/amd
gpu/igpu
gpu/intel
gpu/nvidia
help wanted
kind
bug
kind
chore
kind
docs
kind
feature
kind
performance
kind
question
needs measurements
platform/linux
platform/macos
platform/steam-deck
platform/windows
preview
priority
critical
priority
high
priority
low
regression
risk
high
risk
low
risk
medium
status
blocked
status
confirmed
status
duplicate
status
invalid
status
needs-info
status
wontfix
ci: approved
No milestone
No project
No assignees
3 participants
Notifications
Due date
No due date set.
Dependencies
No dependencies set
Reference
MagicBeanDev/Longshot!2100
Loading…
Reference in a new issue
No description provided.
Delete branch "dyst/Longshot:fix/no-dmabuf-window"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Summary
On Linux,
RX_VK_NO_DMABUF=1is meant to hand the GPU guest memory as host pointers instead of dma-bufs. On RADV itgave no imports at all: every pool was copied, and the game crawled after a save loaded.
The cause is in
reserve_guest(host/hos_main.c). It makes the guest window a memfd's shared mapping whenever/dev/udmabufopens, whateverRX_VK_NO_DMABUFsays. With that switch the renderer makes no dma-bufs(
gpu/vk/vk_memory.cppimport_dma_buf) and asks for host-pointer imports instead. RADV refuses a shared mapping'spages as host pointers (
VK_ERROR_INVALID_EXTERNAL_HANDLE, asreserve_guest's own comment says), so nothing wasimported.
Now, with
RX_VK_NO_DMABUFon, the window is plain anonymous memory, and the log says so:mem: RX_VK_NO_DMABUF: the guest window is anonymous memory, for host-pointer imports.env_on, the rule the renderer reads it by, soRX_VK_NO_DMABUF=0keeps the memfd window andthe dma-bufs.
HOS_GUEST_MEMFDstill decides when it is set.longshot_env.txtis applied before the window is made (maincallsload_longshot_envbeforegame_main), so theswitch works from there too.
To see it on main (Linux, RADV,
/dev/udmabufusable): start withRX_VK_NO_DMABUF=1and load a save. The log showspools
not importedand no live imports, and the game runs at a few fps for a while after the save loads.Found by Mooseca in HeyListen, a fork of Longshot's history:
https://github.com/Sacrappy/HeyListen/pull/75. There, RX 9070 XT on RADV 26.2.6: 170 pools refused and 0 imported with
the switch, 2-5 fps for ~15 s after the save loads; with their fix, 0 refused and about 2 GB imported. Their fix tests
getenv() != NULL, soRX_VK_NO_DMABUF=0would also have turned the memfd off; this one takes the renderer's rule.Related: #1164 falls back to host pointers when udmabuf rejects
a pool. Over a memfd window that fallback meets the same RADV refusal, so the two touch the same case from two sides.
This PR does not change
vk_memory.cpp.Read first:
reserve_guestinhost/hos_main.c, thentests/test_no_dmabuf_window.py.Testing
Tested:
tests/test_no_dmabuf_window.py, on Windows (it cuts the condition out and runs everywhere).HOS_GUEST_MEMFDunset, on or off;RX_VK_NO_DMABUFunset, on (1,yes,on) or a word that says off (0,off, empty).mainfor the bug: with the switch at1,yesoronthe window is still a memfd.getenv() != NULLin its place (HeyListen's reading) it failson
0,offand empty.tests/run_all.py --changed), on Windows: 42 pass, 8 are skipped onWindows. 10 fail, and they fail the same way with
main'shost/hos_main.c:test_crash_record,test_host_guest_writes,test_profiling_mode,test_slow_frames,test_display_startup_failure,test_mods_list,test_vibration_menu,test_crash_dialog,test_perf_report,test_lift_cache.Not run in the game: the change is in the Linux-only branch of
reserve_guest, and my PC is Windows with an RTX 3080.Not compiled on Linux here either; CI's Linux build does that.
Still to test:
Linux, RADV (
radv),/dev/udmabufusable: each build withRX_VK_NO_DMABUF=1and with nothing set. Load a save,Hyrule Field then the Market. Expected:
mainwith the switch: poolsnot importedand a few fps for a while after the load;(@Beanstalk)
Linux, NVIDIA (
linux+nvidia), the same withRX_VK_NO_DMABUF=1: host-pointer imports on anonymous memory, asHOS_GUEST_MEMFD=0gives today (@Beanstalk, or by hand).Measurements
A fix to a switch's start-up; the default is unchanged, so there is no speed to measure on
main's defaults. Thein-game evidence (the imports and the frame times after a save loads, with the switch) comes from the run above.
936d4e0092f236c54ctests/test_no_dmabuf_window.pyRX_VK_NO_DMABUF=1,yes,on: memfd window)RX_VK_NO_DMABUF=1Risks
Only runs with
RX_VK_NO_DMABUFon change. Their guest window is anonymous memory, which is whatHOS_GUEST_MEMFD=0already gives. Those runs lose the dma-buf path, which is what the switch asks for, so on AMD their submissions are
slower than with dma-bufs; anyone who wants the memfd anyway sets
HOS_GUEST_MEMFD=1. A revert is enough. Nothingchanges after updating for anyone who doesn't set the switch.
Checklist
Ready for review
main; the title is a Conventional Commit.mise run testpasses (any failure that also fails onmainis named). (only the checks that read the changed files were run: 10 failures, all also onmain, named above)main, or Measurements says why there is nothing to measure.third_party/untouched; nothing GPL-incompatible.When it applies
mainand passes here.Assisted-by:/Co-Authored-By:trailers.Ready to merge
main, measured again or a comment says why the numbers still hold.🤖 Generated with Claude Code
@Beanstalk only RADV on Linux, with /dev/udmabuf usable: this changes only what RX_VK_NO_DMABUF=1 does at start-up. Run main and this PR each with RX_VK_NO_DMABUF=1 set, and each with nothing set. Load a save in Hyrule Field, stand a minute, then go to the Market. With the switch, main should log pools 'not imported' and run at a few fps for a while after the load; this PR should log 'mem: RX_VK_NO_DMABUF: the guest window is anonymous memory, for host-pointer imports', import its pools and play as it does without the switch. With nothing set the two builds should match (the window a memfd's, guest memory as dma-bufs). Report the import lines ('not imported', 'imports: ... live'), the frame times in the first 30 s after the load, and the 'vk: guest memory as dma-bufs' line of each run.
Possibly related, for context:
2f236c54c0b04ee7b0cb@Beanstalk (again for
b04ee7b, rebased past #2098's build fix) only RADV on Linux, with /dev/udmabuf usable: this changes only what RX_VK_NO_DMABUF=1 does at start-up. Run main and this PR each with RX_VK_NO_DMABUF=1 set, and each with nothing set. Load a save in Hyrule Field, stand a minute, then go to the Market. With the switch, main should log pools 'not imported' and run at a few fps for a while after the load; this PR should log 'mem: RX_VK_NO_DMABUF: the guest window is anonymous memory, for host-pointer imports', import its pools and play as it does without the switch. With nothing set the two builds should match (the window a memfd's, guest memory as dma-bufs). Report the import lines ('not imported', 'imports: ... live'), the frame times in the first 30 s after the load, and the 'vk: guest memory as dma-bufs' line of each run.Beanstalk test farm · ❌ failed on linux+radv
Commit
b04ee7b· approved by dyst (reviewed) · dashboard⚠️ Note: The farm cannot enforce that /dev/udmabuf is usable; without it, the requested main-side failure and default dma-buf path may not be exercised.
⚠️ Note: The first 30-second measurement starts after teleporting to Hyrule Field, not immediately after the Kokiri save loads. Log matches are reported but cannot be made conditional on the switch in these separate scenarios.
Done (started 2026-11-21 23:49 UTC, last report 01:25 UTC). Run 2: re-planned by dyst. Comment
@Beanstalkto run it again.linux+radvWhy it failed
linux+radv:switch-unsetscreenshot.max_diff:limit 0.1Couldn't be judged: the tester's PC could not finish the test. The farm tries another PC, or the same one after a fix; it says nothing about the PR on its own.
linux+radv, report 2:Insufficient frame-time evidence for 4 required comparison(s). See rule results. a step ran out of time: timeout: the save did not load into play within 900 s (then the game was in menu on KokiriForest_Child); a step ran out of time at stepWhat was tested
What will be tested · AI plan (
gpt-6.1-terra)On Linux RADV, compare main and the PR with RX_VK_NO_DMABUF=1 and with the switch unset. Each run loads the Kokiri save, measures the first 30 seconds in Hyrule Field, stands there for another 30 seconds, then visits the Market. The plan reports frame times and import-related log matches, and fails on crashes, substantial frame-time regressions or picture differences. It takes approximately 45 minutes per tester.
kokiri-start(switchesRX_VK_NO_DMABUF=1), then teleport to Hyrule Field, measure field-first-30 for 30 s (after 0 s warm-up), take the picture field to compare with main, teleport to Market, measure market for 10 s (after 5 s warm-up), take the picture market to compare with main; 2 runs on each side.kokiri-start, then teleport to Hyrule Field, measure field-first-30 for 30 s (after 0 s warm-up), take the picture field to compare with main, teleport to Market, measure market for 10 s (after 5 s warm-up), take the picture market to compare with main; 2 runs on each side.Measurements on `linux+radv`, for the PR description
Verdict: fail on linux+radv (2 reports):
switch-unset screenshot.max_difffailed.Hardware: 1: linux+amd+radv+dgpu
AMD Radeon RX 7900 XTX (RADV NAVI31), 26.2.7AMD Ryzen 7 5800X3D 8-Core Processor, 32 GB RAM; 2: linux+amd+radv+igpuAMD Radeon 890M Graphics (RADV STRIX1), 26.1.99AMD Ryzen Z2 Extreme, 32 GB RAMScene: switch-on, from the repository's save kokiri-start (RX_VK_NO_DMABUF=1); switch-unset, from the repository's save kokiri-start; each cell's settings and switches as the table names them
Method: the Beanstalk test farm: main and this PR built and run on the same PC in one session, game starts alternating (ABBA); switch-on: 2 game starts a side, field-first-30 30 s after 0 s warm-up, market 10 s after 5 s warm-up; switch-unset: 2 game starts a side, field-first-30 30 s after 0 s warm-up, market 10 s after 5 s warm-up; each number the median of the runs, then of 2 reports
How to reproduce: the farm's plan above (
@Beanstalkon this PR runs it again)d76d202b04ee7bFlags and settings
None: the plan varied no setting, switch or build option.
As Markdown for the PR description:
The testers' hardware
linux+radvAMD Radeon RX 7900 XTX (RADV NAVI31), 26.2.7AMD Ryzen 7 5800X3D 8-Core Processor, 32 GB RAMlinux+radvAMD Radeon 890M Graphics (RADV STRIX1), 26.1.99AMD Ryzen Z2 Extreme, 32 GB RAM@Beanstalk test it anyway
View command line instructions
Checkout
From your project repository, check out a new branch and test the changes.