fix(linux): guest memory as dma-bufs under Steam and Flatpak, where udmabuf's size_limit_mb can't be read #1155
No reviewers
Labels
No labels
area/accessibility
area/audio
area/input
area/install
area/menu
area/renderer
area/saves
area/tooling
area/upscaling
beanstalk: approved
beanstalk: failed
beanstalk: needs approval
beanstalk: running
beanstalk: tested
ci: approved
good first issue
gpu/amd
gpu/igpu
gpu/intel
gpu/nvidia
help wanted
kind
bug
kind
chore
kind
docs
kind
feature
kind
performance
kind
question
needs measurements
platform/linux
platform/macos
platform/steam-deck
platform/windows
preview
priority
critical
priority
high
priority
low
regression
risk
high
risk
low
risk
medium
status
blocked
status
confirmed
status
duplicate
status
invalid
status
needs-info
status
wontfix
ci: approved
No milestone
No project
No assignees
4 participants
Notifications
Due date
No due date set.
Dependencies
No dependencies set
Reference
MagicBeanDev/Longshot!1155
Loading…
Reference in a new issue
No description provided.
Delete branch "pepe999/Longshot:fix/udmabuf-limit-unreadable"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Type: fix · Risk: low · Tested on: Linux (Bazzite), RX 6700 XT, RADV · Needs: Linux, Flatpak Steam on a real AMD setup (a Steam Deck or an APU)
Why
Under Steam's Flatpak, the game on an AMD GPU runs at a few frames a second, though the same build runs normally from the desktop. The Flatpak shows the game
/dev/udmabufbut hides/sys/module, where udmabuf's size limit lives. Whenhost/hos_main.ccouldn't read that limit, it kept guest memory as host-pointer imports, the slow path on AMD. The renderer never needed that number: when the file can't be read it already uses udmabuf's default of 64 MB.Fixes #1135.
How to reproduce
Seen on: Linux, AMD (the report: ROG Ally X, Flatpak Steam, Game Mode; here: RX 6700 XT, RADV, Mesa 26.2.6, with Flatpak's view of
/sysimitated)Settings and switches: defaults;
HOS_GUEST_MEMFDunsetSteps: start the game with
/sys/modulehidden:bwrap --dev-bind / / --tmpfs /sys/module -- ./bin/longshot, then load a saveExpected:
mem: the guest window is a memfd's ..., thenvk: guest memory as dma-bufs: yes, at the same fps as a start from the desktopActual:
mem: guest memory stays host-pointer imports for the GPU (slow submissions on AMD): udmabuf's size_limit_mb cannot be read; 1-4 fps in Link's houseWhat changed
If
/dev/udmabufopens, the guest window is now a memfd, whether or notsize_limit_mbcan be read. The renderer reads the limit itself and falls back to 64 MB, as before. A guess that is too small only splits pools into more dma-bufs.host/hos_main.c(the oneHOS_GUEST_MEMFDoverrides) no longer reads the limit intolimit_mb. It only opens the file to logmem: udmabuf's size_limit_mb cannot be read (in a container?): its default, 64 MB, assumedwhen that fails. The host-pointer message now covers only/dev/udmabuffailing to open.tests/test_udmabuf_limit_unreadable.py. It cuts that block out ofhos_main.cand runs it with the limit file missing, holding 64, 256, 0, no number and nothing, and with/dev/udmabufnot opened.docs/tested/.Measurements
Verdict: under the sandbox, main runs at 1.0-3.9 fps and this PR at 55.2-58.2 fps (medians), the same as main started without a sandbox (53.3-54.2). The slow path is gone.
Hardware: Bazzite (Fedora 45), KDE Plasma Wayland, kernel 7.2.15; AMD Radeon RX 6700 XT, RADV NAVI22, Mesa 26.2.6; Intel Core i7-14700F, 32 GB
Scene: Link's house (repo save
kokiri-start), standing still; output 2560x1440, quality preset 6, the game's own upscaler,HOS_DYNRES=0,HOS_FPS_MAX=1000, frame generation off, headless, audio offMethod:
scripts/dev_build.pybase at main71b1edd8, this branch as a pack,rtandrt --emptyfrom that base.longshot_ctl.py launch --repo-save kokiri-start --headlesswith--wrapper '["bwrap","--dev-bind","/","/","--tmpfs","/sys/module","--"]'; A pressed untilstateisplay, 20 s warm-up, thenstatus.fpsonce a second for 40 s. One warm-up run of main first, then main and the PR interleaved 3 times; the median of each run.How to reproduce: the commands above, from any Linux PC with
/dev/udmabufaccess and an AMD GPU71b1edd811c805d471b1edd8Pipelines created were about the same on every run (9,421-9,582), so neither side was compiling while it was measured.
Flags and settings
None.
HOS_GUEST_MEMFD=0and=1still force each path, as before.Testing
Tested:
tests/test_udmabuf_limit_unreadable.pyfails on main (got usable=0), passes here, and fails again with the fix'susable = 1turned back to 0.tests/test_udmabuf_warning.py, which cuts the code just above, still passes.tests/run_all.py: 573 passed, 14 failed, 91 skipped. The same 14 fail on main71b1edd8on this PC (missing Vulkan/SPIR-V headers and clang on the host, and Popen mocks):test_linux_update,test_make_game_cache,test_crt_options,test_failed_compile_keeps_objects,test_dlss_feature_size,test_guest_imported,test_bug_report_defaults,test_link_skip,test_missing_nup,test_pending_guest_import,test_mods_size_table,test_run_all_changed,test_walk_key,test_vk_occlusion_queries.dev_build.py.Not tested:
/syswas imitated with bwrap.hos_main.cand is not built there.vk_memory.cpp), whatever the window is.Must be tested:
longshot_log.txt, expectingmem: the guest window is a memfd'sandits default, 64 MB, assumed, with fps near a desktop start's (the reporter saw 6-10 fps before and ~30 withHOS_GUEST_MEMFD=1) (Test Farm: @Beanstalk linux amd steamdeck, start under Flatpak Steam and compare fps with main)Risks
What could break: a container that shows
/dev/udmabufbut whose real limit is below 64 MB would get dma-bufs udmabuf refuses. That is the renderer's existing fallback, untouched here, and it already applies to a readable limit of 0.Turning it off:
HOS_GUEST_MEMFD=0(anonymous memory, as before), or revert the commit.Compatibility: nothing changes: no saves, options, caches, vk ABI or builder tools touched.
Issues
Fixes #1135
Checklist
Everyone
type(scope): description) of the status line's type.main.python -I -B tests/run_all.pypasses (any failure that also fails onmainis named).third_party/untouched (a new folder with aPROVENANCE.mdinstead).When it applies
Assisted-by:/Co-Authored-By:trailers.docs/tested/for each new measured run (format indocs/tested-configurations.md).mainand passing here.Tested in game: RX 6700 XT, RADV NAVI22, Mesa 26.2.6, Bazzite (Fedora 45), kernel 7.2.15: the game started in a Flatpak-like sandbox (bwrap with /sys/module hidden), Link's house from the kokiri-start repo save. main keeps host-pointer imports at 1.0 fps (median of 3 runs); this branch takes the memfd window with 64 MB dma-bufs at 56.2 fps, the same as main started without a sandbox (53.8).
WIP: fix(linux): guest memory as dma-bufs under Steam and Flatpak, where udmabuf's size_limit_mb can't be readto fix(linux): guest memory as dma-bufs under Steam and Flatpak, where udmabuf's size_limit_mb can't be read@Beanstalk linux amd steamdeck: please start the game from Flatpak Steam (Game Mode if the machine has it), load the kokiri-start save and compare main with this PR. Expected: main logs
udmabuf's size_limit_mb cannot be readand runs slowly; this PR logsmem: the guest window is a memfd'sandits default, 64 MB, assumed, at about the fps of a desktop start. I could only imitate the Flatpak's /sys with bwrap, because my Steam is native.@pepe999 only members of
BigBeanscan start the farm. One of them can approve this request by reacting 👍 to it.@Beanstalk steamdeck and handheld: start from Steam in Game Mode (Flatpak), load a save, and compare main with this PR on Death Mountain Trail and Hyrule Field.
Expected on this PR: the log says the guest window is a memfd, and that udmabuf's default of 64 MB is assumed, with fps near a desktop start. On main: udmabuf's size_limit_mb cannot be read, and a few frames a second.
Please report that memfd log line, fps and frame times. The desktop measurement (RX 6700 XT, with /sys hidden) was 1.0 fps on main and 56.2 fps here. The Ally X report was 6-10 fps, becoming about 30 with the memfd path forced on.
@TrashPanda only members of
BigBeanscan start the farm. One of them can approve this request by reacting 👍 to it.Code-only review of
7bc5e522e3; informational findings, with final approval left to another reviewer.P2 — Keep a failed dma-buf capability check from removing the working import path. At
host/hos_main.c:186,usable = 1now also accepts a readablesize_limit_mbcontaining0. Previously that case selected anonymous memory. Zero is a real integer module setting: Linux's udmabuf implementation rejects nonempty creates when their page count exceeds that zero limit. The renderer instead treats zero as its 64 MiB fallback (gpu/vk/vk_memory.cpp:345–351), so every dma-buf request fails. This is not equivalent to the previous fallback: the window is alreadyMAP_SHARED, andreserve_guestexplicitly documents that RADV rejects host-pointer imports of that mapping. If that documented driver behavior applies,import_poolfails too,pool_ofreturns null (line 3126), and binding callers return empty bindings (lines 3964/4200); guest page-table entries are not published. A hidden, nondefault limit below the assumed piece size exposes the same risk for larger pools. This is a source-derived regression case, not a reproduced driver failure, and does not dispute the reported improvement with the normal limit.Preserve anonymous memory for a known unusable limit, and validate a usable dma-buf size before committing to the shared-window path when the limit is unknown (or supply a demonstrated compatible fallback). The fixture currently asserts that zero is usable and stops before the ioctl/import path; extend coverage to rejected creates and the resulting backing choice. The description's claim that readable zero already had this fallback is incorrect for the previous startup gate.
The posted RX 6700 XT bwrap results cover the intended hidden-sysfs case. The already-requested real Flatpak Steam/Game Mode AMD/Deck run remains outstanding; no duplicate request is needed. I checked the full diff, startup mapping/cleanup, renderer import and failure consumers, fixture, recorded runs and current comments. No competing active review/revision claim found; no substantive duplicate identified among the shared-file candidates. No tests, builds or game runs performed, and no source changes made.
View command line instructions
Checkout
From your project repository, check out a new branch and test the changes.