fix(vk): the window's swapchain is let go at the close, not at the process's end #1105
No reviewers
Labels
No labels
area/accessibility
area/audio
area/input
area/install
area/menu
area/renderer
area/saves
area/tooling
area/upscaling
beanstalk: approved
beanstalk: failed
beanstalk: needs approval
beanstalk: running
beanstalk: tested
ci: approved
good first issue
gpu/amd
gpu/igpu
gpu/intel
gpu/nvidia
help wanted
kind
bug
kind
chore
kind
docs
kind
feature
kind
performance
kind
question
needs measurements
platform/linux
platform/macos
platform/steam-deck
platform/windows
preview
priority
critical
priority
high
priority
low
regression
risk
high
risk
low
risk
medium
status
blocked
status
confirmed
status
duplicate
status
invalid
status
needs-info
status
wontfix
ci: approved
No milestone
No project
No assignees
1 participant
Notifications
Due date
No due date set.
Dependencies
No dependencies set
Reference
MagicBeanDev/Longshot!1105
Loading…
Reference in a new issue
No description provided.
Delete branch "omgitsgela/Longshot:fix/deck-exit-display-release"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Type: fix · Risk: low · Tested on: Steam Deck LCD (Jupiter, RADV VANGOGH), SteamOS, Game Mode + desktop; Linux x86-64 game-free checks · Needs: any Game Mode player who hits the post-exit artifact; Wayland/X desktop and Windows near a close
Status
Ready for review and for other Game Mode players to test. On this Deck (LCD): the fix installs and runs, its close marker fires exactly once per quit, and no regression appeared. Three scripted repro conditions (light 1280x800; deep 2304x1440 loop-walk; fresh-area teleport-and-quit) ran paired old-vs-fixed; none reproduced the liftoff ENOMEM/modeset fingerprint even on the unfixed build, despite peaks matching the reported couch session (10.1/10.5 GB vs 9.9 GB). Details and the full protocol in the comments on this PR (latest: #issuecomment-35622) and in the run files
docs/tested/2026-11-16-deck-exit-close-omgitsgela-gm01.mdand…suite01.md. The scripted harness cannot discriminate the race; real-world play from other Game Mode users is now the confirmation path.Why
A player quit Longshot inside the Deck's Game Mode. Afterwards rainbow/garbage pixels lingered on top of the Game Mode UI until they switched to desktop mode; returning to Game Mode later was clean. They nearly thought their GPU had died. (Reported in the couch session of 2026-11-15; machine evidence below.)
The Deck's own logs from that second:
~/.local/share/Steam/logs/console-linux.txt, Game Mode pipes stdout there), 19:59:14: working set 7136 MB (peak 9902), VRAM 3710/5890 MB, the ~9.8 GB memfd guest window shared with the GPU as dma-bufs, last linedisplay: window closed, exiting;liftoff: drmModeAtomicCommit: Cannot allocate memory(×4),xwm: Failed to prepare 1-layer flip (Cannot allocate memory), then a re-detect storm,drm: could not find mode!,xwm: We failed our modeset and have no mode to fall back to!in a loop.The correlation is exact and the mechanism is plausible: at exit this process holds most of the system's memory, and the commit's own allocation fails. A failed modeset leaves the scan-out pointing at buffers the emulator owned — which is what "garbage over the UI" looks like — and a mode switch rebuilds the display config, which is exactly how the player recovered. gamescope's own tracker has prior art for a stuck final frame on game exit (ValveSoftware/gamescope#1822).
What this PR does not claim: that the ENOMEM is proven caused by our allocations (timestamp correlation is not proof), or that it explains every post-exit artifact. Two minutes earlier the same session logged
create_swapchain: Surface already had a gamescope_swapchainat an in-game resolution change, and Mesa/gamescope bugs of their own exist. But our exit path has a real, fixable gap regardless — see How to reproduce.How to reproduce
Seen on: SteamOS (Steam Deck LCD, Jupiter / RADV VANGOGH), Game Mode, session after long heavy play (that one session; scripted arms under this PR's Measurements did not re-trigger it).
Settings and switches: defaults; the couch session ran at native 1280x800 with the launcher's env file switches.
Steps: play a heavy outdoor area until the game's working set approaches the system ceiling (the couch session reached working set ~7.1 GB, peak 9.9 GB of a 14.8 GB pool), then quit the game (window close or menu exit) while still in Game Mode.
Expected: the Game Mode UI returns clean.
Actual: that session (not since reproduced by script): garbage/rainbow pixels over the Game Mode UI until a session switch to desktop mode; the journal shows the liftoff ENOMEM + modeset-failure storm at the quit second. Honest caveat: three scripted paired repro conditions (this PR's Measurements) never triggered the fingerprint even on unfixed main, despite matching the couch memory peaks. The reproducible part is the code gap below; the visual artifact needs a player who hits it.
The code gap, checkable on main without the bug:
mkwvk_close_asked(the deliberate close: window closed, control-bridge quit, menu's Restart now) saves the pipeline cache and stops the shader preparers — and then nothing. No device wait, no swapchain destroy, no semaphore release. Every Vulkan object, including the swapchain's images and whatever they pin in VRAM/GTT, goes down only when the process exits, racing the compositor's final frames. A grep on main:retire_swapchain()exists (device wait idle, drop views, destroy the rendered semaphores, destroy the chain) and is never called from any close path.What changed
At the deliberate close, the renderer now lets go of the window's GPU-side display resources deliberately and in order, instead of leaving process teardown to race the compositor.
gpu/vk/vk_internal.h:dev.closing(atomic bool); externs the present mutex.gpu/vk/vk_main.cpp(mkwvk_close_asked): after preparers stop and caches save — setclosing, takepresent_image_mu(every present, direct or frame-generated, is under it), drain the presenter thread, thenretire_swapchain()underdev.present_mu— the same lock orderpresent_imageitself uses, so no present is mid-acquire on an image being destroyed. Logsthe swapchain is let goonce, so support can verify the path fired.gpu/vk/vk_main.cpp(start): the FIRST swapchain — made while the renderer starts, before any present — takes the same lock and checks the flag first, on both the SDL path and the Windows path (hos_winruns the renderer on its own thread, so the window's close can be asked while the initial chain is being made). A window already closing gets no chain; the review's second pass confirmed the ordering matches the close's exactly.gpu/vk/vk_replay.cpp(present_image): returns before touching the swapchain whileclosing— the window is going away and making a chain again would take the screen back from the compositor mid-transition (the flag discipline the old comment invk_main.cppwarned about: a chain taken from under a frame mid-acquire is a lost device).host/hos_relaunch.c's close ordering is already right (rx_close_asked→ finish → exit). Close is deliberately not the lost-surface path: when the compositor dies on its own,invalidate_lost_surfacestays the route, and its retire is already gated — a close over an already-lost window waits and destroys nothing (retire_swapchain's own first line), now pinned by a fixture case.tests/test_present_semaphores.py: the fixture compiles the realpresent_imageprefix andretire_swapchain, and (the review's ask) extracts the close's wiring and start's guard verbatim from the sources and runs them: deleting either block fromvk_main.cpp, or the closing check from the guard, now fails the test (mutation-checked). New cases: a close retires a live chain (device idled, chain destroyed in order), no present makes a chain after, a closing window gets no initial chain, and a close over an already-lost window waits and destroys nothing.Measurements
Verdict: no fps or memory change by design (the release happens at exit); the fix's close marker fires exactly once per quit on Deck Game-Mode arms; the game-free suite is at parity (no new failures; main lost two since this branch's earlier base); the scripted harness cannot discriminate the bug (see Why's caveat), so the artifact's disappearance awaits a player who hits it.
Hardware: Steam Deck LCD (Jupiter, RADV VANGOGH), SteamOS current, AMD Custom APU 0405, 16 GB.
Scene: Castle Town / Market_Child via the control bridge's teleport, Game Mode, native output; deep arm at
HOS_OUTPUT_RES=2304x1440.Method: three paired scripted rounds (light 1280x800; deep 2304x1440 loop-walk; teleport-to-fresh-area and quit while the census is high), old vs fixed runtimes side by side, journal + game-log captures at the quit second; the game-free suite run side by side on the same Deck the same day (worktrees,
tests/run_all.py).How to reproduce: the runtime wrappers (
scripts/dev_packs.py pack/rtover one base),/home/deck/src/Longshot/scripts/longshot_ctl.pyteleport with the event gate ("event": false) and arrival check; full protocol in #issuecomment-35527 and #issuecomment-35622; run files indocs/tested/.a5b112b608a3bd21the swapchain is let goper quit (Game Mode arms)Why the hardware numbers still hold at the current head: they are labelled with the heads they were
measured at (
a5b112b6/08a3bd21), and main has moved since — five commits between them touch the sametwo files (the Intel uniform snapshot, GPU clock, early reads, VRAM switch, DLSS-FG offer). Checked rather
than assumed: the five functions this fix's campaign turns on —
mkwvk_close_asked,retire_swapchain,present_thread_drain,present_image,present_commit— are byte-identical ata5b112b6and currentmain (
git showbodies compared), so the campaign's fingerprint and marker rows stand as measured. What isnew at the current head (
3803ac69) is the close's bounded patience and the drain's answer, pinned by themutation-checked fixture (deleting either branch fails it) rather than by a fps table; the game-free suite
rows above were re-run side by side against current main the same day. A Deck stack at current main + this
PR builds clean with the pin verified and the compiled renderer carrying the close marker; its Game Mode
arms are the next measured run and will be posted here as their own
docs/tested/file the same way.Flags and settings
None. No switch, menu row, ini key or build flag is added or changed; the release is unconditional on the deliberate-close path.
Testing
Tested:
tests/test_present_semaphores.py(new close/guard cases, verbatim source extraction, mutation-checked) on this branch;tests/test_tested_configurations.pyfor the run files; game-free suite side-by-side on the Deck and side-by-side againstorigin/mainon this host; installed as a Deck runtime and run in Game Mode three paired rounds — clean launches, clean exits, close marker once per quit, no regression; desktop-mode launches unaffected in the same campaign.Not tested: Windows (the guard is in the Windows path but no Windows hardware here); HDR and V-Sync rows near a close; frame generation on during a close; Wayland/X desktop sessions. No local hardware for the first; the rest were not exercised by the scripted arms.
Must be tested:
journalctl --since -2minnoliftoff: drmModeAtomicCommit: Cannot allocate memory/ modeset storm at the quit second (with the fix, the game's log should end withthe swapchain is let gobeforedisplay: window closed, exiting).Risks
What could break: a present racing the retire — closed by taking
present_image_muand draining the presenter beforeretire_swapchain(), under the same lock orderpresent_imageuses; a late present after close — theclosingguard makes it a no-op rather than a recreate; a close racing the first chain — start() takes the close's lock and checks its flag, both platforms. A missed wait would show as a validation error or device-lost at exit, which the Deck arms did not produce.Turning it off: revert the commit (no switch to forget).
Compatibility: Nothing changes for saves, ini, caches, or the vk ABI; the only new player-visible behavior is the log line.
Issues
See ValveSoftware/gamescope#1822 (stuck final frame on game exit — same compositor surface, prior art). See #904 (rainbow artifacts too, but Polaris RX 580 on Windows after a cutscene: different mechanism). See #1043 (WIP presenter-thread swapchain sync — shared ground; if it merges first, rebase and reconcile the retire sequence). See #716 only in spirit: same Deck, different subsystem.
Checklist
Everyone
type(scope): description) of the status line's type.main.python -I -B tests/run_all.pypasses (any failure that also fails onmainis named). On this host: 16 failures, all build-tool checks —test_linux_update,test_make_game_cache,test_bug_report_defaults,test_crt_options,test_dlss_feature_size,test_failed_compile_keeps_objects,test_guest_imported,test_lift_lookups,test_link_skip,test_missing_nup,test_mods_size_table,test_pending_guest_import,test_run_all_changed,test_walk,test_walk_key,test_vk_occlusion_queries— every one also fails onorigin/mainchecked out side by side the same day (main's 18 include these plustest_bug_reportandtest_hardware_report; none is renderer code and none touches this PR's files).third_party/untouched (a new folder with aPROVENANCE.mdinstead).When it applies
Assisted-by:/Co-Authored-By:trailers.mainand passing here. (test_present_semaphores.py's close cases fail on main: the close wiring they extract verbatim is not there.)tests/that reads changed source updated in this PR, not weakened. (The fixture now extracts the close wiring and the startup guard verbatim fromvk_main.cpp; mutation-checked: deleting either fails.)docs/tested/for each new measured run (format indocs/tested-configurations.md).Test run, Deck (Steam Deck LCD, SteamOS, clang): the full game-free suite side by side.
maintipa5b112b6: 577 passed, 15 failed, 74 skipped.6f20d7bd: 576 passed, the same 15 failed, 74 skipped —test_bug_report,test_bug_report_defaults,test_crt_options,test_failed_compile_keeps_objects,test_guest_imported,test_hardware_report,test_linux_preflight,test_linux_update,test_make_game_cache,test_missing_nup,test_mods_size_table,test_pending_guest_import,test_run_all_changed,test_vk_occlusion_queries,test_walk_key— each name fails onmaintoo, so nothing here is introduced by this change. (The 577/576 pass delta is exactlytests/test_dev_build.py:maintip added it with #1095, and this branch is based on the commit before; it is not a test this branch fails.)tests/test_present_semaphores.pypasses on this branch with the new close cases:PASS: a close retires the chain with the device idled, and no present makes a new one after.test_present_semaphores— honestly my fault, the fixture carries the realpresent_imageprefix and needed the new field; the second commit fixed it. The suite was then re-run in full as above.A third commit (
138390a8) adds one log line at the close so a player's log shows the swapchain was let go (the Deck pipes the game's stdout into Steam's log; this makes the fix observable in the field). It does not change behavior beyond the line.Hardware check in flight: this Deck is building the exact PR head as a runtime pack, then a paired Game Mode exit test — the build the couch glitch came from, current main, and this fix — launching, playing, quitting through the control bridge, and reading gamescope's journal for the
drmModeAtomicCommit: Cannot allocate memoryfingerprint at the quit second. Result will land here either way it goes.Hardware update from the paired Game-Mode runs (Deck LCD, Steam session, same user save, same quit path):
Deep run at the couch's memory condition — the arms matched the glitch session's peak and both exited clean:
a4821d56(the build the glitch happened on)6c8d6e6a→e8d02b22)the swapchain is let goin the game log)Method:
HOS_OUTPUT_RES=2304x1440via systemd-run--setenv(the couch session ran at that picture size; the log lineboot: widescreen: picture 2304x1440proves it applied), patient boot wait through the ~216 s shader re-preparation the new resolution triggers, then the walk, thenctl quit. Full scripts + hooks live on the Deck at~/longshot-investigation/deck-gm-exit-{ab,deep,stream}.sh, documented so anyone can re-run them identically.What this tells us. The old build's peak (10,487 MB) matches the glitch session's 9,902 MB — so the deep condition reproduces the size of the allocation. But the scripted exits stayed clean because a loop walk warms caches: by the quit second the working set had settled to ~3.5 GB, while the couch session held 7,136 MB at the exit second — the player was mid-streaming when they quit. The compositor ENOMEM is a race at the exit second: it needs a high working set at the moment of exit, not just a high peak.
Third arm, running now: teleport into a fresh area (
Market_Child, flood the streaming pools on demand) and quit within ~100 s while the working-set census is still climbing — the couch's exact situation manufactured on schedule. Same two arms (old glitch build, then this PR's runtime). If old-now reproduces and fix-now doesn't, that's the clean on-demand discrimination reviewers asked for; if both stay clean, the honest claim stays "evidence chain + fixture proof, no on-demand repro" and the PR text will say so.Suite state unchanged: same 15 pre-existing Deck failures on both main and this branch; the new close-guard case passes at the current head.
Provenance note: the delegated adversarial review of this change is blocked — the review provider has returned HTTP 429 (resets ~Nov 19). The fix has had an inline adversarial pass instead, and I'd genuinely value a human second read on the lock order in
mkwvk_close_asked(present_image_mu → drain → present_mu) against PR #1043's presenter-thread changes, which touch the same paths from the other side.Hardware campaign, final scripted round: the exit fingerprint does not reproduce even on the unfixed build
Third condition tried (
deck-gm-exit-stream.sh, staged on the Deck under~/longshot-investigation/), now finished. Protocol, documented for reproduction:HOS_OUTPUT_RES=2304x1440(the couch picture size), Game-Mode session environment read from the Steam process, same user save, bridge teleport toMarket_Child(fresh-area flood), walk ~100 s into it, then quit while the working-set census is still climbing. One correction the first round forced: the game refuses teleports while an event plays (teleport check: … event playing 1) and the ctl client can still exit 0 — the script now polls--json statusfor"event":falsebefore teleporting, and both arms below actually reached their condition.rt-main(a4821d56, unfixed)rt-fix(head138390a8)So across three scripted conditions and four paired runs — light 1280x800, deep 2304x1440 loop-walk, and fresh-area teleport-and-quit — the liftoff ENOMEM/modeset fingerprint never appeared on the unfixed build either, despite peaks matching the couch session (10.1 GB vs the user's 9.9 GB) and comparable working sets at the exit second. The scripted Game-Mode harness cannot discriminate this race; I'm closing that path rather than burning more Deck sessions on cosmetic variants.
What remains supporting the change:
mkwvk_close_askedreturned without ever letting go of the swapchain or its images, so gamescope kept importing the exiting game's buffers — and gamescope's own tracker has this exact stuck-final-frame/failure family (#1822), with PR #1042/#1043 work circling swapchain lifetime at close;liftoff: drmModeAtomicCommit: Cannot allocate memory×4, failed flip, failed modeset with no fallback), and it shows a mid-session swapchain re-creation two minutes earlier — the game re-handed buffers while gamescope was busy;present_imagecalls that would resurrect it. On hardware the marker fires once per close and neither arm regressed; suites are green (Deck comparison posted earlier: fix = main − 1 test, thetest_dev_build.pymain added via #1095).Honest status: the one user report stands as the only observation of the glitch; scripted repro attempts (including a stronger one than the reported session in peak terms) do not show it. The change makes the renderer do at close what it should always have done, and matches upstream's direction. Moving this PR out of WIP for review; if the race needs gamescope-side cooperation to fix fully (#1822), that's complementary, not competing.
WIP: fix(vk): the window's swapchain is let go at the close, not at the process's endto fix(vk): the window's swapchain is let go at the close, not at the process's endDescription brought current: status now states the real hardware state (three paired scripted rounds, fingerprint never observed even unfixed, close marker clean on the fix; protocol in comments 35527/35622); the suite section replaced with the side-by-side Deck numbers actually posted (34873); 'must still be tested' now asks players who hit the glitch to confirm the fingerprint stops with the fix installed, since scripted repro cannot discriminate.
34fe030e5d08a3bd214eView command line instructions
Checkout
From your project repository, check out a new branch and test the changes.