feat(vr): play in a PC VR headset over OpenXR, with a VR tab in the Insert menu #204
No reviewers
Labels
No labels
area/accessibility
area/audio
area/input
area/install
area/menu
area/renderer
area/saves
area/tooling
area/upscaling
beanstalk: approved
beanstalk: failed
beanstalk: needs approval
beanstalk: running
beanstalk: tested
ci: approved
good first issue
gpu/amd
gpu/igpu
gpu/intel
gpu/nvidia
help wanted
kind
bug
kind
chore
kind
docs
kind
feature
kind
performance
kind
question
needs measurements
platform/linux
platform/macos
platform/steam-deck
platform/windows
preview
priority
critical
priority
high
priority
low
regression
risk
high
risk
low
risk
medium
status
blocked
status
confirmed
status
duplicate
status
invalid
status
needs-info
status
wontfix
ci: approved
No milestone
No project
No assignees
2 participants
Notifications
Due date
No due date set.
Dependencies
No dependencies set
Reference
MagicBeanDev/Longshot!204
Loading…
Reference in a new issue
No description provided.
Delete branch "airdamien/Longshot:feat/vr-openxr"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Type: feat · Risk: high · Tested on: Windows 11, RTX 5080, Meta Quest 3 over Link · Needs: Windows with AMD or Intel, Linux, other headsets
Why
Longshot can only be played on a monitor. This lets it be played in a PC VR headset (tested with a Meta Quest 3 over Quest Link): either on a big flat screen in front of you, or in stereo 3D with the game's camera at your eyes. No issue in the tracker asks for it (searched: "VR", "headset", "OpenXR", "Quest").
What changed
The renderer can run through OpenXR: with VR on, the game shows on a big screen in the headset, or in stereo 3D with one eye drawn a frame and the two eyes handed to the headset as a pair, the HUD on a panel in front of you and the Touch controllers as a gamepad. A VR tab in the Insert menu holds its settings (kept in
longshot_vr.ini). With VR off, the default, nothing of it runs.gpu/vk/vk_xr.cpp/vk_xr.h(new). The renderer's instance and device are made through OpenXR (XR_KHR_vulkan_enable2) when VR is on. Flat-screen mode shows every presented picture on a quad layer. Stereo mode draws one eye a frame (xr_view), matches frames to eyes by the scene camera (xr_note_scene_draw), holds a pair's first eye until its second and gives both to the headset together (xr_copy_acquire,publish_held), one pair perxrWaitFrame(xr_thread). The HUD is redirected to a head-locked panel (xr_hud_draw), with its middle (the aiming reticle) shown where the game aims. A first-person view (left-stick tap or F9). The Touch controllers as a virtual SDL gamepad (pad_attach). On Linux the file is stubs (xr_built()false): it is Win32 only for now.--prepare-shaders,HOS_HEADLESS):xr_wanted().vk_main.cpp(instance, device, shutdown),vk_replay.cpp(present_image: the copies to the headset, the window's mirror),vk_draw.cpp(HUD and scene-camera notes),vk_temporal.cpp(a DLSS/FSR history per eye),vk_framegen.cpp(no made frames with a headset).gpu/vk/vk_menu.cpp.Row::applyis new: a row kept outside the options file (these are kept inlongshot_vr.ini, written byxr_opt_changed).host/hos_main.chos_oot_free_viewhands the game the headset's view and field of view;host/nn_vi.crealtime_stepsteps the game in pairs of frames while stereo runs (both eyes one moment);host/rx_backend.clooks up the renderer's optionalmkwvk_xr_*exports.nx2/oot_wide.py(the projection's field of view at the five matrix-update sites),nx2/oot_patches.py(the step hook records the step a frame read).nx2/build.pyaddsthird_party/openxr/include, and copies Khronos'openxr_loader.dllbeside the game when the tools have it (LONGSHOT_OPENXR_LOADER);nx2/make_game.pycarries it into the game folder (BESIDE).quest/build_vk.ps1builds the renderer alone and installs it with the loader.tools.lockhas a line for Khronos' loader (openxr-loader, Khronos' own release archive, Apache-2.0, SHA-256 pinned), so the Builder downloads it with its other tools;builder/LongshotBuilder/Runner.cspasses its folder to the build. Nothing of it is committed or packaged in a release.third_party/openxr(Khronos' headers, Apache-2.0, unmodified, withPROVENANCE.md).quest/README.md, the README's VR rows, the switches indocs/development.md.tests/test_readme_menu_rows.pycovers the VR tab.Measurements
Verdict: no cost with VR off, the default: the frame times are main's within run-to-run noise (median 16.65-16.66 ms against 16.64-16.65 ms, the game at its 60 fps cap in every run). VR itself is measured in a headset only in the earlier runs below.
Hardware: Windows 11, NVIDIA GeForce RTX 5080 16 GB (driver 591.59), Intel Core Ultra 9 285K, 32 GB
Scene: Castle Town's gate by day from a test save, Link standing still; the defaults of a new game folder (2560x1440 output, DLSS, no frame generation, the game's frame rate cap 60, windowed, V-Sync off); VR off (
RX_VR=0)Method: 150 s runs, the first 60 s left out, two runs each interleaved with main;
RX_VK_FRAME_LOG's frame times, a line per present; the game's own rate from the log'svi:line;HOS_INPUT_EXCLUSIVE=1so only the scripted presses reach the gameHow to reproduce: the builds from the same dump with
nx2/make_game.py, the same options file and save,HOS_INPUT_EXCLUSIVE=1 HOS_INPUT_SCRIPT=12:A,15:A,...,42:A HOS_INPUT_SCRIPT_BACKGROUND=1 RX_VK_FRAME_LOG=<csv>1b2e7555d5b37c06Measured at
d5b37c06on main1b2e7555. The head since is rebased onto main @615b2498, whose five new commits are for the Steam Deck's GPU clock, Intel GPUs' uniforms, speech bubbles at wide picture shapes, the build and Linux audio output (one conflict, ingpu/vk/vk_main.cpp: main'sgpu_clock_device_createdand this PR'sxr_device_readyside by side; this PR's own lines are otherwise unchanged), and adds the run's file,docs/tested/2026-11-16-vr-openxr-airdamien-run03.md. The run on main71b1edd8(the same result) isdocs/tested/2026-11-16-vr-openxr-airdamien-run02.md; the earlier runs, with VR on in a Meta Quest 3, are indocs/tested-configurations.md(2026-11-13).Flags and settings
The VR tab's rows (kept in a new
longshot_vr.ini), switches for one run and for tests, and a build variable for the loader.longshot_vr.inivr=/ VR tab "VR headset"stereo=/ VR tab "View"quest/longshot_vr.iniRX_VR=0/1,RX_VR_STEREO=0/1vr=andstereo=for one runLONGSHOT_OPENXR_LOADER=<dir>openxr_loader.dll, copied beside the gameopenxr-loadertool)RX_VR_TEST=1HOS_HEADLESS,LONGSHOT_PREPARE_ONLY(main's, now also read byxr_wanted())vr=1, 0xc0000409)vr=1. Doesn't help: playRX_VR_TRACE=1Testing
Tested:
python -I -B tests/run_all.pyon Windows 11 (llvm-mingw) at headbb774bb9: 585 passed, 20 failed. The same 20 fail onmain@615b2498on this PC (local tools and packages, line endings in a pipe, fixtures of main's that do not compile here), not this change.nx2/make_game.pyat headsd5b37c06(measured) andbb774bb9(Windows 11, RTX 5080).RX_VK_ONLY=1 nx2/build.py --standalonecompiles and linkslongshot_vk.so.vr=1and no headset connected, and withRX_VR=0. The VR tab opens, a row's change applies and is written tolongshot_vr.ini(xr: menu: fp_height=0.30), the options file is left alone.RX_VR_TEST=1: the game hands out a view per frame and each frame is matched to its eye, 76 s without a crash.tools.py install --only openxr-loaderwith an empty cache downloads Khronos' archive, checks it and unpacksopenxr_loader.dll; amake_game.pybuild withLONGSHOT_OPENXR_LOADERset puts it besidelongshot.exe, and its shader preparation ends normally withvr=1.xr: the session and the instance closed). The author has played stereo since on local builds that carry this code with the controller and NFC pull requests' changes and work not in this PR. Before the port, on Hookshot v2.1.0, the flat screen, stereo, the HUD panel and reticle and the Touch controllers were played over several sessions.Not tested:
main; only the build and the game-free checks ran.Must be tested:
Screenshots
The Insert menu's VR tab, this PR @ 5c269eda (the tab is the same at the head;
mainhas no such tab):Risks
What could break: with VR off (the default) the new code is a handful of calls that return at once; the measured cost at the cap is none. The host changes are in every build:
realtime_stepand the camera hook ask the renderer for a view each frame, and do nothing unless the renderer says a headset runs in stereo. The lifted projection patch (nx2/oot_wide.py) reads host globals that are zero without a headset. Known in stereo: an occasional repeated frame on fast turns; first person shows no body.tools.lock's new line needs mirroring before a release (Reviewer notes).Turning it off:
vr=0inlongshot_vr.ini, the VR tab's VR headset row, orRX_VR=0; otherwise revert the commits (nothing is migrated).Compatibility: the game is lifted again (the projection patch), so the first start after updating translates and prepares as after any lift change.
MkwVkHostandMkwVkRendererare unchanged (MKWVK_API_VERSIONas on main): the host finds the new calls as optional exports. Saves andlongshot_options.iniare unchanged; the VR settings are in a newlongshot_vr.ini. The Builder downloads one more tool (Khronos' loader, 32 MB).Reviewer notes
python -I scripts/tools.py mirror openxr-loader.tools.lock's new line names its copies in Longshot's store first, astests/test_tools_lock.pyasks, and they are not there yet.tools.pyfalls through to Khronos' URL (a fresh cache installs it), but the Windows setup (builder/setup/setup.c) reads only a line's first two URLs, so a Builder release cut from an unmirrored lock could not install;tools.py check --requirerefuses such a release. If a 32 MB download for every Builder install is not wanted for a feature that is off by default, say so and I will make it a build option instead (as--fetch-fsr-runtimeis).Checklist
Everyone
type(scope): description) of the status line's type.can't be run by hand is in Testing's Not testable by hand, with why.
in Risks.
main.python -I -B tests/run_all.pypasses (any failure that also fails onmainis named).third_party/untouched (a new folder with aPROVENANCE.mdinstead).When it applies
Assisted-by:/Co-Authored-By:trailers.docs/tested/for each new measured run (format indocs/tested-configurations.md).tests/that reads changed source updated in this PR, not weakened.🤖 Generated with Claude Code
Rebased onto main @
0c74253dand measured again there (VR off: 236.4 / 236.9 presents a second against main's 236.3 / 237.9, the same median frame time). Added: the Ultrawide menus fix found in a headset run (7b2120c7), the README's VR rows, two tested-configurations rows, and the three test requests (T-N5H4, T-VZY2, T-W3YQ). Still WIP: the head has not been run in a headset yet, and the screenshots are to attach.The README has a line for each VR row, and tests/test_readme_menu_rows.py now covers the VR tab (it failed on the grouped lines of the commit before). run_all on Windows: 443 passed, the 7 failures are main's on this PC.
Run in the headset at the head's code (the build of 7b2120c7, Quest 3 over Link, stereo 3D, Ultrawide menus at Centred, blurred sides): it looks fine, the sides no longer reach the headset, and the session closes cleanly. Screenshots of the VR tab attached. Marking it ready.
WIP: feat(vr): play in a PC VR headset over OpenXR, with a VR tab in the Insert menuto feat(vr): play in a PC VR headset over OpenXR, with a VR tab in the Insert menu843ab23c103f6d4ef86fThe loader no longer needs packaging or a script: tools.lock has a line for Khronos' release archive (openxr-loader), the Builder passes its folder to the build, and the build copies openxr_loader.dll beside the game. A maintainer needs to mirror it before a release (tools.py mirror openxr-loader): the reviewer notes say why. Also: a run with no game window (--prepare-shaders, HOS_HEADLESS) no longer opens a headset session; the preparation after a build died on exit with vr=1. Rebased onto main's head; run_all on Windows: 443 passed, the 7 failures are main's on this PC.
3f6d4ef86f10d2a376c5Rebased onto main @
514591a3(head10d2a376). One conflict,row_shown's line in the menu (main's Damage taken row beside the VR rows' condition);tests/test_damage_debug_menu.py, new on main, gets a stub forvr_row_shown. Built whole and the checks run (454 passed; the 7 that fail also fail on main here). Not measured again: the description says why.10d2a376c540fae344ceRebased onto main @
cf605c85(head40fae344). Conflicts:row_shownin the menu (main's Intel shader compiles row), the host's options (hold-to-toggle), and the measured runs, which are now two files underdocs/tested/as #96 asks.python -I -B tests/run_all.py: 477 passed; the 7 that fail also fail on main here. Built whole. Not measured again: the description says why.40fae344ce0d30abedefThis conflicts with
main(7bacf846) and needs a rebase by hand:gpu/vk/vk_menu.cpp(#729, #550, #735),gpu/vk/vk_replay.cpp(#769, #443),nx2/build.py(#577, #857),docs/development.mdandtests/test_frame_rate_cap_step.py(#488). Please rebase onto currentmain.0d30abedef5595ad0a18Rebased onto main @
a8a30c22(head5595ad0a), as asked. Conflicts:gpu/vk/vk_menu.cpp: the VR tab's helpers beside main's text fields (#51), and Recentre in main's new prompt layout.gpu/vk/vk_replay.cpp: main'sHOS_OPT_SKIP_TITLEandHOS_OPT_RENDER_THREADanswers kept;id == 4still answers on while a headset runs.gpu/vk/vk_temporal.cpp:xr_no_temporal()ahead of main'senv_xess().nx2/build.py: the OpenXR include beside main'stp_vflags(#629).tools.lock,Runner.cs,make_game.py,CONTRIBUTING.md,docs/releases.md,docs/development.md): only theopenxr-loaderlines are added.tests/test_frame_rate_cap_step.pyandtests/upscale_independent_test.cpp: the stubs forrx_xr_syncandxr_enabledbeside main's new ones (#488, #109).Built whole.
python -I -B tests/run_all.py: 579 passed, 25 failed; the same 25 fail on main @a8a30c22on this PC. Not measured again in the headset.The description now follows main's new template (
scripts/check_pr_description.pypasses): the status line, a Verdict, the measurement columns by commit, the flags table's new columns, Must be tested with who runs each item, and Risks with Compatibility. Nothing in it is new but the latest rebase's test run and the items main's template asks for.5595ad0a180d6e78b088Rebased onto main @
a5b112b6. The pull requests that went in since conflicted with this branch:MKWVK_API_VERSION24. The controller pull request's rumble test is now 25 and the NFC pull request is 26.tests/test_audio_output_devices.pypinned the version at exactly 24 and required the playback callbacks to be the last host fields. It now checks for 24 or later, and for the callbacks right afteroptions_path.host/nn_vi.c. Only VR touches this: any cap other than 60 takes the timed step, as on main, and so does a VR frame pair.Built whole with
nx2/make_game.py.tests/run_all.pyfails the same 19 tests as main @a5b112b6on this PC. The description has the new heads.0d6e78b0887df4c09e3dRebased onto main @
71b1edd8and measured again, interleaved with that main. The new run's file is underdocs/tested/, and the description's Measurements section has the numbers. No cost in any run: the game held its 60 fps cap throughout, with median frame times within 0.03 ms of main's.Since the last rebase, main took
MKWVK_API_VERSION25 for the microphone test (#1102), so the controller pull request's rumble test is now 26 and the NFC pull request is 27.tests/test_microphone_test_gain.pypinned the version at exactly 25; it now checks for 25 or later. In the VR pull request, the VR tab's check in the options file's loading loop now sits on a line of its own, because main'stests/test_framegen_by_row.pyreads that loop's text.7df4c09e3d55e1ad8953Rebased onto main @
1b2e7555(eightperf(vk)andperf(lift)commits since71b1edd8) with no conflicts, built whole, and measured again, interleaved with that main. The new run's file is underdocs/tested/, and the description's Measurements section has the numbers. No cost: the game held its 60 fps cap in every run, with median frame times within 0.02 ms of main's.tests/run_all.pyfails the same 21 tests as that main on this PC.Main moved again right after that push (
615b2498, five commits), with one conflict ingpu/vk/vk_main.cpp: main's newgpu_clock_device_createdcall and this PR'sxr_device_readynow sit side by side. Rebased onto it; this PR's own changed lines are identical to before. Built whole atbb774bb9, andtests/run_all.pyfails the same 20 tests as that main on this PC. The measurement stays the one on1b2e7555from an hour earlier: none of the five commits touch the Windows NVIDIA path it measured.55e1ad8953bb774bb9b7Brought up to the ready-for-review standard (#1454): the checklist's Everyone list has two new boxes, added here from what this PR's description and comments show (please check they match what you know).
View command line instructions
Manual merge helper
Use this merge commit message when completing the merge manually.
Checkout
From your project repository, check out a new branch and test the changes.