Motion controls: gyro aiming from SDL pads, with sensitivity and inverts #1114
No reviewers
Labels
No labels
accessibility
autorelease: pending
autorelease: tagged
bug
dependencies
documentation
duplicate
enhancement
github_actions
good first issue
help wanted
invalid
question
testing wanted
wontfix
No milestone
No assignees
1 participant
No due date set.
Dependencies
No dependencies set
Reference
MagicBeanDev/Hookshot!1114
Loading…
Reference in a new issue
No description provided.
Delete branch "hid/motion-controls"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
The game starts the six-axis sensor and reads
GetSixAxisSensorStates, butnn_hid.calways answered with a pad lying still, so gyro aiming did nothing. This reads the controller's gyroscope and accelerometer through SDL and gives the game the state it actually aims with.The game-specific part is the approach spluton found in #112 (tested on a DualSense) and Link-22 corrected (the yaw rate's unit; tested on a GameSir T3 Pro in Switch Pro mode, same feel as Eden). Thanks to both. The first version of this PR sent a plain sensor state, which OoT does not read that way (see the review comment below).
What changed
host/hos_motion.h(new, header-only, no SDL or Windows):host/nn_hid.c:GetSixAxisSensorStatesfills its states from that;GetSixAxisSensorHandlesreturns 3 (the game runs its motion path only with a third source).host/hos_sdl.c,host/pad_sdl.c: SDL3/SDL2HasSensor,SetSensorEnabled,GetSensorData, looked up optionally (an SDL without them still plays, without gyro). The first pad with both sensors is read on the pad thread every poll; sensors are on only while the option is on.On (gyro found),On (no gyro on this pad),On (no controller);These are controls, so changing them does not reset the quality preset.
MkwVkHostgetsgyro_status(API version 8);rx_option18 to 21 are the rows.HOS_MOTION_DEBUG=1logs what the game gets twice a second.On by default because the game expects motion on Switch; happy to flip the default if you prefer it off.
Testing
tests/test_motion.py(game-free, added totests/run_all.py): bias learning, a left turn at 1 turn/s in the yaw slot, a front-up tilt as -0.5 rad in the matrix, the same turn with the pad held upright, each sensitivity scaling only its own axis, the inverts, the up vector settling on gravity, the stall clamp, and the neutral fields throughout. Passes with gcc and clang.SDL3.dll) reads an 8BitDo Ultimate 2 in Switch mode as "Nintendo Switch Pro Controller" with both sensors.🤖 Generated with Claude Code
https://claude.ai/code/session_01BsKC8mjbyEUFAv6uy3SQiW
Tested this PR (
d43ea5c) on Linux (CachyOS, RTX 4070 Super) with a GameSir T3 Pro in Switch Pro Controller mode, over SDL. The PR's tree was built on top of the v2.0.7 AppImage.Result: the gyro is detected and the aim moves, but left/right is inverted and much too fast. This is the same behaviour I saw with my first port of spluton's patch.
HOS_GYRO_FLIP=zdid not fix it. My patch from #112 aims correctly with the same controller, so it isn't a controller or driver issue.From what I found while fixing my port, OoT doesn't seem to read the six-axis state like a plain sensor:
GetSixAxisSensorHandlesreturns 3 handles in my patch. The game gates part of its motion path on that count.The relevant code is in
GetSixAxisSensorStatesinhost/nn_hid.cof my v2.0.7 patch, attached in #112. I'm happy to test any change to this PR with this controller.Thanks a lot for testing, and for the detailed write-up. You're right: this PR delivered the six-axis state as a plain sensor, which isn't how OoT reads it. That also explains what I saw on my 8BitDo Ultimate 2 (Switch mode): roll moved the aim left/right because it landed in the y slot, and up/down came out mirrored.
I'm reworking it to use the approach @spluton found and you corrected in #112, with credit to both of you in the code and the commit:
GetSixAxisSensorHandlesreturning 3It keeps this PR's SDL2/SDL3 plumbing, the Motion controls row (with its gyro found / no gyro status) and the game-free test, and adds Motion sensitivity (10 to 300%) plus Invert motion X / Invert motion Y toggles in the Controls tab, in place of
HOS_GYRO_FLIP.I'm testing it locally on rc3 with my 8BitDo first, then I'll push it here and ping you. A retest on the T3 Pro would be great.
@Link-22 pushed (
5768280): the rework from my last comment. It's still a draft while I try it in game, but if you get a chance to run it on the T3 Pro, I'd love to know how it compares to your patch.Gyro aiming is a great addition. The SDL2/SDL3 sensor wrappers, the units (rad/s ÷ 2π to revolutions/s, rest accel
(0,0,-1)), the 0x60-byte state layout and the option wiring all look right to me. A few things I'd look at, the first three being the important ones:nit: this "not part of the quality preset" pointer list keeps growing (now 14 terms), with parallel lists in
is_sliderand the clamp, and each new Controls row has to be added to all three by hand. A field onRow(or simplyr.tab == TAB_CONTROLS) would replace them.nit: options 18–22 are bare numbers repeated here, in
pad_sdl.cand in thevk_api.hcomment, in a mixed order (19 sens X, 20 inv X, 21 inv Y, 22 sens Y), so swapping 20/22 or 19/21 somewhere would silently cross the settings.vk_api.halready has anHOS_OPT_*enum; addingHOS_OPT_GYRO,HOS_OPT_GYRO_SENS_X, … there and using them on both sides avoids that.@ -0,0 +1,189 @@/* (hookshot) A controller's motion sensors as OoT reads its six-axis state (host/pad_sdl.c fills it from SDL,Any dt ≤ 1 ms (or > 50 ms) is replaced with a full 4 ms. On Linux the pads are polled from the window thread's event loop, which wakes on every SDL event, and turning the sensors on adds controller-sensor events (a DualSense over USB at ~1 kHz, plus mouse motion). So polls can land ~1 ms apart, each counted as 4 ms, and vertical aim moves several times too fast. In the other direction, a 60-100 ms hitch counts as 4 ms and that tilt is lost. Using the real dt for short gaps (only skipping exact duplicates), and clamping long ones rather than replacing them, would keep the integration honest.
"Still" here means under 0.08 rad/s (~4.6°/s) relative to the current bias, so slow, precise aiming gets learned away as bias. At 3°/s the first 256 samples (k = 0.06) cancel the turn within ~70 ms, and later about 63% of a one-second sweep is eaten. When the player stops, the aim drifts back until the bias is relearned. A real offset above 0.08 rad/s is also never learned at all. Gating stillness on low gyro variance over a short window plus a steady ~1 G accelerometer, rather than a threshold on the rate itself, is the usual way to avoid both.
The vertical aim is an open-loop integral of the bias-corrected pitch rate, so any leftover bias or dt error adds up without limit: 0.01 rad/s left over is ~0.6 rad of tilt per minute with the pad held level. The filter already tracks gravity in
up, so pullingpitchgently toward the tilt measured from it (a complementary filter, as done forupitself) would stop the drift.@ -827,24 +828,57 @@ HLE(_ZN2nn3hid21ShowControllerSupportEPNS0_27ControllerSupportResultInfoERKNS0_2/* Handles: six-axis ones are opaque u32s; a vibration handle is {u8 style type (3 = full key), u8 npad id, u8 deviceThis always writes at least 3 handles, whatever count the caller asked for. I get why from the comment (OoT only runs its motion path when three come back), but if the caller's array is sized for the 2 it asks for, the third
G32write lands just past it and corrupts whatever guest data follows. Have you checked that OoT's buffer for this call has room for 3 (e.g. it's part of a larger struct)? If so, a note here saying where that's guaranteed would help. If not, returning 3 while writing onlyARG(1)handles might already be enough to get the motion branch.While live, the sampling number goes up by 1 per call however many new readings arrived, and every entry reports the last poll's interval as its delta time (line 864). With ~4 polls per 16.7 ms frame, anything that counts samples or sums rate × delta_time over new samples sees about a quarter of the real time. Older entries (
s - i) also repeat the current values. Driving the sampling number fromsamples, and giving delta time as the time since the last sample the game was handed, would match how the hardware reports it.Motion always comes from the lowest-numbered pad that has a gyro, not the one being played with. A DualSense charging over USB in slot 0 would take the gyro from the Pro Controller in slot 1. Picking the pad with the latest button/stick input (or the one the npad state already comes from) would fit better. Also, the filter is only reset when the slot number changes, so a different pad that reconnects into the same slot within one scan keeps the old bias/up/pitch.
minor:
getenv("HOS_MOTION_DEBUG")runs on every poll, and this whole update, including the log line's file I/O, happens while holdingstate_lockexclusively, the lock the game's input HLEs take. Reading the env var once into a static, and running the filter outside the lock (taking it only to copymotion.out, the samples and dt), would keep the game thread from waiting on it.@ -90,0 +153,4 @@const int read = pick >= 0 && hos_sdl_pad_sensor(pads[pick].gc, HOS_SDL_SENSOR_ACCEL, a, 3) &&hos_sdl_pad_sensor(pads[pick].gc, HOS_SDL_SENSOR_GYRO, g, 3);/* (the pad that had motion keeps what its filter learned of its gyroscope's bias, for when it has motion again) */if (motion_pad >= 0 && (!read || pick != motion_pad) && pads[motion_pad].gc && pads[motion_pad].serial == motion_serial &&If one of the two enables fails,
motion = 0marks the pad as having no gyro for good. That means thecontinueabove skips it forever, so the accelerometer that did turn on is never turned off, and a temporary failure (e.g. the HIDAPI driver still starting on Bluetooth) is never retried, even after toggling Motion controls. Disabling both on a partial failure and retrying later would avoid that.@hckrman101 thanks for such a careful review. All ten are addressed in
66828d2:GetSixAxisSensorHandlesnow writes only the handles the caller asked for and still returns 3, so a two-handle array is never written past. The handles are opaque andGetSixAxisSensorStatesdoesn't read them, so whatever the game finds in a third slot doesn't matter. It also logs the count the game asked for once, so we can confirm what OoT passes.pitchis now pulled toward the tilt thatupmeasures (the pad's tilt about its x axis since the filter started, scaled by the same sensitivity and invert), with a 1 s time constant. The pull pauses while the pad is shaken or lying on its side. In the test, a 0.01 rad/s leftover bias over 60 s now stays within 0.03 rad.delta_timefor the yaw, a 1 kHz pad would now report about 1 ms. I'll ask Link-22 to compare on the T3 Pro.HOS_MOTION_DEBUGis read once. The filter and the log run outsidestate_lock, which is taken only to publish the new reading.HOS_OPT_GYRO,_SENS_X,_INV_X,_INV_Yand_SENS_Yinvk_api.h, used by bothpad_sdl.candvk_replay.cpp.Rowhas ascaleflag (drawn as a slider, stops at its ends) that replaces both slider lists, and the preset check is nowr.tab != TAB_CONTROLSplus fullscreen, FPS counter and GPU device. One side effect: "Controller in background" no longer turns the preset to Custom, which I think was a bug anyway.tests/motion_test.cnow simulates the pad: gravity turns with the true rotation, and the gyro adds a bias plus either sensor noise or hand tremor. It passes with gcc and clang, and the fullrun_all.pypasses. I'm testing in game on rc3 next.@ -827,24 +828,57 @@ HLE(_ZN2nn3hid21ShowControllerSupportEPNS0_27ControllerSupportResultInfoERKNS0_2/* Handles: six-axis ones are opaque u32s; a vibration handle is {u8 style type (3 = full key), u8 npad id, u8 deviceLooks good now: numbers only move on real readings and each entry carries its own gap.
On the
delta_timequestion: one thing to keep in mind is that a "reading" here is a poll that saw new data, not a raw sensor report, so the gap follows the poll rate rather than the pad's report rate (about 4 ms on the Windows thread, or whatever the window loop runs at on Linux). If OoT does sum rate ×delta_timeover the new samples, the real gap is what keeps the yaw speed right however often we poll. If it just uses the newest rate,delta_timedoesn't matter at all. The old fixed 5 ms came with one sample per call, so the testers' tuning should hold either way. A quick check for Link-22: if the yaw speed stays the same while the poll rate changes (sensors on vs mouse wiggling on Linux, or Windows vs Linux), the game isn't using it.Small follow-up on the new tilt pull (not blocking).
@ -0,0 +176,4 @@pitch about the levelled x (by 1 / sqrt(1 - up[0]^2)), and the pull would drag the aim past each move(hckrman101); what builds up while rolled is corrected once it is level again. Not while shaken either. */if (m->up_ready && valid && fabsf(up[0]) < HOS_MOTION_TILT_ROLL) {const float tilt = atan2f(-up[2], up[1]); /* (it grows at the x rate: d/dt = w[0]) */The pull's target and the integral it corrects don't quite measure the same angle once the pad is rolled.
pitchintegratesrates[0] = w·right(about the levelled x), whileatan2(-up[2], up[1])is the tilt about the pad's own x. Working it through,rates[0] = yz · d(tilt)/dtfor any rotation, so the two only agree whenup[0] ≈ 0.Example: held flat and rolled 45°, tilting the nose up by δ moves
pitchby δ but the target by δ/0.71 ≈ 1.41δ, and the pull then drags the aim the rest of the way over the next second or so. At the 0.5 gate (60° of roll) it's 2×. A slight unconscious roll of 15–25° gives a creep of a few to 10% of each vertical move, more noticeable at high sensitivity.The simplest fix is probably a tighter gate for the pull, e.g. only when
fabsf(up[0]) < 0.25f(or weightkby how level x is). Whatever builds up while rolled then gets corrected once the pad comes back level. It'd be worth a test case: roll 30°, tilt up, hold, and checkpitchdoesn't creep.Thanks for working through all of these. I went over
66828d2and all ten are sorted: the handle count no longer writes past a two-entry array, the filter uses real dt with the hitch cap, bias learning only kicks in when the pad is actually still, the vertical aim is pulled toward gravity, and sampling numbers anddelta_timenow follow real readings. The pad pick, the sensor retry, the lock scope and the option enum all look right too.Leftovers, none blocking:
hos_motion.h); a tighterup[0]gate would fix itHOS_MOTION_DEBUGthat the bias actually gets learned on noisier third-party pads like the 8BitDo and the T3 ProApproving. A retest from Link-22 on the T3 Pro would be nice before merge.
Retested
66828d2on Linux (CachyOS, RTX 4070 Super) with the GameSir T3 Pro in Switch Pro Controller mode, over SDL. The PR's tree was built on top of the v2.0.7 AppImage. Menu defaults: Motion controls on ("On (gyro found)"), both sensitivities at 100%, both inverts off.Everything works:
Great work @wyatt-poole, thanks for picking this up. From my side it's ready to merge.
@hckrman101 thanks for the approval and the follow-ups. Two of them are in
8a5c178:fabsf(up[0]) < 0.25f(about 15° of roll), as you suggested. Anything that builds up while rolled is corrected once the pad is level again. I added the test case: rolled 30°, nose tilted up 0.5 rad, held for 3 s, andpitchstays at -0.5. With the old 0.5 gate it creeps to -0.56.HOS_MOTION_DEBUGthat the bias gets learned on my 8BitDo Ultimate 2 resting on a table, and loosen the thresholds if it doesn't.@Link-22 when you have a moment, a retest on the T3 Pro would be the last thing before this leaves draft. In particular, does the yaw speed match your patch, and does it stay the same between sensors-only and mouse-wiggling polls?
Thanks for the quick follow-ups. Per-pad bias looks right to me: keyed on the slot plus the serial, so a reconnect starts fresh, it's only touched on the poll thread, and turning Motion controls off and on again gets it back. Nice to see the T3 Pro retest too, though that was on
66828d2, before this commit.One thing in the roll gate though (inline):
tilt0is still anchored on the first reading that passes the gate, and now that the gate is 15° instead of about 60°, a filter that starts rolled anchors late, after some pitch has already built up. Easy fix, and a test that rolls back to level would catch it.The branch also conflicts with main now:
gpu/vk/vk_api.h,gpu/vk/vk_menu.cpp,gpu/vk/vk_replay.cpp,host/pad_sdl.c,host/rx_backend.c,readme.mdandtests/test_mouse_settings.py. That's mostly #937 (mic) and the Rumble strength row (option 9, rumble code in pad_sdl.c), plus the menu work. The option ids don't collide (main uses 9, you use 18 to 22), but both branches bumpMKWVK_API_VERSIONfrom 7 to 8 with different struct layouts. After the merge it needs to be 9, withgyro_statusafter the mic_* pointers on both sides, otherwise an old vk dll would pass the version check.@ -0,0 +1,189 @@/* (hookshot) A controller's motion sensors as OoT reads its six-axis state (host/pad_sdl.c fills it from SDL,The anchor here assumes
pitchis 0 when the gate first passes. Before this commit that happened on almost the first reading. Now, if the filter starts with the pad rolled more than 15° (it resets on startup, when Motion controls is turned on, and when switching pads or reconnecting),pitchintegrates while rolled, and when the pad first comes leveltilt0is taken from wherever the pad is pointing then.targetcomes out as 0 there, so the pull drags the aim back by however far it moved while rolled, and from then on the level is off by that much.Anchoring to the current pitch fixes it (sv can't be 0, the menu's minimum is 10%):
@ -0,0 +130,4 @@}NEAR(m.pitch, -0.5, 0.03);run(&m, 0, 0, 0, 3, 1, 1, 0, 0);NEAR(m.pitch, -0.5, 0.03);This case starts rolled and stays rolled for the whole run, so the gate never passes and the pull never runs. It shows the pull stays off while rolled, but not that the aim stays right once it's level again. Could you add a step that rolls back to level with the nose still up (or one that starts level, rolls, aims, and comes back) and check
pitchstays at about -0.5 for a few seconds? Without the anchor fix above, the case that starts rolled should go to 0.@wyatt-poole quick answer on
8a5c178, same setup as before (T3 Pro, Linux, defaults):One thing I hit in game, which may be the late-anchor case @hckrman101 described. If I turn Motion controls on while the pad isn't held level, the gyro aim comes out wrong afterwards: it feels inverted. Turning it off and on again with the pad lying flat on the desk fixes it. So it seems to depend on how the pad is held at the moment the filter starts.
I'll do a full retest on the T3 Pro once the anchor fix and the merge with main are in, and check this case again.
@hckrman101 @Link-22 both points are in, and the branch is merged with main:
76fc99b):tilt0is now anchored to the aim as it is (tilt - pitch / sv), as you suggested, so the pull's target equals the current pitch when the pad first comes level. The rolled test now finishes the job: it starts the filter rolled 30°, aims up, rolls back to level (about the pad's z, which keeps the nose's height), and checks that the aim holds for 3 s. Without the fix it slides from -0.30 to 0. This should also be Link-22's "wrong after turning Motion controls on while not level" case.33d39bc):MKWVK_API_VERSIONis 9, withgyro_statusafter themic_*pointers in bothvk_api.handrx_backend.c's host table.scalefield in favour of main'spreset_exemptandstops_at_ends. The motion rows are preset-exempt, and both sensitivities are sliders that stop at their ends.pad_sdl.ckeeps bothrumble_strengthandmotion_poll.tests/test_rumble_menu.pynow reads everyHOS_OPT_*enum when it checks the option ids for collisions (before, it read only the cheats').run_all.pypasses on Linux (140 passed, 0 failed), andtest_motion.pypasses with both gcc and clang. Taking it out of draft.Re-checked at
76fc99b, looks good.tilt0 = tilt - pitch / sv, the target on the first level reading is(pitch / sv) * sv = pitch, so the pull starts at the aim as it is. That holds for either sign ofsv(inverted or not), and the target keeps moving at the same rate as the integrated pitch after that. The new step (start rolled, aim up, roll back to level about z, hold 3 s) is the case I was after.MKWVK_API_VERSIONis 9.gyro_statuscomes aftermic_selectin bothvk_api.handrx_backend.c's table, and nothing else moved. Ids 18-22 don't collide with main's 0-9/16/17/32-35, andtest_rumble_menu.pynow checks everyHOS_OPT_*enum.pad_sdl.ckeepsrumble_strengthnext tomotion_poll. The sensitivities are sliders that stop at their ends, the same as mouse sens and rumble.One small nit, not blocking: the Motion controls on/off row has no
preset_exempt, so toggling it still sets the Quality preset to Custom. The four rows under it are exempt. Probably just a missing, true.GitHub shows it as mergeable now. No CI checks ran on the branch.
@wyatt-poole @hckrman101 retested
76fc99b(merged with main, built on the v2.0.10 AppImage), same setup: T3 Pro in Switch Pro mode, Linux, menu defaults.Working well:
Still there: the vertical aim depends on how the pad is held when the filter starts. It happens at game start, and when I turn Motion controls off and on while holding the pad tilted. The aim is then off and feels almost inverted. Toggling it off and on with the pad flat on the desk fixes it every time.
I ran it with
HOS_MOTION_DEBUG=1and attached thehid:lines. There are four filter starts, one permotion controls fromline:up)upaim tiltSo with the same hold, start 3 gives a vertical aim about 0.9 rad away from starts 2 and 4. That's roughly the pitch the pad had when the filter started. It looks like
pitchstarts at 0 in whatever pose the pad is in. While the pad is rolled the tilt pull is off, so nothing brings it back to the gravity-level reference. When it comes level, the anchor (tilt - pitch / sv) then keeps that offset for good.A guess at a fix: start
pitchfrom the measured tilt whenupis first set, e.g.sv * asin(-up[2]), the nose's angle above level, which also holds when rolled. Then anchor the pull to the absolute tilt (tilt0 = 0). The vertical would always be relative to level, whatever the pose at start. I'm happy to retest.hookshot-motion-debug-76fc99b.txt