WIP: feat(vk): motion vectors for rigid objects moved by their own matrix #617
No reviewers
Labels
No labels
area/accessibility
area/audio
area/input
area/install
area/menu
area/renderer
area/saves
area/upscaling
beanstalk: approved @ 1697603
beanstalk: approved @ 2c5a62e
beanstalk: approved @ 4fd82b7
beanstalk: approved @ 64d6d09
beanstalk: approved @ 6f91fae
beanstalk: approved @ 70a78f9
beanstalk: approved @ 8d361dc
beanstalk: approved @ 9d8356e
beanstalk: approved @ abf2227
beanstalk: approved @ d82f202
beanstalk: approved @ e5f1e40
beanstalk: approved @ ee8ab5f
ci: approved
good first issue
gpu/amd
gpu/igpu
gpu/intel
gpu/nvidia
help wanted
kind
bug
kind
chore
kind
docs
kind
feature
kind
performance
kind
question
needs measurements
platform/linux
platform/macos
platform/steam-deck
platform/windows
preview
priority
critical
priority
high
priority
low
regression
status
blocked
status
confirmed
status
duplicate
status
invalid
status
needs-info
status
wontfix
testing wanted
ci: approved
No milestone
No project
No assignees
1 participant
Notifications
Due date
No due date set.
Dependencies
No dependencies set
Reference
MagicBeanDev/Longshot!617
Loading…
Reference in a new issue
No description provided.
Delete branch "ballotstomper/Longshot:fix/rigid-object-motion"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Type: feat · Risk: low · Tested on: Linux, RX 6750 XT (RADV) · Needs: Windows or NVIDIA with DLSS; main against this PR, interleaved
Why
With DLSS or FSR 3.1, things that turn or slide by themselves without bones get no motion vectors, so the upscaler
smears them. Kakariko's windmill is the clearest case: the pattern on its sails turns into a blurry coloured mess
as they spin. Actor motion (#59) covers skinned models only; a rigid object that the game moves by changing its own
object matrix was left with the camera's motion alone. No issue covers it (the tracker searched for ghosting,
windmill, motion vectors, DLSS and FSR); #484 is ghosting around Link and Navi with DLSS preset K, another case.
What changed
A scene draw with no joints, whose program reads its object block as far as the origin, is now followed frame to
frame like a one-bone actor. When its object matrix or origin differs from the frame before, it is drawn again into
the motion image with both poses, so the upscaler sees the motion. Draws that stand still add nothing. It is opt-in
with
RX_VK_DLSS_RIGID=1.skin_checkingpu/vk/vk_draw.cppsends a draw with no joints totemporal_note_rigid_drawwhen the vertexprogram reads its object block (the shader's c4, NVN binding 1) to 0x4c and the block is bound that far; skinned
draws go on as before (the bone palette check moved after the joint check).
temporal_note_rigid_drawingpu/vk/vk_temporal.cppreads the 3 x 4 matrix and the origin at the draw(
ub_read, as actor motion reads poses), takes the matrix only when its rows look like a rotation times one scale,and keeps every pose for the next frame. A pose found unchanged in the frame before's sorted list costs one binary
search; a changed one is paired with the nearest copy of its model from the frame before (within 2 units) and becomes
a
Skincarrying both poses.actor_motionswaps and sorts the rigid poses once a frame and draws the rigid skins with their own two poses,outside the pairing by place. The status line adds "rigid draws N seen, M moved, K of them by their origin alone".
dlss_actor.verthas a format bit 512 (dlss_actor::kRigid): every vertex on bone 0 with weight 1. TheSPIR-V header is regenerated with glslang.
RX_VK_DLSS_RIGIDbeside the actor motion switches indocs/development.md.Measurements
Verdict: works: the windmill's sails get motion vectors and keep their pattern while they spin. Against main with
the switch on: the same fps in the second and third walks (41.6 against 41.75, 41.95 against 40.8), about 0.2 ms more
actor motion CPU a frame. The first segments after arrival are noisy on both builds; the first walk was lower with the
PR in both runs (38.9 and 33.1 against 40.7 and 41.5), not explained yet.
Hardware: Arch Linux (kernel 7.3.4), Mesa 26.2.7 (RADV), RX 6750 XT 12 GB, Ryzen 5 5600X, 16 GB RAM
Scene: Kakariko Village at noon (teleport, stand, three walks with teleports between); 2560x1440, render scale fixed at
100%, AMD FSR 3.1, frame generation off, fullscreen; recorder thread, parallel replay 3, submit and presenter threads
Method: scripted runs (
kak_walk3_frames), each build rebuilt before its run: a warm-up on main, then main, PR, main,PR; fps and p99 frame time per segment from the bench's summary, the mean of the two runs; actor motion CPU from the
renderer's status lines;
RX_VK_DLSS_RIGID=1set for all runs (main ignores it)How to reproduce: arrive in Kakariko, look up at the windmill with DLSS or FSR 3.1 on, with and without
RX_VK_DLSS_RIGID=1(andRX_VK_DLSS_DEBUG=motionto see the vectors)6c9edffed2b88782The PR's second run started slowly (stand 24.0 fps with 4 frames over 100 ms, its walk 14.5 s long against ~11.8),
and main's first run's stand was low too (32.4): those segments follow the first load after a rebuild.
Earlier, against the switch off on the same PC and scene (this branch before its rebase: a1edfbe4 is
61d526a4now,47d3e6e0 is
2a34e02e):temporal_note_rigid_draw, share of user samples (walk3)The second commit replaced the per-model maps with sorted lists, which took the rigid path from 0.72% to 0.21% of the
samples.
Motion vector values (
RX_VK_DLSS_DEBUG=motion, red = |x|, green = |y| over 16 px) in the arrival shot, camera still(screenshots in the comment below):
Walking around Kakariko, about 1,500 rigid draws a frame are seen and 40-90 move (3-5%), so the camera's own motion
doesn't mark static objects as moving.
Flags and settings
One new switch.
RX_VK_DLSS_ACTORS=0now turns rigid motion off tooRX_VK_DLSS_RIGID=1turns it on)Testing
Tested: Linux, RX 6750 XT, Mesa 26.2.7, FSR 3.1: the windmill by eye with the switch on and off (the sails' pattern
stays readable through the spin with it on, smears with it off) and in the motion view; scripted Kakariko walks against main
@
6c9edffe, interleaved, with the status line's counts; the game-free suite: 545 passed, 13 failed, 22 skipped, and the same 13 fail on main @6c9edffe(test_appimage_player_build, test_bake_portability, test_cpm_discovery, test_crash_dialog, test_env_on,test_lift_cleanup_error, test_linux_preflight, test_make_game_cache, test_pipeline_tour, test_readback_late,
test_vk_pgo, test_vram_budget_row, test_walk_key).
Not tested: DLSS (no NVIDIA card here), frame generation, Windows. Rigid objects other than
the windmill were not looked at one by one. Things animated inside the game's shaders (the gate banner, wind
vegetation drawn by mesh shaders) are not covered by this PR: their vertex buffers and object matrices stay the same
from frame to frame, and the motion comes from a clock in a shared constant block, so they need the game's vertex
shader run with the frame before's constants (a later PR). Runs on this PC hit GPU device losses (a gfx ring timeout,
the same page fault at 0x820000 read by the scalar data cache) with the switch on and off alike, and on main @
6c9edffe(the warm-up run of the measurement above, the same fault in its GPU dump); they are not from this change and are
being followed separately.
Must be tested:
RX_VK_DLSS_RIGID=1and without,expecting the sails to keep their pattern with it and to smear without (by hand, or Test Farm: @Beanstalk nvidia,
Kakariko windmill with RX_VK_DLSS_RIGID=1 against unset, DLSS on)
RX_VK_DLSS_RIGID=1, two interleaved runs each in Kakariko, expectingno fps loss beyond noise, the first walk after arrival in particular (lower with the PR in both runs here) (by hand)
Risks
What could break: with the switch on, a draw whose object block isn't a placement matrix could get wrong motion
vectors (the rotation-and-scale check is there to leave those alone); about 0.2 ms of CPU a frame across the replay
workers. With it off, nothing: the draw path returns before any of the new work.
Turning it off: leave
RX_VK_DLSS_RIGIDunset (the default), orRX_VK_DLSS_ACTORS=0for all actor motion.Compatibility: Nothing changes: no saves, options, cache or ABI changes.
Checklist
Everyone
type(scope): description) of the status line's type.main.python -I -B tests/run_all.pypasses (any failure that also fails onmainis named).third_party/untouched (a new folder with aPROVENANCE.mdinstead).When it applies
Screenshots: the Kakariko arrival with the windmill in view, AMD FSR 3.1,
RX_VK_DLSS_DEBUG=motion, camera still (RX 6750 XT, Linux). The motion view's red and green are multiplied by 4 and its blue background dropped, so small motion shows; nothing else is changed.After, RX_VK_DLSS_RIGID unset: this PR @ a1edfbe4 (
61d526a4after the rebase): the sails have no motion of their own.After, RX_VK_DLSS_RIGID=1: this PR @ a1edfbe4: the sails show their turn (red one way, cyan the other).
Measured main @
6c9edffeagainst this PR @d2b88782(RX_VK_DLSS_RIGID=1), interleaved after a warm-up on an RX 6750 XT: walk2 and walk3 the same fps (41.95 against 40.8, 41.6 against 41.75), actor motion about 0.2 ms more CPU a frame. The first walk after arrival was lower with the PR in both runs (38.9 and 33.1 against 40.7 and 41.5), not explained yet: added to Must be tested. The GPU device loss seen here also happened on plain main (the warm-up run, same fault). Still open: a Windows build (test request T-ZZJQ) and DLSS (T-39PN).View command line instructions
Checkout
From your project repository, check out a new branch and test the changes.