WIP: feat(vk): FSR 3.1 frame generation for everyone: FSR 3.1 FG on Frame generation by, any multiplier and Auto, the HUD kept still, actor motion, no see-through limbs #1103
No reviewers
Labels
No labels
area/accessibility
area/audio
area/input
area/install
area/menu
area/renderer
area/saves
area/tooling
area/upscaling
beanstalk: approved
beanstalk: failed
beanstalk: needs approval
beanstalk: running
beanstalk: tested
ci: approved
good first issue
gpu/amd
gpu/igpu
gpu/intel
gpu/nvidia
help wanted
kind
bug
kind
chore
kind
docs
kind
feature
kind
performance
kind
question
needs measurements
platform/linux
platform/macos
platform/steam-deck
platform/windows
preview
priority
critical
priority
high
priority
low
regression
risk
high
risk
low
risk
medium
status
blocked
status
confirmed
status
duplicate
status
invalid
status
needs-info
status
wontfix
ci: approved
No milestone
No project
No assignees
2 participants
Notifications
Due date
No due date set.
Dependencies
No dependencies set
Reference
MagicBeanDev/Longshot!1103
Loading…
Reference in a new issue
No description provided.
Delete branch "HumanBean/Longshot:fsr3-fg-everyone"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Type: feat · Risk: medium · Tested on: RTX 3090, Linux (NVIDIA driver; by hand before the smear fix, scripted runs since) · Needs: in-game runs on AMD (RDNA2/3), Intel and NVIDIA pre-RTX 40; Windows and Linux
Why
Frame generation on AMD, Intel and older NVIDIA cards is the renderer's own built-in maker: a 16×16 block match on the
finished picture. It ghosts badly (moving characters, edges, the HUD trailing the camera). AMD FSR 3.1's frame
interpolation was already compiled in and uses the scene's depth and motion, but it stayed opt-in (
RX_VK_FSR_FG=1),made only the 2x midpoint, got camera-only motion (no skinned characters), and moved the HUD with the scene (#1635 and
its duplicates). So almost nobody got it. This makes it good enough to be the default wherever DLSS Frame Generation
doesn't run, and a choice of its own on the Frame generation by row.
See #1075 (HUD artifacts with frame generation),
#1041 (the Frame generation by row, DLSS FG or Built-in
FG; this adds FSR 3.1 FG to it).
What changed
FSR 3.1 frame interpolation now makes the frames at every Frame generation setting (Auto, 2x to 8x), keeps the HUD
still, takes the temporal chain's motion with the skinned characters' own in it (with the game's own upscaler too),
shows a swinging limb once instead of twice see-through, and is a choice of its own on Frame generation by.
scripts/fsr3_shaders.py(FI_PATCHES,patched_fi_headers) compiles AMD's frame-interpolation passes froma patched copy of the SDK's GPU headers;
third_party/is unchanged. The made frame's placetcomes from cbFI'spadding word (
_pad1→fInterpolationT): the vector fields still keep half the motion, scaled where used (2t backto the frame before, 2(1−t) forward), the vector put at the pixel's place at t, the blend weighted t / 1−t. At
t = 0.5 every scale is 1 and the passes are AMD's. The HUD: a pixel identical in both frames while the scene there
moves is the current frame's pixel, and a sample landing on such a pixel takes the other frame's
(
LongshotStaticPixel,LongshotStaticSamples; dispatchFlags bit 16). The ten FI SPIR-V headers are regenerated.gpu/vk/vk_fsr3fi.cpp:generate(cmd, in, times, count)makescountframes: the optical flow andreconstruct-and-dilate once a game frame, setup → interpolate → inpainting once a made frame, each with its own cbFI
block. Made-frame images are allocated as a set asks (
ensure_made), up to 7.asked(): on at Frame generation byFSR 3.1 FG, and at DLSS FG for what DLSS-G does not make (
RX_VK_FSR_FG=0never,=1over the row).gpu/vk/vk_framegen.cpp:fsr3fi_framesreplacesfsr3fi_frame. Fixed multipliers: frames at k/m where DLSS-Gmade none. Auto rate: each refresh's t. FSR is fed every present it is asked (count 0 too) so its history holds; any
set it does not make goes to the built-in maker as before.
gpu/vk/vk_temporal.cpp:temporal_fg_inputs()/fg_inputs(): with no engine drawing (the game's own upscaler, orOff) and FSR FI asked, the chain still takes the camera and skinned draws and makes its motion (prep + actor motion)
at the gameplay upscale, leaving the picture to the network; no jitter is asked of the host. With an engine drawing,
its motion is handed over too.
vk_draw.cpp: the chain's gates admit this mode.gpu/vk/vk_fsr3_inputs.cpp:take_motioncopies the chain's motion over the camera-only motion FSR's own inputsmade at the same dispatch (same units, same camera). PsychoV35 RGBA16F admitted at any multiplier, not only 2x.
AMD's interpolation blends them half and half, so the limb shows twice, see-through. The game's motion is right
(block-matched against the real frames: about 1 px median error on Link), so the patched interpolation takes the
nearer frame's sample there instead, the other frame's where the disocclusion mask says the nearer one doesn't see
the pixel: wherever the two samples differ by more than a fifth, all of it past two fifths (
LongshotNoDoubles,dispatchFlags bit 22;
RX_VK_FSR_FG_DOUBLES=1gives AMD's blend back).actor_motionpairs frame generation's poseswith its own last frame (
fg_inputs). The 600-present log line counts frames with the chain's motion and the inputs'age.
Built-in FG. Where DLSS FG runs: DLSS FG (the default; FSR 3.1 makes what DLSS-G does not, such as Frame generation's
Auto), FSR 3.1 FG, Built-in FG. Where it does not: FSR 3.1 FG (0, the default; a saved 2 is the same) and Built-in FG,
the line at FSR 3.1 FG saying why not DLSS FG; the row is fixed only by
RX_VK_DLSS_FG=1. FSR 3.1 FG's line in all 13translations.
tests/test_framegen_by_row.pyandtests/test_bug_report_defaults.pyfollow the three choices.docs/development.md,docs/temporal-history.md,docs/graphics-and-controls.md; the Graphics guidescreenshot recaptured (
scripts/screenshots/capture_menu.py): the row reads FSR 3.1 FG there now, not a fixed Built-inFG (
docs/images/guide/manifest.json).Measurements
Verdict: not measured yet: the PC's GPU is held for another session's timed runs. Ready to run once it is free
(this PR's runtime and main's at the same base): Kokiri Forest and Hyrule Field at 2x, main's Built-in FG against this
PR's FSR 3.1 FG against
RX_VK_FSR_FG_DOUBLES=1, two interleaved runs each, game fps, frame time and GPU time.Flags and settings
FSR 3.1 frame generation goes from opt-in to a choice of its own on Frame generation by, the default where DLSS FG cannot run; two new A/B switches.
RX_VK_FSR_FG=12x only)=0never,=1over the rowRX_VK_FSR_FG_HUD(new)=0AMD's passes as they wereRX_VK_FSR_FG_DOUBLES(new)=1AMD's blendTesting
Tested:
dev_build.py packover base02903b04), rebased on main1aec0c3c. The FI shaders compile with glslang throughscripts/fsr3_shaders.py. The Graphics guide screenshot wasrendered game-free by
scripts/screenshots/capture_menu.py.characters smeared: Link's hair, arms and legs doubled while strafing left and right, worst at 2x.
Saria, 12 screenshots each way, 20–30 game fps, 70–160 px of motion a frame):
RX_VK_FSR_FG_HUD=0andRX_VK_DLSS_ACTORS=0doubled them the same way, so neither the HUD rule nor actor motion was the cause.
real frames: median error 1.3 px over 3,638 moving points (Link 0.9 px against ~38 px of motion). The doubling is
FSR's blend.
inpainting were tried: see-through arms again, and tearing at limb edges, so they were dropped.
default on this RTX 3090 (no DLSS FG), the chain's motion on 600 of 600.
Not tested:
tests/that read the changed source were updated, not run:test_framegen_by_row.py,test_bug_report_defaults.py,test_psychov35_fsr3_fg.py,test_psychov35_default.py,test_fsr_uniform_ownership.py,present_source_crop_test.cpp(test_present_source_crop.py),test_fsr3_shaders.py(headers regenerated).Known issues: the "A Talk" action prompt is mangled in made frames (HUD).
Must be tested:
character, the default against
RX_VK_FSR_FG_DOUBLES=1. Expect solid limbs without visible tearing.at FSR 3.1 FG saying why not DLSS FG; an options file's
frame_generation_by=1stays Built-in FG.the log's
vk: fsr3fi: AMD FSR 3.1 makes N frame(s)andthe temporal chain's motion ... in place of the camera's alone; the HUD crisp while the camera turns; far less ghosting on Link than with Built-in FG (by hand; Test Farm: amd).RX_VK_FSR_FG_HUD=0against the default at the same spot: the HUD smears without it (by hand).(Test Farm: nvidia).
python -I -B tests/run_all.py(CI).Risks
What could break: frame generation on every non-RTX-40 GPU now goes through FSR 3.1 by default: a fault in it
(a wrong picture, a GPU fault) would hit many players. The chain now runs its inputs pass with the game's own upscaler
while FSR FI is on (one compute pass plus the actor draws a frame). Semi-transparent HUD elements over a moving scene
can still smear a little.
Turning it off: Frame generation by: Built-in FG (or DLSS FG where it runs), or
RX_VK_FSR_FG=0;RX_VK_FSR_FG_HUD=0for the HUD heuristic;RX_VK_FSR_FG_DOUBLES=1for the smear fix.Compatibility: the options file's
frame_generation_bykeeps its values (0 DLSS FG, or FSR 3.1 FG where DLSS FGcannot run; 1 Built-in FG) and gains 2 (FSR 3.1 FG); saves are unchanged. The FI shaders are new SPIR-V (their pipelines are
built again once).
Issues
See #1075, #1041
Checklist
Everyone
type(scope): description) of the status line's type.main.python -I -B tests/run_all.pypasses (any failure that also fails onmainis named). (not run on this PC: CI)third_party/untouched (a new folder with aPROVENANCE.mdinstead).When it applies
Assisted-by:/Co-Authored-By:trailers.tests/that reads changed source updated in this PR, not weakened.🤖 Generated with Claude Code
https://claude.ai/code/session_017vw8dyfXG2rFtkFseYa25y
The incredible update! Could you look into this issue when you have some attention to spare: #596
The DLSS indicator is flashing constantly. It's not detrimental, but I wonder if it would uncover some hidden implementation issues.
Pushed
042e7a8d: a fix for the smearing on characters (Link's arms, legs and hair doubled into see-through copies when strafing, most visible at 2x).RX_VK_FSR_FG_SOURCE=vectors) against block matching between the real frames: median error about 1 px on Link, 5% of the motion. The doubling comes from AMD's interpolation: where the frame before and this frame disagree, it blends them half and half and inpaints the rest.LongshotNoDoubles, dispatchFlags bit 22, on by default).RX_VK_FSR_FG_DOUBLES=1gives AMD's blend back for an A/B.RX_VK_FSR_FG_SOURCE=game|flow|prevwarp|mask|vectors) come out before ready, and the branch still needs reworking onto #1041's row. The "A Talk" prompt is mangled in made frames, a separate HUD issue.042e7a8d1d0d2f95260e0d2f95260e718cb283b8718cb283b83ef7eca7acWIP: feat(vk): FSR 3.1 frame generation for everyone: any multiplier and Auto, the HUD kept still, actor motionto WIP: feat(vk): FSR 3.1 frame generation for everyone: FSR 3.1 FG on Frame generation by, any multiplier and Auto, the HUD kept still, actor motion, no see-through limbsView command line instructions
Checkout
From your project repository, check out a new branch and test the changes.