feat(vk): FSR 3.1 frame generation for everyone: FSR 3.1 FG on Frame generation by, any multiplier and Auto, the HUD kept still, actor motion #1103
No reviewers
Labels
No labels
area/accessibility
area/audio
area/input
area/install
area/menu
area/renderer
area/saves
area/tooling
area/upscaling
beanstalk: approved
beanstalk: failed
beanstalk: needs approval
beanstalk: running
beanstalk: tested
ci: approved
good first issue
gpu/amd
gpu/igpu
gpu/intel
gpu/nvidia
help wanted
kind
bug
kind
chore
kind
docs
kind
feature
kind
performance
kind
question
needs measurements
platform/linux
platform/macos
platform/steam-deck
platform/windows
preview
priority
critical
priority
high
priority
low
regression
risk
high
risk
low
risk
medium
status
blocked
status
confirmed
status
duplicate
status
invalid
status
needs-info
status
wontfix
ci: approved
No milestone
No project
No assignees
2 participants
Notifications
Due date
No due date set.
Dependencies
No dependencies set
Reference
MagicBeanDev/Longshot!1103
Loading…
Reference in a new issue
No description provided.
Delete branch "HumanBean/Longshot:fsr3-fg-everyone"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Type: feat · Risk: medium · Tested on: RTX 3090, Linux (by hand and scripted); RX 580, Linux (scripted, the AMD lab) · Needs: in-game runs on AMD (RDNA2/3), Intel and NVIDIA pre-RTX 40; Windows and Linux
Why
Frame generation on AMD, Intel and older NVIDIA cards is the renderer's own built-in maker: a 16×16 block match on the
finished picture. It ghosts badly (moving characters, edges, the HUD trailing the camera). AMD FSR 3.1's frame
interpolation was already compiled in and uses the scene's depth and motion, but it stayed opt-in (
RX_VK_FSR_FG=1),made only the 2x midpoint, got camera-only motion (no skinned characters), and moved the HUD with the scene (#1635 and
its duplicates). So almost nobody got it. This makes it good enough to be the default wherever DLSS Frame Generation
doesn't run, and a choice of its own on the Frame generation by row.
See #1236 (the limb ghosting, follow-up), #1075 (HUD artifacts with frame generation),
#1041 (the Frame generation by row, DLSS FG or Built-in
FG; this adds FSR 3.1 FG to it).
What changed
FSR 3.1 frame interpolation now makes the frames at every Frame generation setting (Auto, 2x to 8x), keeps the HUD
still, takes the temporal chain's motion with the skinned characters' own in it (with the game's own upscaler too),
and is a choice of its own on Frame generation by.
scripts/fsr3_shaders.py(FI_PATCHES,patched_fi_headers) compiles AMD's frame-interpolation passes froma patched copy of the SDK's GPU headers;
third_party/is unchanged. The made frame's placetcomes from cbFI'spadding word (
_pad1→fInterpolationT): the vector fields still keep half the motion, scaled where used (2t backto the frame before, 2(1−t) forward), the vector put at the pixel's place at t, the blend weighted t / 1−t. At
t = 0.5 every scale is 1 and the passes are AMD's. The HUD: a pixel identical in both frames while the scene there
moves is the current frame's pixel, and a sample landing on such a pixel takes the other frame's
(
LongshotStaticPixel,LongshotStaticSamples; dispatchFlags bit 16). The ten FI SPIR-V headers are regenerated.gpu/vk/vk_fsr3fi.cpp:generate(cmd, in, times, count)makescountframes: the optical flow andreconstruct-and-dilate once a game frame, setup → interpolate → inpainting once a made frame, each with its own cbFI
block. Made-frame images are allocated as a set asks (
ensure_made), up to 7.asked(): on at Frame generation byFSR 3.1 FG, and at DLSS FG for what DLSS-G does not make (
RX_VK_FSR_FG=0never,=1over the row).gpu/vk/vk_framegen.cpp:fsr3fi_framesreplacesfsr3fi_frame. Fixed multipliers: frames at k/m where DLSS-Gmade none. Auto rate: each refresh's t. FSR is fed every present it is asked (count 0 too) so its history holds; any
set it does not make goes to the built-in maker as before.
gpu/vk/vk_temporal.cpp:temporal_fg_inputs()/fg_inputs(): with no engine drawing (the game's own upscaler, orOff) and FSR FI asked, the chain still takes the camera and skinned draws and makes its motion (prep + actor motion)
at the gameplay upscale, leaving the picture to the network; no jitter is asked of the host. With an engine drawing,
its motion is handed over too.
vk_draw.cpp: the chain's gates admit this mode.gpu/vk/vk_fsr3_inputs.cpp:take_motioncopies the chain's motion over the camera-only motion FSR's own inputsmade at the same dispatch (same units, same camera). PsychoV35 RGBA16F admitted at any multiplier, not only 2x.
actor_motionpairs frame generation's poses with frame generation's own last frame (fg_inputs), not the engine's;the 600-present
fsr3filog line counts the frames that took the chain's motion and how old the inputs were.Built-in FG. Where DLSS FG runs: DLSS FG (the default; FSR 3.1 makes what DLSS-G does not, such as Frame generation's
Auto), FSR 3.1 FG, Built-in FG. Where it does not: FSR 3.1 FG (0, the default; a saved 2 is the same) and Built-in FG,
the line at FSR 3.1 FG saying why not DLSS FG; the row is fixed only by
RX_VK_DLSS_FG=1. FSR 3.1 FG's line in all 13translations.
tests/test_framegen_by_row.pyandtests/test_bug_report_defaults.pyfollow the three choices.docs/development.md,docs/temporal-history.md,docs/graphics-and-controls.md; the Graphics guidescreenshot recaptured (
scripts/screenshots/capture_menu.py): the row reads FSR 3.1 FG there now, not a fixed Built-inFG (
docs/images/guide/manifest.json).Measurements
Verdict: on a weak AMD card FSR 3.1 FG costs more than Built-in FG: on an RX 580 at 2560x1440, 3.6 ms more GPU time
a frame standing and 9.6 ms turning, 7% and 18% fewer frames shown. That buys far less ghosting than Built-in FG; on a
card this slow, Built-in FG on the row is the cheaper choice. Not yet measured on a faster card.
Frame generation by at its default (main: Built-in FG, no DLSS FG on this card; this PR: FSR 3.1 FG), Frame pacing on,
V-Sync Off, a fresh options file each run.
game at a time. The median of each phase's
vk: N presents, X fpsandGPU time X ms a framelines (the first of eachphase dropped). This PR ran at
3ef7eca7withRX_VK_FSR_FG_DOUBLES=1, which turned off the blend step since taken out:the same passes as this PR's head. FSR made 598–600 of 600 presents in every PR run.
dev_packs.py packof this branch and of main1aec0c3cover one base,dev_packs.py rteach, thenthe same options and route.
1aec0c3c(Built-in FG)3ef7eca7(FSR 3.1 FG)Flags and settings
FSR 3.1 frame generation goes from opt-in to a choice of its own on Frame generation by, the default where DLSS FG cannot run; one new A/B switch.
RX_VK_FSR_FG=12x only)=0never,=1over the rowRX_VK_FSR_FG_HUD(new)=0AMD's passes as they wereTesting
Tested:
dev_build.py packover base02903b04), rebased on main0cab3548. The FI shaders compile with glslang throughscripts/fsr3_shaders.py. The Graphics guide screenshot wasrendered game-free by
scripts/screenshots/capture_menu.py.characters smeared: Link's hair, arms and legs doubled while strafing left and right, worst at 2x.
Saria, 12 screenshots each way, 20–30 game fps, 70–160 px of motion a frame):
RX_VK_FSR_FG_HUD=0andRX_VK_DLSS_ACTORS=0doubled them the same way, so neither the HUD rule nor actor motion was the cause.
real frames: median error 1.3 px over 3,638 moving points (Link 0.9 px against ~38 px of motion). The doubling is
FSR's blend.
and without skipping the inpainting) was tried and taken out: random screenshots first looked solid, but the made
frames themselves (below) still showed the copies, about as AMD's own blend does. Follow-up: #1236.
DLSS FG), the chain's motion on 600 of 600.
renderer writing every 6th present and the frame made before it:
RX_VK_SHOTS,RX_VK_SHOT_MADE=1): FSR 3.1 FG makesthe frames on AMD (497–600 of 600 presents); real frames clean; made frames show see-through copies of swinging arms
(#1236). The fps runs in Measurements.
Not tested:
with
RX_VK_FSR_FG_DOUBLES=1) and the switch removed; it builds.tests/that read the changed source were updated, not run:test_framegen_by_row.py,test_bug_report_defaults.py,test_psychov35_fsr3_fg.py,test_psychov35_default.py,test_fsr_uniform_ownership.py,present_source_crop_test.cpp(test_present_source_crop.py),test_fsr3_shaders.py(headers regenerated).Known issues: swinging limbs (Link's arms, legs, hair while strafing) show as see-through copies in made frames: #1236. The "A Talk" action prompt is mangled in made frames (HUD).
Must be tested:
does not, and load an options file with
frame_generation_by=1, expecting DLSS FG → Built-in FG → FSR 3.1 FG where DLSSFG runs, FSR 3.1 FG ↔ Built-in FG elsewhere with the line at FSR 3.1 FG saying why not DLSS FG, and the saved 1 still
Built-in FG.
the log's
vk: fsr3fi: AMD FSR 3.1 makes N frame(s)andthe temporal chain's motion ... in place of the camera's alone; the HUD crisp while the camera turns; far less ghosting on Link than with Built-in FG (by hand; Test Farm: amd).RX_VK_FSR_FG_HUD=0against the default at the same spot while the camera turns, expecting the HUD to smearwith it and to stay crisp without it.
@Beanstalkon nvidia): an RTX 40/50 with DLSS at Frame generation 2x–8x and Auto, expecting DLSS FG tomake the frames at 2x–8x and FSR 3.1 only at Auto rate and where DLSS-G makes none.
@Beanstalkon intel): Intel Arc at Frame generation 2x, expecting the log'svk: fsr3fi: AMD FSR 3.1 makes N frame(s)and no fault.game's own upscaler, by design) and the picture's brightness unchanged.
python -I -B tests/run_all.py), expecting them to pass, the updatedtest_framegen_by_row.pyandtest_bug_report_defaults.pyamong them.Risks
What could break: frame generation on every non-RTX-40 GPU now goes through FSR 3.1 by default: a fault in it
(a wrong picture, a GPU fault) would hit many players. The chain now runs its inputs pass with the game's own upscaler
while FSR FI is on (one compute pass plus the actor draws a frame). Semi-transparent HUD elements over a moving scene
can still smear a little. Swinging limbs ghost (#1236). On a slow GPU (an RX 580 at 1440p) it costs 4–10 ms more GPU
time a frame than Built-in FG.
Turning it off: Frame generation by: Built-in FG (or DLSS FG where it runs), or
RX_VK_FSR_FG=0;RX_VK_FSR_FG_HUD=0for the HUD heuristic.Compatibility: the options file's
frame_generation_bykeeps its values (0 DLSS FG, or FSR 3.1 FG where DLSS FGcannot run; 1 Built-in FG) and gains 2 (FSR 3.1 FG); saves are unchanged. The FI shaders are new SPIR-V (their pipelines are
built again once).
Issues
See #1075, #1041
Checklist
Everyone
type(scope): description) of the status line's type.main.python -I -B tests/run_all.pypasses (any failure that also fails onmainis named). (not run on this PC: CI)third_party/untouched (a new folder with aPROVENANCE.mdinstead).When it applies
Assisted-by:/Co-Authored-By:trailers.tests/that reads changed source updated in this PR, not weakened.🤖 Generated with Claude Code
https://claude.ai/code/session_017vw8dyfXG2rFtkFseYa25y
The incredible update! Could you look into this issue when you have some attention to spare: #596
The DLSS indicator is flashing constantly. It's not detrimental, but I wonder if it would uncover some hidden implementation issues.
Pushed
042e7a8d: a fix for the smearing on characters (Link's arms, legs and hair doubled into see-through copies when strafing, most visible at 2x).RX_VK_FSR_FG_SOURCE=vectors) against block matching between the real frames: median error about 1 px on Link, 5% of the motion. The doubling comes from AMD's interpolation: where the frame before and this frame disagree, it blends them half and half and inpaints the rest.LongshotNoDoubles, dispatchFlags bit 22, on by default).RX_VK_FSR_FG_DOUBLES=1gives AMD's blend back for an A/B.RX_VK_FSR_FG_SOURCE=game|flow|prevwarp|mask|vectors) come out before ready, and the branch still needs reworking onto #1041's row. The "A Talk" prompt is mangled in made frames, a separate HUD issue.042e7a8d1d0d2f95260e0d2f95260e718cb283b8718cb283b83ef7eca7acWIP: feat(vk): FSR 3.1 frame generation for everyone: any multiplier and Auto, the HUD kept still, actor motionto WIP: feat(vk): FSR 3.1 frame generation for everyone: FSR 3.1 FG on Frame generation by, any multiplier and Auto, the HUD kept still, actor motion, no see-through limbsWIP: feat(vk): FSR 3.1 frame generation for everyone: FSR 3.1 FG on Frame generation by, any multiplier and Auto, the HUD kept still, actor motion, no see-through limbsto WIP: feat(vk): FSR 3.1 frame generation for everyone: FSR 3.1 FG on Frame generation by, any multiplier and Auto, the HUD kept still, actor motion3ef7eca7ac7d190b64527d190b64526108a22375WIP: feat(vk): FSR 3.1 frame generation for everyone: FSR 3.1 FG on Frame generation by, any multiplier and Auto, the HUD kept still, actor motionto feat(vk): FSR 3.1 frame generation for everyone: FSR 3.1 FG on Frame generation by, any multiplier and Auto, the HUD kept still, actor motion6108a223757bb0590a16