fix(vk): FSR 3.1 frame generation's made frames move again: its HUD-less colour is the presented frame's, sRGB-encoded under the HUD #1538
No reviewers
Labels
No labels
area/accessibility
area/audio
area/input
area/install
area/menu
area/renderer
area/saves
area/tooling
area/upscaling
beanstalk: approved
beanstalk: failed
beanstalk: needs approval
beanstalk: running
beanstalk: tested
ci: approved
good first issue
gpu/amd
gpu/igpu
gpu/intel
gpu/nvidia
help wanted
kind
bug
kind
chore
kind
docs
kind
feature
kind
performance
kind
question
needs measurements
platform/linux
platform/macos
platform/steam-deck
platform/windows
preview
priority
critical
priority
high
priority
low
regression
risk
high
risk
low
risk
medium
status
blocked
status
confirmed
status
duplicate
status
invalid
status
needs-info
status
wontfix
ci: approved
No milestone
No project
No assignees
1 participant
Notifications
Due date
No due date set.
Dependencies
No dependencies set
Reference
MagicBeanDev/Longshot!1538
Loading…
Reference in a new issue
No description provided.
Delete branch "HumanBean/Longshot:fix/fsr-fg-made-frames"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Type: fix · Risk: low · Tested on: Linux, RTX 3090 · Needs: AMD (Linux or Windows), Windows NVIDIA
TL;DR
With AMD FSR 3.1 frame generation the FPS counter went up but the game felt no smoother: every "made" frame was an exact copy of the game's own frame, so at 4x each game frame was simply shown four times. The HUD-less picture FSR 3.1 interpolates was in a different colour encoding from the frame on screen, and FSR 3.1 replaces a made pixel with the on-screen one wherever the two differ, which was everywhere. This PR gives FSR 3.1 a HUD-less picture that matches the on-screen frame exactly outside the HUD. Made frames now sit where they should (0.25, 0.50, 0.75 of the way between game frames, measured), checked in headless captures and played on an RTX 3090, where it felt better though a running character's limbs still ghost.
Breakdown
RX_VK_FSR_FG=1on NVIDIA).Why
A player on Linux (RTX 3090, 240 Hz,
RX_VK_FSR_FG=1, Frame generation 4x, Upscaler Off) saw the counter at ~170 fps but motion no smoother than with frame generation off. The log said FSR 3.1 made 594 of 600 sets and presents were evenly spaced, so the made frames' content was the suspect. Headless shots of the made frames showed them identical to the game frame that ends their set, to the bit. This started with #1487 (FSR 3.1's HUD-less mode, which fixed the HUD smear of #1310); #1487's own test shots show the same (made frame against its game frame: mean difference 0.015 of 255).See #1487, #1310; the limbs still ghosting: #1236.
How to reproduce
Seen on: Linux (CachyOS, sway), RTX 3090, NVIDIA 620.70.06
Settings and switches: Frame generation 4x, Frame generation by DLSS FG (
RX_VK_FSR_FG=1makes FSR 3.1 make them on NVIDIA; any setting where FSR 3.1 makes them), Upscaler Off, V-Sync OffSteps: turn the camera in Kokiri Forest. To see it in numbers:
RX_VK_SHOTS=<dir> RX_VK_SHOT_GO=<file> RX_VK_SHOT_MADE=alland touch the file while turning; compare eachvk_N_madeK.pngwithvk_N.png.Expected: made frames between the game frames, the motion smoother than Off.
Actual: each made frame is the game frame
vk_N.pngitself, bit for bit; the motion steps at the game's rate.What changed
FSR 3.1's HUD-less source now matches the presented frame exactly wherever nothing was drawn over the picture, so FSR 3.1 keeps its interpolation there and lays only the HUD (and effects drawn after the upscale) over it.
gpu/vk/shaders/fg_hud.comphudless_colour(): the presented frame's pixel where the picture before and after the HUD are the same; under the HUD the copy sRGB-encoded (srgb(); the native HDR picture as it is), clamped as the RGBA8 frame is. Linear there, it left a dark ring around the button prompts' see-through parts and the fairies' glow.fg_hud_comp_spv.hregenerated, its first line naming the shader's hash.gpu/vk/vk_framegen.cpphud_source(): tells the pass whether the picture is the native HDR one (push constants 16 → 20 bytes).scripts/fsr3_shaders.pyFI_PATCHES 4,fsr3_fi_inpainting_comp_spv.hregenerated: AMD's overlay factor (which grows with the difference, for a HUD-less colour only near the frame's) is 1 where they differ, since any difference is now the HUD: its fraction cut soft edges and glows off.RX_VK_SHOT_MADE=allshoots each of a set's made frames in turn asvk_N_madeK.png(framegen_made_image,vk_replay.cpp;=1and itsvk_N_made.pngas before).tests/test_fsr3fi_hudless_colour.pycompileshudless_colouras C++: no HUD gives the presented pixel bit for bit (main's gave the linear copy), the HUD gives the sRGB-encoded copy; checks the header is glslang's output for the shader and thathud_sourcepasses the HDR flag.Measurements
Verdict: fixed: made frames land where they are asked to (within 0.003 of 0.25, 0.50 and 0.75), against copies of the game frame on main; spacing and FSR 3.1's frame count are as before.
Hardware: Linux 7.3 (CachyOS), RTX 3090, NVIDIA 620.70.06
Scene: Kokiri Forest (pristine save), camera turn and Link running; 3840x2160 headless, output 2560x1440, Upscaler Off, Frame generation 4x by FSR 3.1 (
RX_VK_FSR_FG=1), V-Sync OffMethod: shots of every present while turning or running (
RX_VK_SHOT_MADE=all); for each made frame, phase correlation over 256-px blocks of the picture gives its shift from the game frame before against the shift between the two game frames: t = projection, the median over blocks. Before: one capture (the made frame of the window); after: three captures, all made frames of each.How to reproduce: the shots as in How to reproduce; the per-block comparison is a short numpy script (phase correlation of each block of the made frame and of the game frame against the game frame before).
Baseline: measured on main @
310e7c9and this change on it; rebased since onto main @4b68c42, which adds #1496 (V-Sync On only) and nothing on this path, so the numbers hold.310e7c9(+RX_VK_SHOT_MADE=all)603db934(b731339a: the same shader and passes, plus the check update)The play session on this PC with the fix (4x, V-Sync Off, the person I work for playing):
Before (left: game frame · middle: made frame at t = 0.5 · right: next game frame), main @
310e7c9: the made frame is the right-hand frameAfter, this PR @
603db934(made frame at t = 0.75): the scene moves, the HUD and prompts stay sharp with no ringFlags and settings
One diagnostic value added to an existing switch; no setting or default changes.
RX_VK_SHOT_MADE=1: the set's first made frame=1the same;=all: each made frame in turn,vk_N_madeK.pngTesting
Tested: on this PC (Linux, RTX 3090, NVIDIA 620.70.06, 240 Hz, sway): the person I work for played it at the reporter's settings (4x, FSR 3.1 by
RX_VK_FSR_FG=1, Upscaler Off, V-Sync Off) and found it better than before, but not completely fixed: characters' limbs still show artifacts while running (below, still open). Headless captures turning the camera and running (Kokiri Forest) as in Measurements; HUD prompts and fairies looked at in made frames: no ring. The in-game run was this change before the last tidy-up (the shader's rule moved intohudless_colour(), the same per-pixel result) on base310e7c9; rebased since on main @4b68c42(no conflict; #1496 in between is V-Sync On's).Not tested: the check suite (
tests/run_all.py): not run on this PC, at the request of the person I work for; CI runs it. Native HDR (thepqpath): not run. AMD and Intel GPUs, Windows.Must be tested:
@Beanstalk linux amd: FSR 3.1 frame generation 3x, Kokiri Forest, turn the camera; made frames between the game frames, the HUD prompts with no dark ring), expecting smoother motion than main and no HUD smear (#1310).RX_VK_FSR_FG=1or AMD: the same, by the Test Farm (windows).Still open, not this bug: a running Link's swinging legs and arms ghost in the made frames (his motion vectors check out on his body; the ghosting is where limbs cross between game frames). That is #1236, a follow-up.
Risks
What could break: FSR 3.1 frame generation's made frames where a HUD copy is made: a game picture that is not linear light (another upscale path) would get a slightly wrong colour under the HUD only; outside the HUD the presented pixel is used as it is.
Turning it off: revert the commit;
RX_VK_FSR_FG_HUDLESS=0gives FSR 3.1 the HUD-marked frame instead, as before #1487.Compatibility: Nothing changes: no saves, options or caches; two shader headers rebuilt in the renderer.
Issues
See #1236
Checklist
Everyone
type(scope): description) of the status line's type.can't be run by hand is in Testing's Not testable by hand, with why.
in Risks.
main.python -I -B tests/run_all.pypasses (any failure that also fails onmainis named). (Not run on this PC at the request of the person I work for; waiting for CI.)third_party/untouched (a new folder with aPROVENANCE.mdinstead).When it applies
Assisted-by:/Co-Authored-By:trailers.mainand passing here. (Added; not run on this PC at the request of the person I work for: waiting for CI.)RX_VK_SHOT_MADE=all: its row in gpu/vk/switches.def; a diagnostic, off unless set.)tests/that reads changed source updated in this PR, not weakened. (test_fsr3fi_hud_mask.py:hud_source's new HDR argument and 20-byte push constants, and a new case that the native HDR picture's flag reaches the pass.)🤖 Generated with Claude Code
Tested in game: RTX 3090, NVIDIA 620.70.06, Linux (CachyOS, sway, 240 Hz): Kokiri Forest, Frame generation 4x by FSR 3.1 (RX_VK_FSR_FG=1), Upscaler Off, V-Sync Off. Played by the person I work for: better than before (made frames now move: headless shots place them at 0.248/0.500/0.750 where main's were copies of the game frame), FSR 3.1 made 599/600 sets, 99% of frames shown 2-7 ms apart; HUD prompts and fairies without the dark ring. Not completely fixed: a running Link's limbs still ghost (#1236).
603db934f2b731339a4bPushed
b731339a:tests/test_fsr3fi_hud_mask.pyfollowshud_source's new native-HDR argument (20-byte push constants) and gains a case that the flag reaches the HUD pass. Description: the measurement's baseline after the rebase, and the checklist's applicable boxes. Not run on this PC at the request of the person I work for: waiting for CI.WIP: fix(vk): FSR 3.1 frame generation's made frames move again: its HUD-less colour is the presented frame's, sRGB-encoded under the HUDto fix(vk): FSR 3.1 frame generation's made frames move again: its HUD-less colour is the presented frame's, sRGB-encoded under the HUD