fix(vk): FSR 3.1 frame generation interpolates the picture from before the HUD and lays the frame's HUD over it #1487
No reviewers
Labels
No labels
area/accessibility
area/audio
area/input
area/install
area/menu
area/renderer
area/saves
area/tooling
area/upscaling
beanstalk: approved
beanstalk: failed
beanstalk: needs approval
beanstalk: running
beanstalk: tested
ci: approved
good first issue
gpu/amd
gpu/igpu
gpu/intel
gpu/nvidia
help wanted
kind
bug
kind
chore
kind
docs
kind
feature
kind
performance
kind
question
needs measurements
platform/linux
platform/macos
platform/steam-deck
platform/windows
preview
priority
critical
priority
high
priority
low
regression
risk
high
risk
low
risk
medium
status
blocked
status
confirmed
status
duplicate
status
invalid
status
needs-info
status
wontfix
ci: approved
No milestone
No project
No assignees
2 participants
Notifications
Due date
No due date set.
Dependencies
No dependencies set
Reference
MagicBeanDev/Longshot!1487
Loading…
Reference in a new issue
No description provided.
Delete branch "HumanBean/Longshot:fix/fsr-fg-hudless"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Type: fix · Risk: medium · Tested on: Linux, RTX 3090 · Needs: AMD (Windows and Linux), Intel
TL;DR
With AMD FSR 3.1 frame generation, the HUD still breaks up in the made frames when the camera turns, on nightly-429 with #1464: hearts and icons speckle, and pieces of the minimap trail behind it. This PR gives FSR 3.1 the picture from just before the game draws its HUD, so it moves only the scene, then lays the real HUD over each made frame. That's how AMD's SDK is meant to handle a HUD. In the game the HUD stayed clean while turning, where #1464's way left red specks under the hearts.
Breakdown
RX_VK_FSR_FG_HUDLESS=0against it: 600 of 600 presents made by FSR 3.1 with the HUD-less picture, no errors, the HUD clean in the made frames. The person I work for played it and confirmed the HUD no longer breaks up.tests/run_all.pyhas not been run (see Testing).RX_VK_FSR_FG_HUDLESS=0goes back to #1464's.Why
#1464 marked the HUD in the alpha of the frame FSR 3.1 interpolates, and patched AMD's passes to keep those pixels still. The HUD still breaks up with nightly-429 (a 10 s clip of Hyrule Field turning, FSR 3.1 FG): in the made frames the hearts, the D-pad and Navi icons dissolve into speckles of the scene behind them, the minimap's arrow smears, and part of the minimap trails beside it.
Two things the mark can't fix: FSR 3.1 keeps the frame before, HUD and all, as its previous source, and a turn drags that HUD with the scene; and its inpainting fills the holes a turn opens from a pyramid built over the made frame, which already has the HUD written in, so the HUD's colours spread into them.
AMD's SDK has a mode for games like this one: give the frame without the HUD as the interpolation source, and the presented frame beside it. The interpolation then never sees a HUD, and the inpainting pass lays the presented frame over the made one wherever the two differ (
HUDLessAttachedFactor). #1464 already copies exactly that picture (framegen_hud_before), so it only had to be handed over.See #1310 (closed by #1464, still seen), #1420, #1464.
How to reproduce
Seen on: nightly-429; reported in a clip from a PC other than this one; on this PC (Linux, RTX 3090, NVIDIA 620.70.06) the red specks under the hearts below
Settings and switches: Frame generation 2x with Frame generation by FSR 3.1 FG (
RX_VK_FSR_FG=1on NVIDIA), the game's own upscalerSteps: Kokiri Forest from the start save, next to Saria ("A Talk" shows), turn the camera with the right stick; or anywhere in Hyrule Field, turning fast
Expected: the HUD stays still and whole in the made frames
Actual: in the made frames, scene speckles in the hearts and icons, the minimap's arrow smeared, HUD pixels trailing below the hearts and beside the minimap
What changed
FSR 3.1 is now given AMD's HUD-less colour in play: the picture copied right after the gameplay upscale, cropped as the presented frame, and the presented frame beside it. History resets when a frame switches between the two kinds of source.
gpu/vk/shaders/fg_hud.compwrites the copy, cropped as the frame, to a second image (hudless) next to the marked source;fg_hud_comp_spv.hregenerated.gpu/vk/vk_framegen.cpphud_source()makes and bindshud.hudless;fsr3fi_frames()gives it asin.colorand the presented frame asin.present(the marked source withRX_VK_FSR_FG_HUDLESS=0, as before).gpu/vk/vk_fsr3fi.hInputs::present;gpu/vk/vk_fsr3fi.cppgenerate()setshudless_attached_factorand bindspresentas the inpainting pass's present backbuffer (binding 2), leaves the HUD patches' flags off (no HUD in either source to keep), and resets when the previous source was the other kind.RX_VK_FSR_FG_HUDLESS(gpu/vk/switches.def, docs/development.md).tests/test_fsr3fi_hud_mask.pyfollows the mask pass's fifth binding and the HUD-less inputs.Measurements
Verdict: the HUD stays clean in FSR 3.1's made frames where #1464's mark left specks; no cost measured (one extra RGBA8 store per pixel in the existing mask pass), frame times not compared.
Hardware: Linux (CachyOS, kernel 7.3.3), RTX 3090, NVIDIA 620.70.06
Scene: Kokiri Forest from the start save next to Saria, 2560x1440 headless, the game's own upscaler, Frame generation 2x by FSR 3.1 FG (
RX_VK_FSR_FG=1)Method: the same build twice,
RX_VK_FSR_FG_HUDLESS=0and unset, a 6 s right-stick turn,RX_VK_SHOTS+RX_VK_SHOT_MADEof the made frames during it, and thefsr3fi600-present lines; one run each (a visual check, not timing)How to reproduce: the steps above, with
RX_VK_SHOTS=<dir> RX_VK_SHOT_MADE=193d11850e,RX_VK_FSR_FG_HUDLESS=093d11850eBaseline: the same build with the switch, so both columns share every other commit; the switch path is #1464's code unchanged.
Made frames during the turn, two each. Top row: #1464's mark, this PR @
93d11850ewithRX_VK_FSR_FG_HUDLESS=0. Bottom row: After, this PR @93d11850e.Flags and settings
One new switch, for an A/B.
RX_VK_FSR_FG_HUDLESS=0gives #1464's markTesting
Tested: Linux, RTX 3090, NVIDIA 620.70.06: the A/B above in the game (Kokiri, turning next to the HUD), made frames compared, no Vulkan errors in either log. The person I work for played the build on this PC with FSR 3.1 FG and confirmed the HUD no longer breaks up.
Not tested:
python -I -B tests/run_all.pywas not run: the checks don't run on this PC.tests/test_fsr3fi_hud_mask.pywas updated for the fifth binding and the HUD-less inputs but has not been compiled. Not run on AMD, Intel or Windows; not with DLSS, FSR 4 or XeSS as the upscaler; not with PsychoV35's RGBA16F picture.Must be tested:
python -I -B tests/run_all.pyon a PC that runs the checks, expectingfsr3fi hud mask: okand no new failures (by hand: the author or a reviewer)with the HUD markednear 600 in thefsr3filog line (Test Farm: @Beanstalk amd, linux and windows)Risks
What could break: FSR 3.1's made frames in play: a HUD element the game draws before its gameplay upscale would be interpolated as scene; a frame that alternates between having the copy and not would reset FSR's history each time (stutter in the made frames).
Turning it off:
RX_VK_FSR_FG_HUDLESS=0, or revert the commit.Compatibility: Nothing changes: no settings, saves or caches; the vk ABI is unchanged.
Issues
See #1310
See #1420
Checklist
Everyone
type(scope): description) of the status line's type.can't be run by hand is in Testing's Not testable by hand, with why.
in Risks.
main. (on main @c7db6c02; the tested build is the same change onb8f3d547, 3 commits earlier, none of them in these files)python -I -B tests/run_all.pypasses (any failure that also fails onmainis named). (not run on this PC: see Testing)third_party/untouched (a new folder with aPROVENANCE.mdinstead).When it applies
Assisted-by:/Co-Authored-By:trailers.tests/that reads changed source updated in this PR, not weakened. (updated, not yet run)🤖 Generated with Claude Code
Tested in game: RTX 3090, NVIDIA 620.70.06, Linux: Kokiri Forest next to Saria, Frame generation 2x by FSR 3.1 FG, turning the camera: A/B against RX_VK_FSR_FG_HUDLESS=0 (#1464's mark), 600/600 presents made by FSR 3.1 with the HUD-less picture, hearts/minimap/prompt clean in the made frames where the mark left red specks under the hearts; also played by hand on the same PC, HUD no longer breaks up.
WIP: fix(vk): FSR 3.1 frame generation interpolates the picture from before the HUD and lays the frame's HUD over itto fix(vk): FSR 3.1 frame generation interpolates the picture from before the HUD and lays the frame's HUD over itFollow-up after #1487, testing nightly 464: the HUD fix now works in my visual test with FSR 4 upsaling, but the HUD still moves/shimmers with FSR 4 Native AA at native 1440p. The remaining case appears to involve the FSR 4 Native AA path. Other FSR 4 quality modes have not been isolated in this report.
During a resolution sweep in one session, the log showed: