fix(vk): FSR 3.1 frame generation lays the HUD over each made frame's own scene: no halo of the real frame's background #1755
No reviewers
Labels
No labels
area/accessibility
area/audio
area/input
area/install
area/menu
area/renderer
area/saves
area/tooling
area/upscaling
beanstalk: approved
beanstalk: failed
beanstalk: needs approval
beanstalk: running
beanstalk: tested
ci: approved
good first issue
gpu/amd
gpu/igpu
gpu/intel
gpu/nvidia
help wanted
kind
bug
kind
chore
kind
docs
kind
feature
kind
performance
kind
question
needs measurements
platform/linux
platform/macos
platform/steam-deck
platform/windows
preview
priority
critical
priority
high
priority
low
regression
risk
high
risk
low
risk
medium
status
blocked
status
confirmed
status
duplicate
status
invalid
status
needs-info
status
wontfix
ci: approved
No milestone
No project
No assignees
2 participants
Notifications
Due date
No due date set.
Dependencies
No dependencies set
Reference
MagicBeanDev/Longshot!1755
Loading…
Reference in a new issue
No description provided.
Delete branch "madebyjordan/Longshot:fix/fg-smear"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Type: fix · Risk: medium · Tested on: Windows 11, RX 6700 XT (Adrenalin 26.10.1) · Needs: RX 9070 XT with FSR 4 Native AA and HDR (the #1565 reporters' setup); NVIDIA and Intel with FSR 3.1 FG; Linux
TL;DR
With AMD FSR 3.1 frame generation, every HUD element still carried a halo in the made frames on v3.0.0: the hearts' soft glow, the item box's shadow, the faint D-pad and Navi icons and the minimap showed the real frame's background through them, while the scene around had moved on. This PR learns how see-through each HUD pixel is from frame to frame and lays the HUD over each made frame's own scene instead. In the game on an RX 6700 XT the halo is gone (pictures below), with no frame-rate cost measured; a new game-free test fails on main and passes here.
Breakdown
B + T × scene, with B (the HUD's colour) and T (how see-through it is) learned from the pictures before and after the HUD over successive frames; where T isn't known yet, the presented pixel as before.RX_VK_FSR_FG_HUD_LAYER=0(main's path) against unset, camera turns in the field, made frames shot and compared; 600 of 600 presents with the HUD layer; three interleaved frame-rate pairs; two game-free tests (one failing on main); a cold review by a second agent, its findings fixed.RX_VK_FSR_FG_HUD_LAYER=0turns it off.Why
#1565 (RX 9070 XT, FSR 4 Native AA, FSR 3.1 FG) still sees "ghosting around UI elements, more noticeable with HDR on" on 3.0.0 RC8, after #1487, #1538 and #1691. On this PC the HUD is marked on 600 of 600 presents with every upscaler at 100% (game's own, FSR 3.1, FSR 4, Off), so it is not a missing capture any more. Shooting the made frames shows what is left: since #1487 FSR 3.1 interpolates the picture from before the HUD and then lays the presented frame over the made frame wherever the two differ (
FI_PATCHES4: the pixel whole). Where the HUD is see-through, the presented pixel is the HUD blended over this frame's scene, so each HUD element drags a patch of the real frame's background into the made frame: a box of different sky around the hearts, the Navi icon and the D-pad, a dark band at the item box's edge, the minimap's see-through parts. Brighter backgrounds (HDR) make the mismatch larger.The HUD itself can't be drawn again on its own here, but it stays where it is while the scene behind it moves. The game blends it as
D = B + T × S(S the picture before the HUD, D after), so from one frame to the nextD1 − D0 = T × (S1 − S0): T can be fitted per pixel from the two pictures the renderer already copies (#1464), and B follows.Fixes #1565. See #1310, #1464, #1487, #1538, #1691; #1236 (limbs, separate).
How to reproduce
Seen on: Windows 11, RX 6700 XT, Adrenalin 26.10.1 (reported on RX 9070 XT, 26.10.1/26.11.1)
Settings and switches: Frame generation 2x, Frame generation by FSR 3.1 FG, any Upscaler (seen with the game's own and FSR 4 at 100%), 60 fps cap
Steps: Hyrule Field by the drawbridge (the bench save), turn the camera with the right stick; to see the made frames:
RX_VK_SHOTS=<dir> RX_VK_SHOT_MADE=all RX_VK_SHOT_GO=<file>and create the file while turningExpected: the HUD's glows, shadows and see-through icons show the same background as the made frame around them
Actual: a halo of the real frame's background around each HUD element in every made frame
What changed
A per-pixel fit of how see-through the HUD is, kept from frame to frame, and the HUD laid over each made frame by it. FSR 3.1 itself is given its HUD-less colour as its present too, so it lays nothing over its made frames.
gpu/vk/shaders/fg_hud_layer.comp(hud_fit: a least-squares fit of T over the colour's channels, decayed so newer frames count more; started again when T no longer explains a frame, i.e. the HUD changed; unknown where the HUD just appeared, nothing moved behind it yet, or the scene is brighter than the frame's 1; a fit that isn't a number starts again) andfg_hud_composite.comp(hud_over:B + T × the made frame's scenein the picture's encoding, the presented pixel where T is unknown); SPIR-V headers made with glslang 16.2.0, their first line naming the shader's hash.gpu/vk/vk_framegen.cppkeeps two pairs of the before/after picture copies (this frame's and the frame before's),hud_layer()runs the fit after the mask pass, andfsr3fi_frames()gives FSR 3.1 the HUD-less colour asInputs::presentand lays the HUD over each made frame after copying it into the set. The 600-presentfsr3filine counts the presents with the HUD layer.RX_VK_FSR_FG_HUD_LAYER=0lays the presented frame as before (gpu/vk/switches.def, docs/development.md); docs/temporal-history.md's FSR 3.1 HUD paragraph.tests/test_fsr3fi_hud_layer.py(new) runshud_fitandhud_over, compiled as C++, over a HUD pixel through frames at B10G11R11's precision;tests/test_fsr3fi_hud_mask.pynow checks the wiring (FSR 3.1's present, the fit's inputs from both frames, the lay-over after the copy, the fallback when the layer's pipelines fail).Measurements
Verdict: fixed on this PC: the halo around the HUD is gone from the made frames while turning, 600 of 600 presents get the HUD layer, and the frame rate is the same within run-to-run noise.
Hardware: Windows 11 (26200), RX 6700 XT 12 GB, Adrenalin 26.10.1, Ryzen 5 5600, 16 GB
Scene: Hyrule Field by the drawbridge (
bench/pinned-fieldslot 02), 1920x1080 window, the game's own upscaler at 100%, Frame generation 2x by FSR 3.1 FGMethod: the control bridge turning the camera (right stick, 4 s) with every present shot (
RX_VK_SHOT_MADE=all); the made frames compared; thefsr3fi600-present line; frame rate: the bridge's game fps each second for 40 s standing still, uncapped, three interleaved pairs, the median of each run.Baseline: the same renderer with
RX_VK_FSR_FG_HUD_LAYER=0, which is main's path (FSR 3.1 given the presented frame; no fit, no lay-over), so both columns share every other commit. The renderer was built from this branch and run on the v3.0.0 game folder (renderer interface 34 on both;vk_framegen.cpp,vk_fsr3fi.cppand the HUD shaders are the same on v3.0.0 and main @8b0ea786). The first pictures (before any change) are v3.0.0's own shipped renderer. The commit after642789ecadds only the run file (docs/tested).How to reproduce: the steps above;
RX_VK_FSR_FG_HUD_LAYER=0for the before side.642789ec642789ectest_fsr3fi_hud_mask.pyin.present == in.color(main @8b0ea786)test_fsr3fi_hud_layer.py8b0ea786)Made frames while turning, the same spot (Hyrule Field by the drawbridge, 1920x1080, the game's own upscaler, FSR 3.1 FG 2x), 2x crops. Top: Before, v3.0.0's own renderer (main's path;
vk_framegen.cppis the same on main @8b0ea786). Bottom: After, this PR @642789ec. Before: a light box of the real frame's sky around the hearts, a ragged dark edge left of the item box. After: the made frame's own sky and wall through them.The minimap in the same two made frames. Left: Before (stone of the real frame along its lower edge). Right: After, this PR @
642789ec.Not affected: build time and size (two small shaders), loading and caches (no pipeline cache change; two compute pipelines made at the first frame with FSR 3.1 FG). VRAM: +24 bytes a pixel of the output while FSR 3.1 FG runs in play (the fit and the layer, RGBA16F, and a second pair of picture copies): about 50 MB at 1080p, 88 MB at 1440p, 200 MB at 4K.
Flags and settings
One new switch, for an A/B.
RX_VK_FSR_FG_HUD_LAYER=0the presented pixel as beforeTesting
Tested: on this PC (Windows 11, RX 6700 XT, 26.10.1): FSR 3.1 FG 2x with the game's own upscaler and with FSR 4 at 100%, turning the camera in Hyrule Field; the made frames shot with the switch off and on: the halo is gone with it on (the drawbridge chain and the clouds continue behind the hearts' glow, the mountain ridge behind the D-pad and the Navi icon); 600/600 presents with the HUD layer in every window; no errors in the log. The game-free tests:
test_fsr3fi_hud_layer.pyandtest_fsr3fi_hud_mask.pypass here, the mask test fails on main @8b0ea786atin.present == in.color(FSR 3.1 given the presented frame), and both fail when the fix's key lines are broken (lay-over always presented; no restart when the HUD changes; no NaN reset; no brightness guard). The 20 checks that readvk_framegen.cpp, the switch table, the docs or the shaders pass; the full suite on Windows: 730 passed, 61 failed, 62 skipped, and the same 61 fail on main @8b0ea786(Windows-only failures, none of them about this change). A second agent reviewed the diff cold; its findings (an uninitialised fit image, a loose tolerance, a scene over 1 behind the HUD, test gaps, memory) are fixed or listed here.Not tested: HDR (Expanded and Native): this monitor offers no HDR10 swapchain; the pq path lays the HUD over in the picture's own encoding, as #1538 takes it. NVIDIA, Intel, Linux. Other areas than the field and the castle gate. 4x and Auto (the lay-over runs once per made frame).
Must be tested:
fsr3filine saying600 with its HUD layer(by hand: a #1565 reporter)600 with its HUD layer(Test Farm: @Beanstalk Frame generation 2x by FSR 3.1 FG, turning the camera in Hyrule Field, on Linux and NVIDIA)Risks
What could break: the HUD's colour in made frames if the fit is wrong somewhere: a T that explains this frame within the picture's precision is used, else the presented pixel; an animated HUD (a beating heart, a count) shows the presented pixel for the frames it changes. A frame's last pass doing more than the plain encode over the picture would make the made frames' HUD differ from the real ones (#1538 already relies on the frame being the picture encoded). VRAM as above.
Turning it off:
RX_VK_FSR_FG_HUD_LAYER=0.Compatibility: nothing changes for a player after updating: no settings, saves, shader or pipeline caches, or renderer interface change.
Issues
Fixes #1565
Checklist
Everyone
type(scope): description) of the status line's type.can't be run by hand is in Testing's Not testable by hand, with why.
in Risks.
main.python -I -B tests/run_all.pypasses (any failure that also fails onmainis named).third_party/untouched (a new folder with aPROVENANCE.mdinstead).When it applies
Assisted-by:/Co-Authored-By:trailers.mainand passing here.tests/that reads changed source updated in this PR, not weakened.642789ecand from v3.0.0 with the same change. Linux not built here yet: the change is portable C++ and GLSL, no platform calls; the Test Farm run below builds it on Linux.)🤖 Generated with Claude Code
Tested in game: Windows 11, RX 6700 XT, Adrenalin 26.10.1: Hyrule Field by the drawbridge, FSR 3.1 FG 2x with the game's own upscaler and FSR 4 at 100%, turning the camera; this branch's renderer (
642789ec) on the v3.0.0 game folder. Made frames shot with RX_VK_FSR_FG_HUD_LAYER=0 and unset: the halo around the hearts, D-pad, Navi icon, item box and minimap is gone with it on; 600/600 presents 'with its HUD layer'; same game fps (median 50.9 off, 51.1 on, three interleaved pairs uncapped); no errors.The before/after pictures (made frames, the same spot) are in the description now. Still open before ready: a Linux build (portable C++ and GLSL; asking the Test Farm next).
@Beanstalk Frame generation 2x by FSR 3.1 FG (RX_VK_FSR_FG=1 on NVIDIA), the game's own upscaler at 100%, turning the camera in Hyrule Field for a few seconds: on Linux (any GPU) and on Windows NVIDIA and Intel. Check it builds and runs, no crash, and the log's fsr3fi 600-present line says '600 with its HUD layer'; frame times against main.
Beanstalk test farm
Commit
e0e5e53· tier reviewed · approved by madebyjordan · dashboardWhat will be tested · AI plan (
gpt-6.1-terra)Build and run main and the PR on Linux, Windows NVIDIA, and Windows Intel; the Windows Intel target will wait for an online tester. Each run uses 2x FSR 3.1 frame generation and the game's own upscaler in Hyrule Field, turns the camera, captures the HUD, and measures frame times. The run takes about 12 minutes per tester, excluding builds, and fails on a crash, a missing 600-present HUD-layer log line, a substantial frame-time regression, or a large screenshot difference. The screenshots are a general visual regression check; they do not specifically verify the halo on generated frames. No tester online now matches windows+intel: that target waits for one.
kokiri-start(settings dlss=0, resolution=0, frame_rate_cap=0, frame_generation=2, frame_generation_by=1; switchesRX_VK_FSR_FG=1), then teleport to Hyrule Field, move the stick, take the picture field-hud to compare with main, measure field for 30 s (after 10 s warm-up); 1 run on each side.Done (started 16:45 UTC, last report 17:27 UTC). Comment
@Beanstalkto run it again.AMD Radeon RX 7900 XTX (RADV NAVI31), 26.2.7; 2: linux+amd+radv+dgpuAMD Radeon RX 9070 XT (RADV GFX1201), 26.2.6NVIDIA GeForce RTX 3050 Ti Laptop GPU, 621.24.0; 2: windows+nvidia+nvidia-proprietary+dgpuNVIDIA GeForce RTX 4070 Ti, 621.14.0Measurements · linux
Verdict: fail on linux (2/1 report):
field-hud log_expected_missing.maxfailed.Hardware: 1: linux+amd+radv+dgpu
AMD Radeon RX 7900 XTX (RADV NAVI31), 26.2.7AMD Ryzen 7 5800X3D 8-Core Processor, 64 GB RAM; 2: linux+amd+radv+dgpuAMD Radeon RX 9070 XT (RADV GFX1201), 26.2.611th Gen Intel(R) Core(TM) i9-11900KB @ 3.30GHz, 32 GB RAMScene: field-hud, from the repository's save kokiri-start (dlss=0, resolution=0, frame_rate_cap=0, frame_generation=2, frame_generation_by=1, RX_VK_FSR_FG=1); each cell's settings and switches as the table names them
Method: the Beanstalk test farm: main and this PR built and run on the same PC in one session, game starts alternating (ABBA); field-hud: 1 game start a side, field 30 s after 10 s warm-up; each number the median of the runs, then of 2 reports
How to reproduce: the farm's plan above (
@Beanstalkon this PR runs it again)8b0ea78e0e5e53Flags and settings
None: the plan varied no setting, switch or build option.
Markdown for the PR description
Measurements · windows+nvidia
Verdict: could not be judged on windows+nvidia (2/1 report).
Hardware: 1: windows+nvidia+nvidia-proprietary+dgpu
NVIDIA GeForce RTX 3050 Ti Laptop GPU, 621.24.0AMD Ryzen 7 5800H with Radeon Graphics, 16 GB RAM; 2: windows+nvidia+nvidia-proprietary+dgpuNVIDIA GeForce RTX 4070 Ti, 621.14.0AMD Ryzen 9 5900X 12-Core Processor, 64 GB RAMScene: field-hud, from the repository's save kokiri-start (dlss=0, resolution=0, frame_rate_cap=0, frame_generation=2, frame_generation_by=1, RX_VK_FSR_FG=1); each cell's settings and switches as the table names them
Method: the Beanstalk test farm: main and this PR built and run on the same PC in one session, game starts alternating (ABBA); field-hud: 1 game start a side, field 30 s after 10 s warm-up; each number the median of the runs, then of 2 reports
How to reproduce: the farm's plan above (
@Beanstalkon this PR runs it again)Nothing was measured: no metric came back.
Flags and settings
None: the plan varied no setting, switch or build option.
Markdown for the PR description
WIP: fix(vk): FSR 3.1 frame generation lays the HUD over each made frame's own scene: no halo of the real frame's backgroundto fix(vk): FSR 3.1 frame generation lays the HUD over each made frame's own scene: no halo of the real frame's background