fix(framegen): FSR 3.1 interpolation built with the SDK's name for low-resolution motion vectors #1943
No reviewers
Labels
No labels
accessibility
autorelease: pending
autorelease: tagged
bug
dependencies
documentation
duplicate
enhancement
github_actions
good first issue
help wanted
invalid
question
testing wanted
wontfix
No milestone
No assignees
1 participant
Due date
No due date set.
Dependencies
No dependencies set
Reference
MagicBeanDev/Hookshot!1943
Loading…
Reference in a new issue
No description provided.
Delete branch "fix/fsr3fi-render-size-blur"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
With FSR 3.1 frame interpolation (
RX_VK_FSR_FG=1) and the 3D drawn smaller than the picture, every made frame is blurred and smeared in the top-left render-size corner. This happens with DLSS or FSR 4, or with the game's own upscale. The cause is a misspelt shader option: the interpolation pass that reads the game's motion was built for motion at the display size.What changed:
scripts/fsr3_shaders.pypasses-DFFX_FRAMEINTERPOLATION_OPTION_LOW_RES_MOTION_VECTORS=1, the name the SDK reads (ffx_frameinterpolation_reconstruct_dilated_velocity_and_previous_depth.h:106, andLOW_RES_in AMD'sCMakeCompileFrameinterpolationShaders.txt). Before, it passed the upscaler's spelling,..._LOW_RESOLUTION_..., which the SDK ignores. Sofi_reconstruct_and_dilateread the render-size motion atComputeHrPosFromLrPos(), a render pixel times display/render.gpu/vk/shaders/fsr3_fi_reconstruct_and_dilate_comp_spv.his regenerated with glslang 16.6.0 (CI's pin). It is the only pass that reads the option, and its bindings and capabilities are unchanged. The other 27 FSR headers recompile byte-identical, apart from glslang's version comment, and are left alone.gpu/vk/vk_fsr3fi.h: theInputscomment named the same wrong macro.tests/test_fsr3_shader_options.py(needs no glslang): every-Dthe script passes for the upscaler, interpolation and optical flow must be tested by an#ifin the SDK files those passes compile from, and every option the interpolation passes test must be set. It fails on main with exactly this name.tests/test_fsr3_shaders.pycould not catch it, because the headers matched their misspelt sources.RX_VK_FSR_FI_RENDER_SIZEand its readme note are gone.Testing:
scripts/fsr3_shaders.py --checkwith glslang 16.6.0: all 28 headers match.tests/run_all.py fsr3 fsr3fi framegen_resources upscale_independenton Windows: 4 passed.test_fsr3_shadersis skipped there because spirv-val is missing; run directly without it, all 28 match.vk_fsr3fi.cppandvk_framegen.cppcompile-checked. Based on maind7a7306.RX_VK_FSR_FG=1, in a build of main (d7a7306) with this fix. FSR 3.1 made 508 of the last 600 made frames, from render-size depth and motion (2240x1260), and the captures below are sharp in the top-left corner, where main's made frames are smeared.Not tested: AMD/Intel, FSR 4 as the upscaler, Linux.
Screenshots (a made frame each, DLSS Balanced to 3840x2160)
The before shot is main's behaviour: a made frame, smeared in the top-left render-size corner (HUD hearts doubled). The after shots are this fix, with the render-size inputs.
Top-left corner at the same scale, main and two captures with this PR:
HDR10 captures, tone-mapped to SDR for this page (paper white to white, highlights rolled off up to 800 nits).
🤖 Generated with Claude Code
Thanks, the before/after captures make the symptom clear. I think I found the root cause, though, and it's a one-line build fix rather than a workaround in the renderer.
The real bug is a misspelled shader define.
scripts/fsr3_shaders.pycompiles the interpolation passes with-DFFX_FRAMEINTERPOLATION_OPTION_LOW_RESOLUTION_MOTION_VECTORS=1, but the vendored SDK checks a different name:ffx_frameinterpolation_reconstruct_dilated_velocity_and_previous_depth.h:106has#if FFX_FRAMEINTERPOLATION_OPTION_LOW_RES_MOTION_VECTORS(it's alsoLOW_RES_in AMD'sCMakeCompileFrameinterpolationShaders.txt). The upscaler's option really is..._LOW_RESOLUTION_..., which is probably where the mix-up came from. Because of that,fi_reconstruct_and_dilateis built with the display-resolution motion branch. That branch reads the input motion atComputeHrPosFromLrPos(iPxLrPos), i.e. render pixel × display/render, out of a render-size motion image. That's exactly the "one of its passes reads a render-size input in the picture's pixels" the PR describes. Blitting depth and motion up to the display size and settingrender_width = uwmakes that mapping the identity, which explains why the screenshots come out sharp. It works around the miscompiled pass, though, and doesn't fix it.Fixing the define (and regenerating
fi_reconstruct_and_dilate's_spv.h, the only pass that reads it) should give the same result without the costs this PR adds:RX_VK_FSR_FI_RENDER_SIZEenv var and a readme note for what is really a build bug.Smaller things, if you do keep this approach:
make_imgfails,fi_scaled.wstays 0, so every following frame doesdevice_wait_idle_locked()and tries the allocation again. The other scaled images in this file work the same way, so it's minor, but a one-shot "failed" flag would avoid stalling every frame.vk_fsr3fi.hcomment onInputsnames the same wrong macro and should change either way.No overlap with the rest of the series that I can see: #1944 touches
vk_fsr3fi.cpp/vk_framegen.cppbut not this path, and #1940–#1942 don't touchfsr3fi_frame. No CI has run on this branch yet.Could you try the define fix (keeping the render-size inputs) with the same DLSS Balanced to 3840x2160 setup? If it's as sharp as your "after" shot, the PR can shrink to the script change, the regenerated header and maybe a test that checks the define against the name in the SDK header.
This works because, with
render_width == width, the miscompiled reconstruct-and-dilate pass'sComputeHrPosFromLrPosbecomes the identity. The pass itself is built with the wrong option name (LOW_RESOLUTION_vs the SDK'sLOW_RES_, see the review body). I'd fix that inscripts/fsr3_shaders.pyand drop this block, which runs all of FSR's render-size passes at display size.Thanks, that was the bug. I replaced the workaround with the define fix and kept the render-size inputs:
scripts/fsr3_shaders.pynow passesFFX_FRAMEINTERPOLATION_OPTION_LOW_RES_MOTION_VECTORS.fsr3_fi_reconstruct_and_dilate_comp_spv.his regenerated with glslang 16.6.0. The other 27 headers recompile byte-identical, apart from the version comment.Inputscomment invk_fsr3fi.his fixed.tests/test_fsr3_shader_options.pychecks every-Dthe script passes against the#ifs in the SDK files the passes compile from. It fails on main with exactly this name.The blits,
RX_VK_FSR_FI_RENDER_SIZE, the readme note and thefi_scaledretry are gone with it.Checked in the game with the same setup (DLSS Balanced to 3840x2160, 2x,
RX_VK_FSR_FG=1): FSR 3.1 made 508 of the last 600 made frames from the render-size inputs, and the top-left corner is as sharp as the earlier "after" shot. The screenshots in the description are now of this build, with a crop of the corner next to main's.Dropped. The script now uses the SDK's
LOW_RES_name, so FSR keeps the render-size inputs.Thanks, this is exactly the fix I hoped for. All of my earlier points are addressed:
FFX_FRAMEINTERPOLATION_OPTION_LOW_RES_MOTION_VECTORS, and FSR keeps the render-size inputs. The display-size blits,RX_VK_FSR_FI_RENDER_SIZE, the readme note and thefi_scaledretry are all gone (the inline thread is answered).Inputscomment invk_fsr3fi.his fixed.tests/test_fsr3_shader_options.pyis a good guard. I checked it against the vendored SDK: every-Din the three option lists is read by an#ifin the compiled GLSL/headers, and the CMake list's extra FI names (*_SAMPLERS_USE_DATA_HALF,JITTER_MOTION_VECTORS, ...) don't show up because only.h/.glslfiles are scanned. It needs noRUN_ALL_*declarations, sorun_all.pypicks it up as is.fi_reconstruct_and_dilate's header changes, which matches the SDK: that's the only pass with aLOW_RES_#if. The new in-game check and the corner crops look convincing.Nit (optional):
compiled_files()scans every GPU header for each family, so a name from one family passed to another (say anFFX_FRAMEINTERPOLATION_OPTION_*in the upscaler's list) would still pass the first test. Narrowing each family to its own subfolder plus the shared headers would catch that too. Fine to leave as is.CI: only the title checks have run. The build and test workflows (Linux AppImage, game-free source regressions, Windows builder) are waiting for a maintainer to approve them. The branch merges cleanly.
A question, not something this PR needs: would you be up for adding FSR 4 frame generation, here or in a follow-up? AMD's SDK v2.4.0, the release
--fetch-fsr-runtimepins (#1866), also shipsamd_fidelityfx_framegeneration_vk.dllwith "FSR Frame Generation (ML)" 4.0.2 on Vulkan 1.2+. According to AMD's docs it's Windows 11 and RX 9000 (RDNA 4) only; other GPUs get 3.1.x. The catch is the reason main made FSR 3.1 interpolation opt-in (#1635: the HUD moves with the scene), and #1831 says FSR 4's frame generation misbehaved the same way in play. AMD's answer to that is a HUD-less colour input or a separate UI surface. So it's only worth doing if it can be fed a HUD-less picture and avoids that problem. If you know a route that does, it'd be very welcome.Same fix tested on a 4070 Ti SUPER (Windows, 2x, 2048x1152 from a 67% render): on main the right third of every made frame is the game's frame unmoved; with the define fixed the whole frame moves, as at 100% render. The soft HUD in made frames is unchanged, at 100% too.
Thanks! Took the nit:
compiled_files()now follows each family's includes from its pass GLSL (beside the including file, then the GPU headers' folder and its family folders, as the script's-Ipaths do), so each family is checked against its own files plus the shared ones (gpu/,spd/, andfsr1/for the upscaler). An upscaler option put intoFI_OPTIONSnow fails the first test; main'sLOW_RESOLUTION_spelling still fails all three. Rebased onto current main.@jarynclouatre Thanks for confirming it on a 4070 Ti SUPER at a 67% render, that's a good second data point. The soft HUD in made frames is a separate thing: OoT draws its HUD into the same picture FSR 3.1 interpolates, so it moves with the scene's motion (the readme's reason this path is opt-in), at any render size.
Helps on Linux as well, which the description lists as not tested: on main, made frames in motion have an edge at two thirds of the width and height, and with this change they move as a whole.
Build: Arch Linux, RTX 4090, NVIDIA 615.78.08, Hyprland (Wayland), a 1920x1080 window at 60 Hz. Main
bdfe09cagainstbdfe09cwith this change on top. I built 93a25f2f; thegpu/andscripts/change of the current head0a043192has the same patch id. Betweenbdfe09cand your base49f75dcthe frame generation code differs only by an HDR image usage flag.Scene: output 3840x2160, render resolution 67 % (2560x1440), FSR 3 as the upscaler, frame generation 2x,
RX_VK_FSR_FG=1. Kokiri Forest from a save, Link running while the camera turns (input script). Made frames captured withRX_VK_SHOT_MADE=1at 1920x1080, one every 120 game frames. Both builds logfsr3fi: 3840x2160 from 2560x1440 inputs, flow 480x270, pyramid 1920x1080 with 11 levelsand595 of the last 600 made frames by FSR 3.1, 0 without its inputs.bdfe09cbdfe09c"Edge" is how much the picture changes across the line x=1280 (top 720 rows) or y=720 (left 1280 columns), divided by the change across the lines next to it: 1.00 means the line looks like its neighbours. A pair is in motion when the made frame differs from the game frame after it by more than 2 of 255 luma levels on average. On main the made frames have an edge on both lines that the game's frames do not have; with this change they do not.
New test at
0a043192:tests/test_fsr3_shader_options.pypasses (3 tests). With main'sscripts/fsr3_shaders.py, 3 of 3 fail ('-DFFX_FRAMEINTERPOLATION_OPTION_LOW_RES_MOTION_VECTORS=1' not found in [...]). At 93a25f2f,scripts/fsr3_shaders.py --checkwith glslang 16.6.0 printedfsr3 shaders: 28 match, and the header is the same at0a043192.Not tested: DLSS as the upscaler (this build has no DLSS), frame generation Auto, Windows, AMD, and the whole
tests/run_all.py. The two runs in motion were not taken at the same load (11 to 8 on main, 73 to 108 with this PR), so I compare only the edges and give no frame rates.Thanks for taking the nit.
compiled_files()now follows each family's includes from its own pass GLSL, so an option passed to the wrong family fails the first test. I read it against the vendored SDK: the lookup order matches the script's-Ipaths closely enough (it also searches the other family folders, but no header name repeats across them, so it can't pick a wrong file), and a header it can't find just goes unscanned, which can only make the test stricter, never let a wrong name through. Since 93a25f2, the regenerated SPIR-V header,vk_fsr3fi.handscripts/fsr3_shaders.pyare byte-identical; only the test changed, plus the rebase.Two more in-game confirmations since then, both showing the same thing: @jarynclouatre on a 4070 Ti SUPER on Windows at a 67% render, and @mooki000 on Linux (RTX 4090, Wayland) with a measured edge at two thirds of the frame on main that goes away with this PR. So the "not tested on Linux" in the description is covered now. The soft HUD in made frames is the known #1635 behaviour, as you said, not something this PR changes.
CI: only the title check has run. The build and test workflows (game-free source regressions, Linux AppImage, Windows builder) are waiting for a maintainer to approve them. The branch merges cleanly.
On my earlier question about FSR 4 frame generation: I didn't see an answer. No pressure, and it isn't needed for this PR. If you're interested in a follow-up, it would need a HUD-less colour input or a separate UI layer to get around the #1635 problem.
Approving. Nothing has changed since my last look at
0a04319. The low-res motion-vector option now uses the SDK's own name, the test follows each family's includes, and jarynclouatre (Windows, 4070 Ti SUPER) and mooki000 (Linux, RTX 4090) both confirmed the seam is gone in made frames. The soft HUD is the known #1635 behaviour, not this PR. The FSR 4 frame generation question isn't needed here, so that can be a separate follow-up.Only the title check has run on this head so far, so the build and test workflows need approving and a green run before merging.
github-sync referenced this pull request2026-11-12 06:22:54 +00:00