feat(vk): FSR 3.1 as an engine of the temporal chain, with the characters' motion #2305
No reviewers
Labels
No labels
accessibility
autorelease: pending
autorelease: tagged
bug
dependencies
documentation
duplicate
enhancement
github_actions
good first issue
help wanted
invalid
question
testing wanted
wontfix
No milestone
No assignees
1 participant
Due date
No due date set.
Dependencies
No dependencies set
Reference
MagicBeanDev/Hookshot!2305
Loading…
Reference in a new issue
No description provided.
Delete branch "feat/fsr3-temporal-chain"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
AMD FSR 3.1 makes its own inputs (
vk_fsr3_inputs.cpp), and the only motion it gets is the camera's. So anything thatmoves differently from the camera is given the motion of what's behind it: Link while he walks, his swinging arm, his
shield, and the people around him. DLSS and FSR 4 already get those characters' own motion on
vk_temporal.cpp's chain.This PR puts FSR 3.1 on that chain too, as a third engine, so all three temporal upscalers share the same inputs.
What changed
vk_fsr3.cpp/.h:kChainEngine, FSR 3.1 as aTemporalEngine(slot 2).dlss_out.compas before.fsr3_chosen()is "the row orRX_VK_FSR=1picks FSR 3.1".fsr3_wanted()now means "FSR 3.1 on its own inputs",which is false while it's on the chain.
fsr3::prepare_sizesandfsr3::releaselet the chain size and free FSR's images.vk_temporal.cpp/.h:temporal_modereturnsFSR3for the row's FSR 3.1 before it falls back to the game's own;engine_ofgets a thirdslot.
TemporalEnginefields:fsr_depth: the chain reverses depth for FSR's passes (built withINVERTED_DEPTH) where the projection puts thenear plane at 0.
retry: a failed evaluation hands that frame to the network instead of turning the engine off for good. FSR 3.1'sconstants come from the frame's stream, which can run out for one frame.
TemporalFramecarries the unjittered projection and the jitter phase count.TRANSFER_DST.RX_VK_FSR_JITTER=0still turns FSR 3.1's jitter off on the chain.shaders/dlss_prep.comp:1 - d).fsr3_prep.comp's rule, because one NaN spreads through anupscaler's history.
glslangValidator -V --target-env vulkan1.3 --vn DLSS_PREP_COMP_SPV, the commandtest_dlss_actor.pychecks.RX_VK_FSR3_CHAIN=0gives FSR 3.1 its own inputs, as before, for comparisons (docs/development.md). The readme's FSR 3.1section describes the new inputs, and says what still trails: things that aren't skinned, such as Navi's body.
temporal_jitter_test.cppcovers FSR 3.1's slot and its failure gate, and a new run (fsr-jitter-off) where onlyRX_VK_FSR_JITTER=0is set: FSR 3.1 on the chain gets no jitter, and DLSS and FSR 4 keep theirs.test_upscaler_rows.pychecks that the row's FSR 3.1 runs on the chain before the fallback to the game's own, thatfsr3_wantedis off while it does, and thatengine_ofhas its slot.fsr-jitter-off.Testing
python -I -B tests/run_all.py: 363 passed, 0 failed, 11 skipped (Linux; the skips are Windows-only, need a Vulkan 1.3device, or need dotnet).
In game: RX 9070 XT 16 GB, Mesa RADV 26.2.6, Gentoo Linux (kernel 7.3.3), KDE Plasma Wayland, windowed, 2560x1440
output, 60 fps cap.
49f75dc8plus this branch.RX_VK_FSR3_CHAIN=0, which is what main does.vk: fsr3 on the chain: 600 frames drawn ... depth reversed for FSR by the chain, with noevaluation failures.
Market runs vary by about 0.9 fps between runs, so the chain costs nothing measurable.
Picture: consecutive 2560x1440 shots at Kokiri's lost-woods path, standing and walking, compared on the same build.
Edge flicker is the mean frame-to-frame change of edge pixels, 0-255, while standing.
Four independent reviewers of the walking sheets saw no visible difference at 100%. At 67%, they found the chain
slightly better: smoother edges on Link's shield and tunic hem, and slightly softer shield texture. No new artefacts.
The clearest evidence is the motion field itself (below). On main, Link isn't in it at all.
Not tested:
RX_VK_FSR_FG=1), which keeps usingvk_fsr3_inputs.cpp;Screenshots (from the game's output image before the present path, 2560x1440)
The motion vectors the upscaler gets while Link walks (
RX_VK_DLSS_DEBUG=motion). Left: camera motion only(
RX_VK_DLSS_ACTORS=0, which is what FSR 3.1 got on main). Right: on the chain, with Link's body, arm and shield carryingtheir own motion:
Link walking: own inputs vs the chain, at 100% and at 67% (2x zoom crops):
Merge notes (from test merges against every open PR that touches these files; textual conflicts show up in git, semantic ones don't)
TemporalFrame::depth_reversedsays so. Anything that reads the chain'sdepth must check it rather than assume plain depth:
ca2e3022passesdepth_reversedtoNGX's
depthInverted, andtest_upscaler_rowschecks it.depthInvertedfromdepth_reversed, not ahard-coded false.
nr_prepassess.depthon, and the NGX backend'sDepthInvertedshould follow it too.bool native_aatoTemporalEngine::prepare.chain_prepareinvk_fsr3.cppneeds the sameparameter, or the build breaks. git merges the two cleanly. FSR 3.1 never has native AA.
7ceae45cmakes the mask inchain_evaluatethe same way DLSS does on the chain, and
test_reactive_mask.pychecks it. Checked in game withRX_VK_REACTIVE=1: the mask ismade, and FSR 3.1 draws all 600 frames on the chain.
a0dfd28b) and is merged into this branch.dlss_prep.compscrubs NaN first, then applies #2243'smax/kSquare/min, and keeps theflags & 8depth flip.The header was regenerated; glslang 16.4.0 gives the same SPIR-V words as CI's 16.6.0 for
dlss_prepanddlss_out.TemporalEngineis orderedmemory,hdr,fsr_depth,retry: DLSS's and FSR 4's engines are positional,and
kChainEngineis designated.hdryet, so it keeps the LDR pair. Giving it HDR colour on #2243's mechanism is a follow-up.fsr3_wanted()to ask whether FSR 3.1 is chosen. After this PR that question isfsr3_chosen(). Itsfsr4 = !fsr3_wanted() && fsr4_wanted()guard and its jitter guard should usefsr3_chosen(). The merge istextually clean, so git won't flag it.
vk_temporal.cpp, the output image's usage should becomeTRANSFER_DST | TRANSFER_SRC, and itsproj[16]andthis PR's
projection[16]should become one field.RX_VK_FSR3_CHAIN=0.docs/tested-configurations.mdanddocs/development.md: rows next to other PRs' rows; keep both.🤖 Generated with Claude Code
Thanks, this is a clear improvement: FSR 3.1 finally gets the skinned draws' motion, and the motion-vector screenshot makes the case well. I went through the inputs and they line up with what FSR 3.1 got on its own path:
vk_fsr3_inputs.cpp, andchain_evaluateleavesmotion_scaleat 1, somotion_vector_scaleis 1/render size as before. Low-res motion, no display-res mixup (FSR 3.1 interpolation's #1943 question doesn't apply here, since FG keeps its own inputs).jitter_to_pixels, and the phase count comes from the samedlss_jitter_phasesthe chain publishes.RX_VK_FSR_JITTER=0still works, the jitter gate has its own slot, andvk_fsr3_inputs' own jitter stays off while FSR 3.1 is on the chain (it is gated onfsr3_wanted), so there's no double jitter.camera_from_projectionon the unjittered projection, the same test the own path makes. Motion is still computed from the unflippedd,actor_motionreads the game's depth view rather thans.depth, anddepth_reversedreaches NGX frame generation.prepare_sizesthen returns 2, so the next frame resets. Theretrypath clearslast_frame, so the frame after a failed evaluation starts a new history.temporal_rejected_network,fsr3_upscalenow returns false andtemporal_wanted()is true for FSR3, so left-out networks go throughtemporal_upscale. (Nit: that path's log line will then say "the temporal upscaler" rather than FSR 3.1.)upscale_lod_biasfalls through totemporal_mode(), which now returns FSR3, so the bias is unchanged. Same for the menu'supscaler().dlss_prep.comp(harmless, arguably a fix) and bit 3, which is never set for them.RX_VK_FSR_FG=1): it keepsvk_fsr3_inputsas it already does while DLSS upscales, so this is the same pairing as DLSS + FSR FG. Fine as untested, but a quick run would be nice.What blocks merging right now:
dlss_prep.comp's colour store (max/kSquare/kColorMax) and its SPIR-V header, thepc.flagsline and the prep/out pipeline selection intemporal_upscale,TemporalEngine(hdraftermemory),TemporalFrame's comment, and the docs rows. Your merge notes for #2243 apply to #2342 almost as written:max(e, 0). GLSL'smaxwith a NaN operand is undefined.kChainEngine's designated order valid (.hdrsits before.fsr_depth).dlss_prep_comp_spv.h.prep_hdrtoo.tests/run_all.pyrun on Windows and Linux.Smaller points:
fsr3_upscale, which is now only theRX_VK_FSR3_CHAIN=0path. Whichever of the two lands second needs to carry the mask throughTemporalFrameintochain_evaluate, or FSR 3.1 loses it by default. Good that you flagged this in the merge notes.The screenshots and numbers are there, thanks. The 67% flicker and outline numbers are convincing. Once it's rebased and CI is green I'd be happy to approve.
Nit: the chain's status line will now show FSR 3.1 using no VRAM of its own, though its internal images (history, luma pyramid, the RGBA16F output) are tens of MB at 1440p. If
fsr3::can sum the image sizes it made, it's cheap to report them here.Done in
514c7ed5c:fsr3::memory()adds up the internal images andkChainEngine'smemoryreports it, so the status line now shows FSR 3.1's own memory (217 MB at 2560x1440 native here). The #2166 log line for a left-out network now says "FSR 3.1 (on the temporal chain)" too.🤖 Addressed by Claude Code
Thanks for the thorough read. Where each point stands:
dlss_prep.compscrubs NaN before themax, then applieskSquare/kColorMax, and keeps theflags & 8flip.TemporalEngineis orderedmemory,hdr,fsr_depth,retry, andkChainEnginestays designated.dlss_prepanddlss_out; I checked against main's headers.depthInvertednow followsdepth_reversed(ca2e3022), and a test covers it.514c7ed5). The status line now shows FSR 3.1's own memory, 217 MB at 2560x1440 native.RX_VK_FSR_FG=1with frame generation at 2x and FSR 3.1 on the chain. It made every interpolated frame in gameplay (600 of 600 per status window); the only frames without its inputs were before gameplay. No device loss.879dd6b1). Everything passes except "Python source checks (windows-2022, part 1/2)", and its one failure istests/test_native_astc_texture_count.py: #2288's fixture usesscanf, which the Windows CRT marks deprecated under-Werror. Main fails that check the same way, and #2397 fixes it, so it should go green once #2397 lands and I merge main again. Locallytests/run_all.pypasses 391 of 391 on Linux.RX_VK_FSR3_CHAIN=0gives a direct comparison.chain_evaluate.Every earlier blocker is handled: #2342 merged in, NaN scrubbed before
max,kChainEngineorder valid, SPIR-V header regenerated, bit 3 reaching both prep pipelines, and #1653's reactive mask carried intochain_evaluate. Theretrypath drops one frame, and projection failures are filtered beforeevaluate, so it doesn't spin. The 11 temporal/FSR/DLSS checks pass locally.If you have one, a Steam Deck or iGPU frame-time number would help, since FSR 3.1 is the low-end upscaler and #2110 would make it the default.
github-sync referenced this pull request2026-11-12 06:22:54 +00:00