AMD FSR 3.1 upscaling and frame generation (split from #1335) #1386

2026-11-10T00:06:38Z

notquiteog commented

2026-11-10 00:06:38 +00:00

(Migrated from github.com)

What

AMD FSR 3.1 upscaling and FSR 3.1 frame generation, split out of #1335 as asked. Nothing else from #1335 is in it.

Upscaling: a fourth value on the existing Upscaler row, AMD FSR 3.1, after DLAA (dlss=3 in hookshot_options.ini; 1 and 2 keep meaning DLSS and DLAA). It takes the gameplay upscale network's dispatch: the same dispatch DLSS takes, found the same way.
Frame generation: with Frame generation at 2x, FSR 3.1's frame interpolation makes the frame in between. It uses the scene's depth and motion. Where it cannot run, the renderer's own interpolation runs as before, and it always handles Auto and 3x and above (see below).
Built from AMD's open shader sources. FidelityFX SDK v1.1.4 (commit c6efa6bf, MIT) is vendored unmodified in third_party/fidelityfx-sdk and compiled into gpu/vk/shaders/fsr3_*_spv.h. Every pass is a dispatch of ours. FSR 3.1 needs no AMD runtime and no DLL: nothing of AMD's is linked or loaded at run time, so it is in every build on every platform. AMD's runtime DLLs are only for FSR 4. FSR 4 is not in this PR (FSR 4.1 is #1008's).
No NVIDIA changes: no DLSS, DLSS-G, DLSS 5, NGX or Streamline code. The only change to vk_dlss.cpp is in dlss_mode(). It used to clamp the row's value into 0..2, so 3 would have turned into DLAA. Now any value other than 1 or 2 means "not DLSS".
No release or packaging changes and no licence acceptance in CI. The only workflow edit adds glslang-tools to source-regressions.yml's apt line, because tests/test_fsr3_shaders.py compiles every vendored pass again and checks the checked-in headers.

Supersedes #1313: its commit is this PR's first commit, unchanged apart from one context line in tests/run_all.py after the rebase.

Commits:

build(vk): the vendored passes, scripts/fsr3_shaders.py, the 28 headers and their test (#1313)
feat(vk): FSR 3.1 upscaling (vk_fsr3.*, vk_fsr3_inputs.*, fsr3_math.h, shaders/fsr3_prep.comp), the jitter in every lift
feat(vk): FSR 3.1 frame interpolation (vk_fsr3fi.*, the chain in vk_framegen.cpp)
feat(menu): the Upscaler row's FSR 3.1 value, the Frame generation hint, and readme.md

Why

Players have asked for FSR 3.1 frame generation, and #1335 was too big to review and test in one piece.

Inputs: what comes from #841 and what does not yet

All of FSR's inputs are made in one file, gpu/vk/vk_fsr3_inputs.cpp, behind one function, prepare(), which returns one Frame struct. Moving FSR 3.1 onto #1008's shared temporal inputs means changing that file alone.

Jitter: #841's, the one jitter in the game. nx2/oot_jitter.py with hos_dlss_jitter_clip. The fork's own jitter (a viewport offset in vk_draw.cpp) is not here.
- nx2/lift.py now installs the jitter in every standalone lift, not only in builds with NVIDIA's SDK. make_game.py no longer sets OOT_DLSS_JITTER, and oot_jitter.wanted() is gone. The lift key covers nx2/, so the first build after this PR lifts the game again. tests/test_make_game_ngx.py no longer checks the old switch.
- The jitter stays zero unless DLSS or FSR 3.1 draws. rx_option(HOS_OPT_DLSS_JITTER_W/_H/_PHASES) returns dlss_jitter_size(), or FSR's fsr3_inputs::jitter_size() (the scene's size while FSR has drawn in the last 30 frames). The phase count is dlss_jitter_phases(), the host's own sequence length, and FSR is given the same count. RX_VK_FSR_JITTER=0 turns it off for a run.
- FSR reads the offset back from the camera's uniform buffer (P[0][2], P[1][2]), as vk_dlss.cpp does. It uses dlss_math.h's jitter_to_pixels and unjitter to convert the offset and take it off the motion.
- #841's names (HOS_OPT_DLSS_JITTER_*, hos_dlss_jitter_clip) are kept, so its interfaces do not churn under #1008.
Camera (near, far, field of view): from the projection, with no hook in the game. nx2/oot_camera.py and host/hos_oot_camera.c are not here.
- The camera is taken from the same uniform buffer and with the same checks as dlss_note_scene_draw.
- fsr3_math.h derives FSR's device_to_view_depth factors and tan(fov/2) from the projection at +0x70. This was checked in a run: p22 -1.0002, p23 -0.20002 gives near 0.1, far 1000.5 and a 45.00 degree field of view.
- The scene's depth is 0 at the near plane (not reversed). The passes are compiled for reversed depth, so FSR is given 1 - d.
- The fork's camera hook published near 1.0 and far 20000 (sead's projection object, not the one the scene is drawn with) and gave FSR reversed-infinite factors. Neither matches the depth FSR was given.
- tests/test_fsr3_math.cpp checks the factors against FSR's own view-distance formula, for both depth directions and for finite and infinite far planes.
Motion: the camera's only, for now.
- shaders/fsr3_prep.comp is the fork's camera-motion pass, reworked to dlss_prep.comp's math: reproject the depth from this frame's jittered clip space to the previous frame's unjittered one, and take this frame's jitter off. The same pass also writes FSR's colour and depth copies.
- #841's actor motion (the skinned draws drawn again with last frame's bones) lives inside vk_dlss.cpp behind HOOKSHOT_NGX, so a build without NVIDIA's SDK has none of it. Moving it out from behind the gate is #1008's refactor (vk_temporal.cpp), and this PR does not duplicate it.
- Until #1008 lands, things that move by themselves can trail with FSR 3.1, as the row's help says. When it lands, vk_fsr3_inputs.cpp's prepare() takes #1008's inputs and FSR gets the actor motion.
Colour: the network's input as it samples it (its square-root encoding), clamped to 0..1, with NaN set to 0. FSR's output is squared back into the network's output by dlss_out.comp, the same pass DLSS's picture goes through.

Left out of the fork's version:

the viewport jitter
the camera hook
upscale_seam.h, upscale_contract.h, upscale_format.h and their tests (the stand-in takes the dispatch the way dlss_stand_in does)
FSR's own RCAS (the Sharpness row already sharpens every upscaler's picture)
the diagnostics (RX_VK_FSR_STAGE, the VP hunt, the MRT census)

Fixed in the port:

The fork's FSR output was blitted into the network's output without being squared back.
The fork's FSR files did not include vk_work.h, so full_barrier could leave out a barrier after their work.
The fork's camera-motion image never had its first layout transition recorded.
When the size changed, the fork destroyed images that frames still in flight could be using. They are now freed a few presents later.

Frame generation

The chain in vk_framegen.cpp is: FSR 3.1, then the renderer's own blend for any present FSR does not take (no inputs for that present, a reset, a GPU that cannot run the passes).

FSR 3.1's interpolation has no phase input: it only makes the midpoint. So it is used at 2x only. Auto and 3x to 8x use the renderer's own.
A present FSR did not see breaks the sequence it keeps its previous frame by, so the next present is a reset. It never interpolates against a stale frame, and each frame's depth and motion are used by one present only.
The Frame generation row's 2x hint says when FSR 3.1 makes the frame. RX_VK_FSR_FG=0 turns FSR's interpolation off for a run (=1 forces it on over the row).

GPUs and drivers

FSR 3.1 is offered where the device has what the passes need. The Upscaler row is shown on such a GPU even in a build without NVIDIA's SDK. These are on top of the renderer's own floor (Vulkan 1.3: dynamic rendering, synchronization2, timeline semaphores).

Upscaler (fsr3::supported()):
- VK_KHR_push_descriptor with maxPushDescriptors of at least 15
- shaderStorageImageExtendedFormats and shaderStorageImageWriteWithoutFormat
- storage and sampled images (optimal tiling) in R8_UNORM, R16_SFLOAT, R8G8B8A8_UNORM, R16G16B16A16_SFLOAT, R16G16_SFLOAT, R32_SFLOAT, R32_UINT, R32G32B32A32_SFLOAT and R16_SNORM
- subgroup basic and quad operations in compute are used where present; without them the two pyramids run as their wave-less variants
Frame interpolation (fsr3fi::capable()):
- push descriptors for the widest pass (19)
- shaderStorageImageReadWithoutFormat, shaderStorageImageWriteWithoutFormat and shaderStorageImageExtendedFormats
- subgroup basic, arithmetic and quad operations in compute
- storage images in R8_UINT, R8G8_UNORM, R16G16_SINT, R16G16_SFLOAT, R32_UINT, R32_SFLOAT, R16G16B16A16_SFLOAT, and the presented R8G8B8A8_UNORM

The log says which way it went: vk: fsr3: AMD FSR 3.1 can run on this GPU, or vk: fsr3: not on this GPU: <why>, and the same for fsr3fi. Tested only on an NVIDIA RTX 5060 Ti with driver 615.71.09 on Linux.

Testing

Linux, RTX 5060 Ti, driver 615.71.09. D=<deps> is build-linux/deps.

Builds (both link):

EMSDK=$D/emsdk RX_CPM=$D/cpm OOT_WORK=... RX_MARCH=x86-64-v3 RX_VK_ONLY=1 python -I nx2/build.py --standalone --jobs 8: compiled, 0 failed, linked hookshot_vk.so (no NVIDIA SDK)
the same with HOOKSHOT_NGX_SDK=<ngx-sdk>: compiled, 0 failed, linked ("with NVIDIA DLSS")
each of commits 2 and 3 on its own: the renderer builds and the tests touching it pass

Tests:

After rebasing onto current main (0828863): the renderer builds and links without NVIDIA's SDK, and tests/run_all.py
(with the CPM headers found, so the SPIR-V and Vulkan header tests run too) gives 202 passed, 0 failed, 5 skipped.
tests/run_all.py on main (a116390): 180 passed, 0 failed, 19 skipped
tests/run_all.py on this branch: 182 passed, 0 failed, the same 19 skipped. The two new ones are test_fsr3_shaders.py and test_fsr3_math.py.
On the earlier base 75a676e: 166 / 168 passed, 0 failed, 17 skipped.

Whole game: make_game.py --update <a copy of the game folder> --jobs 10 built the whole game from this branch in 10 minutes. The lift logged dlss jitter: 3 of 3 sites in a build without NVIDIA's SDK. A run of that build (FSR 3.1 + 2x, pacing on) gave a game fps of 58.4 and 117.2 frames shown a second, with FSR 3.1 making 600 of 600 frames and no refusal lines.

In game:

Setup: the Kokiri Forest scene with Saria, about 1800 draws a frame, 1536x864 to 2048x1152. HOS_INPUT_FRAME_CLOCK=1 HOS_REALTIME_STEP=0 RX_VK_PRECOMPILE=0, the same HOS_INPUT_SCRIPT in every arm, and a stop at 6000 presents. The game's fps is the mean of the vk: N presents, X fps lines between presents 4800 and 6000.
The runs were windowed on Xvfb, not HOS_HEADLESS: frame generation is off without a window (framegen_on() checks !dev.offscreen).
No build ran during an arm. Arms run while another process was using the GPU were discarded and run again.
The arms in the table ran on this branch before its rebase from 75a676e onto a116390. The FSR code is the same; the rebase only resolved conflicts with frame generation Auto, the menu and run_all.py. The whole-game run above is on the rebased branch.

Arm	Game fps (2 runs)	Shown fps	Frames shown 7-9 ms apart
Game's own upscaler	59.7, 59.3
FSR 3.1	59.4, 58.5
FSR 3.1 + FG 2x, pacing on (FSR makes 600 of 600)	58.4, 59.7	116.6, 119.7	1994 of 2332, 2287 of 2394
FSR 3.1 + renderer's own 2x, pacing on	59.3, 59.9	115.8, 120.0	1902 of 2318, 2333 of 2400
FSR 3.1 + FG 2x, pacing off	55.6, 57.4	109, 115
FSR 3.1 + renderer's own 2x, pacing off	58.4, 58.2	118, 117

The game holds its 60 fps cap with FSR, so the cost is clearer in RX_VK_PASS_TIMING=1: FSR 3.1's seven passes take 0.60 ms a frame, against the game's network's 0.078 ms. Frame interpolation's 17 passes take 1.17 ms a frame (0.61 ms of that is the optical-flow search).
Late frames (more than 2 ms after their time): none in any FSR arm, and 3 in one run of the renderer's own interpolation.
With pacing off, the game ran 1 to 3 fps slower with FSR 3.1's interpolation than with the renderer's own (55.6 and 57.4 against 58.4 and 58.2; the lowest line was 45.3). With pacing on, the two were within run-to-run spread.
With pacing off, both interpolators show about half the frames less than 7 ms after the frame before. That is the display thread showing them as they come; it is not FSR's doing.
Log lines in every FSR arm:
- vk: fsr3 1536x864 -> 2048x1152 with assembled depth and camera motion (network fd029a317a9a4b94 ...)
- vk: fsr3 inputs: ... near 0.1000 far 1000.53 fov 45.00 deg, depth 0 at the near plane: reversed for FSR
- every 600 frames, vk: fsr3: N frames drawn ... this one's jitter +0.312 -0.352 px of 14 phases (the jitter changes every frame)
- in the 2x arms, vk: fsr3fi: 600 of the last 600 made frames by FSR 3.1
- no refusal lines once the scene was reached

Visual check:

RX_VK_SHOTS + RX_VK_SHOT_MADE=1 (a shot every 150 presents, 6000 presents): FSR's frames and FSR's made frames look right, with correct colour and no garbage, and the renderer's own flash check wrote no white_* or flash_* file.
RX_VK_DUMP=5400: the HUD pass after the upscale (step 266) draws over FSR's picture at 2048x1152 and looks right.

Not covered

Windows: not built or run here.
AMD and Intel GPUs: not run. The checks above decide where FSR 3.1 is offered; whether they pass on those GPUs is untested.
DLSS with FSR 3.1 frame interpolation (DLSS upscaling, FSR making the frame) is wired, but the NGX build was only linked, not run.
Actor motion for FSR 3.1: waits for #1008.
FSR 3.1's interpolation at Auto or at 3x and above: the renderer's own does those (midpoint only).
The made frame is interpolated from the presented frame, HUD included. The game has no HUD-less colour to give FSR, so the HUD can smear when the scene moves fast.
Long play: the runs are 6000 presents in one place.

🤖 Generated with Claude Code

## What AMD FSR 3.1 upscaling and FSR 3.1 frame generation, split out of #1335 as asked. Nothing else from #1335 is in it. - **Upscaling:** a fourth value on the existing Upscaler row, **AMD FSR 3.1**, after DLAA (`dlss=3` in `hookshot_options.ini`; 1 and 2 keep meaning DLSS and DLAA). It takes the gameplay upscale network's dispatch: the same dispatch DLSS takes, found the same way. - **Frame generation:** with Frame generation at **2x**, FSR 3.1's frame interpolation makes the frame in between. It uses the scene's depth and motion. Where it cannot run, the renderer's own interpolation runs as before, and it always handles Auto and 3x and above (see below). - **Built from AMD's open shader sources.** FidelityFX SDK v1.1.4 (commit c6efa6bf, MIT) is vendored unmodified in `third_party/fidelityfx-sdk` and compiled into `gpu/vk/shaders/fsr3_*_spv.h`. Every pass is a dispatch of ours. **FSR 3.1 needs no AMD runtime and no DLL**: nothing of AMD's is linked or loaded at run time, so it is in every build on every platform. AMD's runtime DLLs are only for FSR 4. FSR 4 is not in this PR (FSR 4.1 is #1008's). - **No NVIDIA changes:** no DLSS, DLSS-G, DLSS 5, NGX or Streamline code. The only change to `vk_dlss.cpp` is in `dlss_mode()`. It used to clamp the row's value into 0..2, so 3 would have turned into DLAA. Now any value other than 1 or 2 means "not DLSS". - **No release or packaging changes and no licence acceptance in CI.** The only workflow edit adds `glslang-tools` to `source-regressions.yml`'s apt line, because `tests/test_fsr3_shaders.py` compiles every vendored pass again and checks the checked-in headers. Supersedes #1313: its commit is this PR's first commit, unchanged apart from one context line in `tests/run_all.py` after the rebase. Commits: 1. `build(vk)`: the vendored passes, `scripts/fsr3_shaders.py`, the 28 headers and their test (#1313) 2. `feat(vk)`: FSR 3.1 upscaling (`vk_fsr3.*`, `vk_fsr3_inputs.*`, `fsr3_math.h`, `shaders/fsr3_prep.comp`), the jitter in every lift 3. `feat(vk)`: FSR 3.1 frame interpolation (`vk_fsr3fi.*`, the chain in `vk_framegen.cpp`) 4. `feat(menu)`: the Upscaler row's FSR 3.1 value, the Frame generation hint, and `readme.md` ## Why Players have asked for FSR 3.1 frame generation, and #1335 was too big to review and test in one piece. ## Inputs: what comes from #841 and what does not yet All of FSR's inputs are made in one file, `gpu/vk/vk_fsr3_inputs.cpp`, behind one function, `prepare()`, which returns one `Frame` struct. Moving FSR 3.1 onto #1008's shared temporal inputs means changing that file alone. - **Jitter: #841's, the one jitter in the game.** `nx2/oot_jitter.py` with `hos_dlss_jitter_clip`. The fork's own jitter (a viewport offset in `vk_draw.cpp`) is not here. - `nx2/lift.py` now installs the jitter in every standalone lift, not only in builds with NVIDIA's SDK. `make_game.py` no longer sets `OOT_DLSS_JITTER`, and `oot_jitter.wanted()` is gone. The lift key covers `nx2/`, so the first build after this PR lifts the game again. `tests/test_make_game_ngx.py` no longer checks the old switch. - The jitter stays zero unless DLSS or FSR 3.1 draws. `rx_option(HOS_OPT_DLSS_JITTER_W/_H/_PHASES)` returns `dlss_jitter_size()`, or FSR's `fsr3_inputs::jitter_size()` (the scene's size while FSR has drawn in the last 30 frames). The phase count is `dlss_jitter_phases()`, the host's own sequence length, and FSR is given the same count. `RX_VK_FSR_JITTER=0` turns it off for a run. - FSR reads the offset back from the camera's uniform buffer (P[0][2], P[1][2]), as `vk_dlss.cpp` does. It uses `dlss_math.h`'s `jitter_to_pixels` and `unjitter` to convert the offset and take it off the motion. - #841's names (`HOS_OPT_DLSS_JITTER_*`, `hos_dlss_jitter_clip`) are kept, so its interfaces do not churn under #1008. - **Camera (near, far, field of view): from the projection, with no hook in the game.** `nx2/oot_camera.py` and `host/hos_oot_camera.c` are not here. - The camera is taken from the same uniform buffer and with the same checks as `dlss_note_scene_draw`. - `fsr3_math.h` derives FSR's `device_to_view_depth` factors and `tan(fov/2)` from the projection at +0x70. This was checked in a run: `p22 -1.0002, p23 -0.20002` gives near 0.1, far 1000.5 and a 45.00 degree field of view. - The scene's depth is 0 at the near plane (not reversed). The passes are compiled for reversed depth, so FSR is given 1 - d. - The fork's camera hook published near 1.0 and far 20000 (sead's projection object, not the one the scene is drawn with) and gave FSR reversed-infinite factors. Neither matches the depth FSR was given. - `tests/test_fsr3_math.cpp` checks the factors against FSR's own view-distance formula, for both depth directions and for finite and infinite far planes. - **Motion: the camera's only, for now.** - `shaders/fsr3_prep.comp` is the fork's camera-motion pass, reworked to `dlss_prep.comp`'s math: reproject the depth from this frame's jittered clip space to the previous frame's unjittered one, and take this frame's jitter off. The same pass also writes FSR's colour and depth copies. - #841's actor motion (the skinned draws drawn again with last frame's bones) lives inside `vk_dlss.cpp` behind `HOOKSHOT_NGX`, so a build without NVIDIA's SDK has none of it. Moving it out from behind the gate is #1008's refactor (`vk_temporal.cpp`), and this PR does not duplicate it. - Until #1008 lands, things that move by themselves can trail with FSR 3.1, as the row's help says. When it lands, `vk_fsr3_inputs.cpp`'s `prepare()` takes #1008's inputs and FSR gets the actor motion. - **Colour:** the network's input as it samples it (its square-root encoding), clamped to 0..1, with NaN set to 0. FSR's output is squared back into the network's output by `dlss_out.comp`, the same pass DLSS's picture goes through. Left out of the fork's version: - the viewport jitter - the camera hook - `upscale_seam.h`, `upscale_contract.h`, `upscale_format.h` and their tests (the stand-in takes the dispatch the way `dlss_stand_in` does) - FSR's own RCAS (the Sharpness row already sharpens every upscaler's picture) - the diagnostics (`RX_VK_FSR_STAGE`, the VP hunt, the MRT census) Fixed in the port: - The fork's FSR output was blitted into the network's output without being squared back. - The fork's FSR files did not include `vk_work.h`, so `full_barrier` could leave out a barrier after their work. - The fork's camera-motion image never had its first layout transition recorded. - When the size changed, the fork destroyed images that frames still in flight could be using. They are now freed a few presents later. ## Frame generation The chain in `vk_framegen.cpp` is: FSR 3.1, then the renderer's own blend for any present FSR does not take (no inputs for that present, a reset, a GPU that cannot run the passes). - FSR 3.1's interpolation has no phase input: it only makes the midpoint. So it is used at 2x only. Auto and 3x to 8x use the renderer's own. - A present FSR did not see breaks the sequence it keeps its previous frame by, so the next present is a reset. It never interpolates against a stale frame, and each frame's depth and motion are used by one present only. - The Frame generation row's 2x hint says when FSR 3.1 makes the frame. `RX_VK_FSR_FG=0` turns FSR's interpolation off for a run (`=1` forces it on over the row). ## GPUs and drivers FSR 3.1 is offered where the device has what the passes need. The Upscaler row is shown on such a GPU even in a build without NVIDIA's SDK. These are on top of the renderer's own floor (Vulkan 1.3: dynamic rendering, synchronization2, timeline semaphores). - **Upscaler** (`fsr3::supported()`): - `VK_KHR_push_descriptor` with `maxPushDescriptors` of at least 15 - `shaderStorageImageExtendedFormats` and `shaderStorageImageWriteWithoutFormat` - storage and sampled images (optimal tiling) in R8_UNORM, R16_SFLOAT, R8G8B8A8_UNORM, R16G16B16A16_SFLOAT, R16G16_SFLOAT, R32_SFLOAT, R32_UINT, R32G32B32A32_SFLOAT and R16_SNORM - subgroup basic and quad operations in compute are used where present; without them the two pyramids run as their wave-less variants - **Frame interpolation** (`fsr3fi::capable()`): - push descriptors for the widest pass (19) - `shaderStorageImageReadWithoutFormat`, `shaderStorageImageWriteWithoutFormat` and `shaderStorageImageExtendedFormats` - subgroup basic, arithmetic and quad operations in compute - storage images in R8_UINT, R8G8_UNORM, R16G16_SINT, R16G16_SFLOAT, R32_UINT, R32_SFLOAT, R16G16B16A16_SFLOAT, and the presented R8G8B8A8_UNORM The log says which way it went: `vk: fsr3: AMD FSR 3.1 can run on this GPU`, or `vk: fsr3: not on this GPU: <why>`, and the same for `fsr3fi`. Tested only on an NVIDIA RTX 5060 Ti with driver 615.71.09 on Linux. ## Testing Linux, RTX 5060 Ti, driver 615.71.09. `D=<deps>` is `build-linux/deps`. **Builds (both link):** - `EMSDK=$D/emsdk RX_CPM=$D/cpm OOT_WORK=... RX_MARCH=x86-64-v3 RX_VK_ONLY=1 python -I nx2/build.py --standalone --jobs 8`: compiled, 0 failed, linked `hookshot_vk.so` (no NVIDIA SDK) - the same with `HOOKSHOT_NGX_SDK=<ngx-sdk>`: compiled, 0 failed, linked ("with NVIDIA DLSS") - each of commits 2 and 3 on its own: the renderer builds and the tests touching it pass **Tests:** - **After rebasing onto current main (0828863):** the renderer builds and links without NVIDIA's SDK, and `tests/run_all.py` (with the CPM headers found, so the SPIR-V and Vulkan header tests run too) gives **202 passed, 0 failed, 5 skipped**. - `tests/run_all.py` on main (a116390): 180 passed, 0 failed, 19 skipped - `tests/run_all.py` on this branch: 182 passed, 0 failed, the same 19 skipped. The two new ones are `test_fsr3_shaders.py` and `test_fsr3_math.py`. - On the earlier base 75a676e: 166 / 168 passed, 0 failed, 17 skipped. **Whole game:** `make_game.py --update <a copy of the game folder> --jobs 10` built the whole game from this branch in 10 minutes. The lift logged `dlss jitter: 3 of 3 sites` in a build without NVIDIA's SDK. A run of that build (FSR 3.1 + 2x, pacing on) gave a game fps of 58.4 and 117.2 frames shown a second, with FSR 3.1 making 600 of 600 frames and no refusal lines. **In game:** - Setup: the Kokiri Forest scene with Saria, about 1800 draws a frame, 1536x864 to 2048x1152. `HOS_INPUT_FRAME_CLOCK=1 HOS_REALTIME_STEP=0 RX_VK_PRECOMPILE=0`, the same `HOS_INPUT_SCRIPT` in every arm, and a stop at 6000 presents. The game's fps is the mean of the `vk: N presents, X fps` lines between presents 4800 and 6000. - The runs were windowed on Xvfb, not `HOS_HEADLESS`: frame generation is off without a window (`framegen_on()` checks `!dev.offscreen`). - No build ran during an arm. Arms run while another process was using the GPU were discarded and run again. - The arms in the table ran on this branch before its rebase from 75a676e onto a116390. The FSR code is the same; the rebase only resolved conflicts with frame generation Auto, the menu and `run_all.py`. The whole-game run above is on the rebased branch. | Arm | Game fps (2 runs) | Shown fps | Frames shown 7-9 ms apart | |---|---|---|---| | Game's own upscaler | 59.7, 59.3 | | | | FSR 3.1 | 59.4, 58.5 | | | | FSR 3.1 + FG 2x, pacing on (FSR makes 600 of 600) | 58.4, 59.7 | 116.6, 119.7 | 1994 of 2332, 2287 of 2394 | | FSR 3.1 + renderer's own 2x, pacing on | 59.3, 59.9 | 115.8, 120.0 | 1902 of 2318, 2333 of 2400 | | FSR 3.1 + FG 2x, pacing off | 55.6, 57.4 | 109, 115 | | | FSR 3.1 + renderer's own 2x, pacing off | 58.4, 58.2 | 118, 117 | | - The game holds its 60 fps cap with FSR, so the cost is clearer in `RX_VK_PASS_TIMING=1`: FSR 3.1's seven passes take 0.60 ms a frame, against the game's network's 0.078 ms. Frame interpolation's 17 passes take 1.17 ms a frame (0.61 ms of that is the optical-flow search). - Late frames (more than 2 ms after their time): none in any FSR arm, and 3 in one run of the renderer's own interpolation. - With pacing off, the game ran 1 to 3 fps slower with FSR 3.1's interpolation than with the renderer's own (55.6 and 57.4 against 58.4 and 58.2; the lowest line was 45.3). With pacing on, the two were within run-to-run spread. - With pacing off, both interpolators show about half the frames less than 7 ms after the frame before. That is the display thread showing them as they come; it is not FSR's doing. - Log lines in every FSR arm: - `vk: fsr3 1536x864 -> 2048x1152 with assembled depth and camera motion (network fd029a317a9a4b94 ...)` - `vk: fsr3 inputs: ... near 0.1000 far 1000.53 fov 45.00 deg, depth 0 at the near plane: reversed for FSR` - every 600 frames, `vk: fsr3: N frames drawn ... this one's jitter +0.312 -0.352 px of 14 phases` (the jitter changes every frame) - in the 2x arms, `vk: fsr3fi: 600 of the last 600 made frames by FSR 3.1` - no refusal lines once the scene was reached **Visual check:** - `RX_VK_SHOTS` + `RX_VK_SHOT_MADE=1` (a shot every 150 presents, 6000 presents): FSR's frames and FSR's made frames look right, with correct colour and no garbage, and the renderer's own flash check wrote no `white_*` or `flash_*` file. - `RX_VK_DUMP=5400`: the HUD pass after the upscale (step 266) draws over FSR's picture at 2048x1152 and looks right. ## Not covered - Windows: not built or run here. - AMD and Intel GPUs: not run. The checks above decide where FSR 3.1 is offered; whether they pass on those GPUs is untested. - DLSS with FSR 3.1 frame interpolation (DLSS upscaling, FSR making the frame) is wired, but the NGX build was only linked, not run. - Actor motion for FSR 3.1: waits for #1008. - FSR 3.1's interpolation at Auto or at 3x and above: the renderer's own does those (midpoint only). - The made frame is interpolated from the presented frame, HUD included. The game has no HUD-less colour to give FSR, so the HUD can smear when the scene moves fast. - Long play: the runs are 6000 presents in one place. 🤖 Generated with [Claude Code](https://claude.com/claude-code)

NotRexed commented

2026-11-10 03:05:39 +00:00

(Migrated from github.com)

This landed in the v2.0.12 branch (merged there for testing; it reaches main with that release), so closing. Thanks!

This landed in the v2.0.12 branch (merged there for testing; it reaches `main` with that release), so closing. Thanks!

NotRexed commented

2026-11-10 03:54:39 +00:00

(Migrated from github.com)

Reopened: closing this was a mistake on my side. It was tested in the v2.0.12 branch, and it will be merged into main here, so it shows as merged. Sorry for the noise.

hckrman101 (Migrated from github.com) reviewed

2026-11-10 04:19:09 +00:00

hckrman101 (Migrated from github.com) left a comment

Thanks for splitting this out, it's much easier to follow than #1335. The FSR side reads well: everything comes from AMD's vendored GLSL with no runtime, the binding tables and constant blocks have size asserts, old images are freed a few frames later instead of right away, the passes are wrapped in full_barrier/descriptors_disturbed, and fsr3_math.h checks out. I worked through the near/far/fov derivation for the p22/p23 in the description and got 0.1 / 1000.5 / 45 deg, same as you. I didn't find a sync or lifetime bug in vk_fsr3.cpp, vk_fsr3_inputs.cpp or vk_fsr3fi.cpp. What's left is about landing it:

CI. The only failure is Python source checks (ubuntu-24.04) -> tests/test_dlss_actor.py: shaders/dlss_actor_frag_spv.h is not made from dlss_actor.frag as it is now. Nothing in the PR touches that shader. The new glslang-tools apt line is what triggers it: that test compares the DLSS headers against a fresh compile only when glslangValidator is on PATH. The runner now gets Ubuntu's glslang 15.1.0, but main's headers were built with 16.6.0 (// 1116.6.0), and 15.1 produces different words for dlss_actor.frag. A local glslang 16.6.0 rebuilds main's header byte for byte, so main is fine. Your own test_fsr3_shaders.py passes on 15.1. So the failure comes from this PR's workflow change, not from its code. Possible fixes: pin a glslang that matches the checked-in headers (a 16.x release tarball, say), make test_dlss_actor.py skip or ignore the generator on a version mismatch the way your strip_generator does (that helps only if the words match), or run the FSR header check in its own step with its own glslang. The other checks were still pending when I looked; the DLSS Windows link check and the AppImage passed.

Conflict with main. One hunk, in vk_replay.cpp's rx_option chain: main added HOS_OPT_GYRO_DEADZONE on the same line. Keep main's ... id == HOS_OPT_GYRO_DEADZONE ? opts.gyro_deadzone : and then your jitter_option(id) line. Nothing else conflicts. The PR adds no new option ids; FSR 3.1 is value 3 under the existing dlss key, so the id list (0-9, 16-22, 24-26, 32-35, 4096+, and 10/11/23/27/28 in other open PRs) isn't affected.

How this fits with #1008. These two can't merge as they stand. They're close in design, though, and that's mostly to your credit: you kept FSR's inputs behind fsr3_inputs::prepare() so they could move over later.

Saved value: #1008 already uses dlss=3 for FSR (4.1) and dlss=4 for FSR native AA (#1469 uses the same 3/4). This PR uses dlss=3 for FSR 3.1. Whichever lands second has to renumber, or a saved dlss=3 silently changes meaning. If #1008 goes first (it's approved), FSR 3.1 would be value 5.
Env var: #1008's RX_VK_FSR is FSR 4 (=0 off, =1/=2 over the row). Here RX_VK_FSR=1 forces FSR 3.1. One of them needs another name, e.g. RX_VK_FSR3.
Inputs: #1008's TemporalFrame is almost exactly your fsr3_inputs::Frame: colour as the network samples it in RGBA16F, R32F depth not reversed, RG16F motion in render pixels with the jitter removed, the jitter in render pixels, reset, and near/far/fov. On top of #1008, FSR 3.1 becomes a third TemporalEngine. evaluate is fsr3::upscale plus your 1 - d depth flip (a tiny pass, or a flag in fsr3_prep.comp). start and prepare wrap supported() and the image setup, and release drops the images. vk_fsr3_inputs.cpp and fsr3_stand_in then go away, and FSR 3.1 picks up #1008's actor motion for free. The dlss_mode() change and jitter_option also go away, because #1008 has temporal_mode() and temporal_jitter_size().
Frame interpolation is the one part #1008 doesn't cover. It needs the chain's last frame (depth, motion, camera, jitter) at present time, which today is fsr3_inputs::take(). That would be a small temporal_last_frame() hook in vk_temporal.cpp, and it would also have to run when the Upscaler is the game's own, since FI-only mode needs the inputs then too.
Jitter in the lift: #1008 keeps oot_jitter.wanted() and sets OOT_DLSS_JITTER for NGX or --fsr builds. This PR installs the jitter in every lift. That's fine and probably what #1008 wants once an SDK-free engine exists, but it's a conflict to resolve in lift.py, make_game.py and oot_jitter.py.

So the cheapest order is probably #1008 first, then this rebased as an engine on it. That also settles #1469, which adds FSR 3.1 a second way: one of the two should go. I'd ask the maintainer which order they want before you do the rebase.

One behaviour point (inline): Frame generation at 2x switches every capable GPU over to FSR's interpolation with no menu option to turn it off.

Nits: jitter_option in vk_replay.cpp is a new non-static function at namespace scope, so make it static. make_image(g.spd_mips, ..., std::max(spd_levels, kSpdStoredMips)) would ask for more mips than the image allows if half the render size were under 32 px. That can't happen behind note_scene_draw's 640x360 floor, but std::min with the real chain length (as vk_fsr3fi.cpp does) is safer.

Thanks for splitting this out, it's much easier to follow than #1335. The FSR side reads well: everything comes from AMD's vendored GLSL with no runtime, the binding tables and constant blocks have size asserts, old images are freed a few frames later instead of right away, the passes are wrapped in `full_barrier`/`descriptors_disturbed`, and `fsr3_math.h` checks out. I worked through the near/far/fov derivation for the p22/p23 in the description and got 0.1 / 1000.5 / 45 deg, same as you. I didn't find a sync or lifetime bug in `vk_fsr3.cpp`, `vk_fsr3_inputs.cpp` or `vk_fsr3fi.cpp`. What's left is about landing it: **CI.** The only failure is `Python source checks (ubuntu-24.04)` -> `tests/test_dlss_actor.py`: `shaders/dlss_actor_frag_spv.h is not made from dlss_actor.frag as it is now`. Nothing in the PR touches that shader. The new `glslang-tools` apt line is what triggers it: that test compares the DLSS headers against a fresh compile only when `glslangValidator` is on PATH. The runner now gets Ubuntu's glslang 15.1.0, but main's headers were built with 16.6.0 (`// 1116.6.0`), and 15.1 produces different words for `dlss_actor.frag`. A local glslang 16.6.0 rebuilds main's header byte for byte, so main is fine. Your own `test_fsr3_shaders.py` passes on 15.1. So the failure comes from this PR's workflow change, not from its code. Possible fixes: pin a glslang that matches the checked-in headers (a 16.x release tarball, say), make `test_dlss_actor.py` skip or ignore the generator on a version mismatch the way your `strip_generator` does (that helps only if the words match), or run the FSR header check in its own step with its own glslang. The other checks were still pending when I looked; the DLSS Windows link check and the AppImage passed. **Conflict with main.** One hunk, in `vk_replay.cpp`'s `rx_option` chain: main added `HOS_OPT_GYRO_DEADZONE` on the same line. Keep main's `... id == HOS_OPT_GYRO_DEADZONE ? opts.gyro_deadzone :` and then your `jitter_option(id)` line. Nothing else conflicts. The PR adds no new option ids; FSR 3.1 is value 3 under the existing `dlss` key, so the id list (0-9, 16-22, 24-26, 32-35, 4096+, and 10/11/23/27/28 in other open PRs) isn't affected. **How this fits with #1008.** These two can't merge as they stand. They're close in design, though, and that's mostly to your credit: you kept FSR's inputs behind `fsr3_inputs::prepare()` so they could move over later. - *Saved value:* #1008 already uses `dlss=3` for FSR (4.1) and `dlss=4` for FSR native AA (#1469 uses the same 3/4). This PR uses `dlss=3` for FSR 3.1. Whichever lands second has to renumber, or a saved `dlss=3` silently changes meaning. If #1008 goes first (it's approved), FSR 3.1 would be value 5. - *Env var:* #1008's `RX_VK_FSR` is FSR 4 (`=0` off, `=1`/`=2` over the row). Here `RX_VK_FSR=1` forces FSR 3.1. One of them needs another name, e.g. `RX_VK_FSR3`. - *Inputs:* #1008's `TemporalFrame` is almost exactly your `fsr3_inputs::Frame`: colour as the network samples it in RGBA16F, R32F depth not reversed, RG16F motion in render pixels with the jitter removed, the jitter in render pixels, reset, and near/far/fov. On top of #1008, FSR 3.1 becomes a third `TemporalEngine`. `evaluate` is `fsr3::upscale` plus your 1 - d depth flip (a tiny pass, or a flag in `fsr3_prep.comp`). `start` and `prepare` wrap `supported()` and the image setup, and `release` drops the images. `vk_fsr3_inputs.cpp` and `fsr3_stand_in` then go away, and FSR 3.1 picks up #1008's actor motion for free. The `dlss_mode()` change and `jitter_option` also go away, because #1008 has `temporal_mode()` and `temporal_jitter_size()`. - *Frame interpolation* is the one part #1008 doesn't cover. It needs the chain's last frame (depth, motion, camera, jitter) at present time, which today is `fsr3_inputs::take()`. That would be a small `temporal_last_frame()` hook in `vk_temporal.cpp`, and it would also have to run when the Upscaler is the game's own, since FI-only mode needs the inputs then too. - *Jitter in the lift:* #1008 keeps `oot_jitter.wanted()` and sets `OOT_DLSS_JITTER` for NGX or `--fsr` builds. This PR installs the jitter in every lift. That's fine and probably what #1008 wants once an SDK-free engine exists, but it's a conflict to resolve in `lift.py`, `make_game.py` and `oot_jitter.py`. So the cheapest order is probably #1008 first, then this rebased as an engine on it. That also settles #1469, which adds FSR 3.1 a second way: one of the two should go. I'd ask the maintainer which order they want before you do the rebase. **One behaviour point** (inline): Frame generation at 2x switches every capable GPU over to FSR's interpolation with no menu option to turn it off. Nits: `jitter_option` in `vk_replay.cpp` is a new non-static function at namespace scope, so make it `static`. `make_image(g.spd_mips, ..., std::max(spd_levels, kSpdStoredMips))` would ask for more mips than the image allows if half the render size were under 32 px. That can't happen behind `note_scene_draw`'s 640x360 floor, but `std::min` with the real chain length (as `vk_fsr3fi.cpp` does) is safer.

.github/workflows/source-regressions.yml

					
				@ -58,3 +58,3 @@

				        if: runner.os == 'Linux'

				        run: sudo apt-get update && sudo apt-get install -y g++ clang spirv-tools

				        run: sudo apt-get update && sudo apt-get install -y g++ clang spirv-tools glslang-tools

This is what fails CI. With glslang on PATH, main's tests/test_dlss_actor.py also compiles the DLSS shaders and compares the words. Ubuntu 24.04's glslang-tools is 15.1.0, but main's headers come from 16.6.0, and dlss_actor.frag compiles differently between them (16.6.0 reproduces main's header exactly). Either pin a glslang that matches the checked-in headers, or keep this check from turning on a version-sensitive one.

This is what fails CI. With glslang on PATH, main's `tests/test_dlss_actor.py` also compiles the DLSS shaders and compares the words. Ubuntu 24.04's `glslang-tools` is 15.1.0, but main's headers come from 16.6.0, and `dlss_actor.frag` compiles differently between them (16.6.0 reproduces main's header exactly). Either pin a glslang that matches the checked-in headers, or keep this check from turning on a version-sensitive one.

gpu/vk/upscaler_rows.h

					
				@ -9,0 +10,4 @@

				enum { GAME = 0, DLSS = 1, DLAA = 2, FSR3 = 3, kUpscalers = 4 };

				/// One bit per upscaler that can run here (1 << GAME always): what the row steps through

				inline unsigned bit(int upscaler) { return 1u << upscaler; }

				/// The DLSS and DLAA bits, for a caller that knows only whether DLSS runs

#1008 (approved) and #1469 both already give dlss=3 to FSR 4 and dlss=4 to its native AA. Whichever lands second has to renumber, or a saved dlss=3 changes meaning between builds. If #1008 goes first, this would be FSR3 = 5.

#1008 (approved) and #1469 both already give `dlss=3` to FSR 4 and `dlss=4` to its native AA. Whichever lands second has to renumber, or a saved `dlss=3` changes meaning between builds. If #1008 goes first, this would be `FSR3 = 5`.

gpu/vk/vk_framegen.cpp

This takes over 2x for everyone on a capable GPU, including players who never picked FSR as the Upscaler. Your own numbers show it costs about 1.2 ms of GPU time plus the prep pass, and 1-3 fps with pacing off. The only way out is RX_VK_FSR_FG=0. I'd make it opt-in: either a value on the Frame generation row, or only when the Upscaler is FSR 3.1. Then a player who's happy with the current 2x doesn't get a different interpolator after an update.

This takes over 2x for everyone on a capable GPU, including players who never picked FSR as the Upscaler. Your own numbers show it costs about 1.2 ms of GPU time plus the prep pass, and 1-3 fps with pacing off. The only way out is `RX_VK_FSR_FG=0`. I'd make it opt-in: either a value on the Frame generation row, or only when the Upscaler is FSR 3.1. Then a player who's happy with the current 2x doesn't get a different interpolator after an update.

gpu/vk/vk_fsr3.h

					
				@ -0,0 +8,4 @@

				// Those inputs are made in vk_fsr3_inputs.cpp; fsr3_upscale below is the dispatch's side: the network's input in, FSR's

				// picture squared back into the network's output, as vk_dlss.cpp's dlss_out.comp does for DLSS.

				//

				// RX_VK_FSR=0: never (the row stays), =1: FSR 3.1 over the Insert menu's Upscaler row.