fix(vk): FSR 4's and XeSS's native AA wait for the native scene when entered from their scaled mode, as DLAA does #1744

Merged
HumanBean merged 2 commits from madebyjordan/Longshot:fix/native-aa-size into main 2026-11-19 15:05:33 +00:00
Member

Type: fix · Risk: low · Tested on: Windows 11, RX 6700 XT (Adrenalin 26.10.1) · Needs: an 8 GB AMD card at its VRAM limit; XeSS (any GPU with libxess)

TL;DR

Switching AMD FSR 4 (or Intel XeSS) from a scaled mode to its Native AA could turn it off for the rest of the session: for the first frames after the switch the game still draws the smaller scene, and the upscaler was rebuilt at that size. On an 8 GB card at its VRAM limit that rebuild failed and the game's own upscaler took over until the player changed the upscaler. Now those first frames are drawn by the scaled mode, as DLSS already does when switching to DLAA, and Native starts once the full-size scene arrives. A new test fails on main and passes here.

Breakdown

  • Who it helps: AMD FSR 4 and Intel XeSS players who switch to Native AA, most of all on 6-8 GB cards near their VRAM limit.
  • What was wrong: only DLAA waited for the full-size scene after a switch to Native; FSR 4's and XeSS's Native AA were made at the size of the last scaled frame.
  • What changes: after a switch from FSR 4 (XeSS) to its Native AA, up to three queued scaled frames are drawn by FSR 4 (XeSS), then Native AA runs at the full size.
  • What does not change: DLAA's behaviour, switching to Native from another engine, Native AA chosen at the start, every scaled mode.
  • How it was checked: the native-transition test, extended to FSR 4 and XeSS, fails on main and passes here (both directions checked by mutation); FSR 4 scaled and Native AA run as before in game (48 fps / 44 fps, same as main).
  • What is still open: the switch itself in game (the control bridge can't select Native) and XeSS on real hardware; FSR 4 still turns off for good after any failed creation (#1612 has the same for DLSS).
  • Risk: low: the same three-frame wait DLAA has had since Hookshot's https://github.com/NotRexed/Hookshot/pull/2145, now for the two other Native modes.

Why

A player on an RX 6600 (8 GB, VRAM at its limit) switched FSR 4 from 92% to Native. The log has AMD FSR 4: its upscaler was not made (3) for 1429x804 (1429x804 is the 92% scene of 1536x864), then AMD FSR 4 Native AA: the game's upscale runs (it failed earlier) for the rest of the session: FSR 4 was off, and the player saw the game's own upscale and lower frame rates when changing the scale. DLAA has waited for the native-size scene since https://github.com/NotRexed/Hookshot/pull/2145 ("Creating DLAA for that queued scene would replace the feature twice"); FSR 4's and XeSS's Native AA never did. With this change FSR 4 makes no new context for the switch at all: the scaled context, made for the output size, is reused for Native.

See #744, #1612.

How to reproduce

Seen on: Windows 11, RX 6600 8 GB, Adrenalin 26.11.1 (the #744 log)
Settings and switches: Upscaler AMD FSR 4, Render resolution 92%; a scene near the VRAM limit
Steps: in the Insert menu, Render resolution: Left to Native (FSR 4)
Expected: FSR 4 Native AA draws (AMD FSR 4 Native AA draws OoT's upscale: 1536x864 -> 1536x864)
Actual: on main AMD FSR 4: FSR 4.1.2 made for outputs of 1429x804 or, at the VRAM limit, its upscaler was not made (3) for 1429x804, then the game's upscale for the rest of the session. On a 12 GB card the creation succeeds, so the visible sign there is the extra creation (~100 ms of CPU, a hitch) at the queued size, followed by a second one at the full size.

What changed

The three-frame wait for the native-size scene, until now DLAA's alone, applies to every Native mode entered from its own engine's scaled mode.

  • vk: draw_engine in gpu/vk/vk_temporal.cpp: entering_native (was entering_dlaa) is any ur::full_scale mode entered from ur::engine(mode); the queued frames are drawn as ur::engine(mode) (was DLSS); the expiry log line names the mode.
  • tests: tests/test_dlss_native_transition.py runs the real dispatch block with FSR 4 → FSR 4 Native AA and XeSS → XeSS Native AA: a queued scaled frame makes no native context; the full-size scene does; a scene that stays small gets Native AA at its size after three frames.

Measurements

Verdict: the test fails on main for the bug's reason and passes here; in game FSR 4 and its Native AA run as on main.

Hardware: Windows 11, RX 6700 XT 12 GB, Adrenalin 26.10.1, Ryzen 5 5600, 16 GB
Scene: Hyrule Field by day, standing still; 2560x1440, uncapped, no frame generation
Method: the game-free test with clang 23 (llvm-mingw); in game, the change applied to the v3.0.0 renderer (renderer built alone, RX_VK_ONLY) in a copy of a v3.0.0 game folder, headless through the control bridge, 20-45 s per setting, median of the bridge's fps
How to reproduce: python -I -B tests/test_dlss_native_transition.py --cxx clang++

Before: main @ 868f6be8 After: this PR @ d5863b06
test_dlss_native_transition FAIL (13: FSR 4's queued frame made a native context at its size) PASS
FSR 4 Performance 50%, fps (in game, see Baseline) 48.8, 49.0 48.1, 48.2
FSR 4 Native AA from the start (RX_VK_FSR4=2), fps (in game, see Baseline) 43.0, 43.9, 43.3 44.0, 44.1, 44.0

Baseline: the in-game rows compare the v3.0.0 renderer (09cbdd97) with the same renderer plus this change, in a copy of a v3.0.0 game folder: main's renderer doesn't build on Windows today (#1729), and v3.0.0's draw_engine block is the same as main's. They show only that nothing else changes; the switch itself is the test's.

main:    FSR 4's queued scaled frame made a native context at its size   (exit 13)
this PR: PASS: queued scenes, bounded native wait, reuse, HDR and engine switch

Flags and settings

None

Testing

Tested: the new test both ways (main: fails with 13; here: passes; the key condition put back to DLAA-only: fails again), with clang++ and llvm-mingw's g++ (clang underneath: no real GCC run). The tests that read vk_temporal.cpp (18 of them) pass, except test_psychov35_fsr3_fg and test_upscaler_rows, which fail on main on this PC too. The renderer builds whole (Windows, llvm-mingw) with a local workaround for main's own Windows build break in descriptor_array_census.h (#1729 fixes it; not in this PR). In game (v3.0.0 + this change): FSR 4 at 50% and FSR 4 Native AA from the start both draw and run at main's frame rate. python -I -B tests/run_all.py: 679 passed, 45 failed, 127 skipped; the same 40 fail on main @ 868f6be8 on this PC, and the other 5 pass here run alone (4 needed MAGICBEAN_TOOLING set, as this worktree has no Tooling beside it; test_longshot_ctl passes alone).

Not testable by hand: the switch to Native itself, here: the control bridge's option_set can't step the Render resolution row onto Native, and this PC's 12 GB card doesn't fail the creation. The game-free test stands in for it.

Not tested: XeSS (no libxess on this PC); Linux.

Must be tested:

  • Windows or Linux, an AMD card that runs FSR 4: Upscaler AMD FSR 4 at 92%, then Render resolution Left to Native, expecting no AMD FSR 4: FSR 4.1.2 made for outputs of line at the 92% size (main makes one, then another at full size) and AMD FSR 4 Native AA draws OoT's upscale at the full size (by hand: the log)
  • Any GPU with Intel XeSS: XeSS at 77%, then Native, expecting XeSS Native AA to draw at the full size and no XeSS context made at the 77% size (Test Farm: @Beanstalk any GPU with XeSS, switch XeSS to Native and back, check the log)

Risks

What could break: for up to three frames after a switch to FSR 4's or XeSS's Native, the picture is drawn by the scaled mode (as DLAA's switch already does); a scene that stays small gets Native AA at its size after those three frames, as before.

Turning it off: revert the commit.

Compatibility: Nothing changes.

Issues

See #744
See #1612

Checklist

Everyone

  • One topic.
  • The title is a Conventional Commit (type(scope): description) of the status line's type.
  • Why gives the reason plainly and links the issues that fit (the tracker searched).
  • Tested on my own PC where it runs (in the game, for a change the game runs), as Testing's Tested says; what
    can't be run by hand is in Testing's Not testable by hand, with why.
  • Measurements has a verdict, or one line saying why there is nothing to measure.
  • Nothing gets worse that I know of: any cost (speed, latency, memory, a setup that worked before) is measured and
    in Risks.
  • Testing names what was tested, what wasn't, and what must still be tested, where and how.
  • Rebased on current main.
  • python -I -B tests/run_all.py passes (any failure that also fails on main is named).
  • No game files, dumps, build output or generated files.
  • third_party/ untouched (a new folder with a PROVENANCE.md instead).
  • The license is OK: GPL-3.0-or-later, nothing incompatible and nothing decompiled.

When it applies

  • AI assistance recorded with Assisted-by: / Co-Authored-By: trailers.
  • A game-free test added for the fix, failing on main and passing here.
  • A check under tests/ that reads changed source updated in this PR, not weakened.

🤖 Generated with Claude Code

**Type:** fix · **Risk:** low · **Tested on:** Windows 11, RX 6700 XT (Adrenalin 26.10.1) · **Needs:** an 8 GB AMD card at its VRAM limit; XeSS (any GPU with libxess) ## TL;DR Switching AMD FSR 4 (or Intel XeSS) from a scaled mode to its Native AA could turn it off for the rest of the session: for the first frames after the switch the game still draws the smaller scene, and the upscaler was rebuilt at that size. On an 8 GB card at its VRAM limit that rebuild failed and the game's own upscaler took over until the player changed the upscaler. Now those first frames are drawn by the scaled mode, as DLSS already does when switching to DLAA, and Native starts once the full-size scene arrives. A new test fails on main and passes here. ## Breakdown - **Who it helps:** AMD FSR 4 and Intel XeSS players who switch to Native AA, most of all on 6-8 GB cards near their VRAM limit. - **What was wrong:** only DLAA waited for the full-size scene after a switch to Native; FSR 4's and XeSS's Native AA were made at the size of the last scaled frame. - **What changes:** after a switch from FSR 4 (XeSS) to its Native AA, up to three queued scaled frames are drawn by FSR 4 (XeSS), then Native AA runs at the full size. - **What does not change:** DLAA's behaviour, switching to Native from another engine, Native AA chosen at the start, every scaled mode. - **How it was checked:** the native-transition test, extended to FSR 4 and XeSS, fails on main and passes here (both directions checked by mutation); FSR 4 scaled and Native AA run as before in game (48 fps / 44 fps, same as main). - **What is still open:** the switch itself in game (the control bridge can't select Native) and XeSS on real hardware; FSR 4 still turns off for good after any failed creation (#1612 has the same for DLSS). - **Risk:** low: the same three-frame wait DLAA has had since Hookshot's https://github.com/NotRexed/Hookshot/pull/2145, now for the two other Native modes. ## Why A player on an RX 6600 (8 GB, VRAM at its limit) switched FSR 4 from 92% to Native. The log has `AMD FSR 4: its upscaler was not made (3) for 1429x804` (1429x804 is the 92% scene of 1536x864), then `AMD FSR 4 Native AA: the game's upscale runs (it failed earlier)` for the rest of the session: FSR 4 was off, and the player saw the game's own upscale and lower frame rates when changing the scale. DLAA has waited for the native-size scene since https://github.com/NotRexed/Hookshot/pull/2145 ("Creating DLAA for that queued scene would replace the feature twice"); FSR 4's and XeSS's Native AA never did. With this change FSR 4 makes no new context for the switch at all: the scaled context, made for the output size, is reused for Native. See #744, #1612. ## How to reproduce **Seen on:** Windows 11, RX 6600 8 GB, Adrenalin 26.11.1 (the #744 log) **Settings and switches:** Upscaler AMD FSR 4, Render resolution 92%; a scene near the VRAM limit **Steps:** in the Insert menu, Render resolution: Left to Native (FSR 4) **Expected:** FSR 4 Native AA draws (`AMD FSR 4 Native AA draws OoT's upscale: 1536x864 -> 1536x864`) **Actual:** on main `AMD FSR 4: FSR 4.1.2 made for outputs of 1429x804` or, at the VRAM limit, `its upscaler was not made (3) for 1429x804`, then the game's upscale for the rest of the session. On a 12 GB card the creation succeeds, so the visible sign there is the extra creation (~100 ms of CPU, a hitch) at the queued size, followed by a second one at the full size. ## What changed The three-frame wait for the native-size scene, until now DLAA's alone, applies to every Native mode entered from its own engine's scaled mode. - vk: `draw_engine` in `gpu/vk/vk_temporal.cpp`: `entering_native` (was `entering_dlaa`) is any `ur::full_scale` mode entered from `ur::engine(mode)`; the queued frames are drawn as `ur::engine(mode)` (was DLSS); the expiry log line names the mode. - tests: `tests/test_dlss_native_transition.py` runs the real dispatch block with FSR 4 → FSR 4 Native AA and XeSS → XeSS Native AA: a queued scaled frame makes no native context; the full-size scene does; a scene that stays small gets Native AA at its size after three frames. ## Measurements **Verdict:** the test fails on main for the bug's reason and passes here; in game FSR 4 and its Native AA run as on main. Hardware: Windows 11, RX 6700 XT 12 GB, Adrenalin 26.10.1, Ryzen 5 5600, 16 GB Scene: Hyrule Field by day, standing still; 2560x1440, uncapped, no frame generation Method: the game-free test with clang 23 (llvm-mingw); in game, the change applied to the v3.0.0 renderer (renderer built alone, `RX_VK_ONLY`) in a copy of a v3.0.0 game folder, headless through the control bridge, 20-45 s per setting, median of the bridge's fps How to reproduce: `python -I -B tests/test_dlss_native_transition.py --cxx clang++` | | Before: main @ 868f6be8 | After: this PR @ d5863b06 | |---|---|---| | `test_dlss_native_transition` | FAIL (13: FSR 4's queued frame made a native context at its size) | PASS | | FSR 4 Performance 50%, fps (in game, see Baseline) | 48.8, 49.0 | 48.1, 48.2 | | FSR 4 Native AA from the start (`RX_VK_FSR4=2`), fps (in game, see Baseline) | 43.0, 43.9, 43.3 | 44.0, 44.1, 44.0 | Baseline: the in-game rows compare the v3.0.0 renderer (09cbdd97) with the same renderer plus this change, in a copy of a v3.0.0 game folder: main's renderer doesn't build on Windows today (#1729), and v3.0.0's `draw_engine` block is the same as main's. They show only that nothing else changes; the switch itself is the test's. ``` main: FSR 4's queued scaled frame made a native context at its size (exit 13) this PR: PASS: queued scenes, bounded native wait, reuse, HDR and engine switch ``` ### Flags and settings None ## Testing **Tested:** the new test both ways (main: fails with 13; here: passes; the key condition put back to DLAA-only: fails again), with clang++ and llvm-mingw's g++ (clang underneath: no real GCC run). The tests that read `vk_temporal.cpp` (18 of them) pass, except `test_psychov35_fsr3_fg` and `test_upscaler_rows`, which fail on main on this PC too. The renderer builds whole (Windows, llvm-mingw) with a local workaround for main's own Windows build break in `descriptor_array_census.h` (#1729 fixes it; not in this PR). In game (v3.0.0 + this change): FSR 4 at 50% and FSR 4 Native AA from the start both draw and run at main's frame rate. `python -I -B tests/run_all.py`: 679 passed, 45 failed, 127 skipped; the same 40 fail on main @ 868f6be8 on this PC, and the other 5 pass here run alone (4 needed `MAGICBEAN_TOOLING` set, as this worktree has no Tooling beside it; `test_longshot_ctl` passes alone). **Not testable by hand:** the switch to Native itself, here: the control bridge's `option_set` can't step the Render resolution row onto Native, and this PC's 12 GB card doesn't fail the creation. The game-free test stands in for it. **Not tested:** XeSS (no libxess on this PC); Linux. **Must be tested:** - Windows or Linux, an AMD card that runs FSR 4: Upscaler AMD FSR 4 at 92%, then Render resolution Left to Native, expecting no `AMD FSR 4: FSR 4.1.2 made for outputs of` line at the 92% size (main makes one, then another at full size) and `AMD FSR 4 Native AA draws OoT's upscale` at the full size (by hand: the log) - Any GPU with Intel XeSS: XeSS at 77%, then Native, expecting XeSS Native AA to draw at the full size and no XeSS context made at the 77% size (Test Farm: @Beanstalk any GPU with XeSS, switch XeSS to Native and back, check the log) ## Risks **What could break:** for up to three frames after a switch to FSR 4's or XeSS's Native, the picture is drawn by the scaled mode (as DLAA's switch already does); a scene that stays small gets Native AA at its size after those three frames, as before. **Turning it off:** revert the commit. **Compatibility:** Nothing changes. ## Issues See #744 See #1612 ## Checklist ### Everyone - [x] One topic. - [x] The title is a Conventional Commit (`type(scope): description`) of the status line's type. - [x] Why gives the reason plainly and links the issues that fit (the tracker searched). - [x] Tested on my own PC where it runs (in the game, for a change the game runs), as Testing's Tested says; what can't be run by hand is in Testing's Not testable by hand, with why. - [x] Measurements has a verdict, or one line saying why there is nothing to measure. - [x] Nothing gets worse that I know of: any cost (speed, latency, memory, a setup that worked before) is measured and in Risks. - [x] Testing names what was tested, what wasn't, and what must still be tested, where and how. - [x] Rebased on current `main`. - [x] `python -I -B tests/run_all.py` passes (any failure that also fails on `main` is named). - [x] No game files, dumps, build output or generated files. - [x] `third_party/` untouched (a new folder with a `PROVENANCE.md` instead). - [x] The license is OK: GPL-3.0-or-later, nothing incompatible and nothing decompiled. ### When it applies - [x] AI assistance recorded with `Assisted-by:` / `Co-Authored-By:` trailers. - [x] A game-free test added for the fix, failing on `main` and passing here. - [x] A check under `tests/` that reads changed source updated in this PR, not weakened. 🤖 Generated with [Claude Code](https://claude.com/claude-code)
The queued scaled frames after a switch to Native are drawn by the scaled
engine, not made into a native context at their size. Fails on main:
FSR 4's queued frame makes a native context at 1720x720.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Assisted-by: Claude Code
fix(vk): FSR 4's and XeSS's native AA wait for the native scene when entered from their scaled mode, as DLAA does (#744)
Some checks failed
CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Has been skipped
CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Has been skipped
CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Has been skipped
Third-party parts / The same as upstream (pull_request) Has been skipped
CI / Conventional Commit title (pull_request) Successful in 8s
PR description / Description follows the template (pull_request_target) Successful in 10s
Guide screenshots / Guide images and local UI source review (pull_request) Failing after 10s
CI / Workflow lint (actionlint) (pull_request) Successful in 12s
Third-party parts / As their notes say (pull_request) Successful in 12s
Website / Website on Cloudflare (pull_request_target) Has been skipped
d5863b0621
The queued scaled frames right after a switch to Native are drawn by the
scaled engine for up to three frames, not made into a native context at
their size. FSR 4's native context made at a queued 1429x804 failed on an
RX 6600 at the VRAM limit (its upscaler was not made (3)), and FSR 4 then
stayed off until the upscaler was changed.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Assisted-by: Claude Code
Author
Member

Tested in game: Windows 11, RX 6700 XT, Adrenalin 26.10.1: the change on the v3.0.0 renderer (same draw_engine as main) in a v3.0.0 game folder, Hyrule Field, headless: FSR 4 at 50% and FSR 4 Native AA from the start both drew and ran at the same fps as without it; the switch to Native itself is covered by the game-free test (the control bridge can't select Native)

Tested in game: Windows 11, RX 6700 XT, Adrenalin 26.10.1: the change on the v3.0.0 renderer (same draw_engine as main) in a v3.0.0 game folder, Hyrule Field, headless: FSR 4 at 50% and FSR 4 Native AA from the start both drew and ran at the same fps as without it; the switch to Native itself is covered by the game-free test (the control bridge can't select Native)
HumanBean deleted branch fix/native-aa-size 2026-11-19 15:05:34 +00:00
Sign in to join this conversation.
No reviewers
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
MagicBeanDev/Longshot!1744
No description provided.