feat(vk): FXAA in place of the full-size anti-aliasing network where the GPU multiplies in software (RX_VK_TAAU_FULL_SIZE=1, off by default) #687

Open

zeldaootr wants to merge 3 commits from zeldaootr/Longshot:deck-native-aa-stand-in into main

pull from: zeldaootr/Longshot:deck-native-aa-stand-in

merge into: MagicBeanDev:main

MagicBeanDev:main

Conversation 4 Commits 3 Files changed 14 +223 -29

zeldaootr commented 2026-11-15 04:40:42 +00:00

Contributor

• Copy link
• Copy Markdown

Type: feat · Risk: low · Tested on: Steam Deck OLED (SteamOS, RADV), Linux game-free checks · Needs: Steam Deck (Linux, RADV)

Why

On a Steam Deck at its own resolution (1280x800, 100% render), the game's own anti-aliasing costs about a fifth of
every frame. At 100% the game runs a small neural network, 9dc98871eb7fc101, over the picture to smooth its edges.
The Deck's GPU has no hardware for the network's matrix maths, so Longshot does it in software: one profiled Deck frame
spent 8.95 ms of its 44.7 ms there, and the network took about 15 s to compile the first time the game started at 100%.

Today a Deck player has two choices: no anti-aliasing at 50% render, or the most expensive anti-aliasing at native.
Longshot already has a cheap replacement for this kind of network: a linear upscale with FXAA on the edges, the default
on these GPUs for the game's 3/4 network. This PR lets a player use it for the full-size pass too, as an opt-in. It is a
picture trade the player should choose: slightly softer textures and weaker smoothing of long, shallow edges, for about
9 ms a frame.

What changed

RX_VK_TAAU_FULL_SIZE=1 (in longshot_env.txt, or the environment) makes the FXAA stand-in take the full-size network
as well. It only applies where the matrix multiplies run in software: AMD, Intel and older NVIDIA cards. With nothing
set, every GPU takes exactly the networks it took before. Native HDR with the PsychoV35 tone mapper keeps the network,
because the stand-in writes no RGBA16F picture yet.

• vk: gpu/vk/taau_default.h (new, no Vulkan) holds the rule. full_size() says when the switch applies, and keys()
  gives the default key list (main's list unless the switch is on).
• vk: taau_key in gpu/vk/vk_draw.cpp takes its default from that header (the switch read with env_on), and leaves
  the full-size pass to the network while PsychoV35 is on. The first time it answers for the full-size pass it logs what
  happens: vk: the full-size network 9dc98871eb7fc101 is stood in for by FXAA (RX_VK_TAAU_FULL_SIZE), or ... runs while PsychoV35 is on (RX_VK_TAAU_FULL_SIZE), once each. An explicit RX_VK_TAAU_KEYS still goes over everything. A
  network taau_key matches is never compiled (unchanged), so with the switch on the network's 15 s first-start compile
  is gone too. taau_stand_in labels the helper with the network it stands in for, not always the 3/4 network's key.
• vk: comments: taau_key's doc comment names both networks.
• tests: tests/test_taau_default.py (new) checks the rule across a table of GPUs with the switch off and on: off
  equals main's list on every GPU, the Deck included. tests/test_taau_env_lookup.py, which cuts taau_key out of the
  source, has seven new cases: unset, =0, =1, =1 with RX_VK_NO_F16, =1 on a GPU with cooperative matrices,
  =1 with PsychoV35 (and PsychoV35 turned on and off in play), and explicit keys with PsychoV35, each checking the log
  line and the helper's label.
• docs: the switch's row in docs/development.md (The GPU), and a sentence on the Upscaler in the README.

Measurements

Verdict: helps. On a Steam Deck at 1280x800 and 100% render, RX_VK_TAAU_FULL_SIZE=1 gives +15.8% standing in
Hyrule Field (15.09 → 17.47 fps) and +10.4% with the camera turning (one run each), with 8.9 ms less GPU time a frame.
With the switch unset, the PR measures the same as main (15.09 against 15.09 fps). These numbers are from the
commits measured, 3fd27a04 (the branch has been rebased since, with its renderer change the same: see Testing).

Hardware: Steam Deck OLED, AMD Custom GPU 0932 (RADV VANGOGH), Mesa 26.2.0-devel, SteamOS 3.9.5, stock GPU clock (1040 MHz median in every run)
Scene: Hyrule Field standing (a Kokiri save, a teleport to the Field at the Lost Woods bridge, 20 s to settle, a 45 s sample); a turning camera in the Field (3 cycles of 420 game frames of pans with Link standing, the last 2 sampled); output 1280x800 (oe: output resolution 1280x800 in every log), render 100%, Quality preset Custom (quality_preset=0), frame generation off, Upscaler on the game's own
Method: headless over ssh, no longshot_env.txt except RX_VK_PRECOMPILE=0 with warm caches; fps from the present trace, GPU time from the unprofiled vk: GPU time line; 2 interleaved runs per arm standing (main, on, unset, then unset, on, main, with other arms of the same session between them), the median; one turning-camera run each for main and the switch
How to reproduce: the settings above, with and without RX_VK_TAAU_FULL_SIZE=1
Baseline: main is the same scripts/dev_packs.py base with no pack. Note: this PR's renderer change was measured applied on main 9c9d69ed, not on its own main 6c9edffe: its final gpu/vk change (git diff 6c9edffe 3fd27a04 -- gpu/vk: gpu/vk/taau_default.h, gpu/vk/vk_draw.cpp, and the comment in gpu/vk/shaders/taau_upscale.comp) applies cleanly there, and was built as a renderer-only pack on that base, the one the Deck had warm. The game binary is the same in every arm; only longshot_vk.so differs.

Hyrule Field, 1280x800 at 100%	Before: main @ 9c9d69ed	After: this PR @ 3fd27a04 (switch unset)	After, RX_VK_TAAU_FULL_SIZE=1: this PR @ 3fd27a04
fps standing (runs)	15.13, 15.05	15.16, 15.01	17.45, 17.49
median	15.09	15.09 (0.0%)	17.47 (+15.8%)
frame time p50 / p95 / p99, standing (median of the runs)	66.2 / 69.6 / 72.7 ms	66.2 / 69.9 / 73.3 ms	57.1 / 60.9 / 63.7 ms
frames over 100 ms, standing	0, 1	0, 0	1, 0
GPU time a frame, standing	44.5, 44.6 ms	44.3, 44.5 ms	35.7, 35.4 ms
readbacks that waited, of 3000	358, 353 at 21.0-21.3 ms each	368, 358 at 20.3-21.0 ms each	366, 366 at 15.8 ms each
first-start compile of 9dc98871eb7fc101	14.4, 14.6 s	14.7, 15.0 s	none
fps, turning camera (one run)	15.10	not run	16.67 (+10.4%)
frame time p50 / p95 / p99, turning camera	62.6 / 91.0 / 119.5 ms	not run	53.9 / 92.4 / 141.5 ms
frames over 100 ms, turning camera	18	not run	30
GPU time a frame, turning camera (last 2 lines)	38.1-41.6 ms	not run	28.5-30.8 ms

switch on (once in each run; neither main nor unset logs it):
  vk: the full-size network 9dc98871eb7fc101 is stood in for by FXAA (RX_VK_TAAU_FULL_SIZE)
  vk: GPU time 35.7 ms a frame (the longest submission 12.3 ms)
main and unset:
  vk: compute pipeline of 9dc98871eb7fc101: SLOW compile, 14400 ms (on a worker)
  vk: GPU time 44.5 ms a frame (the longest submission 21.1 ms)

The run-to-run spread is 0.04-0.15 fps standing. The turning camera has one run per arm, so its spread is not known;
the session before measured +17.5% there (13.97 → 16.42 fps, the pre-review code), and the hitches over 100 ms on that
route come from texture streaming, which this switch does not touch. Every run had the GPU at 1040 MHz (median), no
device loss and no amdgpu kernel lines, and exited normally. In the standing runs the lowest available memory was
973-1912 MiB and the lowest swap free stayed above 5.0 GB. The pre-review code measured the same standing on the same
base the session before (14.94 → 17.34 fps).

Flags and settings

Adds one switch, off by default.

Flag	What it is	Before this PR	After this PR	Effect	Helps / doesn't help / untested	Recommendation
RX_VK_TAAU_FULL_SIZE	The FXAA stand-in also replaces the game's full-size anti-aliasing network (9dc98871eb7fc101) where the multiplies run in software	did not exist (the network ran)	unset: the network runs, as before; 1: FXAA in its place	with 1: about 9 ms less GPU time a frame on a Steam Deck at 1280x800 and 100% render, and no 15 s first-start compile; slightly softer textures and weaker smoothing of long shallow edges; no ghosting	helps where the network is emulated (the Deck, AMD APUs and cards, Intel, GTX 16xx); does nothing where the GPU has cooperative matrices; untested in the game outside the Deck	off by default: the player trades picture for speed, so they should choose it

Screenshots

Same spot, standing in Hyrule Field at the Lost Woods bridge, Steam Deck OLED, 1280x800, 100% render. The crops are 2x,
left to right: Before: main @ 9c9d69ed (the network) | After, RX_VK_TAAU_FULL_SIZE=1: this PR @ 3fd27a04 (FXAA).

Whole frame, Before: main @ 9c9d69ed:

Whole frame, After, RX_VK_TAAU_FULL_SIZE=1: this PR @ 3fd27a04:

• Edges: the network draws dark one-pixel outlines with hard steps along the vine and arch edges against the fog. With
  the switch, those edges are soft and have no outline. Thin leaves and fine texture (moss, specks in the fog) are a little
  softer.
• Lighting: where the right-hand rock wall differs between a pair, it follows Navi's light, not the switch: a main run
  shows the same dark wall.
• Shimmer while turning: not settled. The turning-camera shots came every third frame, too far apart during a pan to
  compare an edge from shot to shot. Each turning frame with the switch shows the same soft edges as standing.

Testing

Tested: Linux, game-free: tests/test_taau_default.py and tests/test_taau_env_lookup.py with g++ and clang. On the
PR's first commit (the tests alone, against main's code) test_taau_env_lookup.py builds main's taau_key and fails on its
checks: in full-on, full-on-psychov35 and full-keys-psychov35 (the full-size pass never stood in for, nothing
logged) and custom (the helper's label); test_taau_default.py does not compile there (main has no
gpu/vk/taau_default.h). Both pass on the second. Run again after the rebase onto main aa95d35f, with g++ 13.3 and with clang
23.1.5: at ba06a950 (the tests alone) both fail the same way, and at e1bd95ee both pass; at the head f69594e3, the 126 checks
that are this PR's or read its changed files: 119 pass, 1 skips (test_precompile_compute_guard: no clang++ on this PC's
path) and 6 fail, each on main aa95d35f too: test_appimage_player_build, test_crash_dialog, test_pipeline_tour,
test_shader_module_guard, test_vk_pgo, test_walk_key. gpu/vk/vk_draw.cpp at f69594e3 compiled whole
(-fsyntax-only -Wall -Wextra) for Linux (clang 23.1.5) and Windows (llvm-mingw 20261104): the same warnings as main
aa95d35f's, none new. Also tests/test_taau_fxaa.py, tests/test_upscale_fallback.py,
test_upscaler_rows, test_env_on (fails as on main) and test_readme_menu_rows. tests/run_all.py, run whole before the
review fixes: 483 passed, 14 failed, 84 skipped (tools not on this PC's path).
The same 14 fail on main 6c9edffe in the same environment (482 passed there; the new check is the difference):
test_appimage_player_build, test_bake_portability, test_cpm_discovery, test_crash_dialog, test_crash_record,
test_env_on, test_lift_cache, test_lift_cleanup_error, test_make_game_cache, test_pipeline_tour,
test_readback_late, test_vk_pgo, test_vram_budget_row, test_walk_key. gpu/vk/vk_draw.cpp compiled whole for Linux
(clang) and Windows (llvm-mingw). The stand-in itself is the one the 3/4 network already runs on these GPUs.

In the game: this PR's renderer change on a Steam Deck OLED (Linux build, RADV), the arms in Measurements, with and without
the switch: the log line, no compile of the network, the fps and the pictures above. The numbers in Measurements are from the
commits measured (3fd27a04's gpu/vk change on main 9c9d69ed), measured again after the review fixes, in a session of
their own: the log line once with the switch and not without, no compile of the network with it, and the same-spot
pictures. An earlier session measured the change before the review fixes the same way (14.94 → 17.34 fps).

The branch has been rebased twice since those runs. Onto main e090cc48, where #608
(PsychoV35 by default) overlapped it: tests/test_taau_env_lookup.py keeps #608's RGBA cases, run for the 3/4 network,
and the comment hunk in gpu/vk/shaders/taau_upscale.comp was left out. Then onto main aa95d35f, with the commits
re-authored under the forge's noreply address and each commit's change identical to the e090cc48 ones by git patch-id.
The changed lines of gpu/vk/vk_draw.cpp and gpu/vk/taau_default.h are those of the measured 3fd27a04, so the
Deck numbers stand for this head (f69594e3).

Not tested: the PR's change on its own main (aa95d35f) in the game: the Deck runs used it on main 9c9d69ed (a
renderer-only pack). Shimmer in motion, Game Mode, frame generation and precompile on. Not run either: Intel, NVIDIA GTX
16xx, AMD discrete cards with the switch on, and Native HDR with PsychoV35 and the switch on.

Must be tested:

• Steam Deck (Linux, RADV), 1280x800, 100% render, Upscaler on the game's own: start with RX_VK_TAAU_FULL_SIZE=1,
  expecting the log line vk: the full-size network 9dc98871eb7fc101 is stood in for by FXAA and no
  compute pipeline of 9dc98871eb7fc101 compile, with the picture matching the RX_VK_TAAU_KEYS arm (Test Farm:
  @Beanstalk steamdeck, RX_VK_TAAU_FULL_SIZE=1 at 100% render in Hyrule Field)

Risks

What could break: only with the switch on. Under the game's dynamic resolution (Render resolution Auto), the 3D is
upscaled linearly with FXAA instead of by the network. If the stand-in cannot draw (no push descriptors), the network,
which was never compiled, is left out and the 3D is missing; the 3/4 stand-in has the same dependency, and every GPU that
runs it has push descriptors. Turning PsychoV35 on mid-game with the switch set starts the network's compile, with the 3D
missing until it is built, as at today's first start at 100%; the log then says vk: the full-size network 9dc98871eb7fc101 runs while PsychoV35 is on. Removing the switch after a first start with it (the one that prepared
the shaders) has the same cost once: that preparation left the network out, and its record (longshot_prepared.bin)
does not say so, so the network compiles in play (about 14 s on a Steam Deck), with the 3D left out meanwhile. Telling it from the record would mean
folding the switch into the driver part's stamp, which makes the whole warm pass run again at each change of the switch,
either way, for one pipeline: not done here.

Threads: taau_key is also asked by the prepare's workers, so with the switch on they read PsychoV35's state
(scene_tonemap_on: dev.hdr and the menu's options) without a lock. The pipeline state list's workers already read it
so (scene_tonemap_for, through seed_program_of); at worst a worker builds or leaves out the network by the state a
moment before, which play then corrects (a compile, or a pipeline never used).

Turning it off: leave RX_VK_TAAU_FULL_SIZE unset, or set it to 0.

Compatibility: Nothing changes: with the switch unset the same networks run on every GPU, and no option, save or cache
changes.

Reviewer notes

Related: #528 gives the same stand-in an RGBA16F output for
PsychoV35. It does not touch taau_key; once it lands, the PsychoV35 carve-out here (in taau_key) can go.

A menu choice (an "FXAA (fast)" value on the Upscaler row, shown only where the multiplies are emulated) was considered
and left out of this PR. The row's values are saved numbers that tests pin (OFF == 6, kUpscalers == 7), and its
engine stepping, upscaler() and the Render resolution steps all key on them. The key list is read once at the start,
while a menu choice changes at any time, and a network that was stood in for has no pipeline to fall back on. On top of
that, the new strings need 13 translations in gpu/vk/menu_i18n.h. That is a follow-up if players want it.

Checklist

Everyone

• One topic.
• The title is a Conventional Commit (type(scope): description) of the status line's type.
• Why gives the reason plainly and links the issues that fit (the tracker searched).
• Measurements has a verdict, or one line saying why there is nothing to measure.
• Testing names what was tested, what wasn't, and what must still be tested, where and how.
• Rebased on current main.
• python -I -B tests/run_all.py passes (any failure that also fails on main is named).
• No game files, dumps, build output or generated files.
• third_party/ untouched (a new folder with a PROVENANCE.md instead).
• The license is OK: GPL-3.0-or-later, nothing incompatible and nothing decompiled.

When it applies

• AI assistance recorded with Assisted-by: / Co-Authored-By: trailers.
• Numbers from the same PC, scene and settings, at least 2 interleaved runs each, the median reported.
• Switches documented (docs/development.md; the README for a menu row) and shipped at their recommended value.
• Screenshots of the same spot, captioned like the Measurements columns, for a change that shows on screen.
• A game-free test added for the fix, failing on main and passing here.
• A check under tests/ that reads changed source updated in this PR, not weakened.
• Host code builds on both Windows (llvm-mingw) and Linux, with UTF-8 paths.
• Docs updated.

**Type:** feat · **Risk:** low · **Tested on:** Steam Deck OLED (SteamOS, RADV), Linux game-free checks · **Needs:** Steam Deck (Linux, RADV) ## Why On a Steam Deck at its own resolution (1280x800, 100% render), the game's own anti-aliasing costs about a fifth of every frame. At 100% the game runs a small neural network, `9dc98871eb7fc101`, over the picture to smooth its edges. The Deck's GPU has no hardware for the network's matrix maths, so Longshot does it in software: one profiled Deck frame spent 8.95 ms of its 44.7 ms there, and the network took about 15 s to compile the first time the game started at 100%. Today a Deck player has two choices: no anti-aliasing at 50% render, or the most expensive anti-aliasing at native. Longshot already has a cheap replacement for this kind of network: a linear upscale with FXAA on the edges, the default on these GPUs for the game's 3/4 network. This PR lets a player use it for the full-size pass too, as an opt-in. It is a picture trade the player should choose: slightly softer textures and weaker smoothing of long, shallow edges, for about 9 ms a frame. ## What changed `RX_VK_TAAU_FULL_SIZE=1` (in `longshot_env.txt`, or the environment) makes the FXAA stand-in take the full-size network as well. It only applies where the matrix multiplies run in software: AMD, Intel and older NVIDIA cards. With nothing set, every GPU takes exactly the networks it took before. Native HDR with the PsychoV35 tone mapper keeps the network, because the stand-in writes no RGBA16F picture yet. - vk: `gpu/vk/taau_default.h` (new, no Vulkan) holds the rule. `full_size()` says when the switch applies, and `keys()` gives the default key list (main's list unless the switch is on). - vk: `taau_key` in `gpu/vk/vk_draw.cpp` takes its default from that header (the switch read with `env_on`), and leaves the full-size pass to the network while PsychoV35 is on. The first time it answers for the full-size pass it logs what happens: `vk: the full-size network 9dc98871eb7fc101 is stood in for by FXAA (RX_VK_TAAU_FULL_SIZE)`, or `... runs while PsychoV35 is on (RX_VK_TAAU_FULL_SIZE)`, once each. An explicit `RX_VK_TAAU_KEYS` still goes over everything. A network `taau_key` matches is never compiled (unchanged), so with the switch on the network's 15 s first-start compile is gone too. `taau_stand_in` labels the helper with the network it stands in for, not always the 3/4 network's key. - vk: comments: `taau_key`'s doc comment names both networks. - tests: `tests/test_taau_default.py` (new) checks the rule across a table of GPUs with the switch off and on: off equals main's list on every GPU, the Deck included. `tests/test_taau_env_lookup.py`, which cuts `taau_key` out of the source, has seven new cases: unset, `=0`, `=1`, `=1` with `RX_VK_NO_F16`, `=1` on a GPU with cooperative matrices, `=1` with PsychoV35 (and PsychoV35 turned on and off in play), and explicit keys with PsychoV35, each checking the log line and the helper's label. - docs: the switch's row in `docs/development.md` (The GPU), and a sentence on the Upscaler in the README. ## Measurements **Verdict:** helps. On a Steam Deck at 1280x800 and 100% render, `RX_VK_TAAU_FULL_SIZE=1` gives **+15.8%** standing in Hyrule Field (15.09 → 17.47 fps) and +10.4% with the camera turning (one run each), with 8.9 ms less GPU time a frame. With the switch unset, the PR measures the same as main (15.09 against 15.09 fps). These numbers are from the commits measured, 3fd27a04 (the branch has been rebased since, with its renderer change the same: see Testing). Hardware: Steam Deck OLED, AMD Custom GPU 0932 (RADV VANGOGH), Mesa 26.2.0-devel, SteamOS 3.9.5, stock GPU clock (1040 MHz median in every run) Scene: Hyrule Field standing (a Kokiri save, a teleport to the Field at the Lost Woods bridge, 20 s to settle, a 45 s sample); a turning camera in the Field (3 cycles of 420 game frames of pans with Link standing, the last 2 sampled); output 1280x800 (`oe: output resolution 1280x800` in every log), render 100%, Quality preset Custom (`quality_preset=0`), frame generation off, Upscaler on the game's own Method: headless over ssh, no `longshot_env.txt` except `RX_VK_PRECOMPILE=0` with warm caches; fps from the present trace, GPU time from the unprofiled `vk: GPU time` line; 2 interleaved runs per arm standing (main, on, unset, then unset, on, main, with other arms of the same session between them), the median; one turning-camera run each for main and the switch How to reproduce: the settings above, with and without `RX_VK_TAAU_FULL_SIZE=1` Baseline: main is the same `scripts/dev_packs.py` base with no pack. Note: this PR's renderer change was measured applied on main 9c9d69ed, not on its own main 6c9edffe: its final `gpu/vk` change (`git diff 6c9edffe 3fd27a04 -- gpu/vk`: `gpu/vk/taau_default.h`, `gpu/vk/vk_draw.cpp`, and the comment in `gpu/vk/shaders/taau_upscale.comp`) applies cleanly there, and was built as a renderer-only pack on that base, the one the Deck had warm. The game binary is the same in every arm; only `longshot_vk.so` differs. | Hyrule Field, 1280x800 at 100% | Before: main @ 9c9d69ed | After: this PR @ 3fd27a04 (switch unset) | After, RX_VK_TAAU_FULL_SIZE=1: this PR @ 3fd27a04 | |---|---|---|---| | fps standing (runs) | 15.13, 15.05 | 15.16, 15.01 | 17.45, 17.49 | | median | **15.09** | **15.09** (0.0%) | **17.47** (+15.8%) | | frame time p50 / p95 / p99, standing (median of the runs) | 66.2 / 69.6 / 72.7 ms | 66.2 / 69.9 / 73.3 ms | 57.1 / 60.9 / 63.7 ms | | frames over 100 ms, standing | 0, 1 | 0, 0 | 1, 0 | | GPU time a frame, standing | 44.5, 44.6 ms | 44.3, 44.5 ms | 35.7, 35.4 ms | | readbacks that waited, of 3000 | 358, 353 at 21.0-21.3 ms each | 368, 358 at 20.3-21.0 ms each | 366, 366 at 15.8 ms each | | first-start compile of `9dc98871eb7fc101` | 14.4, 14.6 s | 14.7, 15.0 s | none | | fps, turning camera (one run) | 15.10 | not run | 16.67 (+10.4%) | | frame time p50 / p95 / p99, turning camera | 62.6 / 91.0 / 119.5 ms | not run | 53.9 / 92.4 / 141.5 ms | | frames over 100 ms, turning camera | 18 | not run | 30 | | GPU time a frame, turning camera (last 2 lines) | 38.1-41.6 ms | not run | 28.5-30.8 ms | ``` switch on (once in each run; neither main nor unset logs it): vk: the full-size network 9dc98871eb7fc101 is stood in for by FXAA (RX_VK_TAAU_FULL_SIZE) vk: GPU time 35.7 ms a frame (the longest submission 12.3 ms) main and unset: vk: compute pipeline of 9dc98871eb7fc101: SLOW compile, 14400 ms (on a worker) vk: GPU time 44.5 ms a frame (the longest submission 21.1 ms) ``` The run-to-run spread is 0.04-0.15 fps standing. The turning camera has one run per arm, so its spread is not known; the session before measured +17.5% there (13.97 → 16.42 fps, the pre-review code), and the hitches over 100 ms on that route come from texture streaming, which this switch does not touch. Every run had the GPU at 1040 MHz (median), no device loss and no amdgpu kernel lines, and exited normally. In the standing runs the lowest available memory was 973-1912 MiB and the lowest swap free stayed above 5.0 GB. The pre-review code measured the same standing on the same base the session before (14.94 → 17.34 fps). ### Flags and settings Adds one switch, off by default. | Flag | What it is | Before this PR | After this PR | Effect | Helps / doesn't help / untested | Recommendation | |---|---|---|---|---|---|---| | `RX_VK_TAAU_FULL_SIZE` | The FXAA stand-in also replaces the game's full-size anti-aliasing network (`9dc98871eb7fc101`) where the multiplies run in software | did not exist (the network ran) | unset: the network runs, as before; `1`: FXAA in its place | with `1`: about 9 ms less GPU time a frame on a Steam Deck at 1280x800 and 100% render, and no 15 s first-start compile; slightly softer textures and weaker smoothing of long shallow edges; no ghosting | helps where the network is emulated (the Deck, AMD APUs and cards, Intel, GTX 16xx); does nothing where the GPU has cooperative matrices; untested in the game outside the Deck | off by default: the player trades picture for speed, so they should choose it | ## Screenshots Same spot, standing in Hyrule Field at the Lost Woods bridge, Steam Deck OLED, 1280x800, 100% render. The crops are 2x, left to right: **Before: main @ 9c9d69ed (the network)** | **After, RX_VK_TAAU_FULL_SIZE=1: this PR @ 3fd27a04 (FXAA)**.     Whole frame, Before: main @ 9c9d69ed:  Whole frame, After, RX_VK_TAAU_FULL_SIZE=1: this PR @ 3fd27a04:  - **Edges:** the network draws dark one-pixel outlines with hard steps along the vine and arch edges against the fog. With the switch, those edges are soft and have no outline. Thin leaves and fine texture (moss, specks in the fog) are a little softer. - **Lighting:** where the right-hand rock wall differs between a pair, it follows Navi's light, not the switch: a main run shows the same dark wall. - **Shimmer while turning: not settled.** The turning-camera shots came every third frame, too far apart during a pan to compare an edge from shot to shot. Each turning frame with the switch shows the same soft edges as standing. ## Testing **Tested:** Linux, game-free: `tests/test_taau_default.py` and `tests/test_taau_env_lookup.py` with g++ and clang. On the PR's first commit (the tests alone, against main's code) `test_taau_env_lookup.py` builds main's `taau_key` and fails on its checks: in `full-on`, `full-on-psychov35` and `full-keys-psychov35` (the full-size pass never stood in for, nothing logged) and `custom` (the helper's label); `test_taau_default.py` does not compile there (main has no `gpu/vk/taau_default.h`). Both pass on the second. Run again after the rebase onto main aa95d35f, with g++ 13.3 and with clang 23.1.5: at ba06a950 (the tests alone) both fail the same way, and at e1bd95ee both pass; at the head f69594e3, the 126 checks that are this PR's or read its changed files: 119 pass, 1 skips (`test_precompile_compute_guard`: no `clang++` on this PC's path) and 6 fail, each on main aa95d35f too: `test_appimage_player_build`, `test_crash_dialog`, `test_pipeline_tour`, `test_shader_module_guard`, `test_vk_pgo`, `test_walk_key`. `gpu/vk/vk_draw.cpp` at f69594e3 compiled whole (`-fsyntax-only -Wall -Wextra`) for Linux (clang 23.1.5) and Windows (llvm-mingw 20261104): the same warnings as main aa95d35f's, none new. Also `tests/test_taau_fxaa.py`, `tests/test_upscale_fallback.py`, `test_upscaler_rows`, `test_env_on` (fails as on main) and `test_readme_menu_rows`. `tests/run_all.py`, run whole before the review fixes: 483 passed, 14 failed, 84 skipped (tools not on this PC's path). The same 14 fail on main 6c9edffe in the same environment (482 passed there; the new check is the difference): `test_appimage_player_build`, `test_bake_portability`, `test_cpm_discovery`, `test_crash_dialog`, `test_crash_record`, `test_env_on`, `test_lift_cache`, `test_lift_cleanup_error`, `test_make_game_cache`, `test_pipeline_tour`, `test_readback_late`, `test_vk_pgo`, `test_vram_budget_row`, `test_walk_key`. `gpu/vk/vk_draw.cpp` compiled whole for Linux (clang) and Windows (llvm-mingw). The stand-in itself is the one the 3/4 network already runs on these GPUs. In the game: this PR's renderer change on a Steam Deck OLED (Linux build, RADV), the arms in Measurements, with and without the switch: the log line, no compile of the network, the fps and the pictures above. The numbers in Measurements are from the commits measured (3fd27a04's `gpu/vk` change on main 9c9d69ed), measured again after the review fixes, in a session of their own: the log line once with the switch and not without, no compile of the network with it, and the same-spot pictures. An earlier session measured the change before the review fixes the same way (14.94 → 17.34 fps). The branch has been rebased twice since those runs. Onto main e090cc48, where https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/608 (PsychoV35 by default) overlapped it: `tests/test_taau_env_lookup.py` keeps #608's RGBA cases, run for the 3/4 network, and the comment hunk in `gpu/vk/shaders/taau_upscale.comp` was left out. Then onto main aa95d35f, with the commits re-authored under the forge's noreply address and each commit's change identical to the e090cc48 ones by `git patch-id`. The changed lines of `gpu/vk/vk_draw.cpp` and `gpu/vk/taau_default.h` are those of the measured 3fd27a04, so the Deck numbers stand for this head (f69594e3). **Not tested:** the PR's change on its own main (aa95d35f) in the game: the Deck runs used it on main 9c9d69ed (a renderer-only pack). Shimmer in motion, Game Mode, frame generation and precompile on. Not run either: Intel, NVIDIA GTX 16xx, AMD discrete cards with the switch on, and Native HDR with PsychoV35 and the switch on. **Must be tested:** - Steam Deck (Linux, RADV), 1280x800, 100% render, Upscaler on the game's own: start with `RX_VK_TAAU_FULL_SIZE=1`, expecting the log line `vk: the full-size network 9dc98871eb7fc101 is stood in for by FXAA` and no `compute pipeline of 9dc98871eb7fc101` compile, with the picture matching the `RX_VK_TAAU_KEYS` arm (Test Farm: @Beanstalk steamdeck, RX_VK_TAAU_FULL_SIZE=1 at 100% render in Hyrule Field) ## Risks **What could break:** only with the switch on. Under the game's dynamic resolution (Render resolution Auto), the 3D is upscaled linearly with FXAA instead of by the network. If the stand-in cannot draw (no push descriptors), the network, which was never compiled, is left out and the 3D is missing; the 3/4 stand-in has the same dependency, and every GPU that runs it has push descriptors. Turning PsychoV35 on mid-game with the switch set starts the network's compile, with the 3D missing until it is built, as at today's first start at 100%; the log then says `vk: the full-size network 9dc98871eb7fc101 runs while PsychoV35 is on`. Removing the switch after a first start with it (the one that prepared the shaders) has the same cost once: that preparation left the network out, and its record (`longshot_prepared.bin`) does not say so, so the network compiles in play (about 14 s on a Steam Deck), with the 3D left out meanwhile. Telling it from the record would mean folding the switch into the driver part's stamp, which makes the whole warm pass run again at each change of the switch, either way, for one pipeline: not done here. **Threads:** `taau_key` is also asked by the prepare's workers, so with the switch on they read PsychoV35's state (`scene_tonemap_on`: `dev.hdr` and the menu's options) without a lock. The pipeline state list's workers already read it so (`scene_tonemap_for`, through `seed_program_of`); at worst a worker builds or leaves out the network by the state a moment before, which play then corrects (a compile, or a pipeline never used). **Turning it off:** leave `RX_VK_TAAU_FULL_SIZE` unset, or set it to `0`. **Compatibility:** Nothing changes: with the switch unset the same networks run on every GPU, and no option, save or cache changes. ## Reviewer notes Related: https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/528 gives the same stand-in an RGBA16F output for PsychoV35. It does not touch `taau_key`; once it lands, the PsychoV35 carve-out here (in `taau_key`) can go. A menu choice (an "FXAA (fast)" value on the Upscaler row, shown only where the multiplies are emulated) was considered and left out of this PR. The row's values are saved numbers that tests pin (`OFF == 6`, `kUpscalers == 7`), and its engine stepping, `upscaler()` and the Render resolution steps all key on them. The key list is read once at the start, while a menu choice changes at any time, and a network that was stood in for has no pipeline to fall back on. On top of that, the new strings need 13 translations in `gpu/vk/menu_i18n.h`. That is a follow-up if players want it. ## Checklist ### Everyone - [x] One topic. - [x] The title is a Conventional Commit (`type(scope): description`) of the status line's type. - [x] Why gives the reason plainly and links the issues that fit (the tracker searched). - [x] Measurements has a verdict, or one line saying why there is nothing to measure. - [x] Testing names what was tested, what wasn't, and what must still be tested, where and how. - [x] Rebased on current `main`. - [x] `python -I -B tests/run_all.py` passes (any failure that also fails on `main` is named). - [x] No game files, dumps, build output or generated files. - [x] `third_party/` untouched (a new folder with a `PROVENANCE.md` instead). - [x] The license is OK: GPL-3.0-or-later, nothing incompatible and nothing decompiled. ### When it applies - [x] AI assistance recorded with `Assisted-by:` / `Co-Authored-By:` trailers. - [x] Numbers from the same PC, scene and settings, at least 2 interleaved runs each, the median reported. - [x] Switches documented (docs/development.md; the README for a menu row) and shipped at their recommended value. - [x] Screenshots of the same spot, captioned like the Measurements columns, for a change that shows on screen. - [x] A game-free test added for the fix, failing on `main` and passing here. - [x] A check under `tests/` that reads changed source updated in this PR, not weakened. - [x] Host code builds on both Windows (llvm-mingw) and Linux, with UTF-8 paths. - [x] Docs updated.

zeldaootr added 3 commits 2026-11-15 04:40:42 +00:00

test: RX_VK_TAAU_FULL_SIZE=1 has the FXAA stand-in take the full-size network 9dc98871eb7fc101; unset is as before ... eb387f7547

Assisted-by: Claude

vk: RX_VK_TAAU_FULL_SIZE=1, FXAA in place of the full-size network where the multiplies run in software (off by default) ... 3fd27a04db

Assisted-by: Claude

docs(tested): RX_VK_TAAU_FULL_SIZE on a Steam Deck OLED at 1280x800 ... 15b434e86a

Assisted-by: Claude

zeldaootr commented 2026-11-15 04:40:56 +00:00

Author

Contributor

• Copy link
• Copy Markdown

Pictures for the description. Steam Deck OLED, Hyrule Field at the Lost Woods bridge, 1280x800 at 100% render. Crops are 2x, left to right: main @ 9c9d69ed (the network) | this PR @ 3fd27a04, RX_VK_TAAU_FULL_SIZE=1 (FXAA).

The arch edge and hanging vines against the fog:

The vines on the right:

The foliage:

Link:

Whole frame, main @ 9c9d69ed:

Whole frame, this PR @ 3fd27a04 with RX_VK_TAAU_FULL_SIZE=1:

Captured by an AI agent (Claude Code) for zeldaootr.

Pictures for the description. Steam Deck OLED, Hyrule Field at the Lost Woods bridge, 1280x800 at 100% render. Crops are 2x, left to right: **main @ 9c9d69ed (the network)** | **this PR @ 3fd27a04, `RX_VK_TAAU_FULL_SIZE=1` (FXAA)**. The arch edge and hanging vines against the fog:  The vines on the right:  The foliage:  Link:  Whole frame, main @ 9c9d69ed:  Whole frame, this PR @ 3fd27a04 with `RX_VK_TAAU_FULL_SIZE=1`:  *Captured by an AI agent (Claude Code) for zeldaootr.*

arch-edge.png

332 KiB

vines.png

334 KiB

foliage.png

257 KiB

link.png

367 KiB

main-full.png

2.9 MiB

fxaa-full.png

2.9 MiB

zeldaootr commented 2026-11-15 04:41:29 +00:00

Author

Contributor

• Copy link
• Copy Markdown

The pictures are in the description now (same spot, main | the switch), and the eight measured runs are under docs/tested/ (commit 15b434e8). On top of #303's RX_VK_READBACK_EARLY=1 and RX_VK_READBACK_LATE=newest the same switch reads 21.35 fps against main's 15.09 on the same Deck and scene (two interleaved runs each; the build was #303's head and this change on main 9c9d69ed), so the two savings add up.

The pictures are in the description now (same spot, main | the switch), and the eight measured runs are under docs/tested/ (commit 15b434e8). On top of #303's RX_VK_READBACK_EARLY=1 and RX_VK_READBACK_LATE=newest the same switch reads 21.35 fps against main's 15.09 on the same Deck and scene (two interleaved runs each; the build was #303's head and this change on main 9c9d69ed), so the two savings add up.

zeldaootr changed title from WIP: feat(vk): FXAA in place of the full-size anti-aliasing network where the GPU multiplies in software (RX_VK_TAAU_FULL_SIZE=1, off by default) to feat(vk): FXAA in place of the full-size anti-aliasing network where the GPU multiplies in software (RX_VK_TAAU_FULL_SIZE=1, off by default) 2026-11-15 04:41:31 +00:00

llun referenced this pull request 2026-11-15 06:12:02 +00:00

WIP: fix(vk): the full-size network's stand-in draws at 100% Render resolution, so GTX 10 cards show the 3D there #718

zeldaootr force-pushed deck-native-aa-stand-in from 15b434e86a to ff06fa036d 2026-11-15 06:27:35 +00:00 Compare

zeldaootr commented 2026-11-15 06:27:41 +00:00

Author

Contributor

• Copy link
• Copy Markdown

Rebased onto main e090cc48. #608 rewrote taau_stand_in for PsychoV35: its RGBA path is kept, with this PR's label (the program's own key) on top, and the check's fixture now has one PsychoV35 switch. The comment-only edit to shaders/taau_upscale.comp is dropped, so no generated header changes. The check fails at the test commit in the same cases as before and passes at the tip (g++, clang), with test_psychov35_taau. The Deck numbers are from before #608, which changes nothing with PsychoV35 off, the case measured. Open question for a reviewer: since #608 the stand-in can write the RGBA16F picture, so this PR's rule that the network runs while PsychoV35 is on may no longer be needed; it is kept as it was, the careful side.

Rebased onto main e090cc48. https://git.magicbean.dev/MagicBeanDev/Longshot/pulls/608 rewrote `taau_stand_in` for PsychoV35: its RGBA path is kept, with this PR's label (the program's own key) on top, and the check's fixture now has one PsychoV35 switch. The comment-only edit to `shaders/taau_upscale.comp` is dropped, so no generated header changes. The check fails at the test commit in the same cases as before and passes at the tip (g++, clang), with `test_psychov35_taau`. The Deck numbers are from before #608, which changes nothing with PsychoV35 off, the case measured. Open question for a reviewer: since #608 the stand-in can write the RGBA16F picture, so this PR's rule that the network runs while PsychoV35 is on may no longer be needed; it is kept as it was, the careful side.

mooki added the

area/renderer

kind

feature

platform/linux

platform/windows

priority

low

labels 2026-11-15 13:58:12 +00:00

zeldaootr force-pushed deck-native-aa-stand-in from ff06fa036d to f69594e348 2026-11-15 17:13:23 +00:00 Compare

zeldaootr commented 2026-11-15 17:13:25 +00:00

Author

Contributor

• Copy link
• Copy Markdown

Rebased onto main aa95d35f (head f69594e3); the commits are re-authored with the forge's noreply address: no code change, the same patches by git patch-id.
Tests re-run on the new base, g++ 13.3 and clang 23.1.5: test_taau_default and test_taau_env_lookup fail at the test-only commit ba06a950 and pass at e1bd95ee; of the 126 checks that are this PR's or read its files, 119 pass, 1 skips and the 6 that fail fail on main too. The Deck numbers stand (the vk_draw.cpp and taau_default.h lines are those of the measured 3fd27a04); the description says so.

Rebased onto `main` aa95d35f (head f69594e3); the commits are re-authored with the forge's noreply address: no code change, the same patches by `git patch-id`. Tests re-run on the new base, g++ 13.3 and clang 23.1.5: `test_taau_default` and `test_taau_env_lookup` fail at the test-only commit ba06a950 and pass at e1bd95ee; of the 126 checks that are this PR's or read its files, 119 pass, 1 skips and the 6 that fail fail on main too. The Deck numbers stand (the `vk_draw.cpp` and `taau_default.h` lines are those of the measured 3fd27a04); the description says so.

Some checks are pending

Hide checks

CI / Conventional Commit title (pull_request) Blocked by required conditions

Details

CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Blocked by required conditions

Details

CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Blocked by required conditions

Details

CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Blocked by required conditions

Details

CI / Workflow lint (actionlint) (pull_request) Blocked by required conditions

Details

Guide screenshots / Guide images and local UI source review (pull_request) Blocked by required conditions

Details

Third-party parts / As their notes say (pull_request) Blocked by required conditions

Details

Third-party parts / The same as upstream (pull_request) Blocked by required conditions

Details

Website / Website on Cloudflare (pull_request) Blocked by required conditions

Details

PR description / Description follows the template (pull_request_target) Successful in 6s

Details

This pull request doesn't have enough approvals yet. 0 of 1 approvals granted.

Some workflows are waiting to be reviewed.

This branch is out-of-date with the base branch

You are not authorized to merge this pull request.

View command line instructions

Checkout

From your project repository, check out a new branch and test the changes.

git fetch -u 'deck-native-aa-stand-in:zeldaootr-deck-native-aa-stand-in'

git switch 'zeldaootr-deck-native-aa-stand-in'

Sign in to join this conversation.

Reviewers

No reviewers

Labels

Clear labels   

area/accessibility

Barriers for people with disabilities

  

area/audio

Sound and music

  

area/input

Controllers, keyboard, mouse

  

area/install

Installing, updating, building from your game

  

area/menu

In-game menu and settings

  

area/renderer

Vulkan renderer, shaders, effects

  

area/saves

Save files

  

area/upscaling

DLSS, FSR, frame generation

  

beanstalk: approved @ 1697603

Set by Beanstalk for show; the farm's database is what counts

  

beanstalk: approved @ 2c5a62e

Set by Beanstalk for show; the farm's database is what counts

  

beanstalk: approved @ 4fd82b7

Set by Beanstalk for show; the farm's database is what counts

  

beanstalk: approved @ 64d6d09

Set by Beanstalk for show; the farm's database is what counts

  

beanstalk: approved @ 6f91fae

Set by Beanstalk for show; the farm's database is what counts

  

beanstalk: approved @ 70a78f9

Set by Beanstalk for show; the farm's database is what counts

  

beanstalk: approved @ 8015c9a

Set by Beanstalk for show; the farm's database is what counts

  

beanstalk: approved @ 8d361dc

Set by Beanstalk for show; the farm's database is what counts

  

beanstalk: approved @ 9d8356e

Set by Beanstalk for show; the farm's database is what counts

  

beanstalk: approved @ abf2227

Set by Beanstalk for show; the farm's database is what counts

  

beanstalk: approved @ d82f202

Set by Beanstalk for show; the farm's database is what counts

  

beanstalk: approved @ e5f1e40

Set by Beanstalk for show; the farm's database is what counts

  

beanstalk: approved @ ee8ab5f

Set by Beanstalk for show; the farm's database is what counts

  

ci: approved

A maintainer approved CircleCI for this fork PR (sticky; removed after 14 quiet days)

  

good first issue

A good start for a new contributor

  

gpu/amd

AMD GPUs

  

gpu/igpu

Integrated GPUs

  

gpu/intel

Intel GPUs

  

gpu/nvidia

NVIDIA GPUs

  

help wanted

Extra hands or knowledge wanted

  

kind

bug

Something doesn't work as it should

  

kind

chore

Build, CI, dependencies, tidying

  

kind

docs

Documentation

  

kind

feature

A new feature or an improvement

  

kind

performance

Frame rate, frame time, loading, memory

  

kind

question

A question rather than a change

  

needs measurements

Waiting for before/after measurements

  

platform/linux

Affects Linux (a change for every system carries both Windows and Linux)

  

platform/macos

macOS (Apple silicon or Intel Macs)

  

platform/steam-deck

Steam Deck

  

platform/windows

Affects Windows (a change for every system carries both Windows and Linux)

  

preview

Maintainers: deploy a preview of this PR's website (read its changes first). Removed after the run.

  

priority

critical

Crashes, data loss, or blocks most players

  

priority

high

Important: next in line

  

priority

low

Nice to have

  

regression

Worked in an earlier build, broken in a newer one: highest priority, above other bugs

  

status

blocked

Waits on something else first

  

status

confirmed

Reproduced or agreed on

  

status

duplicate

Already reported or proposed

  

status

invalid

Not a problem in this project

  

status

needs-info

Waiting for more information from the reporter

  

status

wontfix

Will not be worked on

  

testing wanted

Wanted: a test on other hardware (posts a test request in #test-requests)

  

ci: approved

No labels

area/accessibility

area/audio

area/input

area/install

area/menu

area/renderer

area/saves

area/upscaling

beanstalk: approved @ 1697603

beanstalk: approved @ 2c5a62e

beanstalk: approved @ 4fd82b7

beanstalk: approved @ 64d6d09

beanstalk: approved @ 6f91fae

beanstalk: approved @ 70a78f9

beanstalk: approved @ 8015c9a

beanstalk: approved @ 8d361dc

beanstalk: approved @ 9d8356e

beanstalk: approved @ abf2227

beanstalk: approved @ d82f202

beanstalk: approved @ e5f1e40

beanstalk: approved @ ee8ab5f

ci: approved

good first issue

gpu/amd

gpu/igpu

gpu/intel

gpu/nvidia

help wanted

kind

bug

kind

chore

kind

docs

kind

feature

kind

performance

kind

question

needs measurements

platform/linux

platform/macos

platform/steam-deck

platform/windows

preview

priority

critical

priority

high

priority

low

regression

status

blocked

status

confirmed

status

duplicate

status

invalid

status

needs-info

status

wontfix

testing wanted

ci: approved

Milestone

Clear milestone

No items

No milestone

Projects

Clear projects

No items

No project

Assignees

Clear assignees

No assignees

1 participant

Notifications

Subscribe

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference

MagicBeanDev/Longshot!687

No description provided.