AMD/RADV: Resolution Auto colour noise, one-frame sunlight flicker, huge pages, per-tap gathers #487

Closed

chuppatricky wants to merge 4 commits from amd-flicker-noise-fixes into main

chuppatricky commented

2026-11-08 05:48:00 +00:00

(Migrated from github.com)

AMD/RADV fixes for Ocarina of Time: the colour noise with Resolution Auto, a one-frame sunlight flicker, slower submits from MADV_NOHUGEPAGE, and per-tap gathers with runtime offsets.

Tested on an RX 7900 GRE (RADV, Mesa 26.2.6, kernel 7.3 CachyOS, THP always), in Kokiri Forest from a save. Most measurements are from headless runs with the same scripted input (HOS_HEADLESS, HOS_INPUT_SCRIPT). The fixes were developed and measured on 184564d and rebased onto ef2e1c4. On ef2e1c4 the renderer builds (0 failures), loads with no unresolved symbols, and host/hos_main.c passes a syntax check. The rebased build has not been run in game.

Colour noise with Resolution Auto (issues #361, #312)

When Auto drops below about 75%, the game switches to one of five smaller upscaler networks (0775215f8da720e6, 19346f9c9299279e, 72ba090c70fe2614, f10782f9e58b6a7c, 6676619641320ba7). These are left out on AMD and replaced by upscale_stub.

The stub's input is the scene in RGB9E5. Because textures share images by bytes per texel, that scene sits in an image created as R11G11B10. vkCmdBlitImage decodes the image's own format, so the stub stretched RGB9E5 bits read as R11G11B10 over the whole 3D picture. That's the psychedelic noise.

The fix: when the texture's format and its image's format differ and both are 4 bytes per texel, the stub copies the texels unchanged into an image of the texture's own format, then blits from that.

Reproduced in game: noise in the frames while 0775215f was stood in for.
Standalone test on the same GPU (RGB9E5 bits in an R11G11B10 image, linear 2x blit):
- mean colour error, old path: 143,000%
- mean colour error, new path: 0.38%
Not re-observed in game after the fix: I couldn't get Auto to drop that low again on demand. Most of the evidence is the standalone test above.

One-frame sunlight flicker (most visible at load-in)

The faked occlusion count (16 per draw since the reset) goes up and down from frame to frame, and the game's sun visibility follows it. For about one frame in 70, the direct sunlight leaves the ground and the haze brightens across the whole picture.

The fix: every samples-passed report now reads 4096 (everything visible). RX_VK_OCC_FIXED=<n> sets the value; -1 restores the count.

A/B results, one-frame brightness spikes per gameplay run (same script, run in both orders, about 2,600–2,900 gameplay frames each):

Occlusion report	Spikes per run
per-draw count (before)	30, 52
fixed 4096 (this PR)	6, 4

No visible change otherwise: lighting, shadows and Navi's glow look the same.

Transparent huge pages (`40c3f851`, issue #312)

MADV_NOHUGEPAGE made RADV slower here, not faster. Each submission makes amdgpu check every page of the imported (userptr) memory. In a minimal test this cost about 0.5 ms per GB even when the memory is untouched, and with 4 KB pages the check took three times as long:

Measurement	Huge pages allowed	`MADV_NOHUGEPAGE`
queue submit per frame, title screen	1.25 ms	3.8 ms
fps in Kokiri	8–10	3–5

These numbers are from before batched submits (aa1a4ce). Batching cuts the number of submissions, so the gap should be smaller now, but each remaining submission still pays the per-page check.

The guest window now keeps the system's huge-page setting. HOS_THP=0 turns huge pages off as before.

Per-tap gathers with runtime offsets

textureGatherOffsets with offsets computed at run time lost those offsets: Vulkan only accepts the four offsets as ConstOffsets, so ImageOperands dropped them ("Not all arguments in PTP are immediate, ignoring").

They now become four gathers, each with one runtime Offset (shaderImageGatherExtended), keeping texel i0j0 (component w) from each, which is how Mesa lowers it. Constant offsets keep the single ConstOffsets gather. Ported from Lichii (1a8f736f, 64293e7c).

No new translation failures: 105 of 21,609 programs fail with the stock renderer and with this one.
Unknown whether OoT uses this path: neither renderer logged the warning, so this may not be reached in OoT at all.

Not included

Sending each frame whole instead of in 120-draw parts: this roughly doubled fps here, but the batched submits now on with RADV (aa1a4ce) address the same per-submission cost, so I left it out.
Diagnostics: the per-frame brightness and white-flash logging I used to find the flicker. Available if useful.

🤖 Generated with Claude Code

AMD/RADV fixes for Ocarina of Time: the colour noise with Resolution Auto, a one-frame sunlight flicker, slower submits from `MADV_NOHUGEPAGE`, and per-tap gathers with runtime offsets. Tested on an RX 7900 GRE (RADV, Mesa 26.2.6, kernel 7.3 CachyOS, THP `always`), in Kokiri Forest from a save. Most measurements are from headless runs with the same scripted input (`HOS_HEADLESS`, `HOS_INPUT_SCRIPT`). The fixes were developed and measured on `184564d` and rebased onto `ef2e1c4`. On `ef2e1c4` the renderer builds (0 failures), loads with no unresolved symbols, and `host/hos_main.c` passes a syntax check. **The rebased build has not been run in game.** ## Colour noise with Resolution Auto (issues #361, #312) When Auto drops below about 75%, the game switches to one of five smaller upscaler networks (`0775215f8da720e6`, `19346f9c9299279e`, `72ba090c70fe2614`, `f10782f9e58b6a7c`, `6676619641320ba7`). These are left out on AMD and replaced by `upscale_stub`. The stub's input is the scene in RGB9E5. Because textures share images by bytes per texel, that scene sits in an image created as R11G11B10. `vkCmdBlitImage` decodes the image's own format, so the stub stretched RGB9E5 bits read as R11G11B10 over the whole 3D picture. That's the psychedelic noise. The fix: when the texture's format and its image's format differ and both are 4 bytes per texel, the stub copies the texels unchanged into an image of the texture's own format, then blits from that. - **Reproduced in game:** noise in the frames while `0775215f` was stood in for. - **Standalone test on the same GPU** (RGB9E5 bits in an R11G11B10 image, linear 2x blit): - mean colour error, old path: **143,000%** - mean colour error, new path: **0.38%** - **Not re-observed in game after the fix:** I couldn't get Auto to drop that low again on demand. Most of the evidence is the standalone test above. ## One-frame sunlight flicker (most visible at load-in) The faked occlusion count (16 per draw since the reset) goes up and down from frame to frame, and the game's sun visibility follows it. For about one frame in 70, the direct sunlight leaves the ground and the haze brightens across the whole picture. The fix: every samples-passed report now reads 4096 (everything visible). `RX_VK_OCC_FIXED=<n>` sets the value; `-1` restores the count. A/B results, one-frame brightness spikes per gameplay run (same script, run in both orders, about 2,600–2,900 gameplay frames each): | Occlusion report | Spikes per run | |---|---| | per-draw count (before) | 30, 52 | | fixed 4096 (this PR) | 6, 4 | No visible change otherwise: lighting, shadows and Navi's glow look the same. ## Transparent huge pages (`40c3f851`, issue #312) `MADV_NOHUGEPAGE` made RADV slower here, not faster. Each submission makes amdgpu check every page of the imported (userptr) memory. In a minimal test this cost about 0.5 ms per GB even when the memory is untouched, and with 4 KB pages the check took three times as long: | Measurement | Huge pages allowed | `MADV_NOHUGEPAGE` | |---|---|---| | queue submit per frame, title screen | 1.25 ms | 3.8 ms | | fps in Kokiri | 8–10 | 3–5 | These numbers are from before batched submits (`aa1a4ce`). Batching cuts the number of submissions, so the gap should be smaller now, but each remaining submission still pays the per-page check. The guest window now keeps the system's huge-page setting. `HOS_THP=0` turns huge pages off as before. ## Per-tap gathers with runtime offsets `textureGatherOffsets` with offsets computed at run time lost those offsets: Vulkan only accepts the four offsets as `ConstOffsets`, so `ImageOperands` dropped them ("Not all arguments in PTP are immediate, ignoring"). They now become four gathers, each with one runtime `Offset` (`shaderImageGatherExtended`), keeping texel i0j0 (component w) from each, which is how Mesa lowers it. Constant offsets keep the single `ConstOffsets` gather. Ported from Lichii (`1a8f736f`, `64293e7c`). - **No new translation failures:** 105 of 21,609 programs fail with the stock renderer and with this one. - **Unknown whether OoT uses this path:** neither renderer logged the warning, so this may not be reached in OoT at all. ## Not included - **Sending each frame whole instead of in 120-draw parts:** this roughly doubled fps here, but the batched submits now on with RADV (`aa1a4ce`) address the same per-submission cost, so I left it out. - **Diagnostics:** the per-frame brightness and white-flash logging I used to find the flicker. Available if useful. 🤖 Generated with [Claude Code](https://claude.com/claude-code)

RufflezAU commented

2026-11-09 03:51:30 +00:00

(Migrated from github.com)

Feedback from a Steam Deck (RADV VANGOGH), in case the default is not what you want everywhere.

With the 4096 default (everything visible) the sun flicker is indeed gone, but the occlusion culling goes with it: in the starting room, geometry from outside the room flashes through the walls — glowing bits of objects that should be culled, every frame. That is very visible on a small screen.

RX_VK_OCC_FIXED=-1 restores the per-draw count and the culling (and the flashing stops), but then the one-frame sunlight flicker is back. On this GPU the two are tied: the count is what the game's sun visibility follows.

No action needed if 4096 is the intended default for desktop GPUs — just flagging that on the Deck it trades a subtle one-frame flicker for visible geometry bleeding through walls, and there is no middle value in the switch as it stands.

Feedback from a Steam Deck (RADV VANGOGH), in case the default is not what you want everywhere. With the 4096 default (everything visible) the sun flicker is indeed gone, but the occlusion culling goes with it: in the starting room, **geometry from outside the room flashes through the walls** — glowing bits of objects that should be culled, every frame. That is very visible on a small screen. `RX_VK_OCC_FIXED=-1` restores the per-draw count and the culling (and the flashing stops), but then the one-frame sunlight flicker is back. On this GPU the two are tied: the count is what the game's sun visibility follows. No action needed if 4096 is the intended default for desktop GPUs — just flagging that on the Deck it trades a subtle one-frame flicker for visible geometry bleeding through walls, and there is no middle value in the switch as it stands.

NotRexed commented

2026-11-09 05:29:36 +00:00

(Migrated from github.com)

Closing to keep the queue clear — please rebase on current main and reopen; thanks! (The per-tap gathers and samples-passed parts are on main as e16b6f7 and 5257559; the Resolution Auto colour noise, sunlight flicker and huge-page parts are still new.)

Closing to keep the queue clear — please rebase on current main and reopen; thanks! (The per-tap gathers and samples-passed parts are on main as e16b6f7 and 5257559; the Resolution Auto colour noise, sunlight flicker and huge-page parts are still new.)