fix(vk): Sharpness samples one picture pixel apart, RCAS-style, the same at any window size #1256

2026-11-09T19:10:29Z

hckrman101 commented

2026-11-09 19:10:29 +00:00

(Migrated from github.com)

Sharpness: neighbours one picture pixel apart, contrast-adaptive in the manner of AMD FidelityFX RCAS

The Insert menu's Sharpness runs in the window pass (gpu/vk/shaders/fg_scale.comp, the only sharpen on the Vulkan path: direct presents and frame generation's made frames both go through present_image -> framegen_scale, same shader, same push constant). It took its four neighbours 1.0 / vec2(d) apart, one window pixel. When the window is larger than the picture that is a fraction of a picture pixel (0.53 for the 2048 x 1152 picture on a 3840 x 2160 screen, 0.33 for 1280 x 720), so what a given setting did depended on the window's size rather than on the picture. This was found in #1071, which makes the same change in its copy of the shader; this PR is a separate implementation of it, on main, with measurements.

What changes

The neighbours are one picture pixel apart (1.0 / vec2(textureSize(src, 0))), or one window pixel when the window is the smaller of the two, so a downscale does not sharpen detail the window cannot show.
The sharpen is a contrast-adaptive one in the manner of AMD FidelityFX RCAS (written from RCAS's published math): a negative lobe on the four neighbours, the largest that keeps the result inside 0..1 for that neighbourhood (smaller near black and white), capped at RCAS's maximum (0.25 - 1/16), then normalized. The slider maps 0..100% onto 0..that maximum.
The result is then held to the lightest and darkest of the pixel and its neighbours, as before, so the sharpen steepens edges without halos beside them.
RX_VK_UPSCALE_LOG's line says what the number means now (sharpness 40% (of RCAS's strongest, neighbours one source pixel apart)).
fg_scale_comp_spv.h regenerated as the other headers are: glslangValidator -V --target-env vulkan1.2 --vn FG_SCALE_COMP_SPV (glslang 16.6.0, which reproduces main's header byte for byte before the change).

Measured

Headless 3840 x 2160 output, the default 2048 x 1152 picture, Catmull-Rom, frame generation off, resolution 100%, Kokiri Forest, same save each run. Metric: mean |Laplacian| of luma, at window scale and at picture scale (the shot box-averaged back to 2048 x 1152). In-game shots, 3 per run; Sharpness 0 is the average of three runs (0.0339 / 0.0725), run-to-run noise about 3%:

Sharpness	main, window	this PR, window	main, picture	this PR, picture
40%	+14%	+11%	+10%	+7%
80%	+24%	+29%	+16%	+17%
100%	+27%	+39%	+17%	+20%

Since animation moves things a little between runs, the same numbers were also taken on one frame: the game's 2048 x 1152 picture from a run, put through main's and this PR's shader math on the CPU, so every variant sees the same input.

One frame, 2048 x 1152 -> 3840 x 2160	40%	80%	100%
main, window \|Laplacian\|	+13%	+23%	+27%
this PR, window \|Laplacian\|	+11%	+26%	+35%
main, pixels newly clipped to 0 or 255	0.037%	0.057%	0.065%
this PR, pixels newly clipped	0.0004%	0.0010%	0.0012%
main, edge pixels past the unsharpened 5x5 light range by more than 1/255	0.010%	0.023%	0.025%
this PR, same	0.005%	0.022%	0.030%

At this window size main's sharpen did work: its neighbours went through the linear sampler, which blends the picture pixels around them. The gap grows with the window-to-picture ratio. The same frame at 1280 x 720 on 3840 x 2160 (3x): main +11 / +21 / +27% at 40 / 80 / 100%, this PR +20 / +59 / +87% (picture scale +7 / +13 / +15% against +10 / +20 / +24%). With Upscaling Off (the linear filter) at 2048 x 1152: main +16 / +30 / +36%, this PR +16 / +39 / +54%. Neither adds halos (both hold the result to the neighbourhood), and the RCAS lobe no longer pushes pixels next to black or white into clipping.

For players following the DLSS advice to set Sharpness to about 80%: main's 80% at 2048 x 1152 on a 4K screen is about this PR's 70 to 80% (70% by window-scale detail, 80% by picture-scale detail). About 75% is a good default for that advice after this change, and the value now carries over between window sizes.

Tests

tests/test_fg_scale_sharpen.py (in run_all.py): the sharpen's neighbour spacing is the picture's size; the shader's own sharpen() compiled as C++ against 200,000 random neighbourhoods at 0 / 10 / 40 / 80 / 100%: unchanged at 0, never outside the pixel's neighbourhood or 0..1, flat areas and black or white left as they are, and a blurred edge steeper as the setting rises. With glslangValidator present the shader compiles, and with the version the header names the header matches its output. python -I -B tests/run_all.py: 150 passed, 0 failed, 7 skipped (Linux). Linux build of vk + game.

Credits

The finding, and the idea of an RCAS-style sharpen here, come from #1071. The sharpen follows the math of AMD FidelityFX RCAS (MIT license), written from its description, not copied.

Presets and existing settings

The Quality presets' Sharpness values were tuned for the old spacing. Each is re-tuned to give about the same sharpening as before on a 3840 x 2160 window at that preset's picture (2048 x 1152 docked, 1536 x 864 with the Handheld profile; Render resolution does not change the picture the window pass sees), on the slider's steps: Minimum 50 -> 30, Low 35 -> 30, Balanced 25 -> 30, High 30 -> 40, Ultra 40 -> 50. A Sharpness already saved in hookshot_options.ini is kept as it is: it is stronger than before when the window is much larger than the picture (most of all with the Handheld profile on a 4K screen), and the readme says so.

The window pass's SPIR-V header now starts with the sha256 of the fg_scale.comp it was made from, so the test checks it against the shader in CI as well (python -I -B tests/test_fg_scale_sharpen.py --write makes it).

## Sharpness: neighbours one picture pixel apart, contrast-adaptive in the manner of AMD FidelityFX RCAS The Insert menu's Sharpness runs in the window pass (`gpu/vk/shaders/fg_scale.comp`, the only sharpen on the Vulkan path: direct presents and frame generation's made frames both go through `present_image` -> `framegen_scale`, same shader, same push constant). It took its four neighbours `1.0 / vec2(d)` apart, one window pixel. When the window is larger than the picture that is a fraction of a picture pixel (0.53 for the 2048 x 1152 picture on a 3840 x 2160 screen, 0.33 for 1280 x 720), so what a given setting did depended on the window's size rather than on the picture. This was found in #1071, which makes the same change in its copy of the shader; this PR is a separate implementation of it, on main, with measurements. ### What changes - The neighbours are one picture pixel apart (`1.0 / vec2(textureSize(src, 0))`), or one window pixel when the window is the smaller of the two, so a downscale does not sharpen detail the window cannot show. - The sharpen is a contrast-adaptive one in the manner of AMD FidelityFX RCAS (written from RCAS's published math): a negative lobe on the four neighbours, the largest that keeps the result inside 0..1 for that neighbourhood (smaller near black and white), capped at RCAS's maximum (0.25 - 1/16), then normalized. The slider maps 0..100% onto 0..that maximum. - The result is then held to the lightest and darkest of the pixel and its neighbours, as before, so the sharpen steepens edges without halos beside them. - `RX_VK_UPSCALE_LOG`'s line says what the number means now (`sharpness 40% (of RCAS's strongest, neighbours one source pixel apart)`). - `fg_scale_comp_spv.h` regenerated as the other headers are: `glslangValidator -V --target-env vulkan1.2 --vn FG_SCALE_COMP_SPV` (glslang 16.6.0, which reproduces main's header byte for byte before the change). ### Measured Headless 3840 x 2160 output, the default 2048 x 1152 picture, Catmull-Rom, frame generation off, resolution 100%, Kokiri Forest, same save each run. Metric: mean |Laplacian| of luma, at window scale and at picture scale (the shot box-averaged back to 2048 x 1152). In-game shots, 3 per run; Sharpness 0 is the average of three runs (0.0339 / 0.0725), run-to-run noise about 3%: | Sharpness | main, window | this PR, window | main, picture | this PR, picture | |---|---|---|---|---| | 40% | +14% | +11% | +10% | +7% | | 80% | +24% | +29% | +16% | +17% | | 100% | +27% | +39% | +17% | +20% | Since animation moves things a little between runs, the same numbers were also taken on one frame: the game's 2048 x 1152 picture from a run, put through main's and this PR's shader math on the CPU, so every variant sees the same input. | One frame, 2048 x 1152 -> 3840 x 2160 | 40% | 80% | 100% | |---|---|---|---| | main, window \|Laplacian\| | +13% | +23% | +27% | | this PR, window \|Laplacian\| | +11% | +26% | +35% | | main, pixels newly clipped to 0 or 255 | 0.037% | 0.057% | 0.065% | | this PR, pixels newly clipped | 0.0004% | 0.0010% | 0.0012% | | main, edge pixels past the unsharpened 5x5 light range by more than 1/255 | 0.010% | 0.023% | 0.025% | | this PR, same | 0.005% | 0.022% | 0.030% | At this window size main's sharpen did work: its neighbours went through the linear sampler, which blends the picture pixels around them. The gap grows with the window-to-picture ratio. The same frame at 1280 x 720 on 3840 x 2160 (3x): main +11 / +21 / +27% at 40 / 80 / 100%, this PR +20 / +59 / +87% (picture scale +7 / +13 / +15% against +10 / +20 / +24%). With Upscaling Off (the linear filter) at 2048 x 1152: main +16 / +30 / +36%, this PR +16 / +39 / +54%. Neither adds halos (both hold the result to the neighbourhood), and the RCAS lobe no longer pushes pixels next to black or white into clipping. For players following the DLSS advice to set Sharpness to about 80%: main's 80% at 2048 x 1152 on a 4K screen is about this PR's 70 to 80% (70% by window-scale detail, 80% by picture-scale detail). About 75% is a good default for that advice after this change, and the value now carries over between window sizes. ### Tests `tests/test_fg_scale_sharpen.py` (in `run_all.py`): the sharpen's neighbour spacing is the picture's size; the shader's own `sharpen()` compiled as C++ against 200,000 random neighbourhoods at 0 / 10 / 40 / 80 / 100%: unchanged at 0, never outside the pixel's neighbourhood or 0..1, flat areas and black or white left as they are, and a blurred edge steeper as the setting rises. With glslangValidator present the shader compiles, and with the version the header names the header matches its output. `python -I -B tests/run_all.py`: 150 passed, 0 failed, 7 skipped (Linux). Linux build of vk + game. ### Credits The finding, and the idea of an RCAS-style sharpen here, come from #1071. The sharpen follows the math of AMD FidelityFX RCAS (MIT license), written from its description, not copied. ### Presets and existing settings The Quality presets' Sharpness values were tuned for the old spacing. Each is re-tuned to give about the same sharpening as before on a 3840 x 2160 window at that preset's picture (2048 x 1152 docked, 1536 x 864 with the Handheld profile; Render resolution does not change the picture the window pass sees), on the slider's steps: Minimum 50 -> 30, Low 35 -> 30, Balanced 25 -> 30, High 30 -> 40, Ultra 40 -> 50. A Sharpness already saved in hookshot_options.ini is kept as it is: it is stronger than before when the window is much larger than the picture (most of all with the Handheld profile on a 4K screen), and the readme says so. The window pass's SPIR-V header now starts with the sha256 of the fg_scale.comp it was made from, so the test checks it against the shader in CI as well (`python -I -B tests/test_fg_scale_sharpen.py --write` makes it).

hckrman101 (Migrated from github.com) reviewed

2026-11-09 19:22:55 +00:00

hckrman101 (Migrated from github.com) left a comment

Self-review notes.

Checked and fine:

0% is exactly main: pc.sharp > 0.0 still gates the whole block, and sharpen() at amount 0 returns c bit for bit anyway (lobe -0, divide by 1, clamp no-op).
Push constants unchanged ({int mode; float sharp;}, 8 bytes on both sides); the C++ change is only the log line.
Spacing: the crop is blitted into scale.cropped before the pass, so textureSize(src, 0) is the cropped picture and 1/size is one picture pixel in uv for any ratio, per axis (non-integer and ultrawide fine). Downscale falls back to one window pixel. CLAMP_TO_EDGE linear sampler handles borders.
RCAS maths: lo/hi ring, (1-hi)/(4-4*mn4) white room, min over channels, lobe capped at 0.25-1/16, (c + lobe*sum)/(1+4*lobe); the 1/1024 floors keep it finite at black/white. Same 4 taps as before, a few extra ALU ops: no perf concern.
Still sharpens UNORM (display-encoded) values, as before and as RCAS normally runs. Same shader/push constant for direct present and frame generation, all three Upscaling modes.
New test passes on both CI runners; the two Python failures are main's own (test_output_resolution, test_program_cache_cleanup).

Worth a look before merge:

Presets were tuned against the old window-pixel spacing (Minimum = 50%, Low = 35% at reduced resolution). With large window/picture ratios the same number is now much stronger (the 3x numbers in the description: +87% vs +27% at 100%). Either re-tune apply_preset() in vk_menu.cpp or call out in the release notes that existing Sharpness values get stronger at low Resolution.
The header check only compares bytes when the local glslang's version line matches the header's (// 1116.6.0), so CI (no matching glslang) never verifies fg_scale_comp_spv.h is the .comp compiled. Fine as-is since it reproduced main byte for byte locally, but the header is effectively unchecked in CI.
Minor deviations from RCAS noted inline (to_black denominator, saturated colours); neither is a bug.

#1166 (closed as duplicate of #1007) is about the game's own sharpening at native 4K, not this slider, so this PR neither fixes nor worsens it.

Self-review notes. Checked and fine: - 0% is exactly main: `pc.sharp > 0.0` still gates the whole block, and `sharpen()` at amount 0 returns `c` bit for bit anyway (lobe -0, divide by 1, clamp no-op). - Push constants unchanged (`{int mode; float sharp;}`, 8 bytes on both sides); the C++ change is only the log line. - Spacing: the crop is blitted into `scale.cropped` before the pass, so `textureSize(src, 0)` is the cropped picture and 1/size is one picture pixel in uv for any ratio, per axis (non-integer and ultrawide fine). Downscale falls back to one window pixel. CLAMP_TO_EDGE linear sampler handles borders. - RCAS maths: lo/hi ring, `(1-hi)/(4-4*mn4)` white room, min over channels, lobe capped at 0.25-1/16, `(c + lobe*sum)/(1+4*lobe)`; the 1/1024 floors keep it finite at black/white. Same 4 taps as before, a few extra ALU ops: no perf concern. - Still sharpens UNORM (display-encoded) values, as before and as RCAS normally runs. Same shader/push constant for direct present and frame generation, all three Upscaling modes. - New test passes on both CI runners; the two Python failures are main's own (test_output_resolution, test_program_cache_cleanup). Worth a look before merge: 1. Presets were tuned against the old window-pixel spacing (Minimum = 50%, Low = 35% at reduced resolution). With large window/picture ratios the same number is now much stronger (the 3x numbers in the description: +87% vs +27% at 100%). Either re-tune `apply_preset()` in vk_menu.cpp or call out in the release notes that existing Sharpness values get stronger at low Resolution. 2. The header check only compares bytes when the local glslang's version line matches the header's (`// 1116.6.0`), so CI (no matching glslang) never verifies fg_scale_comp_spv.h is the .comp compiled. Fine as-is since it reproduced main byte for byte locally, but the header is effectively unchecked in CI. 3. Minor deviations from RCAS noted inline (to_black denominator, saturated colours); neither is a bug. #1166 (closed as duplicate of #1007) is about the game's own sharpening at native 4K, not this slider, so this PR neither fixes nor worsens it.

gpu/vk/shaders/fg_scale.comp

					
				@ -39,2 +56,4 @@

				    return clamp(sharp, lo, hi);

				}

				void main() {

				    const ivec2 g = ivec2(gl_GlobalInvocationID.xy);

RCAS divides by 4*mx4 (neighbours only); this uses hi, which includes c. That is never larger room than RCAS, so only more conservative, and the final clamp to [lo,hi] already guarantees 0..1. Fine, just not literally RCAS; maybe say so in the comment.

RCAS divides by `4*mx4` (neighbours only); this uses `hi`, which includes `c`. That is never larger room than RCAS, so only more conservative, and the final clamp to [lo,hi] already guarantees 0..1. Fine, just not literally RCAS; maybe say so in the comment.

gpu/vk/shaders/fg_scale.comp

					
				@ -43,11 +62,13 @@ void main() {

				    const vec2 uv = (vec2(g) + 0.5) / vec2(d);

Taking the min room over r/g/b means any pixel with a channel near 0 or 1 (saturated colours, not just black/white) gets almost no lobe. RCAS does the same, so it's faithful, but it explains why the effect is weaker on strongly coloured areas than the luma metric suggests.

gpu/vk/shaders/fg_scale.comp

Checked: src is the cropped picture when a crop is requested (blit into scale.cropped first), so this is one picture pixel per axis at any ratio. Neighbours are bilinear at the output pixel's sub-texel phase while c is Catmull-Rom/Lanczos, same as main; no new issue.

Checked: `src` is the cropped picture when a crop is requested (blit into scale.cropped first), so this is one picture pixel per axis at any ratio. Neighbours are bilinear at the output pixel's sub-texel phase while `c` is Catmull-Rom/Lanczos, same as main; no new issue.

tests/test_fg_scale_sharpen.py

					
				@ -0,0 +1,182 @@

				"""The window pass's Sharpness (gpu/vk/shaders/fg_scale.comp): its neighbours are one picture (source) pixel apart, not one

With a different glslang this only checks that the shader compiles, so CI never compares the committed header against the source. Acceptable, but it means the .h is trusted from the local regen.

hckrman101 commented

2026-11-09 20:04:48 +00:00

(Migrated from github.com)

Presets: re-tuned in 488ec95. The window-pass picture is the output (2048 x 1152, or 1536 x 864 with the Handheld profile; Resolution % only changes the 3D inside it), so I matched each preset on a 3840 x 2160 window at its own picture: one game frame through main's and this PR's shader maths on the CPU, mean |Laplacian| of the window and of the picture (box-averaged back), the old value against this PR's on the slider's steps. Minimum 50 -> 30, Low 35 -> 30, Balanced 25 -> 30, High 30 -> 40, Ultra 40 -> 50 (the old handheld presets drop, the docked ones rise a step, since RCAS's lobe at low percentages is gentler than the old linear amount). On a 1920 x 1080 window the same values are a little softer than main's (0.6 to 0.7 of main's gain for the handheld presets, about 0.75 to 0.9 for the docked ones), which I preferred to over-sharpening 4K. Existing ini values are left as they are; the readme now says a Sharpness kept from an older version is stronger when the window is much larger than the picture.
Header check: fg_scale_comp_spv.h's first line is now the sha256 of the fg_scale.comp it was made from (line ends normalised to LF), and the test checks it against the shader as checked out, with or without glslang, so CI fails when the .comp changes without the header. With a matching glslang the rest of the header is still compared byte for byte. python -I -B tests/test_fg_scale_sharpen.py --write makes the header (glslang's output plus that line). In b48cb14.
RCAS comments: the shader now says where it differs: the room to black divides by 4 x the lightest of the pixel and its neighbours (RCAS: neighbours only), never more room than RCAS, so only more careful; and like RCAS the room is the smallest of the three channels, so saturated colours (a channel near 0 or 1) are sharpened little. Also b48cb14.
Merged newest main (4a4a187), which carries the fixes for test_output_resolution and test_program_cache_cleanup.

1. Presets: re-tuned in 488ec95. The window-pass picture is the output (2048 x 1152, or 1536 x 864 with the Handheld profile; Resolution % only changes the 3D inside it), so I matched each preset on a 3840 x 2160 window at its own picture: one game frame through main's and this PR's shader maths on the CPU, mean |Laplacian| of the window and of the picture (box-averaged back), the old value against this PR's on the slider's steps. Minimum 50 -> 30, Low 35 -> 30, Balanced 25 -> 30, High 30 -> 40, Ultra 40 -> 50 (the old handheld presets drop, the docked ones rise a step, since RCAS's lobe at low percentages is gentler than the old linear amount). On a 1920 x 1080 window the same values are a little softer than main's (0.6 to 0.7 of main's gain for the handheld presets, about 0.75 to 0.9 for the docked ones), which I preferred to over-sharpening 4K. Existing ini values are left as they are; the readme now says a Sharpness kept from an older version is stronger when the window is much larger than the picture. 2. Header check: fg_scale_comp_spv.h's first line is now the sha256 of the fg_scale.comp it was made from (line ends normalised to LF), and the test checks it against the shader as checked out, with or without glslang, so CI fails when the .comp changes without the header. With a matching glslang the rest of the header is still compared byte for byte. `python -I -B tests/test_fg_scale_sharpen.py --write` makes the header (glslang's output plus that line). In b48cb14. 3. RCAS comments: the shader now says where it differs: the room to black divides by 4 x the lightest of the pixel and its neighbours (RCAS: neighbours only), never more room than RCAS, so only more careful; and like RCAS the room is the smallest of the three channels, so saturated colours (a channel near 0 or 1) are sharpened little. Also b48cb14. 4. Merged newest main (4a4a187), which carries the fixes for test_output_resolution and test_program_cache_cleanup.

Commenting is not possible because the repository is archived.

No reviewers