fix(vk): widen RGB rows with each format's own alpha one #923
No reviewers
Labels
No labels
area/accessibility
area/audio
area/input
area/install
area/menu
area/renderer
area/saves
area/tooling
area/upscaling
beanstalk: approved
beanstalk: failed
beanstalk: needs approval
beanstalk: running
beanstalk: tested
ci: approved
good first issue
gpu/amd
gpu/igpu
gpu/intel
gpu/nvidia
help wanted
kind
bug
kind
chore
kind
docs
kind
feature
kind
performance
kind
question
needs measurements
platform/linux
platform/macos
platform/steam-deck
platform/windows
preview
priority
critical
priority
high
priority
low
regression
risk
high
risk
low
risk
medium
status
blocked
status
confirmed
status
duplicate
status
invalid
status
needs-info
status
wontfix
ci: approved
No milestone
No project
No assignees
3 participants
Notifications
Due date
No due date set.
Dependencies
No dependencies set
Reference
MagicBeanDev/Longshot!923
Loading…
Reference in a new issue
No description provided.
Delete branch "jcabarca/Longshot:fix/vk-widen-alpha"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Type: fix · Risk: low · Tested on: Windows, AMD Radeon RX 9070 XT · Needs: none
Why
RGB textures that are widened to RGBA for upload got an alpha chosen by component size alone, so several formats read a wrong alpha: R8G8B8_SNORM -0.008, R16G16B16_UNORM 0.23, R16G16B16_SNORM 0.47, and the integer formats 255, 15360 or 0x3F800000 instead of 1. Anything the game blends or tests with that alpha comes out wrong.
How to reproduce
Seen on: not seen in the game: found by reading
widen_rows()ingpu/vk/vk_texture.cpp; it happens for every texture in one of those RGB formats on a device that cannot sample the three-component format (most devices)Settings and switches: defaults
Steps: game-free:
python -I -B tests/test_widen_alpha.py --cxx clang++ --headers <Vulkan-Headers include>with--sourcepointing at a main checkout (the test from this PR); it compileswiden_rows()out of the source and checks the alpha bytes of the twelve widened formatsExpected: each widened texel gets one in its own format as alpha
Actual: on main there is no per-format alpha (
widen_alpha(VkFormat)does not exist); the test failsWhat changed
A new
widen_alpha(VkFormat)returns the bit pattern of one for each formatwidened()widens, andwiden_rows()writes that pattern instead of picking one by component size.gpu/vk/vk_texture.cpp:widen_alpha(VkFormat)(new): 0x7F for R8G8B8_SNORM, 1 for the UINT and SINT formats, 0x3C00 for R16G16B16_SFLOAT, 0xFFFF for R16G16B16_UNORM, 0x7FFF for R16G16B16_SNORM, 0x3F800000 for R32G32B32_SFLOAT, 0xFF otherwise (R8G8B8_UNORM and SRGB).widen_rows()takes the alpha as an argument and copies its lowto - frombytes after each texel (little endian).upload_inner()keeps the source format insrc_vfand passeswiden_alpha(src_vf).tests/test_widen_alpha.py(new): compileswiden_alpha()andwiden_rows()out of the source and checks all twelve formats.The bug, step by step:
upload_inner()askswidened()for an RGBA format and widens each row withwiden_rows().widen_rows()chose the alpha by component size alone:0xFFfor 1-byte components, half-float 1.0 (0x3C00) for 2-byte ones, float 1.0 for 4-byte ones.This breaks no Vulkan rule: the bytes are valid, only the alpha they hold is wrong.
Before:
After:
Measurements
Verdict: it fixes the bug:
tests/test_widen_alpha.pyfails on main (no per-format alpha) and passes with this PR, in both interleaved runs. No speed change is expected: onememcpyof the alpha per texel instead of a branch and a store.Hardware: AMD Radeon RX 9070 XT, Windows; the test itself is game-free (it compiles a cut-out of the source with clang++ from llvm-mingw and runs it on the CPU); Python 3.12
Scene: none, game-free
Method:
tests/test_widen_alpha.pyfrom this PR with--sourceset to each side's checkout, 2 runs each, interleaved (main, PR, main, PR); the result is pass/fail, so the median is the result both runs gaveHow to reproduce:
python -I -B tests/test_widen_alpha.py --cxx clang++ --headers <Vulkan-Headers include> --source <checkout>for each sideb8d2b04etests/test_widen_alpha.py, run 1tests/test_widen_alpha.py, run 2What the test checks:
vk_texture.cpphasuint32_t widen_alpha(VkFormat f)andupload_inner()callswiden_alpha(src_vf).widen_alpha()andwiden_rows()widens two texels of each of the twelve formats into a buffer pre-filled with0xEE, and checks that the RGB bytes are copied unchanged and the alpha is exactly that format's one.Flags and settings
None.
Testing
Tested: on this branch, Windows, Python 3.12, clang++ from llvm-mingw:
test_widen_alphapasses (fails on main), and the other checks that readwidened()pass:test_image_view_retry,test_pipeline_state_portable.Built and played: this commit builds on Windows (llvm-mingw back end) and on Linux (
longshot_vk.so, without and with AMD FSR 4). Played on Windows, AMD Radeon RX 9070 XT: the game runs normally.Not tested: a texture in one of the affected formats looked at in the game: not checked in the game (which textures use them was not traced). Linux was built, not run.
Must be tested: None: the change is the same on every GPU and OS, and the game-free test covers it.
Risks
What could break: textures in the affected formats now read alpha one where they read the wrong value before; if a scene happened to look right with the wrong alpha it now changes, but one is what the format's missing component means. R8G8B8_UNORM/SRGB, R16G16B16_SFLOAT and R32G32B32_SFLOAT write the same bytes as before.
Turning it off: revert the commit.
Compatibility: Nothing changes.
Checklist
Everyone
type(scope): description) of the status line's type.can't be run by hand is in Testing's Not testable by hand, with why.
in Risks.
main.python -I -B tests/run_all.pypasses (any failure that also fails onmainis named). (the checks that read the changed code, on this PR, Windows:test_widen_alpha,test_image_view_retry,test_pipeline_state_portable, all pass)third_party/untouched (a new folder with aPROVENANCE.mdinstead).When it applies
Assisted-by:/Co-Authored-By:trailers.mainand passing here. (tests/test_widen_alpha.py)tests/that reads changed source updated in this PR, not weakened. (tests/test_widen_alpha.pyadded; no check removed)longshot_vk.sowithout and with AMD FSR 4; no OS-specific code or paths)WIP: fix(vk): widen RGB rows with each format's own alpha oneto fix(vk): widen RGB rows with each format's own alpha oneCode-only review of f045f945db045bb0a6ce914da3c38beb23c27510; informational findings, with final approval left to another reviewer.
No new correctness defect found in the diff. The alpha encodings cover all twelve formats currently handled by
widened(), and the 1/2/4-byte copies match their component widths on the supported x86-64 targets. The source format is preserved before widening; GPU unswizzle and the all-zero clear shortcut already exclude widened formats, so those routes do not bypass this correction.The added fixture checks real output bytes for two texels of every supported widened format, including preservation of RGB. Reverting a corrected alpha value would fail those cases. Its failure on main is a missing-helper check rather than an execution of the old faulty algorithm, so that baseline result alone is not a before/after rendering measurement. The reported Windows RX 9070 XT playthrough and Windows/Linux builds are useful smoke coverage; an affected texture was not traced in-game, as the description states.
Read the full diff, upload/creation paths, test and related texture-fix diffs. The ASTC, sparse-unswizzle and packed-format fixes in 920, 925 and 924 address distinct paths; no dependency on them was found. No tests, builds or game runs performed for this review.
Brought up to the ready-for-review standard (#1454): the checklist's Everyone list has two new boxes, added here from what this PR's description and comments show (please check they match what you know).
Rebased onto main @
7daafa7a; the related tests pass on it.c415a10738ded2aa72efCode review of
ded2aa72(read only, nothing run).From the code and the evidence, this looks ready.
Correctness.
widen_alpha()covers all twelve formatswidened()widens, and each one gets its own "one": UNORM8 0xFF, SNORM8 0x7F, UNORM16 0xFFFF, SNORM16 0x7FFF, half 0x3C00, float 0x3F800000, and 1 for every UINT/SINT. Thememcpyof the lowto - frombytes (1, 2 or 4) matches the component width, and it is right on every target we build (x86-64 and ARM64 are both little endian).src_vfis the same expressionwidened()was given before, so no other texture takes the new path. The vertex-format widening invk_draw.cppis a separate path, and this PR doesn't touch it.Cost. Only RGB textures take this loop, and it runs after the unswizzle, which costs far more. Swapping a well-predicted branch and a constant store for a second
memcpywhose size is only known at run time won't matter there. The claim "no speed change" comes from reasoning, not a measurement, and that is fine for a path this narrow.Test. The fixture runs the real
widen_alpha()+widen_rows()from the source. It checks both the RGB bytes and the alpha bytes against a table of its own, so reverting any single format's value fails it. (On main it fails at the missing-helper text check, as already noted above.)Suggestions (non-blocking, also inline):
gpu/vk/vk_texture.cpp:1101:default: return 0xFFgives the wrong alpha, with no warning, to a format added towidened()later (say a USCALED/SSCALED one). Acase VK_FORMAT_R8G8B8_UNORM:label in front of the default, or a debug assert thatwidened(f, &from)givesfrom == 3there, would catch that.gpu/vk/vk_texture.cpp:1108: the comment says little endian. Astatic_assert(std::endian::native == std::endian::little)(from<bit>) would make the code enforce that assumption.tests/test_widen_alpha.py:21: AGENTS.md asks new checks to cut source withtests/srccut.py(block), which gives one clear message when a marker is missing or appears twice. This file has its own brace counter instead.Measurements: for a fix, the evidence is a game-free test that fails on main and passes here (two interleaved runs each, Windows, RX 9070 XT). The change is the same on every GPU and OS, so the missing hardware coverage doesn't matter. Not shown: a texture in an affected format checked in the game. The description says so.
@ -1092,0 +1098,4 @@case VK_FORMAT_R16G16B16_UNORM: return 0xFFFF;case VK_FORMAT_R16G16B16_SNORM: return 0x7FFF;case VK_FORMAT_R32G32B32_SFLOAT: return 0x3F800000;default: return 0xFF;Nit: if
widened()gains a format later, this default gives it 0xFF without warning. Suggestcase VK_FORMAT_R8G8B8_UNORM: default: return 0xFF;plus a debug assert thatwidened(f, &from)givesfrom == 3here, or keep the alpha next to the format inwidened()itself.@ -1095,3 +1108,1 @@if (comp == 1) a[0] = 0xFF;else if (comp == 2) a[0] = 0x00, a[1] = 0x3C; // (half 1.0; UNORM16 would want 0xFFFF: close enough for alpha)else { const float one = 1.0f; std::memcpy(a, &one, 4); }std::memcpy(dst + i * to + from, &alpha, n); // (little endian: the low n bytes)Nit:
static_assert(std::endian::native == std::endian::little);(from<bit>) would make the code enforce the "low n bytes" assumption rather than leave it to the comment.@ -0,0 +18,4 @@ROOT = Path(__file__).resolve().parents[1]def block(text, marker):Nit: AGENTS.md asks new checks to cut with
tests/srccut.py(from srccut import blockaftersys.path.insert(0, str(Path(__file__).resolve().parent))) rather than a brace counter of their own; it gives one clear message when a marker is missing or appears twice.mooki referenced this pull request2026-11-19 08:23:56 +00:00