fix(vk): widen RGB rows with each format's own alpha one #923
No reviewers
Labels
No labels
area/accessibility
area/audio
area/input
area/install
area/menu
area/renderer
area/saves
area/tooling
area/upscaling
beanstalk: approved
beanstalk: failed
beanstalk: needs approval
beanstalk: running
beanstalk: tested
ci: approved
good first issue
gpu/amd
gpu/igpu
gpu/intel
gpu/nvidia
help wanted
kind
bug
kind
chore
kind
docs
kind
feature
kind
performance
kind
question
needs measurements
platform/linux
platform/macos
platform/steam-deck
platform/windows
preview
priority
critical
priority
high
priority
low
regression
risk
high
risk
low
risk
medium
status
blocked
status
confirmed
status
duplicate
status
invalid
status
needs-info
status
wontfix
ci: approved
No milestone
No project
No assignees
2 participants
Notifications
Due date
No due date set.
Dependencies
No dependencies set
Reference
MagicBeanDev/Longshot!923
Loading…
Reference in a new issue
No description provided.
Delete branch "jcabarca/Longshot:fix/vk-widen-alpha"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Type: fix · Risk: low · Tested on: Windows, AMD Radeon RX 9070 XT · Needs: none
Why
RGB textures that are widened to RGBA for upload got an alpha chosen by component size alone, so several formats read a wrong alpha: R8G8B8_SNORM -0.008, R16G16B16_UNORM 0.23, R16G16B16_SNORM 0.47, and the integer formats 255, 15360 or 0x3F800000 instead of 1. Anything the game blends or tests with that alpha comes out wrong.
How to reproduce
Seen on: not seen in the game: found by reading
widen_rows()ingpu/vk/vk_texture.cpp; it happens for every texture in one of those RGB formats on a device that cannot sample the three-component format (most devices)Settings and switches: defaults
Steps: game-free:
python -I -B tests/test_widen_alpha.py --cxx clang++ --headers <Vulkan-Headers include>with--sourcepointing at a main checkout (the test from this PR); it compileswiden_rows()out of the source and checks the alpha bytes of the twelve widened formatsExpected: each widened texel gets one in its own format as alpha
Actual: on main there is no per-format alpha (
widen_alpha(VkFormat)does not exist); the test failsWhat changed
A new
widen_alpha(VkFormat)returns the bit pattern of one for each formatwidened()widens, andwiden_rows()writes that pattern instead of picking one by component size.gpu/vk/vk_texture.cpp:widen_alpha(VkFormat)(new): 0x7F for R8G8B8_SNORM, 1 for the UINT and SINT formats, 0x3C00 for R16G16B16_SFLOAT, 0xFFFF for R16G16B16_UNORM, 0x7FFF for R16G16B16_SNORM, 0x3F800000 for R32G32B32_SFLOAT, 0xFF otherwise (R8G8B8_UNORM and SRGB).widen_rows()takes the alpha as an argument and copies its lowto - frombytes after each texel (little endian).upload_inner()keeps the source format insrc_vfand passeswiden_alpha(src_vf).tests/test_widen_alpha.py(new): compileswiden_alpha()andwiden_rows()out of the source and checks all twelve formats.The bug, step by step:
upload_inner()askswidened()for an RGBA format and widens each row withwiden_rows().widen_rows()chose the alpha by component size alone:0xFFfor 1-byte components, half-float 1.0 (0x3C00) for 2-byte ones, float 1.0 for 4-byte ones.This breaks no Vulkan rule: the bytes are valid, only the alpha they hold is wrong.
Before:
After:
Measurements
Verdict: it fixes the bug:
tests/test_widen_alpha.pyfails on main (no per-format alpha) and passes with this PR, in both interleaved runs. No speed change is expected: onememcpyof the alpha per texel instead of a branch and a store.Hardware: AMD Radeon RX 9070 XT, Windows; the test itself is game-free (it compiles a cut-out of the source with clang++ from llvm-mingw and runs it on the CPU); Python 3.12
Scene: none, game-free
Method:
tests/test_widen_alpha.pyfrom this PR with--sourceset to each side's checkout, 2 runs each, interleaved (main, PR, main, PR); the result is pass/fail, so the median is the result both runs gaveHow to reproduce:
python -I -B tests/test_widen_alpha.py --cxx clang++ --headers <Vulkan-Headers include> --source <checkout>for each sideb8d2b04ef045f945tests/test_widen_alpha.py, run 1tests/test_widen_alpha.py, run 2What the test checks:
vk_texture.cpphasuint32_t widen_alpha(VkFormat f)andupload_inner()callswiden_alpha(src_vf).widen_alpha()andwiden_rows()widens two texels of each of the twelve formats into a buffer pre-filled with0xEE, and checks that the RGB bytes are copied unchanged and the alpha is exactly that format's one.Flags and settings
None.
Testing
Tested: on this branch, Windows, Python 3.12, clang++ from llvm-mingw:
test_widen_alphapasses (fails on main), and the other checks that readwidened()pass:test_image_view_retry,test_pipeline_state_portable.Built and played: this commit builds on Windows (llvm-mingw back end) and on Linux (
longshot_vk.so, without and with AMD FSR 4). Played on Windows, AMD Radeon RX 9070 XT: the game runs normally.Not tested: a texture in one of the affected formats looked at in the game: not checked in the game (which textures use them was not traced). Linux was built, not run.
Must be tested: None: the change is the same on every GPU and OS, and the game-free test covers it.
Risks
What could break: textures in the affected formats now read alpha one where they read the wrong value before; if a scene happened to look right with the wrong alpha it now changes, but one is what the format's missing component means. R8G8B8_UNORM/SRGB, R16G16B16_SFLOAT and R32G32B32_SFLOAT write the same bytes as before.
Turning it off: revert the commit.
Compatibility: Nothing changes.
Checklist
Everyone
type(scope): description) of the status line's type.main.python -I -B tests/run_all.pypasses (any failure that also fails onmainis named). (the checks that read the changed code, on this PR, Windows:test_widen_alpha,test_image_view_retry,test_pipeline_state_portable, all pass)third_party/untouched (a new folder with aPROVENANCE.mdinstead).When it applies
Assisted-by:/Co-Authored-By:trailers.mainand passing here. (tests/test_widen_alpha.py)tests/that reads changed source updated in this PR, not weakened. (tests/test_widen_alpha.pyadded; no check removed)longshot_vk.sowithout and with AMD FSR 4; no OS-specific code or paths)WIP: fix(vk): widen RGB rows with each format's own alpha oneto fix(vk): widen RGB rows with each format's own alpha oneCode-only review of
f045f945db; informational findings, with final approval left to another reviewer.No new correctness defect found in the diff. The alpha encodings cover all twelve formats currently handled by
widened(), and the 1/2/4-byte copies match their component widths on the supported x86-64 targets. The source format is preserved before widening; GPU unswizzle and the all-zero clear shortcut already exclude widened formats, so those routes do not bypass this correction.The added fixture checks real output bytes for two texels of every supported widened format, including preservation of RGB. Reverting a corrected alpha value would fail those cases. Its failure on main is a missing-helper check rather than an execution of the old faulty algorithm, so that baseline result alone is not a before/after rendering measurement. The reported Windows RX 9070 XT playthrough and Windows/Linux builds are useful smoke coverage; an affected texture was not traced in-game, as the description states.
Read the full diff, upload/creation paths, test and related texture-fix diffs. The ASTC, sparse-unswizzle and packed-format fixes in 920, 925 and 924 address distinct paths; no dependency on them was found. No tests, builds or game runs performed for this review.
View command line instructions
Checkout
From your project repository, check out a new branch and test the changes.