fix(vk): name the view's format in custom border colours of packed 16-bit textures #1232
No reviewers
Labels
No labels
area/accessibility
area/audio
area/input
area/install
area/menu
area/renderer
area/saves
area/tooling
area/upscaling
beanstalk: approved
beanstalk: failed
beanstalk: needs approval
beanstalk: running
beanstalk: tested
ci: approved
good first issue
gpu/amd
gpu/igpu
gpu/intel
gpu/nvidia
help wanted
kind
bug
kind
chore
kind
docs
kind
feature
kind
performance
kind
question
needs measurements
platform/linux
platform/macos
platform/steam-deck
platform/windows
preview
priority
critical
priority
high
priority
low
regression
risk
high
risk
low
risk
medium
status
blocked
status
confirmed
status
duplicate
status
invalid
status
needs-info
status
wontfix
ci: approved
No milestone
No project
No assignees
3 participants
Notifications
Due date
No due date set.
Dependencies
No dependencies set
Reference
MagicBeanDev/Longshot!1232
Loading…
Reference in a new issue
No description provided.
Delete branch "jcabarca/Longshot:fix/vk-border-color-format"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Type: fix · Risk: low · Tested on: Windows, AMD Radeon RX 9070 XT · Needs: none
Why
A texture in a packed 16-bit format (NVN RGB565 or RGB5) sampled with a custom border colour uses a sampler whose custom border colour names no format, which Vulkan forbids for those formats (VUID-VkSamplerCustomBorderColorCreateInfoEXT-format-04015): the colour seen past the texture's edge is then undefined and can differ between drivers.
How to reproduce
Seen on: not seen in the game: found by reading
sampler_make()ingpu/vk/vk_draw.cpp; it happens on any device with custom border colours (customBorderColorsandcustomBorderColorWithoutFormat) whenever the game samples an RGB565 or RGB5 texture with a sampler whose border colour is not transparent black, opaque black or opaque whiteSettings and switches: defaults
Steps: game-free:
python -I -B tests/test_sampler_admission.py --cxx clang++ --headers <Vulkan-Headers include>with--sourcepointing at a main checkout (the test from this PR); its newborder-formatcase asks for a sampler for a view of each of the four packed formats and checks the format its custom border colour namesExpected: the custom border colour names the view's format, and its colour is within what that UNORM format holds
Actual: on main the custom border colour names
VK_FORMAT_UNDEFINEDfor every view; the test fails 3 of 153 cases (theborder-formatcase on each of its three routes)What changed
The sampler now knows the format of the view it samples. For the four packed 16-bit formats VUID 04015 names, its custom border colour names that format (and the colour is clamped to [0, 1], what a UNORM format holds); for every other format it stays
VK_FORMAT_UNDEFINED, as before.gpu/vk/vk_draw.cpp:border_format(VkFormat)(new): returns the view's format forB4G4R4A4_UNORM_PACK16,B5G6R5_UNORM_PACK16,A1B5G5R5_UNORM_PACK16andB5G5R5A1_UNORM_PACK16, elseVK_FORMAT_UNDEFINED.sampler_make(id, compare, view_format)andsampler_for(id, compare, view_format)take the view's format.sampler_make()mixes the border format into its cache key, setsVkSamplerCustomBorderColorCreateInfoEXT::formatto it and clamps the colour when it is set;sampler_for()'s per-thread table keeps the border format in each entry, so a sampler made for one view format is not handed to a view of another.SamplerInfo(the sampler book's key) records the custom border colour's format (19 to 20 words, still without padding).texture_view_of()gains aformat_outargument;HandleHitkeeps the view's format, so a cached hit returns it too.build_descriptors()passes each texture's view format tosampler_for()(repeated table entries reuse it; dummy views passVK_FORMAT_UNDEFINED).tests/sampler_admission_test.cppandtests/test_sampler_admission.py: aborder-formatcase on the three routes (by id,sampler_make()directly, the cache off) with a colour past [0, 1]; the fixture records the format each custom border colour names.The bug, step by step:
vk_formats.cppmapsNVN_FORMAT_RGB565toVK_FORMAT_B5G6R5_UNORM_PACK16andNVN_FORMAT_RGB5toVK_FORMAT_B5G5R5A1_UNORM_PACK16.vk_main.cppturns custom border colours on only withcustomBorderColorWithoutFormat(dev.custom_border_color).sampler_make()setsVK_BORDER_COLOR_FLOAT_CUSTOM_EXTwithbc.format = VK_FORMAT_UNDEFINED.build_descriptors()pairs that sampler with whatever view the handle names (sampler_for((int)(handle >> 20), d.is_depth)), including views of the formats in step 1. Every condition of 04015 holds.The rules:
VK_BORDER_COLOR_FLOAT_CUSTOM_EXT.VK_FORMAT_UNDEFINEDstays valid for every other view.Before:
After:
Measurements
Verdict: it fixes the bug:
tests/test_sampler_admission.pyfails 3 of 153 cases on main and passes 153 of 153 with this PR, in both interleaved runs. No speed change is expected: one more word in the sampler's key, and a new sampler only for a packed 16-bit view with a custom border colour.Hardware: AMD Radeon RX 9070 XT, Windows; the test itself is game-free (it compiles a cut-out of the source with clang++ from llvm-mingw and runs it on the CPU); Python 3.12
Scene: none, game-free
Method:
tests/test_sampler_admission.pyfrom this PR with--sourceset to each side's checkout, 2 runs each, interleaved (main, PR, main, PR); the result is pass/fail, so the median is the result both runs gaveHow to reproduce:
python -I -B tests/test_sampler_admission.py --cxx clang++ --headers <Vulkan-Headers include> --source <checkout>for each sideac778751e279a9b2tests/test_sampler_admission.py, run 1tests/test_sampler_admission.py, run 2What the test checks: for a view of each of the four packed formats, the custom border colour names that format and its colour {1.5, -0.5, 0.75, 1} becomes {1, 0, 0.75, 1}; for any other view it names
VK_FORMAT_UNDEFINEDand keeps the colour; a sampler made for one view format is not returned for a view of another.Flags and settings
None.
Testing
Tested: on this branch, Windows, Python 3.12, clang++ from llvm-mingw:
test_sampler_admissionpasses (fails on main), and the other checks that read the changed code pass:test_flat_map,test_handle_lookup,test_prof_symbols,test_taau_env_lookup(its seven cases pass; on Windows it then fails to delete its temporaryfixture.exe, the same on main).Built and played: this commit builds on Windows (llvm-mingw back end) and on Linux (
longshot_vk.so, without and with AMD FSR 4). Played on Windows, AMD Radeon RX 9070 XT: the game runs normally.Not tested: a packed 16-bit texture sampled past its edge with a custom border colour, looked at in the game (which textures and samplers the game pairs that way was not traced). Linux was built, not run.
Must be tested: None: the change is the same on every GPU and OS, and the game-free test covers it.
Risks
What could break: a packed 16-bit view with a custom border colour now gets its own sampler, so up to four more samplers can exist per custom border colour.
VUID-VkSamplerCreateInfo-None-04012limits how many samplers with custom border colours a device can have (maxCustomBorderColorSamplers); main does not check that limit either, and this PR does not change that. A border colour outside [0, 1] on those views now reads clamped, which is all the format can hold.Turning it off: revert the commit.
Compatibility: Nothing changes.
Checklist
Everyone
type(scope): description) of the status line's type.can't be run by hand is in Testing's Not testable by hand, with why.
in Risks. Not ticked yet: Risks names up to four extra samplers against a device limit that main does not check, and this is unmeasured.
main.python -I -B tests/run_all.pypasses (any failure that also fails onmainis named). (the checks that read the changed code, on this PR, Windows:test_sampler_admission,test_flat_map,test_handle_lookup,test_prof_symbols,test_taau_env_lookup; the last one's cleanup error on Windows also happens onmain)third_party/untouched (a new folder with aPROVENANCE.mdinstead).When it applies
Assisted-by:/Co-Authored-By:trailers.mainand passing here. (theborder-formatcase oftests/test_sampler_admission.py)tests/that reads changed source updated in this PR, not weakened. (tests/sampler_admission_test.cppfollows the new arguments; no case removed)longshot_vk.sowithout and with AMD FSR 4; no OS-specific code or paths)Tested in game: Played on Windows, AMD Radeon RX 9070 XT: the game runs normally.
WIP: fix(vk): name the view's format in custom border colours of packed 16-bit texturesto fix(vk): name the view's format in custom border colours of packed 16-bit texturesA packed 16-bit texture with a custom border colour now names that texture's format on the sampler. I read head
e279a9b2. Approved.Vulkan forbids a custom border colour with no format when the view is one of
B4G4R4A4,B5G6R5,A1B5G5R5orB5G5R5A1. The sampler was built withVK_FORMAT_UNDEFINEDfor every view.border_formatreturns the view's format for those four andVK_FORMAT_UNDEFINEDfor the rest, whichcustomBorderColorWithoutFormatstill allows. The colour is clamped to [0, 1] only when a format is named. The sampler cache, the per-thread table and the sampler book all keep that format, so a sampler made for one of those views is not handed to another.The view's format comes back from
texture_viewon the thread that made it, and a cached handle keeps the format from when the view was made. A missing view still asks for no format. The check compilessampler_makeand records the format the custom border colour names: each of the four formats, the clamp of {1.5, -0.5, 0.75, 1} to {1, 0, 0.75, 1}, and an ordinary view left undefined. It fails on main. The Windows play on an RX 9070 XT did not look at a packed texture past its edge.Brought up to the ready-for-review standard (#1454): the checklist's Everyone list has two new boxes, added here from what this PR's description and comments show (please check they match what you know).
View command line instructions
Manual merge helper
Use this merge commit message when completing the merge manually.
Checkout
From your project repository, check out a new branch and test the changes.