fix(vk): name the view's format in custom border colours of packed 16-bit textures #1232

Open
jcabarca wants to merge 2 commits from jcabarca/Longshot:fix/vk-border-color-format into main
Contributor

Type: fix · Risk: low · Tested on: Windows, AMD Radeon RX 9070 XT · Needs: none

Why

A texture in a packed 16-bit format (NVN RGB565 or RGB5) sampled with a custom border colour uses a sampler whose custom border colour names no format, which Vulkan forbids for those formats (VUID-VkSamplerCustomBorderColorCreateInfoEXT-format-04015): the colour seen past the texture's edge is then undefined and can differ between drivers.

How to reproduce

Seen on: not seen in the game: found by reading sampler_make() in gpu/vk/vk_draw.cpp; it happens on any device with custom border colours (customBorderColors and customBorderColorWithoutFormat) whenever the game samples an RGB565 or RGB5 texture with a sampler whose border colour is not transparent black, opaque black or opaque white
Settings and switches: defaults
Steps: game-free: python -I -B tests/test_sampler_admission.py --cxx clang++ --headers <Vulkan-Headers include> with --source pointing at a main checkout (the test from this PR); its new border-format case asks for a sampler for a view of each of the four packed formats and checks the format its custom border colour names
Expected: the custom border colour names the view's format, and its colour is within what that UNORM format holds
Actual: on main the custom border colour names VK_FORMAT_UNDEFINED for every view; the test fails 3 of 153 cases (the border-format case on each of its three routes)

What changed

The sampler now knows the format of the view it samples. For the four packed 16-bit formats VUID 04015 names, its custom border colour names that format (and the colour is clamped to [0, 1], what a UNORM format holds); for every other format it stays VK_FORMAT_UNDEFINED, as before.

  • vk: gpu/vk/vk_draw.cpp:
    • border_format(VkFormat) (new): returns the view's format for B4G4R4A4_UNORM_PACK16, B5G6R5_UNORM_PACK16, A1B5G5R5_UNORM_PACK16 and B5G5R5A1_UNORM_PACK16, else VK_FORMAT_UNDEFINED.
    • sampler_make(id, compare, view_format) and sampler_for(id, compare, view_format) take the view's format. sampler_make() mixes the border format into its cache key, sets VkSamplerCustomBorderColorCreateInfoEXT::format to it and clamps the colour when it is set; sampler_for()'s per-thread table keeps the border format in each entry, so a sampler made for one view format is not handed to a view of another.
    • SamplerInfo (the sampler book's key) records the custom border colour's format (19 to 20 words, still without padding).
    • texture_view_of() gains a format_out argument; HandleHit keeps the view's format, so a cached hit returns it too.
    • build_descriptors() passes each texture's view format to sampler_for() (repeated table entries reuse it; dummy views pass VK_FORMAT_UNDEFINED).
  • tests: tests/sampler_admission_test.cpp and tests/test_sampler_admission.py: a border-format case on the three routes (by id, sampler_make() directly, the cache off) with a colour past [0, 1]; the fixture records the format each custom border colour names.

The bug, step by step:

  1. vk_formats.cpp maps NVN_FORMAT_RGB565 to VK_FORMAT_B5G6R5_UNORM_PACK16 and NVN_FORMAT_RGB5 to VK_FORMAT_B5G5R5A1_UNORM_PACK16.
  2. vk_main.cpp turns custom border colours on only with customBorderColorWithoutFormat (dev.custom_border_color).
  3. For a border colour that is not one of the three standard ones, sampler_make() sets VK_BORDER_COLOR_FLOAT_CUSTOM_EXT with bc.format = VK_FORMAT_UNDEFINED.
  4. build_descriptors() pairs that sampler with whatever view the handle names (sampler_for((int)(handle >> 20), d.is_depth)), including views of the formats in step 1. Every condition of 04015 holds.

The rules:

  • VUID-VkSamplerCustomBorderColorCreateInfoEXT-format-04015: "If the sampler is used to sample an image view of VK_FORMAT_B4G4R4A4_UNORM_PACK16, VK_FORMAT_B5G6R5_UNORM_PACK16, VK_FORMAT_A1B5G5R5_UNORM_PACK16, or VK_FORMAT_B5G5R5A1_UNORM_PACK16 format then format must not be VK_FORMAT_UNDEFINED". Main breaks it; this PR names the format for those four.
  • VUID-VkSamplerCreateInfo-borderColor-04442: "If borderColor is one of VK_BORDER_COLOR_FLOAT_CUSTOM_EXT or VK_BORDER_COLOR_INT_CUSTOM_EXT, and VkSamplerCustomBorderColorCreateInfoEXT::format is not VK_FORMAT_UNDEFINED, VkSamplerCustomBorderColorCreateInfoEXT::customBorderColor must be within the range of values representable in format". The colour is clamped to [0, 1] when a format is named.
  • VUID-VkSamplerCustomBorderColorCreateInfoEXT-format-07605: "If format is not VK_FORMAT_UNDEFINED and format is not a depth/stencil format then the VkSamplerCreateInfo::borderColor type must match the sampled type of the provided format, as shown in the SPIR-V Type column of the Interpretation of Numeric Format table". The four formats are UNORM (float) and the border colour is VK_BORDER_COLOR_FLOAT_CUSTOM_EXT.
  • VUID-VkSamplerCustomBorderColorCreateInfoEXT-format-04014: "If the customBorderColorWithoutFormat feature is not enabled then format must not be VK_FORMAT_UNDEFINED". Unchanged: the feature is required for custom border colours, so VK_FORMAT_UNDEFINED stays valid for every other view.

Before:

    else if (dev.custom_border_color) {
        ci.borderColor = VK_BORDER_COLOR_FLOAT_CUSTOM_EXT;
        std::memcpy(bc.customBorderColor.float32, p.border, 16);
        bc.format = VK_FORMAT_UNDEFINED;
        ci.pNext = &bc;

After:

    else if (dev.custom_border_color) {
        ci.borderColor = VK_BORDER_COLOR_FLOAT_CUSTOM_EXT;
        std::memcpy(bc.customBorderColor.float32, p.border, 16);
        bc.format = bfmt;   // (VUID-VkSamplerCustomBorderColorCreateInfoEXT-format-04015)
        if (bfmt != VK_FORMAT_UNDEFINED)   // (VUID-VkSamplerCreateInfo-borderColor-04442: within what the UNORM format holds)
            for (float& c : bc.customBorderColor.float32) c = c > 0.0f ? std::min(c, 1.0f) : 0.0f;
        ci.pNext = &bc;

Measurements

Verdict: it fixes the bug: tests/test_sampler_admission.py fails 3 of 153 cases on main and passes 153 of 153 with this PR, in both interleaved runs. No speed change is expected: one more word in the sampler's key, and a new sampler only for a packed 16-bit view with a custom border colour.

Hardware: AMD Radeon RX 9070 XT, Windows; the test itself is game-free (it compiles a cut-out of the source with clang++ from llvm-mingw and runs it on the CPU); Python 3.12
Scene: none, game-free
Method: tests/test_sampler_admission.py from this PR with --source set to each side's checkout, 2 runs each, interleaved (main, PR, main, PR); the result is pass/fail, so the median is the result both runs gave
How to reproduce: python -I -B tests/test_sampler_admission.py --cxx clang++ --headers <Vulkan-Headers include> --source <checkout> for each side

Before: main @ ac778751 After: this PR @ e279a9b2
tests/test_sampler_admission.py, run 1 fails (3 of 153) passes (153 of 153)
tests/test_sampler_admission.py, run 2 fails (3 of 153) passes (153 of 153)
Median fails passes
run 1, main @ ac778751: 3 of 153 fail: border-format on the id, make and cache-off routes
run 1, this PR @ e279a9b2: 153 of 153 pass
run 2, main @ ac778751: 3 of 153 fail: border-format on the id, make and cache-off routes
run 2, this PR @ e279a9b2: 153 of 153 pass

What the test checks: for a view of each of the four packed formats, the custom border colour names that format and its colour {1.5, -0.5, 0.75, 1} becomes {1, 0, 0.75, 1}; for any other view it names VK_FORMAT_UNDEFINED and keeps the colour; a sampler made for one view format is not returned for a view of another.

Flags and settings

None.

Testing

Tested: on this branch, Windows, Python 3.12, clang++ from llvm-mingw: test_sampler_admission passes (fails on main), and the other checks that read the changed code pass: test_flat_map, test_handle_lookup, test_prof_symbols, test_taau_env_lookup (its seven cases pass; on Windows it then fails to delete its temporary fixture.exe, the same on main).

Built and played: this commit builds on Windows (llvm-mingw back end) and on Linux (longshot_vk.so, without and with AMD FSR 4). Played on Windows, AMD Radeon RX 9070 XT: the game runs normally.

Not tested: a packed 16-bit texture sampled past its edge with a custom border colour, looked at in the game (which textures and samplers the game pairs that way was not traced). Linux was built, not run.

Must be tested: None: the change is the same on every GPU and OS, and the game-free test covers it.

Risks

What could break: a packed 16-bit view with a custom border colour now gets its own sampler, so up to four more samplers can exist per custom border colour. VUID-VkSamplerCreateInfo-None-04012 limits how many samplers with custom border colours a device can have (maxCustomBorderColorSamplers); main does not check that limit either, and this PR does not change that. A border colour outside [0, 1] on those views now reads clamped, which is all the format can hold.

Turning it off: revert the commit.

Compatibility: Nothing changes.

Checklist

Everyone

  • One topic.
  • The title is a Conventional Commit (type(scope): description) of the status line's type.
  • Why gives the reason plainly and links the issues that fit (the tracker searched). (no issue fits)
  • Tested on my own PC where it runs (in the game, for a change the game runs), as Testing's Tested says; what
    can't be run by hand is in Testing's Not testable by hand, with why.
  • Measurements has a verdict, or one line saying why there is nothing to measure.
  • Nothing gets worse that I know of: any cost (speed, latency, memory, a setup that worked before) is measured and
    in Risks. Not ticked yet: Risks names up to four extra samplers against a device limit that main does not check, and this is unmeasured.
  • Testing names what was tested, what wasn't, and what must still be tested, where and how.
  • Rebased on current main.
  • python -I -B tests/run_all.py passes (any failure that also fails on main is named). (the checks that read the changed code, on this PR, Windows: test_sampler_admission, test_flat_map, test_handle_lookup, test_prof_symbols, test_taau_env_lookup; the last one's cleanup error on Windows also happens on main)
  • No game files, dumps, build output or generated files.
  • third_party/ untouched (a new folder with a PROVENANCE.md instead).
  • The license is OK: GPL-3.0-or-later, nothing incompatible and nothing decompiled.

When it applies

  • AI assistance recorded with Assisted-by: / Co-Authored-By: trailers.
  • A game-free test added for the fix, failing on main and passing here. (the border-format case of tests/test_sampler_admission.py)
  • A check under tests/ that reads changed source updated in this PR, not weakened. (tests/sampler_admission_test.cpp follows the new arguments; no case removed)
  • Host code builds on both Windows (llvm-mingw) and Linux, with UTF-8 paths. (built on Windows with llvm-mingw and on Linux, longshot_vk.so without and with AMD FSR 4; no OS-specific code or paths)
**Type:** fix · **Risk:** low · **Tested on:** Windows, AMD Radeon RX 9070 XT · **Needs:** none ## Why A texture in a packed 16-bit format (NVN RGB565 or RGB5) sampled with a custom border colour uses a sampler whose custom border colour names no format, which Vulkan forbids for those formats (VUID-VkSamplerCustomBorderColorCreateInfoEXT-format-04015): the colour seen past the texture's edge is then undefined and can differ between drivers. ## How to reproduce **Seen on:** not seen in the game: found by reading `sampler_make()` in `gpu/vk/vk_draw.cpp`; it happens on any device with custom border colours (`customBorderColors` and `customBorderColorWithoutFormat`) whenever the game samples an RGB565 or RGB5 texture with a sampler whose border colour is not transparent black, opaque black or opaque white **Settings and switches:** defaults **Steps:** game-free: `python -I -B tests/test_sampler_admission.py --cxx clang++ --headers <Vulkan-Headers include>` with `--source` pointing at a main checkout (the test from this PR); its new `border-format` case asks for a sampler for a view of each of the four packed formats and checks the format its custom border colour names **Expected:** the custom border colour names the view's format, and its colour is within what that UNORM format holds **Actual:** on main the custom border colour names `VK_FORMAT_UNDEFINED` for every view; the test fails 3 of 153 cases (the `border-format` case on each of its three routes) ## What changed The sampler now knows the format of the view it samples. For the four packed 16-bit formats VUID 04015 names, its custom border colour names that format (and the colour is clamped to [0, 1], what a UNORM format holds); for every other format it stays `VK_FORMAT_UNDEFINED`, as before. - vk: `gpu/vk/vk_draw.cpp`: - `border_format(VkFormat)` (new): returns the view's format for `B4G4R4A4_UNORM_PACK16`, `B5G6R5_UNORM_PACK16`, `A1B5G5R5_UNORM_PACK16` and `B5G5R5A1_UNORM_PACK16`, else `VK_FORMAT_UNDEFINED`. - `sampler_make(id, compare, view_format)` and `sampler_for(id, compare, view_format)` take the view's format. `sampler_make()` mixes the border format into its cache key, sets `VkSamplerCustomBorderColorCreateInfoEXT::format` to it and clamps the colour when it is set; `sampler_for()`'s per-thread table keeps the border format in each entry, so a sampler made for one view format is not handed to a view of another. - `SamplerInfo` (the sampler book's key) records the custom border colour's format (19 to 20 words, still without padding). - `texture_view_of()` gains a `format_out` argument; `HandleHit` keeps the view's format, so a cached hit returns it too. - `build_descriptors()` passes each texture's view format to `sampler_for()` (repeated table entries reuse it; dummy views pass `VK_FORMAT_UNDEFINED`). - tests: `tests/sampler_admission_test.cpp` and `tests/test_sampler_admission.py`: a `border-format` case on the three routes (by id, `sampler_make()` directly, the cache off) with a colour past [0, 1]; the fixture records the format each custom border colour names. The bug, step by step: 1. `vk_formats.cpp` maps `NVN_FORMAT_RGB565` to `VK_FORMAT_B5G6R5_UNORM_PACK16` and `NVN_FORMAT_RGB5` to `VK_FORMAT_B5G5R5A1_UNORM_PACK16`. 2. `vk_main.cpp` turns custom border colours on only with `customBorderColorWithoutFormat` (`dev.custom_border_color`). 3. For a border colour that is not one of the three standard ones, `sampler_make()` sets `VK_BORDER_COLOR_FLOAT_CUSTOM_EXT` with `bc.format = VK_FORMAT_UNDEFINED`. 4. `build_descriptors()` pairs that sampler with whatever view the handle names (`sampler_for((int)(handle >> 20), d.is_depth)`), including views of the formats in step 1. Every condition of 04015 holds. The rules: - VUID-VkSamplerCustomBorderColorCreateInfoEXT-format-04015: "If the sampler is used to sample an image view of VK_FORMAT_B4G4R4A4_UNORM_PACK16, VK_FORMAT_B5G6R5_UNORM_PACK16, VK_FORMAT_A1B5G5R5_UNORM_PACK16, or VK_FORMAT_B5G5R5A1_UNORM_PACK16 format then format must not be VK_FORMAT_UNDEFINED". Main breaks it; this PR names the format for those four. - VUID-VkSamplerCreateInfo-borderColor-04442: "If borderColor is one of VK_BORDER_COLOR_FLOAT_CUSTOM_EXT or VK_BORDER_COLOR_INT_CUSTOM_EXT, and VkSamplerCustomBorderColorCreateInfoEXT::format is not VK_FORMAT_UNDEFINED, VkSamplerCustomBorderColorCreateInfoEXT::customBorderColor must be within the range of values representable in format". The colour is clamped to [0, 1] when a format is named. - VUID-VkSamplerCustomBorderColorCreateInfoEXT-format-07605: "If format is not VK_FORMAT_UNDEFINED and format is not a depth/stencil format then the VkSamplerCreateInfo::borderColor type must match the sampled type of the provided format, as shown in the SPIR-V Type column of the Interpretation of Numeric Format table". The four formats are UNORM (float) and the border colour is `VK_BORDER_COLOR_FLOAT_CUSTOM_EXT`. - VUID-VkSamplerCustomBorderColorCreateInfoEXT-format-04014: "If the customBorderColorWithoutFormat feature is not enabled then format must not be VK_FORMAT_UNDEFINED". Unchanged: the feature is required for custom border colours, so `VK_FORMAT_UNDEFINED` stays valid for every other view. Before: ```cpp else if (dev.custom_border_color) { ci.borderColor = VK_BORDER_COLOR_FLOAT_CUSTOM_EXT; std::memcpy(bc.customBorderColor.float32, p.border, 16); bc.format = VK_FORMAT_UNDEFINED; ci.pNext = &bc; ``` After: ```cpp else if (dev.custom_border_color) { ci.borderColor = VK_BORDER_COLOR_FLOAT_CUSTOM_EXT; std::memcpy(bc.customBorderColor.float32, p.border, 16); bc.format = bfmt; // (VUID-VkSamplerCustomBorderColorCreateInfoEXT-format-04015) if (bfmt != VK_FORMAT_UNDEFINED) // (VUID-VkSamplerCreateInfo-borderColor-04442: within what the UNORM format holds) for (float& c : bc.customBorderColor.float32) c = c > 0.0f ? std::min(c, 1.0f) : 0.0f; ci.pNext = &bc; ``` ## Measurements **Verdict:** it fixes the bug: `tests/test_sampler_admission.py` fails 3 of 153 cases on main and passes 153 of 153 with this PR, in both interleaved runs. No speed change is expected: one more word in the sampler's key, and a new sampler only for a packed 16-bit view with a custom border colour. Hardware: AMD Radeon RX 9070 XT, Windows; the test itself is game-free (it compiles a cut-out of the source with clang++ from llvm-mingw and runs it on the CPU); Python 3.12 Scene: none, game-free Method: `tests/test_sampler_admission.py` from this PR with `--source` set to each side's checkout, 2 runs each, interleaved (main, PR, main, PR); the result is pass/fail, so the median is the result both runs gave How to reproduce: `python -I -B tests/test_sampler_admission.py --cxx clang++ --headers <Vulkan-Headers include> --source <checkout>` for each side | | Before: main @ ac778751 | After: this PR @ e279a9b2 | |---|---|---| | `tests/test_sampler_admission.py`, run 1 | fails (3 of 153) | passes (153 of 153) | | `tests/test_sampler_admission.py`, run 2 | fails (3 of 153) | passes (153 of 153) | | Median | fails | passes | ``` run 1, main @ ac778751: 3 of 153 fail: border-format on the id, make and cache-off routes run 1, this PR @ e279a9b2: 153 of 153 pass run 2, main @ ac778751: 3 of 153 fail: border-format on the id, make and cache-off routes run 2, this PR @ e279a9b2: 153 of 153 pass ``` What the test checks: for a view of each of the four packed formats, the custom border colour names that format and its colour {1.5, -0.5, 0.75, 1} becomes {1, 0, 0.75, 1}; for any other view it names `VK_FORMAT_UNDEFINED` and keeps the colour; a sampler made for one view format is not returned for a view of another. ### Flags and settings None. ## Testing **Tested:** on this branch, Windows, Python 3.12, clang++ from llvm-mingw: `test_sampler_admission` passes (fails on main), and the other checks that read the changed code pass: `test_flat_map`, `test_handle_lookup`, `test_prof_symbols`, `test_taau_env_lookup` (its seven cases pass; on Windows it then fails to delete its temporary `fixture.exe`, the same on main). **Built and played:** this commit builds on Windows (llvm-mingw back end) and on Linux (`longshot_vk.so`, without and with AMD FSR 4). Played on Windows, AMD Radeon RX 9070 XT: the game runs normally. **Not tested:** a packed 16-bit texture sampled past its edge with a custom border colour, looked at in the game (which textures and samplers the game pairs that way was not traced). Linux was built, not run. **Must be tested:** None: the change is the same on every GPU and OS, and the game-free test covers it. ## Risks **What could break:** a packed 16-bit view with a custom border colour now gets its own sampler, so up to four more samplers can exist per custom border colour. `VUID-VkSamplerCreateInfo-None-04012` limits how many samplers with custom border colours a device can have (`maxCustomBorderColorSamplers`); main does not check that limit either, and this PR does not change that. A border colour outside [0, 1] on those views now reads clamped, which is all the format can hold. **Turning it off:** revert the commit. **Compatibility:** Nothing changes. ## Checklist ### Everyone - [x] One topic. - [x] The title is a Conventional Commit (`type(scope): description`) of the status line's type. - [x] Why gives the reason plainly and links the issues that fit (the tracker searched). (no issue fits) - [x] Tested on my own PC where it runs (in the game, for a change the game runs), as Testing's Tested says; what can't be run by hand is in Testing's Not testable by hand, with why. - [x] Measurements has a verdict, or one line saying why there is nothing to measure. - [ ] Nothing gets worse that I know of: any cost (speed, latency, memory, a setup that worked before) is measured and in Risks. Not ticked yet: Risks names up to four extra samplers against a device limit that main does not check, and this is unmeasured. - [x] Testing names what was tested, what wasn't, and what must still be tested, where and how. - [x] Rebased on current `main`. - [x] `python -I -B tests/run_all.py` passes (any failure that also fails on `main` is named). (the checks that read the changed code, on this PR, Windows: `test_sampler_admission`, `test_flat_map`, `test_handle_lookup`, `test_prof_symbols`, `test_taau_env_lookup`; the last one's cleanup error on Windows also happens on `main`) - [x] No game files, dumps, build output or generated files. - [x] `third_party/` untouched (a new folder with a `PROVENANCE.md` instead). - [x] The license is OK: GPL-3.0-or-later, nothing incompatible and nothing decompiled. ### When it applies - [x] AI assistance recorded with `Assisted-by:` / `Co-Authored-By:` trailers. - [x] A game-free test added for the fix, failing on `main` and passing here. (the `border-format` case of `tests/test_sampler_admission.py`) - [x] A check under `tests/` that reads changed source updated in this PR, not weakened. (`tests/sampler_admission_test.cpp` follows the new arguments; no case removed) - [x] Host code builds on both Windows (llvm-mingw) and Linux, with UTF-8 paths. (built on Windows with llvm-mingw and on Linux, `longshot_vk.so` without and with AMD FSR 4; no OS-specific code or paths)
VUID-VkSamplerCustomBorderColorCreateInfoEXT-format-04015: a sampler with a custom border colour that samples a B4G4R4A4, B5G6R5, A1B5G5R5 or B5G5R5A1 view must name that format. The new border-format case fails on main: the format stays VK_FORMAT_UNDEFINED.

Assisted-by: GitHub Copilot
fix(vk): name the view's format in custom border colours of packed 16-bit textures
Some checks failed
CI / Conventional Commit title (pull_request) Blocked by required conditions
CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Blocked by required conditions
CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Blocked by required conditions
CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Blocked by required conditions
CI / Workflow lint (actionlint) (pull_request) Blocked by required conditions
Guide screenshots / Guide images and local UI source review (pull_request) Blocked by required conditions
Third-party parts / As their notes say (pull_request) Blocked by required conditions
Third-party parts / The same as upstream (pull_request) Blocked by required conditions
Website / Website on Cloudflare (pull_request_target) Has been skipped
PR description / Description follows the template (pull_request_target) Failing after 7s
e279a9b28b
VUID-VkSamplerCustomBorderColorCreateInfoEXT-format-04015: a sampler with a custom border colour that samples a B4G4R4A4, B5G6R5, A1B5G5R5 or B5G5R5A1 view must not leave format VK_FORMAT_UNDEFINED. The descriptor site passes the view's format to sampler_for and sampler_make; those four formats join the sampler's key and set the border's format, with the colour clamped to [0,1] (VUID-VkSamplerCreateInfo-borderColor-04442). Other views keep no format.

Assisted-by: GitHub Copilot
Author
Contributor

Tested in game: Played on Windows, AMD Radeon RX 9070 XT: the game runs normally.

Tested in game: Played on Windows, AMD Radeon RX 9070 XT: the game runs normally.
jcabarca changed title from WIP: fix(vk): name the view's format in custom border colours of packed 16-bit textures to fix(vk): name the view's format in custom border colours of packed 16-bit textures 2026-11-16 21:09:06 +00:00
TrashPanda approved these changes 2026-11-17 00:56:07 +00:00
TrashPanda left a comment

A packed 16-bit texture with a custom border colour now names that texture's format on the sampler. I read head e279a9b2. Approved.

Vulkan forbids a custom border colour with no format when the view is one of B4G4R4A4, B5G6R5, A1B5G5R5 or B5G5R5A1. The sampler was built with VK_FORMAT_UNDEFINED for every view. border_format returns the view's format for those four and VK_FORMAT_UNDEFINED for the rest, which customBorderColorWithoutFormat still allows. The colour is clamped to [0, 1] only when a format is named. The sampler cache, the per-thread table and the sampler book all keep that format, so a sampler made for one of those views is not handed to another.

The view's format comes back from texture_view on the thread that made it, and a cached handle keeps the format from when the view was made. A missing view still asks for no format. The check compiles sampler_make and records the format the custom border colour names: each of the four formats, the clamp of {1.5, -0.5, 0.75, 1} to {1, 0, 0.75, 1}, and an ordinary view left undefined. It fails on main. The Windows play on an RX 9070 XT did not look at a packed texture past its edge.

A packed 16-bit texture with a custom border colour now names that texture's format on the sampler. I read head `e279a9b2`. Approved. Vulkan forbids a custom border colour with no format when the view is one of `B4G4R4A4`, `B5G6R5`, `A1B5G5R5` or `B5G5R5A1`. The sampler was built with `VK_FORMAT_UNDEFINED` for every view. `border_format` returns the view's format for those four and `VK_FORMAT_UNDEFINED` for the rest, which `customBorderColorWithoutFormat` still allows. The colour is clamped to [0, 1] only when a format is named. The sampler cache, the per-thread table and the sampler book all keep that format, so a sampler made for one of those views is not handed to another. The view's format comes back from `texture_view` on the thread that made it, and a cached handle keeps the format from when the view was made. A missing view still asks for no format. The check compiles `sampler_make` and records the format the custom border colour names: each of the four formats, the clamp of {1.5, -0.5, 0.75, 1} to {1, 0, 0.75, 1}, and an ordinary view left undefined. It fails on main. The Windows play on an RX 9070 XT did not look at a packed texture past its edge.
Owner

Brought up to the ready-for-review standard (#1454): the checklist's Everyone list has two new boxes, added here from what this PR's description and comments show (please check they match what you know).

  • Tested on my own PC: ticked: Testing's Tested and the author's comment: played in game on Windows, AMD Radeon RX 9070 XT.
  • Nothing gets worse that I know of: not ticked: Risks names up to four extra samplers against a device limit that main does not check, and this is unmeasured. Measure that cost and state it in Risks, then tick it.
Brought up to the [ready-for-review standard](https://git.magicbean.dev/MagicBeanDev/Longshot/src/branch/main/CONTRIBUTING.md#ready-for-review-out-of-wip) (#1454): the checklist's Everyone list has two new boxes, added here from what this PR's description and comments show (please check they match what you know). - *Tested on my own PC*: ticked: Testing's Tested and the author's comment: played in game on Windows, AMD Radeon RX 9070 XT. - *Nothing gets worse that I know of*: not ticked: Risks names up to four extra samplers against a device limit that main does not check, and this is unmeasured. Measure that cost and state it in Risks, then tick it.
Some checks failed
CI / Conventional Commit title (pull_request) Blocked by required conditions
CI / Python source checks (ubuntu-24.04, part 1/2) (pull_request) Blocked by required conditions
CI / Python source checks (ubuntu-24.04, part 2/2) (pull_request) Blocked by required conditions
CI / SPIR-V fixtures (global narrow load/store, direct surface atomics, F2I immediate) (pull_request) Blocked by required conditions
CI / Workflow lint (actionlint) (pull_request) Blocked by required conditions
Guide screenshots / Guide images and local UI source review (pull_request) Blocked by required conditions
Third-party parts / As their notes say (pull_request) Blocked by required conditions
Third-party parts / The same as upstream (pull_request) Blocked by required conditions
Website / Website on Cloudflare (pull_request_target) Has been skipped
PR description / Description follows the template (pull_request_target) Failing after 7s
This pull request has changes conflicting with the target branch.
  • gpu/vk/vk_draw.cpp
  • tests/sampler_admission_test.cpp
  • tests/test_sampler_admission.py
Some workflows are waiting to be reviewed.
View command line instructions

Manual merge helper

Use this merge commit message when completing the merge manually.

Checkout

From your project repository, check out a new branch and test the changes.
git fetch -u 'fix/vk-border-color-format:jcabarca-fix/vk-border-color-format'
git switch 'jcabarca-fix/vk-border-color-format'
Sign in to join this conversation.
No reviewers
No milestone
No project
No assignees
3 participants
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
MagicBeanDev/Longshot!1232
No description provided.